Presentation generation is an intermediate stage of the Flick idl compiler process. Presentation generator modules (typically) take as input an aoi interface representation, and output an interface representation that is specific to a programming language. The output also specifies a default presentation (see Section 11.1) for stubs and data types. This chapter explains the Flick presentation generation system in detail for the C and C++ languages.
The translation from aoi to pres_c moves the interface representation toward stub code generation by introducing a message format (mint) and a language-specific representation (cast) of data types and functions in the interface. As described in Chapter 7, pres_c structures contain mint and cast structures. Utilizing these mint and cast structures is a third part of pres_c, an array of stub definitions.
The presentation of an interface encompasses everything the application programmer must know to program to and implement the interface. This includes the stub signatures, data type names and structure (if transparent), and specialized support functions directly related to the interface. This also includes any semantics that may be associated with passing values in the system (by reference or value), runtime routines for initialization/destruction of the data types, and when/where/how allocation and deallocation is expected to take place. The presentation generator is responsible for completely describing these requirements without dictating an implementation for them. The implementation is left to the back end which takes into account the message encoding, link-layer details, and the target runtime's API.
The typical rpc presentation follows a general pattern, regardless of the specific presentation. This pattern is a set of automatically-generated stubs that execute communication between a client program and a server program. Each operation includes a request, or message sent by the client to the server with any data to be acted upon, and a reply, or message sent by the server back to the client with any data that resulted from the action performed. There is almost always some sort of object model, where the operations belong to a specific object (or server entity), and they are invoked as methods on an object instance (via a reference to the object instance). These object references contain the information necessary for the runtime to direct messages to the appropriate object for invocation.
In general, there are four major types of parameters that may be a part of the request or reply messages. Note that not all presentation use all of these major parameter types. The following list briefly describes each type.
In addition to the major types of parameters, there are a set of communication stubs produced for every function declared in the input idl file. These include:
In addition to these generated functions, there are prototypes generated for the work functions that implement the operations defined in the idl.
Some work has been done on specializing Flick to generate non-traditional communication stubs for a specific class of distributed applications. These stubs reflect a message-passing methodology rather than rpc or rmi, however they still have notions of requests and replies. We call them "decomposed" because we have split apart the separate main tasks of a typical rpc stub into their own stubs, namely marshaling, unmarshaling, sending, and receiving. Note that while the presentation of these stubs is significantly different, the definition of the interfaces and operations (e.g., corba idl) has not been modified. We can produce either regular rpc stubs or our decomposed stubs from a single .IDL file.
By decomposing the separate tasks of an rpc call, we gain finer grain control over when and how the system creates and exchanges messages. For example, one could construct a single message and send it to multiple servers without paying for the marshaling cost multiple times. Further, when a message is sent, it is not necessary to idly wait for its reply (thus allowing for asynchronous communications). Messages can be forwarded (by resending them) to another server without paying the cost of decoding and reencoding the message contents. Finally, the handling of a received message (on either side) can be postponed until a later time (e.g., if other data is not yet available).
The decomposed stubs presentation generates the following types of communication stubs:
As in the default presentation, there are prototypes generated for the work functions that implement the operations described in the idl. The prototypes are significantly different, however, in that they take a marshaled message rather than the unmarshaled parameters. This allows the application control over when (or even if) the message is unmarshaled. Another difference is that work function prototypes are generated for replies (on the client side) in addition to the expected ones for requests (on the server side). This completes the symmetric handling of requests and replies on each side of communication (thus the relation to message-passing versus standard rpc) and allows for complete asynchronous communication.1
The decomposed stub presentation is currently only supported in the corba C presentation generator and the iiop back end and runtime. The Khazana runtime, when finished, will fully support the decomposed presentation as well. Support exists (in fact, much is implemented) in the main presentation generator library, but the presentation itself was a modification of the C mapping for corba, and thus was not carried into other presentations. To try them out, issue the command-line option "-a" (for asynchronous) to the presentation generator.
Finally, it was intended that the resulting stubs for the decomposed stubs presentation be compatible and interchangeable with the standard corba presentation stubs. Unfortunately, our goal could not be fully met, as the decomposed stubs required some additional information be sent in both request and reply messages, and the communication model is slightly different (the server contacts the client to send the reply message, rather than replying on the communication socket initiated by the client. A sufficiently intelligent runtime system would iron out the details and allow a standard/decomposed stub mismatch between client and server.
The process of translating aoi to pres_c involves the generation of mint, cast and stub definitions. This section discusses these data translation steps in more detail. For detailed information regarding the structure of the internal representations themselves, see their respective chapters in this guide.
Translation of aoi to pres_c involves the translation of each element of aoi.defs<>, the sequence of aoi definitions. This is accomplished in two main stages. The first stage is building the mint. The PG starts by creating a list of "primitive" mint definitions, corresponding to the primitive types supported by the presentation (those that are expected to exist without being defined, such as "int" or "float"). It then makes an optional initial "preprocess" of the aoi, possibly translating or modifying information to make the pres_c conversion easier or more straightforward. For example, corba's interface attributes are presented as accessor functions, so the preprocess phase in the corba PGs translate these attributes into operations, so the second stage can do the obvious translation on an operation. Finally, it traverses the entire aoi tree making a one-to-one translation of the aoi definitions into their mint counterparts.
The second stage requires building the necessary cast and linking it to the mint through the pres_c structures. This is (broadly) the function of pg_state::gen. First, the cast is initialized similar to mint with the primitive definitions that are expected to exist by default in the presentation. Second, the aoi definitions are walked, building the pres_c definitions hierarchically.
The pres_c definitions generated are specific to either the client or server side of the communication, based on parameters given to the presentation generator. Currently the infrastructure is not sufficient to generate both the client and server presentations at the same time. Perhaps this is a future direction that can be explored.
Other parameters that can provide information, direction, and guidance to a PG belong to the pg_flags class, which is shared by all presentation generators. These include the flags to choose client or server presentation (as described above), those for inhibiting generation of inherited operations (for when they are generated separately), one to enable the use of security identifiers, and finally one to enable the "decomposed" stub presentation style. In addition to the pg_flags, there is a large set of options that can modify the naming conversion between the idl and the generated code.
One of the differences between presentations is the names assigned to data types and operations defined in the idl. There is typically a well defined pattern which can be used to determine the name of an operation or data type one can use in the implementation from that described in the idl specification. For example, the corba C mapping dictates that a data type bar inside of an interface foo can be used by the implementation as foo_bar. Under the corba C++ mapping, this becomes foo::bar. Flick has a relatively simple method of describing and handling these name conversions, very similar to C's printf: calc_name.
For more information on how to redefine or define new calc_name mappings, see the code documentation available in mom/c/pfe.hh and c/pfe/lib/p_calc_name.cc. The default set of name mappings can be found in c/pfe/lib/pg_state.cc (and likewise the sets of mappings for each presentation can be found in the presentation's .cc file).
Another common difference between presentations is the mapping from the IDL-defined operations to the actual functions that are created for use by the idl compiler. In general, there is a one-to-one correspondence between the two sets; however that need not be the case (e.g., the decomposed stubs presentation). In addition, there are often "special" parameters that need to be added before and/or after the normal parameters given in the idl (e.g., corba's object reference and environment parameters). These differences cannot be expressed solely by name translation.
There are high-level generation functions that allow one to control the stubs or skeletons that are generated. p_client_stubs handles all of the client-side stubs that are to be generated. For the decomposed stubs, this is modified to produce a request marshaler and reply unmarshaler for every operation, rather than just a single client stub. p_skel is the counterpart that produces a skeleton, or dispatch routine, again modified for the decomposed stubs to also produce request unmarshalers and reply marshalers for every operation on the server side. The decomposed presentation also uses p_skel to generate the client-side dispatch routine for replies.
Other high-level generation function include p_marshal_stub and p_unmarshal_stub to generate marshal and unmarshal functions for a specific data type, and p_server_func to generate the prototype for a server work function. More exist that are currently specific to the decomposed presentation, including p_continue_stub, p_send_stub, and p_receive_func.
Each of the high-level functions are similar in that they create a new stub description in the cast and pres_c (simultaneously) and establish the link between the the description and the mint. As part of creating the function description, one of the process_*_params functions is used to create and handle each parameter of the function, including special parameters (ones that must exist for the presentation but do not appear explicitly in the idl) such as the object reference or the corba environment. From there, parameters are handled individually by p_param, and dispatched to a specific data type presentation via p_type.
Most parameters are explicit, meaning they appear in the presentation. However, there are some cases where a parameter is implicit, meaning it may be constant or exist at the global level, but it isn't a formal parameter to the stub. An example of this is errno in the onc rpc presentation: it is a global error property that must be set within the stub, but from the standpoint of the stubs, it can be viewed as an implicit parameter.
Data types are generated by the p_*_type family of functions, some of which may recursively call each other to construct more complex types. Each operates on a type collection that potentially holds more than a single (default) cast and pres_c mapping tree for the type. This is particularly necessary for the corba C++ presentation that yields multiple presentation types from a single idl type. In general, however, only the default type is used.
Following are brief descriptions of the p_type_collection and related interfaces.
The p_scope_node class is used with the p_type_collection class to allow type declarations to be separated into separate C++ scopes. The scope node holds a reference to the cast_scope and the list of types in this scope.
The p_type_node class is used to hold any information about a type in the presentation generator. The types it represents can be real (i.e., they end up defined in cast) or just a different version of another type (i.e., it has a different interpretation than the standard version).
An additional aspect of presentation generation is the semantic relationship between the entities of the presentation (at least as far as organization). For example, an idl file may include other idl files for definitions of common data types or interfaces from which to inherit. Because they are defined, the presentation generator creates the necessary pres_c for them. However, it may be useful to know the origin of these inherited operations or included data structures, especially if they are compiled separately and should not be included with the native interfaces.
For this purpose, the metadata provided in the aoi is copied into the pres_c, and associations between aoi entities and the metadata are preserved when generating corresponding pres_c entities. Further, presentation-only entities (necessary constructions that are not based on the input) can be linked appropriately to the metadata structures by the presentation generator. For more information on metadata, see Chapter 3.
Presentation generation programs in Flick are built using a base library containing the main procedure for the program and a C++ class, pg_state, which maintains the presentation generation state information. The top-level member function in pg_state is gen, which starts presentation generation for the aoi stored in the class member in_aoi (input aoi data structure).
IDL-specific presentation generation functionality is achieved through inheritance of pg_state and overriding some or all of its virtual functions as needed. Two additional libraries exist that have specialized the base presentation generator library to the standard corba C and C++ mappings, providing a starting ground for generators that implement or extend these corba standards. These are named pg_corba and pg_corbaxx, respectively.
Presentation generation code can make use of library routines in the libaoi, libmeta, libmint, libcompiler, c/libcast, and c/libpres_c directories.
The base Presentation Generator Library for the C language is in the directory c/pfe/lib, and the class declaration can be found in mom/c/pfe.hh. To extend the base library, one must inherit from the pg_state and write a simple getGenerator function to return an instance of the new class. By overriding the p_* methods, one can change the structure of the generated pres_c. The other methods are higher-level and either create or return additional state information or guide the pres_c generation at a higher level (e.g., process_client_params guides the generation of all the parameters of a client stub).
In general, this base library and other presentation generators are specific to the C language (the exception being the corba C++ PG library). Typically, support for a language other than C will require the development of a language-specific abstract syntax tree (analogous to cast) and presentation structure (analogous to pres_c). Since C++ is not far distant (structurally) from C, it was possible to extend parts of cast and pres_c to include the necessary information to describe and generate C++ code. However, it is not in harmony with the original intention of either cast or pres_c to be extended in this way.
The corba C PG implements the standard corba mapping for the C language.
As mentioned above, the corba C mapping is implemented as a library and can be found in c/pfe/libcorba. The class declaration is in mom/c/pg_corba.hh. The corba C PG implements the pg_corba class.
corba-style typed allocators are not yet implemented in the corba C language presentation. The corba C language presentation generator creates presentations in which heap memory is allocated with the type-independent functions CORBA_flick_alloc and CORBA_flick_free. (CORBA_alloc and CORBA_free are aliases for these functions.) To release an object reference, one must explicitly call CORBA_object_release.
The corba C++ PG implements the standard corba mapping for the C++ language. It should be noted that exception handling is performed as in the corba C mapping, rather than using C++ exceptions. Since the C-style mapping for corba exceptions is permitted by the corba C++ mapping specification, it was decided to reuse the current infrastructure rather than spend the time developing new infrastructure to perform C++ exception handling.
Like the corba C mapping, the C++ mapping is also implemented as a library in c/pfe/libcorbaxx. The class declaration is in mom/c/pg_corbaxx.hh. The corba C++ PG implements the pg_corbaxx class, and is currently the only generator that handles C++. At a high level, this PG modifies the base library in much the same way that the corba C PG does, since the two presentations are closely related. The specific details, however, are different for C++, and are thus reflected in the specific cast and pres_c generated.
Ideally, a presentation generator should not be specific to or customized for any particular runtime; that's the job of the back end. However, there may be a few things we have overlooked that are specific to the TAO runtime, since we support no other C++ runtime.
As mentioned above, C++ exception handling cannot currently be expressed in Flick. Adding this infrastructure is not trivial, but may simplify some of the other code generation, such as handling communication or allocation errors. It would be nice, in the future, to make use of C++ exceptions, not only for the presentation, but in the stub and runtime layers as well.
The onc rpc presentation is provided by pg_sun that specializes pg_state. The class declaration is in c/pfe/sun. This is a relatively simple PG in that all operations take at most one "in" parameter, and return at most one value. The only errors possible are those defined by the system and allocation and deallocation are straightforward.
The onc rpc presentation generator does not produce rpcgen-like marshal and unmarshal stubs (xdr translation routines). It would likely not be hard to implement such stubs, but it has not been a priority for the development team.2
Fluke is the operating system being developed by the Flux project at the University of Utah. All of Fluke's communication stubs are produced by Flick.
The class declaration for the Fluke presentation generator is in c/pfe/fluke/pg_fluke.hh. The Fluke PG specializes the pg_corba class to produce pg_fluke).
The Fluke (MOM) presentation generator may produce bad code for recursive data types. This is because all references are "unqualified"; for example, we always refer to struct foo as foo, never struct foo. We do this for esoteric reasons. Eventually, we can fix self-referential data types by changing the PG library to separate the name definition from the structure definition:
typedef struct foo foo;
struct foo {
...
};
|
The Mach Interface Generator (mig) for the Mach OS provides a special presentation that is closely related to the input idl. Or rather, the idl allows the user some control over the presentation in addition to defining interfaces. As such, it is not possible to simulate mig's behavior through the normal function of the presentation generator. For this reason, Flick's mig PG is implemented as a combined front-end and presentation generator to enable the presentation control the mig idl allows. Rather than cluttering our aoi representation with hints and hacks to allow for mig, the mig PG directly translates a representation of the input idl (.defs) file to a pres_c file. The mig presentation generator can be found in fe/mig.
The mig PG is based on mig's lex- and yacc based parser that converts mig idl into an internal parse tree. The Flick-specific "back end" then walks over this parse tree in order to produce the appropriate mint representations of the messages and pres_c representations of the to-be-generated client stubs and server skeletons. The Flick-specific code is contained in the following files:
Presentation generation for mig is organized differently from the other presentation generators, although the algorithm is the similar. The main difference is that we generate pres_c and mint simultaneously from the special mig idl parse tree, rather than separately from an aoi tree. The bulk of this translation is coded in xlate_pres.c.
The guiding function is make_routine which handles the creation of the mint and pres_c for exactly one rpc routine. It fills in several "out" data structures that are used by the higher-level translate_routines to create a pres_c_stub.
The high-level mint and pres_c is constructed by make_top_inlines, which produces everything from the union of all idl's (the top-most mint union) down to the union of the operation's request and reply, and returns the necessary hooks to produce the request or reply parameters for the operation. The pres_c for each parameter is then processed by make_arg_inlines, which is in charge of expanding a single idl argument into multiple presented arguments if needed, e.g., an array and its length. Note that these multiple presented arguments are still contained within a single inline because they are related in some fashion. The mig presentation is unique in this regard, where idl entities are expressed through multiple arguments (as opposed to corba or onc rpc where they are encapsulated within a single C structure). Finally, make_arg guides the specific pres_c construction for a single presented parameter.
There are three routines which make the lowest-level pres_c_mapping for a particular parameter type, as follows:
If the parameter type is an array, the element type will be created, and an array will be "interposed" on top of the element's mapping. Other things may be interposed on the mapping, such as the parameter direction, translation (between user-defined and compatible native types), allocation semantics, and the parameter's root tag. These functions are named interpose_* or pres_c_interpose_*, depending on whether or not they belong to the pres_c library.
Two important functions regarding allocation and deallocation are make_arg_server_dealloc_in (to deallocate "in" parameters on the server side) and make_arg_server_alloc_out (to allocate "out" parameters on the server side). The latter corresponds directly to pg_state::p_param_server_alloc_out from the PG library. There should be a corresponding pg_state::p_param_server_dealloc_in, however most other presentations rely on stack-based allocation of "in" parameters on the server side and thus require no explicit deallocation. mig however has the capability to send data out-of-line, and such data will be allocated automatically by the system in the receiving process's address space. In some cases the rpc system determines whether data should be sent inline or out-of-line, but must properly deallocate it to preserve a consistent presentation. There are also a few get_*_allocation functions to return the common pres_c_allocation structures describing mig's allocation semantics.
Flick's mig PG still cannot handle everything mig itself supports. Most of the unsupported features, however, shouldn't take long to implement; it simply hasn't been a high priority for the development team. Following is a list of unsupported migisms, and a brief description of how Flick might be extended to implement them. Note that many of the unsupported features will likely require modification to both the mig PG and the Mach3/mig back end. This list is also contained (briefly) in the file doc/BUGS. For more information regarding these concepts, refer to the MIG Server Writer's guide
Presentation generation is only half of the story. In many instances, a given presentation is almost what the user wants, with the exception of only a few minor tweaks, such as name changes or slight variations on semantics. For example, a corba-like presentation might be desired, but with the allocation semantics such that a zero-copy environment is possible (e.g., out parameters are always preallocated by the caller on the client side, so the data can be placed exactly where it needs to be rather than in freshly allocated space). While it is relatively simple to implement this functionality by deriving a new class from the corba PG library, it would be far easier and more useful to have a presentation modification system, where (at least some) pieces of the presentation can be tweaked according to some description. Ideally, even more complex presentations such as decomposed stubs could be expressible in a presentation modification language.
At one time, Flick had such a utility, named pdl, which can be found in the c/pdl directory. It even served as a driving utility, invoking the other stages of Flick as needed to produce stubs in a single step. The code is severely out-of-date, however, and has been largely ignored as advancements in Flick have been made. In the future, it is intended to resurrect this relic (or at least its intention) to provide a simple means whereby application writers may modify a presentation to their needs and liking, rather than limiting extension only to developers familiar with Flick's internal design.