1 Introduction

This guide assumes that the reader is familiar with the basics of distributed application programming: RPCs, RMIs, IDLs, and the various distributed computing standards such as CORBA and ONC RPC. Readers who are unfamiliar with distributed computing should review these terms before proceeding through this manual.

1.1 What is Flick?

Flick is a flexible and optimizing compiler for interface definition languages (idls). The name "Flick" is an acronym for "Flexible idl Compiler Kit." Like a traditional idl compiler, Flick can read a high-level specification describing an interface to a software component or module, and from that, produce special functions called stubs that implement the interface in the C or C++ programming language. The stubs implement remote procedure call (rpc) or remote method invocation (rmi) semantics and allow separate client and server processes to communicate through what appear to be ordinary, local procedure calls or method invocations.

Unlike traditional idl compilers, however, Flick is implemented as a set of programs, each handling a single phase of compilation. These separate programs allow Flick to support features beyond the capabilities of ordinary idl compilers:

• Multiple idls. Most rpc and distributed object systems, including corba and onc rpc (Sun rpc), specify an idl and from then on refer to it as the idl. Flick, however, does not introduce a new idl but instead supports a number of existing ones side-by-side. Any idl supported by Flick can be used to generate stubs for any environment (with some limitations). Supporting multiple idls allows for backward compatibility and migration of existing programs, and also allows developers to choose the best idl for the task at hand.
• Flexible language mappings (a.k.a. "presentations"). Flick is designed to allow the appearance and behavior of generated stubs to be altered to suit the needs of the particular client or server in which those stubs are used, without altering the "on-the-wire" protocol between client and server. In other words, Flick provides a framework for creating and modifying language mappings from idls to target programming languages (currently, C and C++).

  A language mapping is implemented by what Flick calls a presentation generator. The current framework allows Flick users to tweak a language mapping through command line options to a presentation generator, e.g., to change the patterns used to build names, or to tell the presentation generator to create "decomposed" stubs instead of ordinary stubs (as described in Section 11.1.2). Large-scale modifications, however, currently require a Flick programmer to write C++ code to implement the modified language mapping (as a new presentation generator). In the future, we hope to make it possible for "ordinary users" to make more and more complicated changes to language mappings, e.g., through some sort of special-purpose "presentation definition language."

• Sophisticated code generation and optimization. Flick was originally targeted toward communication in microkernel-based systems. In that environment, Flick provides the user-level glue between the microkernel's primitive ipc facilities and user-level application code. This target environment implies that Flick-generated stubs must be very fast: good microkernels always have extremely fast and tightly optimized primitive ipc paths, so inefficient stubs could easily raise the effective ipc time by an order of magnitude. Today, Flick produces optimized code for all of its supported transport facilities including tcp-based transports and lower-level, microkernel-based ipc.
• Multiple back ends. Finally, while many rpc and distributed object systems claim to support multiple transport protocols, the protocols they support are generally only slight variations of each other. For example, onc rpc supports multiple transports in the sense that it can pass messages across either tcp or udp links. However, the basic data encoding scheme used (xdr) is the same in each case, as are the stubs generated by the idl compiler. The multiple protocol support is really just a stream-based abstraction layer in the rpc runtime support library. There is little hope, for example, that the onc rpc system could easily be made to support the corba Internet Inter-orb Protocol (iiop) even though both are based on tcp.

  Flick, on the other hand, aggressively supports multiple protocols by allowing different stub code generators (back ends) to be used interchangeably. Each back end has complete control over the rpc protocols that it implements: the back end determines not only the underlying transport protocol (e.g., tcp, udp, Mach 3 messages, direct function call, etc.) but also the data encoding scheme used (e.g., Sun's xdr, dce's ndr, corba's cdr, "dumb" architecture-specific encoding, etc.) and the high-level rpc protocol layered above the low-level stream or message protocol (e.g., onc rpc, corba iiop, Mach 3's mig protocol, etc.). Flick contains several back ends supporting extremely diverse protocol combinations.

Flick is being developed as part of the Flux Operating Systems Project at the University of Utah. However, Flick is intended to be an open "kit" that others will extend and adapt for their own projects.

1.2 Organization of this Document

Part I of this manual, which you are reading now, introduces Flick. Chapter 2 contains an overview of Flick's components, a catalog of the directories in the Flick source tree, and a summary of the coding style used in the Flick sources.

Part II of this manual describes the data structures that Flick uses at different stages of idl compilation. These data structures are the intermediate "languages" used by different parts of Flick and include:

• meta, for keeping track of the files that Flick reads and writes (Chapter 3);
• aoi, to represent idl specifications (Chapter 4);
• mint, to represent the abstract structure of on-the-wire messages (Chapter 5);
• cast, to describe C and C++ code (Chapter 6);
• pres_c, to represent the behavior of generated stubs and skeletons (Chapter 7); and
• scml, to define templates for other functions that Flick must generate as part of an interface presentation (Chapter 8).

Understanding these intermediate languages -- and the associated libraries for using them -- is critical to understanding Flick overall.

In Part III, this manual describes the inner workings of Flick's front ends (Chapter 10), presentation generators (Chapter 11), and back ends (Chapter 12). The chapters in this part of the manual describe the individual implementations of each compiler stage, as well as the libraries that are common to each stage. The guide concludes with a summary of the runtime libraries that provide support to the code generated by Flick's various back ends (Chapter 13).

In summary, this document is intended to be a programmer's guide to the inner workings of Flick: the overall architecture, the intermediate data structures, and the individual Flick compiler programs and libraries. This "road map" can only take you so far, however, and it is not a substitute for actually reading and working through the Flick source code. Flick is complex but not obscure -- don't be afraid to read the code!