The FastCGI interface, as specified in [FastCGISpec], is a packet-oriented communications protocol. Even though the underlying transport mechanism is connection-oriented, the FastCGI protocol is not. (Yes, there is a very good reason for this. Just read on.) Each message consists of a header and a body. The header contains — among others — the following fields:

Furthermore, the following message types are defined:

The typical conversation between the web server and the FastCGI application looks like this:

Some of you may wonder now why the message header contains a request ID. After all, if we have one connection per request, that type of information is redundant. That's right, only that there may be more than one request per connection! That is the reason why FastCGI uses a packet-oriented protocol rather than a single data stream: Multiple requests can be intermixed in one connection; software engineers will know that as “multiplexing”. Because every packet contains a unique request ID, the server can deliver an arbitrary number of requests simultaneously — using only a single connection. Furthermore, the FastCGI application can receive data for an arbitrary number of connections, each of which can carry an arbitrary number of requests.

Anyway, the description above misleadingly suggests that the communication takes place sequentially. That is not the case: The server may well send say twenty BEGIN_REQUEST packets in a row, then follow up with a few PARAMS packets from each request, then send some STDIN packets, then send some PARAMS packets again, and so on.

To make this whole thing even more flexible, the web server may also maintain multiple connections to one application, while continuing to multiplex multiple requests on each connection. The web server may just as well maintain multiple connections to several distinct FastCGI applications, all of them working in parallel. And since the communication may take place via TCP, these FastCGI applications may also run on different machines.

At first glance, this architecture looks like a nightmare to implement, but in fact writing an application capable of multiplexing several requests is pretty easy. We shall study a generic framework for FastCGI applications in the section called “Writing Web-based Applications using FastCGI” of this document, which allows the developer to write even complex applications without having to worry about every detail of the FastCGI protocol. But before we come to that, one more word about why anyone might want to use such an architecture at all.

Multiplexing several I/O channels within a single process is a technique that has been somewhat forgotten in recent years — probably because nowadays almost every operating system supports preemptive multitasking. Most developers seem to assume that with preemptive multitasking available, implementing applications capable of internal multiplexing isn't worthwhile anymore. This is wrong. Without a doubt, preemptive multitasking has innumerable benefits, and certainly nobody would ever again want to work on an operating system without it. But, having three separate processes running simultaneously just to read and write from three different I/O channels is a waste of resources.

When we compare an implementation of an I/O-bound application that uses one process per channel to an implementation that is capable of multiplexing the channels internally, we'll find some major advantages of the latter approach:

With FastCGI, we have an interface available that allows us to use internal multiplexing between several HTTP requests. That is one reason why well-designed FastCGI applications outperform “normal” web-based applications by an order of magnitude. An application capable of multiplexing can, upon receipt of a new request, send out the database query. While the database engine fetches the requested records, the application can send out a few pages that were requested a bit earlier and have been created in memory already. Then, it checks for results from the database again … nothing yet. OK, there is a new request coming in anyway, so we process that one first and send out the next query to the database. Meanwhile, the first query has returned results, so we start reading those via IPC. Reading the results may block after a kilobyte, though, because we can read those results faster than the database can come up with them. So we look at the second query again: no results yet. OK, then we start processing the results we already have from the first query. Once that has been accomplished, we try to read from the database again, etc. …

All this happens within a single process; no context switches by the OS are necessary to achieve that. Of course, multiplexing will not make the CPU run any faster than it did before — even with multiplexing, the machine can handle only so much load, but we can use that available processing power more efficiently. Consequently, the response time for the user improves. As soon as his results are available, they will be processed. And while the results are not available, we process somebody else's.

Apache 1.x does not come with out-of-the-box FastCGI support, but there is a third-party module available that adds FastCGI support to Apache: mod_fastcgi. The module can be found at [ModFastCGI].

Unfortunately, mod_fastcgi is not capable of multiplexing several requests over one connection; it needs one connection per request. It cannot even re-use that connection after a request has ben processed; mod_fastcgi will connect to the application every time a new request is received.

We hereby express our hope that this will be remedied in the near future. In defense of the mod_fastcgi author, though, it should be noted that Apache itself is unable to handle multiple requests within one process, so obviously there is little that the mod_fastcgi author can do about that shortcoming. Luckily, there are — contrary to what some people might believe — other web servers available that don't suffer from this limitation.

The scheduler library contains a single class, declaration of which is shown in Figure 1, “Declaration of the scheduler class”.

class scheduler
    {
  public:
    class event_handler
        {
      public:
        event_handler() { }
        virtual ~event_handler() = 0;
        virtual void fd_is_readable(int) = 0;
        virtual void fd_is_writable(int) = 0;
        virtual void read_timeout(int) = 0;
        virtual void write_timeout(int) = 0;
        };
    struct handler_properties
        {
        short        poll_events;
        unsigned int read_timeout;
        unsigned int write_timeout;
        };
    void register_handler(int fd, event_handler& handler,
                          const handler_properties& properties);
    void remove_handler(int fd);
    void schedule();
    };

The objective of scheduler is to manage a set of file descriptors and corresponding event handlers, and to notify the event handlers every time an event occurs on “their” file descriptor. An “event” is the file descriptor becoming readable or writable as well as a a file descriptor not becoming readable or writable within a certain amount of time (timeout).

An “event handler” is anything that satisfies the scheduler::event_handler interface. Since all member functions in event_handler are pure virtual, the programmer may register a derived version of that class as event handler, thereby providing the appropriate code for the four callback functions.

Registering a file descriptor and its event handler takes place via the register_handler() member function, which expects an additional third parameter: the scheduler::handler_properties. That is a structure containing the flags that determine what events the handler is interested in, plus the appropriate timeouts (in seconds) that apply when no event occurred for that amount of time.

Once at least one file descriptor (and its event handler) has been registered, the member function schedule() may be called. schedule() will use the UNIX function poll(2) to determine the state of the registered file descriptors. When poll() returns, schedule() will iterate through all registered descriptors and execute the appropriate callbacks depending on whether the descriptor is readable or writable.

Readers who are familiar with the concept of “design patterns” will have recognized this functionality: scheduler implements a “reactor” pattern — often referred to as “dispatcher”, too.

Since grey is, young friend, all theory, let us take a look at a specific example how scheduler might be used in practice. Figure 2, “An example event handler for the scheduler class” shows a callback that will read from file descriptor 0 and write anything it read to descriptor 1.

class my_handler : public scheduler::event_handler
    {
  public:
    my_handler(scheduler& sched) : mysched(sched) { }
    ~my_handler()                                 { }
    scheduler::handler_properties prop;
  private:
    virtual void fd_is_readable(int fd)
        {
        int rc = read(fd, &tmp, sizeof(tmp));
        if (rc == 0)
            mysched.remove_handler(0);
        else if (rc > 0)
            {
            buffer.append(tmp, rc);
            prop.poll_events = POLLOUT;
            mysched.register_handler(1, *this, prop);
            }
        }
    virtual void fd_is_writable(int fd)
        {
        if (buffer.empty())
            mysched.remove_handler(1);
        int rc = write(fd, buffer.data(), buffer.length());
        if (rc > 0)
            buffer.erase(0, rc);
        }
    char tmp[1024];
    string buffer;
    scheduler& mysched;
    };

Not all the code necessary to actually run that example is shown here, of course; we cut the program down to the parts essential for understanding the underlying principle. The interested reader will find the complete source code for this example at [Libscheduler], though.

Anyway, assume that the callback has been registered to be called when file descriptor 0 becomes readable. Assume furthermore that we don't care about timeouts. Then schedule() will use poll() to wait until file descriptor 0 becomes readable and then call the fd_is_readable() function of our callback.

The function now reads as many bytes from the descriptor as it can store in its internal buffer. (It won't matter if we don't read all available data in one go; if there is additional data pending on that stream, we'll be called again right away.) The data we read is now appended to a dynamically growing string instance, which we use as our output buffer, as will be seen later. Once the data has been appended to that string, we register ourself as callback for file descriptor 1, waiting for the “writable” event.

So the fd_is_readable() function will go on and on until the data stream associated with file descriptor 0 ends: Then the read(2) call will fail. On that occasion, we simply unregister ourself. Please note that this does not mean the process as a whole is finished now: There might still be some data waiting in the output buffer for descriptor 1 becoming writable!

The fd_is_writable() function behaves as follows: It will check whether the output buffer is empty. If it is, it unregisters itself, because obviously there is nothing to be done. If the output buffer is not empty, it will write as much data to file descriptor 1 as it can. Any data that has been written successfully will be removed from the output buffer.

That's it; all the intelligence is in that short class definition shown in Figure 2, “An example event handler for the scheduler class”. The main() routine of the program does nothing but instantiate the scheduler and my_handler class, register the latter as is-readable-callback for file descriptor 0, and call schedule() — if you are interested, you can see the complete source code at [Libscheduler].

It is easy to design a state machine that fulfils a certain task like reading, processing, and writing some data — exactly what the majority of FastCGI applications will do —, but using the scheduler class, it is even easier to have an arbitrary number of instances of that state machine run simultaneously, serving different channels.

This link library provides two classes: FCGIProtocolDriver and FCGIRequest. The protocol driver class is basically an interpreter of the FastCGI protocol, which frees the application developer from worrying about byte counting, endian conversion, and creating or interpreting packet headers. You will usually have one instance of this class per connection to the web server.

The second class, FCGIRequest, is spawned by the protocol driver every time a BEGIN_REQUEST packet is received. It contains the decoded information of the FastCGI request, such as a map of the parameters and the contents of the STDIN stream. It also contains a reference to the actual handler of the request — the code provided by the application developer.

class FCGIProtocolDriver
    {
  public:
    struct OutputCallback
        {
        virtual ~OutputCallback() = 0;
        virtual void operator() (const void*, size_t) = 0;
        };

    FCGIProtocolDriver(OutputCallback& cb);

    void process_input(const void* buf, size_t count);
    FCGIRequest* get_request();
    bool have_active_requests();
    };

class FCGIRequest
    {
  public:
    enum role_t { RESPONDER, AUTHORIZER, FILTER };

    const u_int16_t id;
    const role_t role;
    const bool keep_connection;
    bool aborted;
    std::map<std::string,std::string> params;
    std::string stdin_stream, data_stream;
    bool stdin_eof, data_eof;

    enum ostream_type_t { STDOUT, STDERR };
    void write(const std::string& buf, ostream_type_t stream = STDOUT);
    void write(const char* buf, size_t count, ostream_type_t stream = STDOUT);

    enum protocol_status_t
        {
        REQUEST_COMPLETE,
        CANT_MPX_CONN,
        OVERLOADED,
        UNKNOWN_ROLE
        };
    void end_request(u_int32_t appStatus, protocol_status_t protStatus);

    struct handler
        {
        virtual ~handler() { }
        virtual void operator()(FCGIRequest*) = 0;
        };
    handler* handler_cb;
    };

The example program that comes with this paper, and which can be found at [Simons2002], contains the header file infrastructure.hh. This file defines a generic framework for FastCGI application developers. It takes care of the low-level socket communication (I/O), error recovery in case of run-time errors, and instantiation and handling of the appropriate FCGIProtocolDriver and FCGIRequest objects. It provides the following classes:

template<class T>
class Listener : public scheduler::event_handler
    {
  public:
    Listener(scheduler& sched);
    virtual ~Listener()
    };

template<class T>
class ConnectionHandler : public scheduler::event_handler,
                          public FCGIProtocolDriver::OutputCallback
    {
  public:
    ConnectionHandler(scheduler& sched, int sock);
    };

This FastCGI application echoes all received information back to the user in an HTML formatted page. This includes:

That in itself is arguably not very useful, but hopefully the program is a useful example of how all the components of the FastCGI application framework can be put together.

The echo.fcgi program uses the helper classes described in the section called “FastCGI Application Framework” to provide its functionality. It consists only of a very short main function, which constructs the scheduler and Listener objects, and the class RequestHandler, which is derived from FCGIRequest::handler.

One major shortcoming of PHP and alikes is that the developer is forced to think in web pages: The application starts when a web page is requested, and it terminates as soon as the web page has been delivered. But you don't really want to design applications around the user interface; the focus of the design should be on the task the application is supposed to do.

Having a permanently running process makes it easier for the developer to design the application in a more structured way, because he has a clear separation between the program doing the actual work and the mark-up of the results of that work.

But the most important advantage, gained by FastCGI programs outliving the web page, is a different one: You don't lose your context. The application does not need to read its configuration file again and again, it doesn't have to open a connection to the database every time a request comes in, and it certainly doesn't need to pass a myriad of different variables from page to page just because all variables are gone by the time the user requests the next page. One can implement efficient caching, share results between requests, etc.

Numerous advantages result from your application no longer being tied to web pages — most of which will be described in this section. But when reading the following paragraphs, keep in mind that all these optimizations are possible only because FastCGI applications don't lose their context.

FastCGI applications scale easily. One could have several separate machines running the application, using the web server to balance the load between these machines. With the patch available at [Armbruster], the web server will even ensure that subsequent requests from the same user are handed over to the same FastCGI instance every time, which makes life a lot easier if the application needs some user context that would otherwise have to be shared among all running instances.

Alternatively, one might choose not to rely on the web server for the load balancing, but to build a small dispatcher application instead. Such a dispatcher would accept all incoming requests from the web server and forward them to the actual applications according to some clever algorithm. The possibilities are endless. Because the application is inherently capable of communicating over the network, it is easy to distribute the load between multiple servers.

In the section called “The Wonders of Multiplexing” we talked about this point already. “Ordinary” web-based applications, as we know them today, need one process per request. FastCGI applications don't. This allows for more efficient use of the available CPU power and system resources. As a result, FastCGI applications will usually be capable of handling hundreds of simultaneous requests, a number the process-based architecture cannot achieve.

Caching is about the most powerful tool to increase the performance of a web-based application. With FastCGI, all incoming requests will go through the same process, so you can cache much more efficiently than you could in CGI or PHP applications. In fact, in those latter architectures, you can hardly cache at all, because the applications terminate after each request. If a request dispatcher as described in the section called “Scalability” is deployed, it could be used to cache entire requests, too.

There are some high-quality implementations of caching algorithms available on the Internet — C++ programmers might want to look at the “Cache Associative Containers” available at [Cerniglia]. Using this class, adding caching capabilities to your applications is really simple.

What types of information an application should cache depends very much on the application and on the data, so no universally valid recommendation can be given. Here are some ideas, though, on what you might want to look at:

The architecture described in this paper uses the C++ programming language. Of course, it is possible to implement FastCGI applications in any programming language that supports interprocess communication, be it Perl, TCL, or even shell scripts. We nonetheless chose C++, because we feel that “real” programming languages are more appropriate for the implementation of complex applications. Writing web-based applications in C or C++ gives the developer more control over the machine. Especially under the UNIX operating system, numerous mechanisms are available only to software writers who use C.

Another advantage is that writing web-based applications in C or C++ gives you access to billions of lines of code freely available on the Internet, no matter whether it is an arbitrary-precision math library, a library for the creation and manipulition of images, or a set of routines implementing strong cryptography. It's all there — just use it!

Also, the applications are self-contained; if the latest PHP version has a nasty bug, what do you care? All the bugs in the code are your own bugs — assuming that your operating system has no major shortcomings. (Don't laugh, there seem to be people on this planet who put up with buggy operating systems!) There are no weird limitations on how much memory your program may use or how long it may run forced upon your process by the run-time system. Heck, there ain't no such thing as a run-time system!

Last but not least, a good reason for C-language applications is that your code runs faster than any of the interpreted languages.

Developing FastCGI applications within the framework described in the section called “Writing Web-based Applications using FastCGI” is actually comparatively easy. Once you get used to designing your applications like a state machine, you'll find that you can program web-based applications almost like an interactive process. In principle, there is no difference: The application receives some input from the user, processes it, writes back some output, and waits for the next event.

Having a single process that takes care of all the requests has other unsuspected benefits: record locking and synchronization are easier to implement. One problem every web-based application we know of has is that they can't cope with race conditions: User A requests a page that will enable him to edit the contents of some database fields. Instead of editing the contents and submitting the changes immediately, though, he is interrupted and leaves his terminal with the browser still open. Meanwhile, user B requests the very same page, edits the contents of the fields, and submits his changes. A few minutes later, user A returns to his machine and submits his changes, too, thereby overwriting the changes made by user B.

In FastCGI applications, you can use semaphores, record locks, and other mechanisms to avoid that race condition, because these locks won't be lost when the request is finished — your application doesn't terminate. You can even implement timeouts for locks; something that other web developers can't do trivially.

Finally, writing web-based applications using the FastCGI interface promotes the strict separation of content and layout, because you really don't want to have your HTML pages compiled into the application itself. Instead, build yourself some kind of template mechanism; add magic words to the HTML page that your application will replace with the appropriate data before delivering it.

An even more sophisticated approach would be to have the FastCGI application gather all data required to create the page. Then call PHP and provide a “fake” request, which passes those values to the PHP interpreter via POST. Now all the formatting can be done in PHP and all the processing in the FastCGI application. An ideal combination, because the C programmers don't have to know HTML, JavaScript, etc., and the web designers don't have to know how to program. Most of them don't anyway.