DistWork - Implementation

Implementation Overview

Implementation Languages

The DistWork Core is implemented as a GObject (part of GLib, see http://www.gtk.org) based library. It is supplemented by Guile Scheme bindings, driven by a Scheme program providing an easy-to-use configuration language. Additionally, there are clients for different tasks. Since the core and the main loop (from GLib) are written in C, this approach combines the speed of C with the high-level features and power of Scheme.

Peer-to-Peer

Distwork nodes form a peer-to-peer network, modelled after Gnutella. Each nodes offers a set of services, identified by strings; one might image oggenc (OGG compression) bzip2 (bzip2 compression) or gcc-3.3-i386-linux (GNU C compiler for Linux oni386) services being offered.

Clients connect to a node (most likely localhost) and request a service. If the node is already saturated with work, other nodes offering he same services are searched for and the data transferred transparently to a node willing to accept the job.

Data Exchange

The DistWork nodes, like Gnutella, have two different "connection types" between them: P2P connections, using a Gnutella-like protocol, and data exchange connections. Since data exchange in Distwork is potentially bi-directional and multi-channel, a slightly adapted HTTP scheme as used by Gnutella is not sufficient.

Just think of a distcc like service: one input stream (preprocessed source) and standard output, standard error and the object file as output streams. Therefore, Distwork uses a multiplexed bi-directional streams of data on top of TCP.

Connection Pooling

Each service (or rather class of services) can freely define the semantics of the 2^16 available channels, thus allowing for high flexibility in the range of services DistWork can potentially support.

Since DistWork was primarily intended for distribution of compiler-like tasks, it has a "job-management" layer on top of the bare multiplexed channels, which allows for connection pooling to keep latencies down. In this layer, each service caches the connections it builds up when reacting to nodes offering the service. Since these nodes have a relativly high propability of being willing to take over a job, even if queried for takeover a (short) time period after the original positive response.