|
Date: Fri, 16 Aug 2002 01:20:23 +0000
From: Julien Gilli
To: James A Morrison
Subject: Re: DMS and distcc
On Thu, 15 Aug 2002 17:20:01 -0400 (EDT)
James A Morrison wrote:
> I'm curious how does the work you are doing compare to distcc [1].
Don't be afraid by the length of the text, if you want a short answer,
skip the first paragraphs :-).
Well, I will probably say a lot of things that _probably_ are, or
_probably_ will be (because the project has just started), and since I
don't speak english fluently, you might misunderstand me sometimes, so
don't give too much sense to my words, and feel free to ask if something
is not clear.
This project is currently a school project. We (three friends and me)
are studying in a computer science school, and we are free, this time,
to choose the project we would want to work on.
A project where you can do something interesting in 1 month and a
half, that is still challenging and that let a lot more challenges
opened after is not easy to find. We found that a software that could
distribute jobs within a network would best suit our needs. Some time
before our decision, we went to a compagny where people were using
TeamBuilder. We really enjoyed the design of this program, and we
thought that this could be really fun and usefull to write. Just after
we decided to write such a software, we began searching for similar
sofwtare and we found that actually a lot of people have worked on such
tools. Teambuilder and distcc were the most interesting for us, because
the others were patched version of *Make, or used middlewares such as
PVM or MOSIX.
So the first millestone of our project will be very similar to
TeamBuilder or distcc. We want to be able to distribute compilation jobs
within a LAN, in order to decrease the compilation time. We want to
support at least the C and C++ language, and the GCC and ICC compilers
collection. The first millestone should also include a graphical
monitoring tool, like the one which is provided by teambuilder.
One this millestone is rock solid, we want concentrante on several
topics that will lead us (hopefully) to support more compilers, more
programming languages, to deal with security problems, to increase the
performance and scalability (distribute jobs across the internet),
support building binaries for other architectures and more. However, we
want to concentre first and mostly on the first millestone objectives,
the objectives mentionned just above are just thoughs.
Currently, we manage to compile huge projects (emacs, perl, python,
etc.) in a client/server fashion. We just need a basic scheduler to be
able to distribute the jobs on several machines. Once the basic
scheduler written (probably tomorrow night), we will test it and compare
our results with the ones of distcc and teambuilder. If they are good
enough, then we'll know that we are on the right way, and will begin to
add features (C++ and ICC support, enhanced scheduler, stability, better
overall source code, GUI and support for more OSes, probably linux,
OpenBSD, FreeBSD first, and then IRIX, MacOS X and Solaris).
On the other hand, we don't want to duplicate work, and it is possible
that when our project won't be a school project anymore (by the end of
September), we will ask the author of distcc if he agree to merge our
projects. Indeed, by that time, the goals of the two projects may be
quite similar, and maybe we wil find that we can better work together.
The major problem that could prevent us to work together is that we use
to very different programming languages. We use Python and distcc use
plain c. Although Python has a good C API, it would be quite difficult
to reuse code from the both projects, and merge it into a new one.
Well, I realize that I wrote all this text without answering your
question. So how does our work compare to distcc ? Currently, it is
difficult to say, because we've just started. But I think that the
Python programming language will help us to add features really fast,
and to concentrate only on what matters : scalability, portability,
compilers and programming languages supports, etc. And we can see it
everyday. The code is easylly readable and we can add features _really
fast_. So we hope that some contributors will join us by the end of
September.
I hope that the fact that Python is interpreted won't be a too big
issue, since we follow the same design than distcc and TeamBuilder :
each compilation command line invoke one of the client wich will ask a
deamon to do the job. It means that if you distribute the job on fivteen
computers, the client that called make will have fivteen Python
interpreters running at the same time at most, which could lead to bad
performance. However, we alredy thought about workarounds to this
eventual problem.
We'll make a snapshot of the CVS when we'll manage to build, in a
distributed way, a sufficient number of programs, so that you can check
what's going on. it should be done before the end of the week-end. We'll
also post news to tell about the evolution of the project, and
if you want, I can keep you informed of important millestones.
I hope I didn't bothered you, and that I answered your question. Feel
free to ask questions and to bring ideas. The only thing I really can't
do is to accept source code contributions from you. Since it is a school
project, we'll accept source code contributions by the end of September.
Regards.
P.S : you can find TeamBuilder at :
http://www.trolltech.com/products/teambuilder/index.html
--
Julien Gilli
|
|
