distcc is a clever little program that allows you to distribute compilations to many machines, across the network. If you have a bunch of machines running the same operating systems and compilers, it’s quite easy to use: start the daemon on all the machines you wish to serve compilation jobs to, tell the host doing the compiling to use distcc rather than gcc or g++ and compile away!

The problem comes about when the systems aren’t running the same OS. For example: I have an older Linux laptop that I occasionally build things on, but a much faster MacOS X desktop machine. The solution in this case is a cross-compiler. A full-fledged cross-compiler is a complicated thing and requires an obnoxious amount of preparation, but for the purposes of distcc, one can avoid most of the headache.

It’s worth mentioning that doing the reverse (targeting OS X, using Linux servers to speed up the compilation) is much harder because building the cross-assembler requires heavy hackery (Apple does not invest any effort in making their assembler portable). I had something that worked back in the 10.3 days but never updated it for newer releases.

1. Build binutils for the target, on the server. The only piece that distcc needs is the assembler, but it’s easier to just build the whole thing. Since my target is an 64-bit x86 Ubuntu machine, I’ll use x86_64-linux-gnu as the target in this example.
   

   wget -c http://ftp.gnu.org/gnu/binutils/binutils-2.21.tar.bz2
tar -xjf binutils-2.21.tar.bz2
./configure --target=x86_64-linux-gnu --disable-nls --prefix=/opt/local/cross-linux
make
sudo make install
export PATH="$PATH:/opt/local/cross-linux/bin

2. Build gcc partially. You need the compilers, but the core libraries and other pieces are unnecessary. Note that it is important that the version of the compiler match the version on your target. My Ubuntu 10.10 machine uses 4.4.5, so that’s what I used for the cross compiler. I only build C and C++ compilers, as those are the languages that distcc supports.
   

   wget -c http://ftp.gnu.org/gnu/gcc/gcc-4.4.5/gcc-4.4.5.tar.bz2
tar -xjf gcc-4.4.5.tar.bz2
mkdir objdir
cd objdir
../gcc-4.4.5/configure --target=x86_64-linux-gnu --prefix=/opt/local/cross-linux --disable-nls --with-libiconv=/opt/local/ --with-gmp=/opt/local/ --without-cloog --enable-languages=c,c++
make
sudo make -k install


   Somewhere in the make process you will get an error. So long as it happens in the library building stage after the compilers have been built, it’s fine.

3. Verify that the compiler and assembler work. A dummy file works fine here.
   

   touch hello.c
x86_64-linux-gnu-gcc -c hello.c

   This should not produce any errors!

4. Enable the distcc daemon on the server. If it is not already installed at this point, get it. You’ll need to specify the IPs of the hosts that are allowed to connect. Also, make sure the compiler is in $PATH at this point.
   

   distccd --allow=192.168.1.1/24

5. On the client (target) machine, also make sure that distcc is installed. Set the variable DISTCC_HOSTS to a list of IPs of servers running distccd. Then set CC and CXX appropriately, and you’re ready to build away!
   

   export DISTCC_HOSTS='serverip localhost'
export CC="distcc x86_64-linux-gnu-gcc"
export CXX="distcc x86_64-linux-gnu-g++"

The way that distcc works is that it farms out the compilation and assembly process, but keeps linking and preprocessing local. This is why a full cross-compiler is not necessary. distcc works best in cases where the build process is dominated by compilation time. Since linking is not distributed, distcc will only have an effect on the steps of the build that are explicitly compilation, i.e. Makefile rules that use gcc -c.