Software Dependency Complexity: Fink, Unison

By Xah Lee. Date: 2010-06-04

This essay documents a computer industry complexity, in particular, a problem of installing a single software, caused by software dependency.

Today, i got a project to update a website of a university. The first job, is to install the Unison software, which allows 2-way file synchronization.

The remote machine is Mac OS X 10.6 running on a recent server version of mini mac. On the mac, there are 2 primary ways to systematically install a large number of unix software. One is [ Fink ] [ https://en.wikipedia.org/wiki/Fink ], and another is [ Macports ] [ https://en.wikipedia.org/wiki/Macports ] .

I've been using Fink since it started in ~2002. I've casually tried MacPorts once or twice in the past, but didn't have a good experience with it. Over the years i've been using Fink, largely i'm a happy user, but i've had some problems so i don't regard it as near perfect. So, i did some web reading to see if one has really surpassed the other. After a hour, it seems there is no conclusive evidence whether one is much better than the other. Basically, MacPort has more updates, but Fink has more packages and is stable.

So, i decided to use Fink. (long story short, i've worked on this server before, and have installed unix tool set thru Fink, but recently the professor upgrated the machine from 10.4 server to 10.6 with new hardware, so i removed all Fink stuff /sw/, necessarily for a fresh start.)

After installing Fink, and run thru numerous commands such as selfupdate, and other preparations such as checking Unison versions on the machines i need to work on and available versions on Fink and enable unstable branch the latest Unison at Fink is in, i started to install Unison.

fink desc unison
sudo fink install unison-2.27.57-1008

Then, i got this:

Pick one: [1] 
The following package will be installed or updated:
 unison
The following 129 additional packages will be installed:
 atk1 atk1-shlibs autoconf2.6 automake1.11 avahi avahi-dev cairo cairo-shlibs dbus
 dbus-glib1.2-dev dbus-glib1.2-shlibs dbus1.3-dev dbus1.3-shlibs docbook-utils enchant1-dev
 enchant1-shlibs esound esound-bin esound-common esound-shlibs gconf2 gconf2-dev gconf2-shlibs
 gd2 gd2-shlibs gettext gettext-dev glib2-dev glib2-shlibs gnome-desktop2.11-dev
 gnome-desktop2.11-shlibs gnome-doc-utils gnome-icon-theme gnome-keyring gnome-keyring-dev
 gnome-keyring-shlibs gnome-menus gnome-menus2-dev gnome-menus2-shlibs gnome-mime-data
 gnome-panel-dev gnome-panel-shlibs gnome-python2-py26 gnome-vfs2-unified gnome-vfs2-unified-dev
 gnome-vfs2-unified-shlibs gnutls26 gnutls26-shlibs gtk+2 gtk+2-dev gtk+2-shlibs gtk-doc
 gtkglarea2 gtkglarea2-shlibs gtksourceview-1.0-data gtksourceview-dev gtksourceview-shlibs
 gtkspell2-dev gtkspell2-shlibs gweather-data jadetex lablgtk lablgtk2 libavahi-client3-shlibs
 libavahi-common3-shlibs libavahi-core6-shlibs libavahi-glib1-dev libavahi-glib1-shlibs
 libbonobo2 libbonobo2-dev libbonobo2-shlibs libbonoboui2 libbonoboui2-dev libbonoboui2-shlibs
 libcroco3 libcroco3-shlibs libgettext8-dev libglade2 libglade2-shlibs libgnome2 libgnome2-dev
 libgnome2-shlibs libgnomecanvas2 libgnomecanvas2-dev libgnomecanvas2-shlibs libgnomecups-dev
 libgnomecups-shlibs libgnomeprint2.2-dev libgnomeprint2.2-shlibs libgnomeprintui2.2-dev
 libgnomeprintui2.2-shlibs libgnomeui2-dev libgnomeui2-shlibs libgsf1.114-dev libgsf1.114-shlibs
 libgweather libgweather-shlibs libidl2 libidl2-shlibs libjpeg8 libpng14 librsvg2
 librsvg2-shlibs libsoup2.4.1-ssl libsoup2.4.1-ssl-shlibs libwnck1.22 libwnck1.22-shlibs
 libxml2-py26 numpy-py26 orbit2 orbit2-dev orbit2-shlibs pango1-xft2-ft219 pango1-xft2-ft219-dev
 pango1-xft2-ft219-shlibs ptex-base ptex-texmf pycairo-py26 pycairo-py26-dev pygobject2-py26
 pygobject2-py26-dev pygobject2-py26-shlibs pygtk2-gtk-py26 pygtk2-gtk-py26-dev pyorbit2-py26
 pyorbit2-py26-dev scipy-core-py26 shared-mime-info texlive-base
Do you want to continue? [Y/n]

I was a bit surprised. Installing Unison requires 129 additional software? and things like gnome-icon-theme? For a moment i was thinking perhaps i did something wrong. I didn't have time to ponder on it. After some re-checking, i went ahead.

The installation and compilation started about 8:30 pm, and ended at 10:37 pm, with thousands lines of output, that ends with this message:

…
New package: dists/unstable/main/binary-darwin-i386/x11/xinitrc_1.5-1_darwin-i386.deb
Failed: Fink::SysState: Could not resolve inconsistent dependencies

What the faak? A web search quickly found this official Fink FAQ at: http://www.finkproject.org/faq/comp-general.php?phpLang=en#alternates. Quote:

Q6.22: Whenever I try to build from source, Fink keeps waffling between alternate versions of the same library.

A: Often, in a complicated build tree, you may find that some of the packages depend on a particular version of a library, and other depend on a different one (For example, db47 vs. db44). Consequently, Fink may try to switch to whichever one isn't currently installed in order to satisfy the build dependency for the current package that you're trying to update.

Unfortunately, due to limitations in the build-dependency engine, you may wind up with the dreaded
Fink::SysState: Could not resolve inconsistent dependencies
message when trying a sufficiently complicated update-all. This generally gives you a command to try to resolve the issue:
fink scanpackages
sudo apt-get update
sudo apt-get install foo=1.23-4	
but this may not work for sufficiently complicated updates. You might need to update packages one-by-one, at least for a while.

So, there it is. Installing a single software, with expert know-how, ends up in failure with 4 hours spent. That is the state of the art in software today.

One might think that perhaps it is easier to just compile Unison myself. Note that Unison is written in OCaml. So, to compile Unison you'll need to have the whole OCaml platform. If you are lucky, you might succeed with half a day on that.

MacPort Try

Ok, at this point i can maybe try to fix it or try MacPort. So, MacPort it is.

vmm:~ xahlee$ which port
/opt/local/bin/port
vmm:~ xahlee$ sudo port install unison
--->  Computing dependencies for unison
--->  Fetching ocaml
--->  Attempting to fetch ocaml-3.11.2.tar.bz2 from http://distfiles.macports.org/ocaml
--->  Verifying checksum(s) for ocaml
--->  Extracting ocaml
--->  Configuring ocaml
--->  Building ocaml
--->  Staging ocaml into destroot
--->  Installing ocaml @3.11.2_0
--->  Activating ocaml @3.11.2_0
--->  Cleaning ocaml
--->  Fetching unison
--->  Attempting to fetch unison-2.32.52.tar.gz from http://distfiles.macports.org/unison
--->  Verifying checksum(s) for unison
--->  Extracting unison
--->  Applying patches to unison
--->  Configuring unison
--->  Building unison
--->  Staging unison into destroot
--->  Installing unison @2.32.52_0
--->  Activating unison @2.32.52_0
--->  Cleaning unison
vmm:~ xahlee$ which unison
/opt/local/bin/unison
vmm:~ xahlee$

Success in 10 min.

What is The Problem?

This is not a story of MacPort vs Fink. They are pretty similar in quality. Also, MacPort by default installs binary, while Fink by default compiles, which i've chosen to not change.

In this essay, i wanted to document the complexity of software. Why is it complex, why so many seemingly simple things are fraught with problems, what's the cause, why is software engineering so complicated.

In this particular case, i think the problem traces to software dependencies. It is really a issue that cannot be practically solved. In order to have a robust dependency system, you have to have a dependency architecture to start with, that each software can document or communicate as a element in the system. This ideal framework is simply not possible from existing diverse software, that are built from different languages and different operating systems, from different decades. Instead, a dependency system is somewhat cooked up by practical means. That means, there's lots of flaws. In my case, the practical consequences is that, installing a command line file syncing software spuriously requires one to install 128 other software, including a desktop system with images for icons, and caused me 4 hours. LOL.

(For another documentation on the issues of Unison and Cygwin, see: Complexity of Software Engineering; Emacs, Unicode, Unison)