Unix Pipe as Functional Language

By Xah Lee. Date: . Last updated: .

interview with Alfred Aho (designer of awk, and author of the compiler Dragon Book), on unix pipe as function composition.

Is there a connection between the idea of composing programs together from the command line throught pipes and the idea of writing little languages, each for a specific domain?

Alfred Aho: I think there's a connection. Certainly in the early days of Unix, pipes facilitated function composition on the command line. You could take an input, perform some transformation on it, and then pipe the output into another program. …

When you say “function composition”, that brings to mind the mathematical approach of function composition.

Alfred Aho: That's exactly what I mean.

Was that mathematical formalism in mind at the invention of the pipe, or was that a metaphor added later when someone realized it worked the same way?

Alfred Aho: I think it was right there from the start. Doug McIlroy, at least in my book, deserves the credit for pipes. He thought like a mathematician and I think he had this connection right from the start. I think of the Unix command line as a prototypical functional language.

It is from a interview with Alfred Aho, one of the creators of AWK. The source is from this book:

Masterminds of Programming: Conversations with the Creators of Major Programming Languages By Federico Biancuzzi et al. Buy at amazon

Since about 1998, when i got into the unix programing industry, i see the pipe as a postfix notation, and sequencing pipes as a form of functional programing, but finding it overall extremely badly designed. I've wrote a few essays explaining the functional programing connection and exposing the lousy syntax. (mostly in years around 2000) However, i've never seen another person expressing the idea that unix pipes is a form of postfix notation and functional programing. It is a great satisfaction to see one of the main unix author state so.

funsh functional bash
A comic by Mike Ledger 2012-05-21, from fun.sh http://quasimal.com/posts/2012-05-21-funsh.html

Unix Pipe as Functional Programing

The following email content (slighted edited) is posted to Mac OS X mailing list. [source http://www.omnigroup.com/mailman/archive/macosx-talk/2002-May/064059.html ]

From: xah@xahlee.org
Subject: Re: mail handling/conversion between OSes/apps
Date: May 12, 2002 8:41:58 PM PDT
Cc: macosx-talk@omnigroup.com

Yes, unix have this beautiful philosophy. The philosophy is functional programing. For example, define:

power(x,y) := x^y

so “power(3,2)” returns “9”.

Here “power” is a function that takes 2 arguments. First parameter is the number to multiply by itself, the second is the number of times of doing so.

functions can be nested,


or composed


Here the “compose” itself is a function, which take other functions as arguments, and the output of compose is a new function that is equivalent to nesting f g h.

Nesting does not necessarily involve nested syntax. For example, f(g(h)) in Mathematica's postfix notation is written like this:

x // h // g // f

or prefix notation:

f @ g @ h @ x

The principle is that everything is either a function definition or function application, and function's behavior is strictly determined by its arguments.

Apple around 1997 or so have this OpenDoc technology, which is similar idea applied more broadly across OS. That is, instead of one monolithic browser or big image editors or other software, but have lots of small tools or components that each does one specific thing and all can call each other or embedded in a application framework as services or the like. For example, in a email app, you can use BBEdit to write you email, use Microsoft's spell checker, use XYZ brand of recorder to record a message, without having to open many applications or use the Finder the way we would do today. This multiplies flexibility. (OpenDoc was killed when Steve Jobs became the iCEO around 1998 and did some serious house-cleansing, against the ghastly anger of Mac developers and fanatics, I'm sure many of you remember this piece of history.)

The unix pipe syntax “|”, is a postfix notation for nesting. example

ps auwwx | awk '{print $2}' | sort -n | xargs echo

in conventional syntax it might look like this:

xargs(  echo, sort(n, awk('print $2', ps(auwwx)))  )

So when you use “pipe” to string many commands in unix, you are doing supreme functional programing. That's why it is so flexible and useful, because each component or function does one thing, and you can combine them in myriad of ways. However, this beautiful functional programing idea, when it is implemented by the unix heads, becomes a faaking mess. Nothing works and nothing works right.

I don't feel like writing a comprehensive exposition on this at the moment. Here's a quick summary:

Maybe some other day when i'm pissed, i'll write a better exposition on this issue. I've been wanting to write a final-say essay on this for long. Don't feel like it now.

Unix Pipe, Dot Notation, Postfix Notation

Unix Shell