Why Clojure is Dense

By Xah Lee. Date: . Last updated: .

clojure is hard to learn. very hard. Forget about fan boy things you've heard. This essay tells you why.

clojure Rich Hickey a stateful transducer 2014-09-17
A Stateful Transducer. [image source https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Transducers/00.36.36.jpg]

clojure is quite a beast. I've seen nothing like it. Perl, python, ruby, they are similar, introduced no new concepts, really. Of lisps i know, 10 years of emacs lisp and some scheme/racket lisp, clojure is rather quite different from them, and introduced lots of clojure's own concepts, such as transducer, and clojure's metadata, clojure's “protocol”, clojure's refs and transactions, atoms, agents.

I know a thing or two about programing languages, my tutorials to wit:

i also coded a lisp language the Wolfram Language intensively for 7 years (1992 to ~1999), and recently read golang

see

but Clojure, in comparison, is somewhat just opaque.

why is clojure hard?

after some introspection, it's because:

Lisp Stuff: Parens, Macros, “Reader”

• it's got lisp stuff, like parens, and macros, and “reader”. Parens you can get in a day. Macros takes a few months to sink in, “lisp reader” you may never get. [see Can Lisp Macro Change Lisp Syntax?]

Clojure's Idiosyncratic Concepts and Tech

• clojure's got transducer, clojure's odd concept of metadata, clojure's own concept of “protocol”, clojure's refs and transactions, atoms, agents. (i don't think any lang before clojure had them, at least not java, perl, python, ruby, php, JavaScript, nor Wolfram Language. Haskell, Ocaml, got their own things, but it feels like clojure's stuff is the weird inventitive ones.)

Abuse of Syntax, and Terse Function Names

• that's not enough. Clojure got abusive with syntax.

See Clojure Sigils (Magic Characters)

One major advantage of lisp's nested parenthesis syntax is that it's easy to understand. Basically everything is the form (f …), and any newbie can lookup function name f. Not clojure lisp.

And, clojure function names are heavily abbreviated. For example, function composition is just comp. (and you have: { conj, assoc, coll?, seq, rem, pr, … } )

In classic lisps (Common Lisp, Scheme Lisp, Emacs Lisp), and Mathematica (Wolfram Language), function names follow the lisp tradition of full long names.

Intermingle with Java the Language

Now, this is the worst part.

• clojure the language is heavily intermixed with Java's concepts and practices. You can't understand clojure well unless you understand java well, and java, is one elephantine tedious object oriented complexity.

For example, clojure refers itself as hosted language. And, for the Java Virtual Machine implementation, clojure doesn't have its own regex [see Clojure: regex] , nor math lib, nor much of string functions. Users are told to use Java class and methods via Clojure. [see Clojure: Call Java Method]

This dependency aspect of Clojure can also be seen in Clojure docs, where inline function doc usually directly explain or refer to Java language concepts.

Here is clojure doc on sequence:

seq
function

Usage: (seq coll)

Returns a seq on the collection. If the collection is empty, returns
nil. (seq nil) returns nil. seq also works on Strings, native
Java arrays (of reference types) and any objects that
implement Iterable. Note that seqs cache values, thus seq
should not be used on any Iterable whose iterator
repeatedly returns the same mutable object.

Note the mention of Java array and Java Iterable.

To understand what it means, you have to know the Java language's feature of Interface, and you have to know that Iterable is a interface which the Collection interface is extended from, and the Collection interface is effectively a list-like “datatype” in Java.

[see Java: Interface]

[see Java Tutorial: Collection, Map]

Note, Java array is a list-like data structure, but is not like other objects with methods, and array is not part of set of data structure objects that implement the Collection interface. (but by Java spec, array is an object). There creeps in Java complexity.

[see Java: Array]

Now look at the function seq?:

seq?
function

Usage: (seq? x)

Return true if x implements ISeq

The ISeq there, is a Java Interface. It is purely a Clojure implementation detail, but such implementation detail is widely used as documentation in Clojure the language.

In case you don't know, Java interface is a way to specify what methods and fields a class can have. You define a interface as you do writing a method or class. Then, when you define a class, you can declare that your class C implement interface X Y or Z that you defined previously. When a programer knows that a class C implements a interface T, he can be sure that the C has all the methods of T.

You can also see Java dependence in Rich Hickey's Language References essays, where, Hickey talks glibly in terms of Java, as if everybody had 100 years of experience coding Enterprise Java.

let's look at Riche Hickey's reference essay. For example, see http://clojure.org/reference/data_structures .

Here is a more pertinent example, see http://clojure.org/reference/sequences. Read this, together with the collection essay. Nobody understand it.

Let's see:

Clojure defines many algorithms in terms of sequences (seqs). A seq is a logical list, and unlike most Lisps where the list is represented by a concrete, 2-slot structure, Clojure uses the ISeq interface to allow many data structures to provide access to their elements as sequences. The seq function yields an implementation of ISeq appropriate to the collection. Seqs differ from iterators in that they are persistent and immutable, not stateful cursors into a collection. As such, they are useful for much more than foreach - functions can consume and produce seqs, they are thread safe, they can share structure etc.

to understand the above, you need to:

the reference essay on sequence continues:

Most of the sequence library functions are lazy, i.e. functions that return seqs do so incrementally, as they are consumed, and thus consume any seq arguments incrementally as well. Functions returning lazy seqs can be implemented using the lazy-seq macro. See also lazy.

To understand the above paragraph, you need to:

we read on:

When seq is used on objects that implement Iterable, the resulting sequence is still immutable and persistent, and will represent a single pass across the data. Because that pass might happen lazily, the pass might see changes that happen after seq has been called. Also, if the backing iterator is subject to ConcurrentModificationException, then so too is the resulting seq. When seq is used on native Java arrays, changes to the underlying array will be reflected in the seq - you must copy the source array to get full immutability. That said, there is still a lot of utility to using seq on Iterables and arrays since seqs support multi-pass and lazy algorithms. Robust programs should not mutate arrays or Iterables that have seqs on them.

To understand the above paragraph, you need to:

in any other lang, such as {perl, python, ruby, lisp, JavaScript, php}, even java, you have basically 3 types of data structures as builtin language datatypes. (1) linked list type, (2) array/vector/tuple type, (3) a key/value pairs type aka dictionary, hash table. Each with some algorithmic performance properties, and with other limitations such as if items can be added/removed, whether item can be modified, whether element must have same type, whether ordered, etc. But in clojure, it seems way more complicated, and it's hard to pin down concretely. You start with “abstractions”, of which Hickey says is the main dish, then you have concrete types underneath, such that a operation may do different things to the same interface thing (e.g. adding a element to a sequence may add to front or end depending what's the actual type), then you have java's technical concept of interface (good luck if you are 100 years old and have 10 years experience with each of {C, perl, python, ruby, php, lisp, Wolfram Language, APL}, and have no idea what Java's tech concept of Interface is), and you have Java tech detail stuff like iterable and iterator… looks like basically you have 2 abstract types, collection (which hickey unhelpfully like to abbrev to “coll”) and sequence (“seq”). Each has a few concrete implementations. And, yet, seems any coll is also a seq, because due to if the thing has a method implementing ISeq interface. (you see? here's Hickey's terse incomprehensible+java talk start to show) and Hickey's reference doc is fluidly ample with these jargons, and dropping in lisp's cons and haskell's lazy concepts like gangsta rap.

[see Ice Cube - “Gangsta Rap Made Me Do It”]

y'know? it's strange. In ocaml [see OCaml Tutorial] or haskell, they are also known as difficult, but, stuff there are typically math based, or, from strong tradition of functional programing practices in academia. But am not sure that can be said of clojure.

Rich Hickey the Clojure Inventor

clojure Rich Hickey
image from Rich Hickey fanclub at https://github.com/tallesl/Rich-Hickey-fanclub

clojure's background…, it's basically Hickey's personal cookie, from a background of practical coding of enterprise software, with perhaps mostly Java.

(in some language, when you do the language, you are kinda idoling one guy. A “Cult of personality” phenomenon. This is so for perl, python, ruby, to various degrees. Clojure too. However, in contrast, for example, when you code Java, you don't think much of James Gosling, nor C of Dennis Ritchie, nor C++ of Bjarne Stroustrup, nor JavaScript of Brendan Eich, and you don't even know who created PHP.)

PS from reading Rich Hickey's Language References essays (and watching his video presentations), although clojure seems dense and has lots of its own inventive concepts, and Rich's writing has its own particular style. But i think Hickey's concepts and his language is a solid one. This is in sharp contrast to some other language inventors, such as Larry Wall of Perl and Guido of Python, who are sputtering ignorant trivial personal opinions, and in case of Larry Wall, selling snake oil as humor. (disclaimer: all opinion only)

For clojure coder who don't know Java, see also: Java Tutorial: Collection, Map .

Addendum

i had a revelation yesterday, on why clojure doc is intertwined with Java. In fact, it's highly advantageous for clojure's growth.

See: Clojure's Popularity, a Tango with Java .