Thoughts on Common Lisp Scheme Lisp Based Emacs

By Xah Lee. Date:

Over the past decade, before i really got my hands down with Emacs Lisp in the last couple of years, i have heard many times of wishes and projects that want to replace elisp with Common Lisp or Scheme lisp. At those times with my ignorance of deep elisp, i wished alone the same line, blithely hoping that one day we'll have a emacs with a better lisp (without me needing to put anything in). (this is the “i want to believe” syndrome typical with Free Software Foundation and Open Source youngsters) However, in the past 2 years i studied elisp, and chanced to actually spend maybe 30 minutes about this issue of modernizing elisp and looked at some websites about such projects that use Scheme lisp or Common Lisp. I think now that these efforts are not as important as they seem.

Learn Just One Lisp Argument

To create a emacs with Common Lisp or Scheme Lisp as extension language, is not going to work as one might thought. Because, the basis of such wish is that a programer would just learn one language (and a better one at that) and not to have to learn one for X lisp and one for Emacs lisp. This line of thinking is reasonable, but has hidden, unexpected flaws. Because, to create such a extension language in a text-processing system, you necessarily have to need a lot concepts and data types extraneous to the language. Specifically, you'll need data type of buffer, point, frame, kill-ring, keymap, syntax table, fontification infrastructure, etc. So, even if one day we have a emacs with Scheme Lisp as the extension lang, but the deeper fact is that it will actually have perhaps hundreds, of functions inherent in the system that has nothing to do with Scheme the language or general programing. (and, there will be needs or complexities, to separate what's editing-related function from the language's core functions. Essentially, creating a intermediate language) So, the final effect is that, to write any program for text processing or editor-extension for the New-Lisp Emac, the programer is really, factually, spending most of his time learning about the working of these functions in this text-processing system, even if he is already a Scheme expert. In short, given a Scheme Emacs (or Common Lisp emacs), it will not make that much of a difference as with current Elisp Emacs, with regards to the effort of writing code in emacs. (And if this new Scheme Emacs or Common Lisp Emacs does not have a intermediate language, such that to extend the editor the programer has to write code and re-compile with the full source code, then it ceases to be emacs. It is just another text editor.)

A Better Lisp Language Argument

The fact a somewhat technically better lisp (CL, Scheme) replacing elisp, has very little practical benefits for a emacs extension language, even when we consider just the technical aspects. It is certainly true, that Common Lisp and Scheme Lisp are technically better than Emacs Lisp, in different ways. For example, elisp uses dynamic scope. Elisp has a data type that is both character and integer. Elisp does not support exact arithmetics. These defects are not present in CL or Scheme. And, CL supports multi-threads, whereas elisp has single thread. (practically meaning that emacs users must wait for each operation to finish) CL has many tried-and-true libraries. Scheme provide the same language power with a simpler, cleaner language.

However, there are also considerable technical disadvantage. For example, CL is fret with baggage and ugliness, with its voluminous language specification. Scheme does not have a library system (same situation as elisp) untill the R6RS specification in late 2007, and fully conformal implementation are scarce even for R5RS. Both CL and Scheme does not universally support Unicode, and while in emacs lisp a infrastructure exist that is already employed to support full Unicode for all practical purposes.

When we survey most uses of the extension language in emacs, they are predominantly text processing related tasks, with some other uses such as embedding a text-based client for ftp, irc, mail, web browser, OS command line interface, file manager. The benefits of fixing elisp's weaknesses by CL or Scheme for small-scale programing as these are not major, and is offset by the various technical problems they introduce.

The Real Advantage

The real advantage of a CL emacs, is not “learn just one lisp” argument or “better lisp” argument. The advantage is a social one, and with far greater impact. That is: a integrated general purporse language development system, analogous to the symbiotic relations of Visual Studio with C++, Xcode and Objective-C; Eclipse/NetBeans with Java.

The force of this is that, any coder who use emacs, will transparently be introduced to lisp as a general purpose programing language. This will open the doors of lisp to the average programers.

Effort Required

It is a fact that the effort needed to create such as system basically hampered all such projects (although such system with CL and Scheme already exists (and to my knowledge, usable to some extent).). And, the need to advertise such a system to get people to switch from existing system (emacs) is nigh impossible. The 2 facts that the effort needed for such system and the effort needed to cover legacy elisp so people will switch, basically puts a death knell to such projects (or, long, long time down the road, maybe 5, 10 years, if without commercial incentive. (and with such long gestation, in today's technological speeding, i do not think lisp language itself (in particular, CL, Scheme, or anything with the cons business), are suitable or competitive as a high-level general language of the 2010s)).

Note: This essay is not meant to discourage the development of a new lisp emacs, especially because they already exist. This essay is meant to put off some common wishful thinking. As noted before, a major benefit of a emacs with a general purpose Lisp is that they opens the doors of lisp to professional programers in a transparent way, much like the symbiotic relationships of C++ and Visual Studio, Java and Eclipse, Objective C and XCode. Here are the home pages of the most prominent on-going efforts:

Note: Professional quality Emacs-like IDE for CL already exist commercially. Franz Inc's Allegro CL and LispWork Ltd's LispWorks. Free quality IDE for CL also exist, namely: SLIME (which is a add-on to emacs).

(Thanks to Rainer Joswig for pointing out Common Lisp's multi-thread advantage, and pointing out Hemlock.)

Emacs Modernization