Emacs Switching to Guile Scheme Lisp? (2010)

By Xah Lee. Date: .

Recently, there's discussion on Emacs dev forum about switching to Guile, a Scheme lisp. Of course, such discussion always sparks controversy. Namely, the Common Lisp camp isn't happy. Among the posts in the emacs dev list, those by Thomas Lord has some good info of emacs and Guile history around late 1990s. See

I'd like to correct some accounts of history.

On Mon, 2010-03-08 at 22:19 -0500, Richard Stallman wrote:

RS>

If it only supports something like Emacs Lisp then it is not much of an advance. Guile supports Scheme and Javascript as well as Emacs Lisp.

It also supports multithreading, which IMHO is a big deal as well, perhaps more important than bignums.

I think the support for multiple languages is the biggest advance. I hope support for other popular extension languages will be implemented eventually -- that was the original goal of Guile.

Your recollection there seems to me to be mistaken. It was a long time ago but I distinctly remember things differently. As I recall:

The original GNU Manifesto described a unix-like foundation with a heavy emphasis on a Lisp-centric user-space.

In the early 1990s, when I worked at the FSF, several of the hackers there (including me) understood the GNU vision to imply that many interactive programs - not just Emacs - would be extensible in a dialect of Lisp. We mostly shared the opinion that Scheme was an appropriate dialect.

Before I got to the FSF people had already started on a GNU Extension Language Library - an embeddable Scheme interpreter. As the (bad) "joke" goes: they code-named the project "revoc" but upon reflection that's just a "cover".
(Get it? Spooky, ain't it?)

Revoc was (at least back then) going nowhere fast. Later, while working on a spreadsheet program, I attempted to start a new attempt at a scheme-based extension language library. You were (rightly, I suppose) upset that there were higher priorities. You also pointed out that I was probably wasting time by starting from scratch and that I ought to have started from some existing Scheme implementation. I don't recall if you specifically suggested SCM but I think you might have. So there were two attempts at a Scheme-based GNU extension library down - and one to go.

A year or two later I went to work for Cygnus and we, there, initially agreed to invest in making a scheme-based extension language library for GNU programs. This was originally named "GEL" (for GNU Extension Language) until the company lawyers warned that "GEL" was a trademark for another program. It was renamed Guile (a cognitive relative of "scheme" and a pun for "Guy L."). You were informed of this work and were encouraging. I forget at which stage of this process you declared Guile to be a "GNU project" but I'm sure the extant public record has it somewhere.

Around that time, Sun was beginning to announce and promote Java. They also hired John Ousterhout and began declaring "Tcl is to be the scripting language for the Internet!" Back then, Sun was a particularly influential company. On a technical level, Tcl was a horribly regressive language - a giant step backwards in many subtle but important ways. Its license was a free software license but it was uncomforable at the time to have Sun pushing it so heavily because of the technical problems and because of doubts about Sun's motives and control over the software.

A faction arose within Cygnus that argued that Cygnus should divest from the Guile project and adopt Tcl instead. I made you aware of that and of Sun's swagger, regarding Tcl.

Around the same time, two of Prof. Ousterhout's graduate students, John Blow and Adam Sah, were proposing that the TCL language be altered in some small but significant ways that would facilitate its translation into reasonable efficient Scheme code. Their altered version of Tcl was largely but not completely upward compatible with standard Tcl. By applying a Scheme->C compiler to the generated Scheme code, they were able to benchmark far better than standard Tcl in many cases.

And around that same time I was independently proposing similar things. Mssrs. Blow, Sah, and I compared notes and so forth.

Around that time, also, Python was beginning to gain recognition in the US (it already had a foothold in Europe). Blow, Sah, and I had dinner with Von Rossum and tried to persuade him to add LAMBDA and proper tail calls and perhaps continuations to the language so that it could be usefully translated to Scheme. He did not agree. Back at Cygnus, the oppositional factions had fun lampooning lisp syntax and pointing to examples like Python of what was better.

Some of my notes to you mysteriously transmogrified into a USENET post which the archives show as having you in the "From:" line and with my signature at the bottom of the message. This sparked what people, to this day, still call "The TCL war".

So:

The "original goal" of Guile was most definitely not to support multiple languages. Rather, that was a goal that was established as a tactic in response to early competition with Guile.

Moreover, it was never, in the early days, a goal that Guile support other popular extension languages. That was something that you added and that I (regrettably) agreed to in response to the perceived threat of Tcl and to a lesser extent Python.

And: it was never in those years a goal to support any of those other languages exactly. It was always a goal to have a Tcl-like syntax, a Python-like syntax, and a C-like syntax for Guile Scheme. But tight compatibility with those languages was not the goal. The notion wasn't so much to "support Tcl" (or any other language) as to have a hyper-flexible syntax and to support, via libraries, environments with all of the convenience features of a shell-like language like Tcl.

Early on after we adopted that tactic you and I and a few others had some lengthy discussions about adding Emacs Lisp to the list of languages that Guile could run. We ran into some severe issues such as the difference between 'NIL in Emacs lisp and '() and Scheme. Perhaps not to you but to several other of us it became fairly clear, back then, that a strictly compatible Emacs lisp would never sit comfortably alongside a proper Scheme environment. Just as we would need to make a "Tcl-like" language that would break some Tcl code, we would need to break some Emacs lisp code, if indeed we ultimately wanted to bother trying to support any. (That is part of why, at Cygnus, I built a from-scratch multi-buffer, self-documenting, extensible text editor in Scheme with multi-font and proportionally-spaced font capabilities that, at the time, GNU Emacs wasn't close to achieving. The notion was to see how far I could get just leapfrogging over GNU Emacs instead of trying to retrofit it.)

Now, years have passed. In recent years, I gather, the implementation of Guile has been fairly radically altered so that it is now more of a bytecode VM with primitive LISP-ish types. In some sense, it has begun to drift away from being primarily a Scheme to being more in the category of JVM or the Mono VM. It is easier to target multiple languages to such a VM but no less easy to make them interoperate cleanly in a way one would want to live with for the long run.

So Guile's goals have shifted. It was once (originally) to be a tight, clean, fun Scheme environment with some alternative syntaxes and specialized environments -- and only later did it become the kind of heterogenous language environment we see it moving towards today.

So, again:

I hope support for other popular extension languages will be implemented eventually -- that was the original goal of Guile.

Nah. The heck it was. That's just not true.

Now, there is a separate question: is that goal to support other popular extension languages a better idea or a worse idea (or not importantly different) than the original goal of a nice, tight, fun Scheme-based system?

I have my opinion ("far worse!") but it's not my intent to argue for that opinion here. Just to set history straight.

That multi-lingual stuff was not the original goal of Guile. On the contrary, the multi-lingual goals only came up at all because of a perceived crisis sparked by Sun's announcement that Tcl was to become the ubiquitous scripting language of the Internet.

-t

from http://lists.gnu.org/archive/html/emacs-devel/2010-04/msg00507.html

local copy Guile_in_Emacs_Thomas_Lord.txt

Here's the post by Andy_Wingo , Guile's current maintainer, offers his help on this and why he thinks it's a good choice.

Greetings, all!

My name is Andy, and together with Ludovic Courtès I co-maintain Guile.

I meant to put off writing this note until some months in the future, but as people seem to be interested in the topic, now is as good a time as any.

The topic? Emacs and Guile. (Cue cheering and derisions!) I'll be telling some history here, from the perspective of one who wasn't there, but I believe I have the essentials right.

Emacs and Guile have a long and lovely history, longer in fact than Guile's existence itself. Hacking Emacs is the core of GNU -- as you know, Emacs is alive, it's fungible, it's free. The GNU system grew out of Emacs and the hack-experience that it represents.

(I have a friend who regularly chides me when I have to reboot and shut down my Emacs -- it's like I'm killing a friend, he says!)

But when it was decided that GNU should be like a Unix, it was a fall from grace, in a sense. The system no longer had the Emacs nature. Time passed, and people looked to see how to spread the Emacs nature from Emacs itself to the rest of the programs in the GNU system.

One possibility would be to extract elisp from emacs and use it as an extension language. But given that the language would have to be extracted anyway, the thought was to use a more modern Lisp dialect, Scheme. Thus Guile was born, as the GNU extension language.

(Its birth coincided unfortunately with the so-called "Tcl wars", but Guile was not made because of Tcl. Guile was made out of a need that existed outside the polemic. For example, Neil Jerram -- recently retired Guile maintainer, may he hack in peace -- mentioned to me once that he had contacted Richard quite early on, before the "Tcl wars", discussing the need for a GNU extension language, as Elisp was to Emacs.)

But I digress! Tom Lord was the one to make the initial Guile, derived from SCM as you know, and though his time with Guile was too short, we are privileged to have traces of his code in Guile, and his ethereal presence here on this list.

Since then, about 15 years ago now, Guile has had many maintainers. It was stabilized, and documented, and extended, and hacked, and fixed, but eventually we realized that Guile's interpreter -- which traversed trees of expressions to produce its effects and values -- was just too slow. We needed to do some pre-computation on the source; we needed to compile it to something.

Given that the U is for ubiquitous (not Unique, all you salt-n-pepa fans), we couldn't do native compilation, not directly anyway -- we needed something portable. So, with the help of a few hackers, I updated some work that Keisuke Nishida did on writing a VM and associated compiler for Guile, and that's what we're going to release hopefully next month as Guile 2.0.

We've been trying hard to maintain backwards compatibility for Guile, and we've done well, I think. The move from interpreted to compiled code has all of the eval-when pitfalls that you all know, but most extensions just compile fine with 2.0, and practically all correct Scheme code -- a number of errors were not caught with 1.8 and before -- practically all of that code runs fine without modification.

If you really want the whole story, I'd advise you to see my presentation at November's GNU Hackers Meeting in Sweden, which can be viewed or downloaded here:

http://wingolog.org/archives/2010/04/02/recent-developments-in-guile

Hi! Still with me? Cool :)

I've said all of this to give some credibility to what I'm going to say now.

Guile can implement Emacs Lisp better than Emacs can.

We can compile Emacs Lisp to Guile's VM, and make C shims to present a suitably compatible C interface to the rest of Emacs' C primitives.

No one will notice! Except that after a switch, Emacs would be faster, more powerful, and have the ability to access all of Guile's facilities -- the Scheme language, other languages implemented for Guile (Javascript, Lua, ...), a proper ffi, dynamically loadable libraries, a module system, the numeric tower (rationals, bignums, etc), Guile's existing libraries, delimited continuations (!), fast bytevector access, native threads, etc.

On a language level, I am convinced that not only can Guile implement Emacs Lisp, but it can provide a coherent story for tight interaction between e.g. minor modes in Javascript and Scheme and Elisp, etc. More details will come later, though feel free to bring up specific points.

There have been some suggestions on the list that "Guile people want to rewrite Emacs in Scheme". Though Scheme does have some facilities to offer, personally I do not believe in rewriting software. I've failed enough times, thank-you-very-much -- and if we could retrofit a compiler into Guile in two years via correctness-preserving transformations, surely we can replace the implementation of Elisp without causing problems.

My goal is to make it the obvious decision for the Emacs maintainers to say, "yes, let's switch to Guile's implementation of elisp, because it will make Emacs better and have no drawbacks". We're not there yet -- and that's why I wanted to put off this mail -- but we'll get there.

I guess that's all I have to say, for now -- my bandwidth is not with Emacs right now, as there are enough things to take care of in Guile itself. But, I'll be responding to this thread, and lurking on the list, so do get in contact if you have a concern.

In the meantime, happy hacking!

Andy

http://wingolog.org/

Whisper to myself: please don't fuck up, please don't fuck up.

I hope the emacs maintainers would just let Andy do his work. In non commercially backed open source projects (For example, not Google, Firefox), there's always huge number of opinions, ideologies, arguing and arguing for years, nothing gets done. What actually happens in the end is whoever wrote code and made things work, ends up as real progress; bad design choices or other problems not withstanding. Ideology and “good” design oriented intentions usually don't go anywhere.

From my own experiences, i find it true of Thomas's account of the late 1990s history of gnu language. I also think moving to Guile is the best practical choice for emacs, in particular because there's someone willing and able to do the work.

It's been 10 years. Sun Microsystems, once was trying to kill Microsoft, is now dead. (am very, very glad. There's no commercial software company i hate more than Sun Microsystems) Perl, once was the craze and number 1 scripting language, is now barely heard (am also very, very glad). For detail, see: Programing Languages, Purity, Cult, and Deception. Scheme and Haskell (2009) .

on scheme lisp