A Syntax Highlighting Language?

By Xah Lee. Date: 2012-10-03

This page describes these related issues:

is there a grammar-language to specify syntax coloring and related features for a programer's text editor? What would such language be like? What would it cover? How would it be different from something like BNF or parsing expression grammar?
A comparison of 3 extension packages written for {emacs, vim, Sublime Text}, for the language TypeScript.

so, Microsoft created TypeScript language, with bundled extensions for {emacs, vim, Sublime Text} editors. Download them at [Sublime Text, Vi, Emacs: TypeScript enabled! 2012-10-01 By Olivier Bloch. At ~~http://blogs.msdn.com/b/interoperability/archive/2012/10/01/sublime-text-vi-emacs-typescript-enabled.aspx~~ , accessed on 2012-10-03 ].

Am curious and looked at the code of these extensions. Here's some comparison:

Sublime Text: typescript.tmlanguage.xml → XML file of Mac OS X's Property list format, 132 lines.
vim: typescript.vim → Vim script. 321 lines.
Emacs: TypeScript.el → emacs lisp. 3351 lines.

There are many interesting aspects:

The extension language offered/used by each editor.
The number of lines required for each.
The ease of writing such extension.
The power of such extension in each editor.

It seems to me, apparently the easiest is the Sublime Text one. Just 132 lines. Is it some kinda standard format/protocol for specifying syntax highlighting? As far as i know, am not aware of any widely-used standard to specify a language syntax for the purpose of syntax coloring. Although, such standard could easily exist. For example, at the most basic level, a simple standardized format of list of language's keywords (For example, built-in function names, variables names, constant names, or regex for such, and list of character categories (such as comma, colon, semicolon, brackets, operators, etc))

Here is the doc about how to define syntax definition in Sublime: ~~http://docs.sublimetext.info/en/latest/extensibility/syntaxdefs.html~~ (it's a XML file of Mac OS X's Property list format)

Now, let's look at the vim extension, 321 lines. It's written in Vim script.

It's interesting that Microsoft has listed the people who wrote the vim extension: “Credits: Zhao Yi, Claudio Fleiner, Scott Shattuck, Jose Elera Campana”. But not for the sublime or emacs extension.

The most interesting one is the emacs one, at 3.3k lines, a very sizable code. That's 10 times the vim script one, and 25 times the Sublime one.

I have written a few emacs mode for languages. It isn't trivial. [see How to Write a Emacs Major Mode for Syntax Coloring] My first one, took about 2 months to finish, counting learning time. (though it's fairly feature-complete, including keyword completion, comment/uncomment, custom keys, doc lookup, GUI menu. [see Emacs LSL Mode (xlsl-mode) for Linden Scripting Language]) It's only 1.1k lines, but half of it is list of keywords one on each line.

At 3.3k lines, it's not a trivial package. Looking at the code, it seems to be written by elisp expert, not some quick hacked up noob job. But nowhere it did mention who wrote it. I recall, about 2 years ago, there were rumors going around social networks that Microsoft is hiring related to emacs. Maybe it's for this.

Of the 3, their README file says they do syntax coloring, except the emacs one, which also claims to do indentation.

Now, the interesting question is what degree of power each of these 3 extension language provides. (i think it's correct to say, without any doubt, that the emacs lisp is most powerful, by far.)

But, clearly, the emacs lisp one is also the most difficult and time-consuming to write. A more interesting question is the ratio of power/“ease of coding” the extension. Although the Sublime and Vim one didn't claim indentation feature, i'm sure the editor does it already (to what degree?).

Imagine, if there is such a lang spec for the purpose of syntax coloring and programer editor features (the spec would also include info about indentation, keyword completion, and others. But for short, let's just call it “syntax coloring spec”). Then, we write just 300 lines like the Sublime Text extension, in pure brainless inert format, without thinking, and it would have all features of fancy emacs language modes of 3k or 10k lines. And, all editors can use this file, a universal standard. And, to create a new lang mode would be trivial, without much programing skill.

I haven't tried any of these, as i'm not really familiar with Sublime or Vim, and it takes some effort to download the TypeScript and test it in emacs with the extension to see just how good is the syntax coloring, or indentation feature, or perhaps other features.

Has anyone looked at TypeScript using one of these editors? How good are they (the syntax coloring)? What features? Can anyone do screenshot of each?