How Mathematica does Unicode?

This page explains some tech detail about how Mathematica uses Unicode.

Mathematica supports Unicode, but does not use Unicode when saving to file. [see Unicode Basics: Character Set, Encoding, UTF-8]

Mathematica files use 7-bits ASCII only. http://www.wolfram.com/technology/nb/

How does it support Unicode if it uses only ASCII?

Mathematica's Named Characters \[Name]

Mathematica has a set of special characters with the syntax \[name]. For example:

So, when you type \[Alpha], it is displayed as “α”. (All built-in symbols in Mathematica starts with capital letter.)

You can think of them as HTML's “named character entities”. [see Character Sets and Encoding in HTML] There are about 900 named chars. http://reference.wolfram.com/mathematica/guide/ListingOfNamedCharacters.html

Many of the named chars are also in Unicode, but not all. Similarly, many Math Symbols in Unicode are not in this list. Also, Unicode's Chinese chars, Arabic alphabets etc, are not in Mathematica's named chars.

Map Between Unicode and Named Chars

When you paste a Unicode char into Mathematica, Mathematica will try to interpret the Unicode as one of the named char.

So, for example, if you paste “α” (GREEK SMALL LETTER ALPHA; “U+x3b1”), it automatically becomes Mathematica's \[Alpha], and displayed as “α”.

Syntax for Arbitrary Unicode Char

For any Unicode that's not one of Mathematica's named char (such as Chinese chars), their syntax is this: \:xxxx, where the xxxx is Unicode's 4 digit hexidecimal representation of the char. For example, the Chinese char “水” (water), Unicode hex is “6c34”, in Mathematica is: \:6c34.

The above roughly summarize how Mathematica takes Unicode as input.

Some Named Chars has Built-in Meaning

Of the named chars, many has special meaning in Mathematica. For example, π \[Pi] is automatically considered identical to the built-in symbol Pi, which means the mathematical constant. (So, if you type \[Pi] or \:03c0, they are displayed as π with meaning of Pi.). Here's some examples of special meaning named chars.

Note: it appears that it is possible to over-ride the default interpretation of named char to built-in symbol (function, constant), for all or some of the named char. (i haven't investigated on how yet.)

• http://reference.wolfram.com/mathematica/ref/MakeExpression.html
• http://reference.wolfram.com/mathematica/tutorial/Operators.html

Alias Shortcut for Named Chars

Some of the named char has one or more aliases for ease of input. For example, to enter α, you can type EscapeaEscape or EscapealphaEscape. Here's some examples:

• http://reference.wolfram.com/mathematica/tutorial/Introduction-ListingOfNamedCharacters.html
• http://reference.wolfram.com/mathematica/guide/SpecialCharacters.html

Inputting Special Chars

You can input a special character by:

• Use one of the graphical palettes.
• Type Escape, then the char's alias name, then Escape again.
• Copy the Unicode char somewhere and pasting it in Mathematica.
• Type it like this: \[Name].
• Type the Unicode hexadecimal like this: \:xxxx