Meaning of Lisp List, Function Type, and Syntax Coloring

Reddit question:

Why does emacs lisp syntax highlighting not color the car of an s-expressoin differently than the cdr? I would think that would make differentiating proper lists from s-expressions much easier, no? I don't understand why coloring all elements of the s expression equally is useful.

[source: https://www.reddit.com/r/emacs/comments/5n2vji/why_does_emacs_syntax_highlighting_not_color_the/dc8q4yn/]

my answer:

emacs lisp manual distinguishes symbols by the following categories

.
    Kind           Predicate           Example
    ----------------------------------------
    Special Form    special-form-p    progn
    Macro           macrop              when
    Command         commandp           count-words
    Prefix Command  none            help-command
    Function        functionp       file-name-directory
    Constant        none            dir-locals-file
    User Option     none            after-save-hook
    Variable        boundp          buffer-file-name

you can see that in each symbol doc entry in elisp manual is prefixed by one of the above “kind” words with two dash. For example,

-- Command: find-file filename &optional wildcards

;; eval this in emacs to jump to elisp manual page
    (info "(elisp) Visiting Functions")

or see Visiting Functions (ELISP Manual)

Note: there are only 1 Prefix Command in elisp manual: help-command

Note: there are only 2 constants in elisp manual: • dir-locals-file • display-buffer-fallback-action

Note: these “kinds” are mostly not mutually exclusive. For example, special form, macro, command, are all functions too. Rather, this “categorization” is helpful on what the function is most useful for, or give indication about some behavior of the function. (For example, special form and macro don't eval args in the normal way, and special form is written in C, while macro is user-define in lisp, while command is callable by user as M-x.)

so, to me, it'd be helpful that emacs lisp mode color symbols by the above categories.

for some reason, the emacs-lisp-mode does not. (i asked about how emacs-lisp-mode chose to color some decade ago, but haven't got satisfactory answer.)

if anyone wants a elisp mode colored by the “kinds of symbol” approach above, try Emacs: Xah Elisp Mode (xah-elisp-mode.el).

as to your question, as other said, lists are s-expressions, proper list or not. List is made of atoms separated by space. Atom itself is a s-expression. s-expression is either atom or list.

the first atom in a list is sometimes called the “head” of expression. It is special, in the sense that this atom is presumed to be function.

also, it's critical to understand that there are 2 senses of “list” in lisp literature. One is linked list datatype, as in many but not all other programing language's sense (and this is implemented by nesting cons in a proper way (with last con's last element being nil), called proper list). The other sense of list, is lisp's internal syntax, it is the output of lisp “reader”. In summary: lisp reader takes the source code, output lisp “syntax” (lists), and this “syntax” is feed into lisp compiler.

To really understand it, you have to understand lisp “reader” in detail. (lisp has a 2-level sense “syntax”. One sense is like all other programing languages, which is what programers type to be valid code. The other sense, is the output of lisp reader. Lisp reader reads in what you typed, and return (in some internal sense) a textual “list”.) Lisp calls this “syntax”.

GNU Emacs Lisp Reference Manual: Read and Print Read and Print (ELISP Manual)

in practice, understanding lisp reader isn't much useful for most lisp coding tasks. It's useful when you want to write a language, or do advanced meta-programing.

PS of this thread so far, i recommend /u/RobThorpe 's answer. His perspective is more of a low-level with implementation.