Emacs Lisp: Process File line-by-line

By Xah Lee. Date: . Last updated: .

This page gives a example of how to use emacs lisp to process a file line by line, in a buffer, not as string of lines.

Problem

Given a file of many lines, like this:

at_target(integer tnum, vector targetpos, vector ourpos)

For each line, create a file of the same name as first part of the line. For example: at_target.txt

The file content should be the whole line, with other static text, like this:

# --
at_target(integer tnum, vector targetpos, vector ourpos)
{
$0
}

Detail

I'm writing a major mode for Linden Script Language (LSL). LSL is a scripting language used for the virtual world Second Life. It has few hundred functions, and each one has parameters that is unusual as compared to normal programing languages. For example, this is a LSL function:

at_rot_target(integer tnum, rotation targetrot, rotation ourrot) {
// …
}

So, i want a function template feature in my major mode. If a programer has typed “at_rot_target”, then, he can press a button, and it expands to:

at_rot_target(integer tnum, rotation targetrot, rotation ourrot) {
▮
}

There is a easy-to-use template system package for emacs, called YASnippet. [see YASnippet tutorial] So, i decided to use this instead of implementing my own template system.

With yasnippet, it uses a plain text for template definition. To define a template, you need to create a file. For example, in LSL there's a function named “collision” with this syntax collision(integer num_detected) {…}. This means, i must have a file named “collision” in the template dir, and the file content must be like this:

# --
collision(integer num_detected)
{
$0
}

This means, when my mode xlsl-mode is on, and yasnippet minor mode is on, then user can type “collision” followed by a hotkey for template completion, then the function form will be inserted and cursor will be placed between the braces.

I have prepared a file that is over 300 lines that are the LSL functions and parameters. For example, part of the file looks like this:

at_rot_target(integer tnum, rotation targetrot, rotation ourrot)
at_target(integer tnum, vector targetpos, vector ourpos)
attach(key id)
changed(integer change)
collision(integer num_detected)
collision_end(integer num_detected)
collision_start(integer num_detected)
control(key id, integer held, integer change)

(Save the above text in a file name it xx_event_forms.txt for later testing of elisp code.)

Now, the task is to parse this file, and for each line, create the template file for it.

Solution

The task has these steps:

These are simple tasks. There are a lot ways to do this in elisp. We can for example grab the whole file's text, then use split-string by newline char to get a list of lines. Then we loop thru the list.

Read File Content as List of Lines

Emacs Lisp: Read File Content as String or List of Lines

Process Each Line in a Buffer

Another way more idiomatic to emacs lisp, is to simply open the file in a buffer, then move cursor one line at a time, each time grab the line and do what we need to do.

For this task, the split lines into a list method is probably simpler. But since we are learning emacs lisp, let's use the emacs buffer method.

First, we define few global vars.

;; input file
(setq inputFile "xx_event_forms.txt")

;; other vars
(setq splitPos 0) ;; cursor position of split, for each line
(setq fName "")
(setq restLine "")
(setq moreLines t ) ;; whether there are more lines to parse

Now, we open the file, like this:

;; open the file
(find-file inputFile)
(goto-char (point-min)) ;; needed in case the file is already open.

Now, we loop thru the lines, like this:

(while moreLines
  (search-forward "(")

  (setq splitPos (1- (point)))
  (beginning-of-line)
  (setq fName (buffer-substring-no-properties (point) splitPos))

  (end-of-line)
  (setq restLine (buffer-substring-no-properties splitPos (point) ))

  ;; create the file
  (find-file fName)
  (insert "# --\n")
  (insert fName restLine "\n{\n$0\n}" )
  (save-buffer)
  (kill-buffer (current-buffer))

  (setq moreLines (= 0 (forward-line 1)))
)

In the above, we use search-forward to move cursor to the opening paren. Then, save the position to splitPos. Everything before that should be the template file name, so we save it in fName. Everything after that is restLine.

Now, we create the file fName using (find-file fName), then, insert the content, save it, close it.

Lastly, we move cursor to the next line by (forward-line 1). Note that if the cursor is at the last line, and when forward-line is unable to move forward, it will return a number indicating how many lines it failed to pass. So, normally it returns 0. If not, that means we are on the last line.

After we processed the lines, we just close the input buffer, like this:

(kill-buffer (current-buffer)) ;; close the input file

To test the above, first create a sample input file. Take the sample input lines above and save it as xx_event_forms.txt. Then, grab all the above lisp code and save it in a file test_line_process.el. Now, open the lisp file and Alt+x eval-buffer. Then all the template files will be created in the same dir.

Emacs 🧡

Emacs Lisp, Process Lines