Clojure Tutorial: nREPL

nREPL is a Clojure library. The name “nREPL” means “network REPL”. It lets applications to connect to a Clojure engine via network (remotely or locally). For example, it can be used to let editor/IDE such as emacs or web browser based app to evaluate Clojure code.

nREPL home page: https://github.com/clojure/tools.nrepl

nREPL API documentation: http://clojure.github.io/tools.nrepl/

Here's a basic tutorial for writing app using nREPL.

This page is WORK IN PROGRESS.

nREPL includes both server and client. That is, you can use it to create/start a Clojure code evaluation server. It also includes a client. That is, you can use it to query a Clojure code evaluation server and get result.

using nREPL as Client

start a test nrepl server

first, lets use lein to start a nrepl server.

to start a nrepl server using lein, in terminal type

#start a nrepl server using lein
lein repl :start :port 55555

[see Clojure Leiningen Tutorial]

(defproject xah-clojure-nrepl-client "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
                  :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [org.clojure/tools.nrepl "0.2.5"]]
  :main ^:skip-aot xah-clojure-nrepl-client.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}})

(ns xah-clojure-nrepl-client.core
    (:require [clojure.tools.nrepl :as nrepl] )
    (:gen-class))

(defn -main
  [& args]
  (with-open [tp (nrepl/connect :port 55555)]
             (pr
              (nrepl/response-values
               (nrepl/message
                (nrepl/client
                 tp
                 1000)
                {:op "eval" :code "(+ 3 4)"}))))
  ) ; [7]

nREPL largely are made up of three parts: {handler, middleware, transport}

nREPL is message-oriented and asynchronous.

nrepl messages

nrepl server and client send each other “messages”.

A message is a clojure “map” data type.

for example, here's a nrepl message:

{"op" "eval" "code" "(+ 3 4)"}

A message has many keys. Some keys are required. The key/value pairs and meaning depends on “transport” used. (“transport” is explained later.)

the "op" key means the operation to perform.

A key may imply existance of other key. For example, "op" key with value of "eval" requires a key named "code".

The result of performing each operation may be sent back to the nREPL client in one or more response messages. The content depend on the operation.

Depending on its :op, a message might be required to contain other slots, and might optionally contain others. It is generally the case that request messages should contain a globally-unique :id. Every request must provoke at least one and potentially many response messages, each of which should contain an :id slot echoing that of the provoking request.

Once a handler has completely processed a message, a response containing a :status :done must be sent. Some operations necessitate that additional responses related to the processing of a request are sent after a :status :done is reported (e.g. delivering content written to *out* by evaluated code that started a future).

Other statuses are possible, depending upon the semantics of the :op being handled; in particular, if the message is malformed or incomplete for a particular :op, then a response with a :status :error should be sent, potentially with additional information about the nature of the problem.

nrepl transports

“transport” is a layer that specify what protocol to use to transmit messages. The default transport is Network socket, and message content is encoded by Bencode (basically a scheme of transmitting bytes using ASCII characters. [see ASCII characters])

nrepl handlers

nrepl handler is a function that accept a single incoming message as argument.

Handler takes a message (map) as argument. The argument given to handler has keys of datatype “keyword” (that is, of the form {:x1 …, :x2 …, etc}).

the argument given to handler are guaranteed to have a key :transport. Its value is a transport object, and is used to send response via that object.

handler return values are ignored.

middleware

“Middleware” are functions that accept a handler function and return a new handler.

“Middleware” is used for adding additional functionality onto or around the original handler.

Middleware are wrapers, for extending server's functionality, in other words.

All of nREPL's default functionality are made of many middleware. You can see these at clojure.tools.nrepl.server/default-middlewares.

default middleware are merged with any user-specified middleware provided to clojure.tools.nrepl.server/default-handler.

The merge is done in server configuration step. Server configuration are done in a format called “descriptor”. (“descriptor” is explained later.)

For example, here's a middleware that handles a message that contains {… :op "time?" …}. It replies with the local time on the server. Here's the code:

(require '[clojure.tools.nrepl.transport :as tp])
(use '[clojure.tools.nrepl.misc :only (response-for)])

(defn my-middleware-current-time
  "takes a function and returns a function."
  [xx]
  (fn [{:keys [op transport] :as msg}]
      (if (= "time?" op)
          (tp/send transport
                   (response-for msg :status :done :time (System/currentTimeMillis)))
        (xx msg))))

;; the arg, xx, is a function (a “handler”)

In the above, we defined a function “my-middleware-current-time”. It takes a function “xx” (a handler), and returns a function (a new handler).

this new handler adds a extra functionality. If a handler's input message contains {:op "time?"} pair, then it sends the current time (as side effect).

the (clojure.tools.nrepl.transport/send transport message) sends the message message via the transport object transport. It returns a transport object.

nrepl Server Middleware configuration: Middleware Descriptors

clojure.tools.nrepl.middleware/set-descriptor!

(set-descriptor! middleware_var descriptor)

Sets the given [descriptor] map as the ::descriptor metadata on the provided [middleware-var], after assoc'ing in the var's fully-qualified name as the descriptor's "implemented-by" value.

Sets the given [descriptor] map as the ::descriptor metadata on the provided [middleware-var], after assoc'ing in the var's fully-qualified name as the descriptor's “implemented-by” value.

nrepl supported operations (message keys)

for list and detail of nREPL's default middleware stack, what each operation expects in request messages, and what they emit for responses, see https://github.com/clojure/tools.nrepl/blob/master/doc/ops.md

Other nREPL middlewares are provided by the community.

Middleware descriptors and nREPL server configuration

It is generally the case that most users of nREPL will expect some minimal REPL functionality to always be available: evaluation (and the ability to interrupt evaluations), sessions, file loading, and so on. However, as with all middleware, the order in which nREPL middleware is applied to a base handler is significant; e.g., the session middleware's handler must look up a user's session and add it to the message map before delegating to the handler it wraps (so that, for example: evaluation middleware can use that session data to stand up the user's dynamic evaluation context). If middleware were “just” functions, then any customization of an nREPL middleware stack would need to explicitly repeat all of the defaults, except for the edge cases where middleware is to be appended or prepended to the default stack.

To eliminate this tedium, the vars holding nREPL middleware functions may have a descriptor applied to them to specify certain constraints in how that middleware is applied. For example, the descriptor for the clojure.tools.nrepl.middleware.session/add-stdin middleware is set thusly:

(set-descriptor! #'add-stdin
  {:requires #{#'session}
   :expects #{"eval"}
   :handles {"stdin"
             {:doc "Add content from the value of \"stdin\" to *in* in the current session."
              :requires {"stdin" "Content to add to *in*."}
              :optional {}
              :returns {"status" "A status of \"need-input\" will be sent if a session's *in* requires content in order to satisfy an attempted read operation."}}}})

Middleware descriptors are implemented as a map in var metadata under a :clojure.tools.nrepl.middleware/descriptor key. Each descriptor can contain any of three entries:

• :requires, a set containing strings or vars identifying other middleware that must be applied at a higher level than the middleware being described. Var references indicate an implementation detail dependency; string values indicate a dependency on any middleware that handles the specified :op.
• :expects, the same as :requires, except the referenced middleware must exist in the final stack at a lower level than the middleware being described.
• :handles, a map that documents the operations implemented by the middleware. Each entry in this map must have as its key the string value of the handled :op and a value that contains any of four entries:
• :doc, a human-readable docstring for the middleware
• :requires, a map of slots that the handled operation must find in request messages with the indicated :op
• :optional, a map of slots that the handled operation may utilize from the request messages with the indicated :op
• :returns, a map of slots that may be found in messages sent in response to handling the indicated :op

The values in the :handles map is used to support the “describe” operation, which provides “a machine- and human-readable directory and documentation for the operations supported by an nREPL endpoint” (see clojure.tools.nrepl.middleware/describe-markdown, and the results of “describe” and describe-markdown here).

The :requires and :expects entries control the order in which middleware is applied to a base handler. In the add-stdin example above, that middleware will be applied after any middleware that handles the “eval” operation, but before the clojure.tools.nrepl.middleware.session/session middleware. In the case of add-stdin, this ensures that incoming messages hit the session middleware (thus ensuring that the user's dynamic scope — including *in* — has been added to the message) before the add-stdin's handler sees them, so that it may append the provided stdin content to the buffer underlying *in*. Additionally, add-stdin must be “above” any eval middleware, as it takes responsibility for calling clojure.main/skip-if-eol on *in* prior to each evaluation (in order to ensure functional parity with Clojure's default stream-based REPL implementation).

The specific contents of a middleware's descriptor depends entirely on its objectives: which operations it is to implement/define, how it is to modify incoming request messages, and which higher- and lower-level middlewares are to aid in accomplishing its aims.

nREPL uses the dependency information in descriptors in order to produce a linearization of a set of middleware; this linearization is exposed by clojure.tools.nrepl.middleware/linearize-middleware-stack, which is implicitly used by clojure.tools.nrepl.server/default-handler to combine the default stack of middleware with any additional provided middleware vars. The primary contribution of default-handler is to use clojure.tools.nrepl.server/unknown-op as the base handler; this ensures that unhandled messages will always produce a response message with an :unknown-op :status. Any handlers otherwise created (For example, via direct usage of linearize-middleware-stack to obtain a ordered sequence of middleware vars) should do the same, or use a similar alternative base handler.

response-values will return only the values of evaluated expressions, read from their (by default) pr-encoded representations via read. You can see the full content of message responses easily:

(with-open [conn (repl/connect :port 33623)]
            (-> (repl/client conn 1000)
                (repl/message {:op :eval :code "(time (reduce + (range 1e6)))"})
                doall ;; `message` and `client-session` all return lazy seqs
                pprint)) ;; nil

({:out "\"Elapsed time: 68.032 msecs\"\n",
       :session "2ba81681-5093-4262-81c5-edddad573201",
       :id "3124d886-7a5d-4c1e-9fc3-2946b1b3cfaa"}
 {:ns "user",
      :value "499999500000",
      :session "2ba81681-5093-4262-81c5-edddad573201",
      :id "3124d886-7a5d-4c1e-9fc3-2946b1b3cfaa"}
 {:status ["done"],
          :session "2ba81681-5093-4262-81c5-edddad573201",
          :id "3124d886-7a5d-4c1e-9fc3-2946b1b3cfaa"})

Each message must contain at least an :op (or "op") slot, which specifies the "type" of the operation to be performed. The operations supported by an nREPL endpoint are determined by the handlers and middleware stack used when starting that endpoint; the default middleware stack (described below) supports a particular set of operations, detailed here.

using nREPL as Server

(use '[clojure.tools.nrepl.server :only (start-server stop-server)]) ; nil

(defonce server (start-server :port 7888))