Building Middleware

This page is very incomplete and a work in progress.

Overview

This part of the documentation aims to guide you in the process of implementing nREPL middleware. We’ll cover the basics, the best practices and some of the common pitfalls you might encounter.

This page complements the middleware design documentation. Make sure you’re familiar with it before proceeding to implement any middleware.

Basics

The basic structure of most middleware is pretty simple:

  • An op dispatch function (a.k.a. handler) that intercepts requests to the middleware

  • Some functions that do the actual work for each op and provide the responses

  • A middleware descriptor that specifies which middlewares are provided/required and documents the API of the middleware

Optionally you can have some dynamic vars for configuration purposes. Bellow is a complete example:

(ns nrepl.middleware.completion
  "Code completion middleware."
  (:require
   [nrepl.util.completion :as complete]
   [nrepl.middleware :as middleware :refer [set-descriptor!]]
   [nrepl.misc :refer [response-for] :as misc]
   [nrepl.transport :as t])
  (:import nrepl.transport.Transport))

;; middleware configuration
(def ^:dynamic *complete-fn*
  "Function to use for completion. Takes three arguments: `prefix`, the completion prefix,
  `ns`, the namespace in which to look for completions, and `options`, a map of additional
  options for the completion function."
  complete/completions)

;; the business logic for the "completions" op
(defn completion-reply
  [{:keys [session prefix ns complete-fn options] :as msg}]
  (let [ns (if ns (symbol ns) (symbol (str (@session #'*ns*))))
        completion-fn (or (and complete-fn (misc/requiring-resolve (symbol complete-fn))) *complete-fn*)]
    (try
      (response-for msg {:status :done :completions (completion-fn prefix ns options)})
      (catch Exception e
        (response-for msg {:status #{:done :completion-error :namespace-not-found}})))))

;; the handler
(defn wrap-completion
  "Middleware that provides code completion.
  It understands the following params:

  * `prefix` - the prefix which to complete.
  * `ns`- the namespace in which to do completion. Defaults to `*ns*`.
  * `complete-fn` – a fully-qualified symbol naming a var whose function to use for
  completion. Must point to a function with signature [prefix ns options].
  * `options` – a map of options to pass to the completion function."
  [h]
  (fn [{:keys [op ^Transport transport] :as msg}]
    (if (= op "completions")
      (t/send transport (completion-reply msg))
      (h msg))))

(set-descriptor! #'wrap-completion
                 {:requires #{"clone"}
                  :expects #{}
                  :handles {"completions"
                            {:doc "Provides a list of completion candidates."
                             :requires {"prefix" "The prefix to complete."}
                             :optional {"ns" "The namespace in which we want to obtain completion candidates. Defaults to `*ns*`."
                                        "complete-fn" "The fully qualified name of a completion function to use instead of the default one (e.g. `my.ns/completion`)."
                                        "options" "A map of options supported by the completion function."}
                             :returns {"completions" "A list of completion candidates. Each candidate is a map with `:candidate` and `:type` keys. Vars also have a `:ns` key."}}}})
Not all middleware respond directly to ops. You can have middleware that simply modify requests or responses.

It’s important to make sure you don’t let some exceptions slide, as some clients might block when waiting for a response (as some clients will try to handle nREPL messages in a synchronous fashion due to various limitation).

Notice that you can have multiple statuses for one response - e.g. a combination of :done and some error status.

Also notice that the example middleware depends on the session middleware, as it’s fetching some data from the session. In practice this means that the session middleware has already been applied to the message for the completion middleware before the message made its way to it, and the required session data has been added to it.

Testing

Typically we’d be testing nREPL middleware with integration tests that generate middleware requests and check their responses. nREPL bundles a few helper functions to streamline the process. Here’s a simple example:

(ns nrepl.middleware.completion-test
  {:author "Bozhidar Batsov"}
  (:require
   [clojure.test :refer :all]
   [nrepl.core :as nrepl]
   [nrepl.core-test :refer [def-repl-test repl-server-fixture project-base-dir clean-response]])
  (:import
   (java.io File)))

(use-fixtures :each repl-server-fixture)

(defn dummy-completion [prefix _ns _options]
  [{:candidate prefix}])

(def-repl-test completions-op
  (let [result (-> (nrepl/message session {:op "completions" :prefix "map" :ns "clojure.core"})
                   nrepl/combine-responses
                   clean-response
                   (select-keys [:completions :status]))]
    (is (= #{:done} (:status result)))
    (is (not-empty (:completions result)))))

(def-repl-test completions-op-error
  (let [result (-> (nrepl/message session {:op "completions"})
                   nrepl/combine-responses
                   clean-response
                   (select-keys [:completions :status]))]
    (is (= #{:done :completion-error :namespace-not-found} (:status result)))))

(def-repl-test completions-op-custom-fn
  (let [result (-> (nrepl/message session {:op "completions" :prefix "map" :ns "clojure.core" :complete-fn "nrepl.middleware.completion-test/dummy-completion"})
                   nrepl/combine-responses
                   clean-response
                   (select-keys [:completions :status]))]
    (is (= #{:done} (:status result)))
    (is (= [{:candidate "map"}] (:completions result)))))

Best Practices

In this section we’ll go over some of the best practices for implementing nREPL middleware.

Code Structure

It’s best to keep the business logic decoupled from the middleware code and have middleware serve as a thin wrapper around it. This means that ideally you should have the business logic in a different namespace (or even a different library). A good example would be the completion middleware bundled with nREPL:

  • The actual completion logic lives in nrepl.util.completion

  • The middleware code lives in nrepl.middleware.completion and it simply delegates to nrepl.util.completion

This separation makes it easier to test the business logic in isolation and to re-use it outside of nREPL (a good example here would be the orchard library that cider-nrepl uses heavily).

As a corollary - you should avoid creating middleware that provides a lot of unrelated functionality. Ideally all ops within some middleware should be closely linked by their purpose.

Naming conventions

It’s recommended to prefix middleware names with wrap - e.g. wrap-complete, wrap-lookup, etc. This naming convention came from Ring middleware, which was a major influence on the design of nREPL.

nREPL itself breaks this convention with names like add-stdin and interruptible-eval, but those names are historical and hard to change at this point.

When it comes to ops, ideally their names should be verbs - e.g. complete, lookup, test, etc. Related ops can be grouped under some common prefix - e.g. test-var, test-ns, test-all.

It’s also prudent to prefix op names with some "namespace"-like prefix to avoid conflicts between different middleware - e.g. project-name/op-name.

That’s the reason why nREPL’s completion op is named completion instead of complete. cider-nrepl already has an op named complete and adding such an op to nREPL would have introduced some non-deterministic behaviour when it comes to the ordering of the two competing completion middleware. As a result, in some cases you’d be invoking nREPL’s op and in other cases cider-nrepl’s op. If `cider-nrepl had named the op cider/complete instead, that would have prevented this unfortunate situation.

Additional Resources