A lot of our internal code base at Exoscale is written in the Clojure programming language. While its expressiveness and functional nature makes the language attractive, its lack of full-application development tutorials makes it a little more difficult to approach than other languages.

Here is such a tutorial, exploring how to build a simple web application with the stack we at Exoscale have grown to love.

a complete Clojure turorial

Enter exopaste: a pastebin application in Clojure

As an example, we’ll write a simple pastebin clone, exposing a service on the HTTP port, and storing the pastes in an in-memory database.

The principle of pastebin services, in a nutshell, is to allow you to paste snippets of text, into a web form and retrieve the content later on, via an invariant URL. This allows to quickly share text and code snippets across the internet.

The Ubuntu pastebin is an example of such a service.

Choosing the right Clojure libraries

We’ll use the same stack we tend to use at Exoscale, leveraging the following libraries:

  • Aleph for the webserver itself. Aleph is a great webserver with very good performance characteristics, and we tend to converge towards it for new projects. It also comes with a convenient HTTP client that can leverage the same thread pool (if needed), which makes it a very attractive option. We won’t need all of its power for our simple pastebin service, but it’s good to know that that power is available should the need arise.
  • Bidi for the routing library. There are other good options here, such as Compojure, but we chose to go with Bidi, because it also works in ClojureScript out of the box, and offers a convenient bi-directional (hence the name) approach to routes, allowing us to know what the route of a particular handler is.
  • Component to fit everything together. Component is similar to “dependency injection frameworks” of the Java world, allowing us to write modular and decoupled applications. We will leverage its power through this articles to make our paste service able to “grow” into more and more robust back-ends.
  • Hiccup for HTML “templating”. Hiccup allows programmers to define HTML code as a normal Clojure data structure, and takes care of the rendering into HTML for you. The advantage of this approach is that your view code is pure Clojure structures that you can use your usual tools and functions to create.

Writing the exopaste application

The full code for this example application is freely available online on github, and it is encouraged to refer to the full source code there while reading this post. The snippets provided here generally do not show the whole file but rather highlight specific parts.

Get started with Leiningen

Leiningen is the most popular way to manage Clojure projects, and deals with the project’s release and dependency management for us. It serves as the entry point to your Clojure project as a developer. A few of the (many) useful commands:

  • lein run will run your project’s “main” function after making sure all the dependencies are up to date.
  • lein test will run your project’s tests
  • lein uberjar probably the least intuitive here, this command will bundle up your project and all its dependencies in a single executable jar, allowing your to run it with java -jar directly.

The command that interests us most right now however is the lein new command, which will write out the skeleton of a Clojure project for us to start from:

lein new app exopaste

We won’t need the “core” module the template generates for us, so we can get rid of it now:

rm src/exopaste/core.clj
rm test/exopaste/core_test.clj

Let’s replace the default project file with our own, including some of the dependencies we have chosen:

;; Full source code can be found here:
;; https://github.com/exoscale/exopaste/blob/0.1.0/project.clj
(defproject exopaste "0.1.0-SNAPSHOT"
  :description "A pastebin clone storing data in-memory"
  :url "https://github.com/exoscale/exopaste"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.9.0"]
                 [com.stuartsierra/component "0.3.2"]
                 [aleph "0.4.6"]
                 [hiccup "1.0.5"]
                 [bidi "2.1.3"]])

Building a Clojure application with components

We use components a lot at Exoscale, since they allow flexible run time composition of applications.

For our simple pastebin clone, we’ll need just a couple of components:

  1. A webserver component that will handle the aleph startup and bidi routes.
  2. An in-memory “store” for our pasted data to live in. The reason we’ll make it a component here (as opposed to a simple module with functions) is to allow further articles to build upon this foundation and make our application save its data in a real database.

What are components exactly?

Components are implemented as clojure records, which you can think of as a Map where the fields are declared, and on which functions can be bound.

Furthermore, Clojure allows the definition of protocols, which declare a group of functions that records must implement.

;; A protocol declaring two functions. Note that there are no implementations.
(defprotocol Openable
  (open [x])
  (close [x]))

;; This record implements our protocol:
(defrecord Door []

  Openable

  ;; Implementations are provided here.
  (open [x] (println "opening door!"))
  (close [x] (println "closing door!")))

Our components are records that implement the “Lifecycle” protocol, that defines two simple functions: start and stop.

;; This is what the Lifecycle component looks like. You don't need to write this
;; as it's already part of the component library.
(defprotocol Lifecycle
  (start [x])  ;; What to do when the component is started.
  (stop [x]))  ;; What to do when the component is stopped.

Those functions define what should happen when the component starts (typically, setup some external state such as a database connection), and what should be done when it is stopped (closing the connection).

The system map

In order for our components to be started, we need to build a system map using the aptly named (component/system-map) function. What we do here is tell the component system which components to actually use, a little bit like what inversion of control frameworks of the java world would do (conceptually similar to the Spring framework, for example).

This allows us to build very simple components first (such as our in-memory store), that we will later be able to easily swap for another implementation, for instance using a real database as a back-end instead of simply storing data in-memory.

;; Full source code can be found here:
;; https://github.com/exoscale/exopaste/blob/0.1.0/src/exopaste/system.clj

(defn build-system
  "Defines our system map."
  []
  (try
    (-> (component/system-map
         :server (server/make-server)
         :store (store/make-store))
        (component/system-using {:server [:store]}))
    (catch Exception e
      (error "Failed to build system" e))))

While the (system-map) call describes the list of components we have, the next call, (system-using) describes the relationship between them. In our case, for now the only relationship we declare is that our server component should know about and be allowed to access our storage component.

Behind the scenes, the components library will add a :store entry to the server component (a component is conceptually a map).

A simple Clojure webserver component

In order to serve our pastes, we’ll need a server component that is in charge of serving HTTP. We need very few things here: an URL to which to paste a new paste to, a URL with which to read pastes (by identifier), and a top-level form with which to present the form.

In its simplest form, that component wraps the aleph webserver:

;; Full source code can be found here:
;; https://github.com/exoscale/exopaste/blob/0.1.0/src/exopaste/server.clj

(defrecord HttpServer [server]

  component/Lifecycle

  (start [this]
    (assoc this :server (http/start-server (handler (:store this)) {:port 8080})))

  (stop [this]
    (assoc this :server nil)))

(defn make-server
  []
  (map->HttpServer {}))

As you notice, we pass it the result of the (handler) function, that builds the bidi handler. The handler is what maps the URLs your users will input to functions (that produce the content to display in the client).

The handler function also receives the store, as an entry in the server component map. This allows us to “forward” a reference to the store to our paste handler:

;; Full source code can be found here:
;; https://github.com/exoscale/exopaste/blob/0.1.0/src/exopaste/server.clj

(defn paste-handler
  [store request]
  (let [paste (store/get-paste-by-uuid store (:uuid (:route-params request)))]
    (res/response (view/render-paste paste))))

(defn handler
  "Get the handler function for our routes."
  [store]
  (make-handler ["/" {"" (partial index-handler store)
                      [:uuid] (partial paste-handler store)}]))

This will return the result of the index-handler and paste-handler functions to the HTTP client when launched via the system map, and we can immediately see how the paste’s uuid parameter will be passed to our handling function.

Notice that the handler makes use of the (partial) function around the handler functions. The reason for this is that our handlers would like to have two parameters - the request, and the store. However the ring specification says that handlers will only get a single parameter - the request. The partial function binds the store to our handler, and returns another function that now takes a single argument.

The store component

We need a place to store the data we send to our exopaste server. For now, we’ll build a very simple storage component that leverages a Clojure “atom” - a Clojure concurrency primitive wrapping arbitrary state - as an in-memory data-store.

During our store component’s initialization, we set its “data” field to a fresh new (atom). If we had a database connection (for example), we would also need to close it at program shutdown in the (stop) function.

;; Full source code can be found here:
;; https://github.com/exoscale/exopaste/blob/0.1.0/src/exopaste/store.clj

(defrecord InMemoryStore [data]

  component/Lifecycle

  (start [this]
    (assoc this :data (atom {})))

  (stop [this] this))

(defn make-store
  []
  (map->InMemoryStore {}))

The component itself is not the full story however - we need to add behavior as well.

For now, we’ll simply add a couple of functions that take our component as a parameter, and perform the required operations:

;; Full source code can be found here:
;; https://github.com/exoscale/exopaste/blob/0.1.0/src/exopaste/store.clj

(defn add-new-paste
  "Insert a new paste in the database, then return its UUID."
  [store content]
  (let [uuid (.toString (java.util.UUID/randomUUID))]
    (swap! (:data store) assoc (keyword uuid) {:content content})
    uuid))

(defn get-paste-by-uuid
  "Find the paste corresponding to the passed-in uuid, then return it."
  [store uuid]
  ((keyword uuid) @(:data store)))

Hiccup: Clojure HTML templating and rendering

Our pastebin app still needs to render actual HTML to the client’s web browser. For this, we’ll create a new module, that will leverage the Hiccup library to render HTML.

Hiccup is a neat Clojure library that makes it convenient to create HTML with Clojure syntax, and since all of the data you write is just that - data - you can use all the power of Clojure to compose your view.

For our simple pastebin application, we will need just two views: one to render a paste, and one to display the paste form to our users.

;; Full source code can be found here:
;; https://github.com/exoscale/exopaste/blob/0.1.0/src/exopaste/view.clj

(defn render-form
  "Render a simple HTML form page."
  []
  (html5 [:head
          [:meta {:charset "UTF-8"}]]
         [:body
          (form-to [:post "/"]
                   (text-area {:cols 80
                               :rows 10} "content")
                   [:div]
                   (submit-button "Paste!"))]))

Running it all: lein run

As we mentioned before, running our project is a simple case of executing lein run.

For now, our little project will start a server listening on localhost, so pointing your browser to “http://localhost:8080” should get you a running, minimal pastebin using our stack!

Let’s try to paste a simple Clojure hello world example:

(println "hello, world!")

Pressing the submit button will redirect us to a paste, with Clojure syntax highlighting!