A Tutorial To Stuart Sierra's Component

clojure tutorial

A lot of people have already heard that they should use Stuart Sierra’s component library to manage state in their Clojure applications. But most don’t know why and how you would incorporate it into your software.

This how to guide will give you all the tools you need to start managing state in your Clojure software.

What is component?

Component is a Clojure library that can be thought of as a micro dependency injection framework. I use it in all my web projects to manage runtime state.

What are alternative(s) to component?

  • Mount is an alternate way of managing state in Clojure that’s picking up steam.

Component Strengths

  • Component enables the reloaded workflow, which makes it much easier to reload code with runtime state in a REPL. Imagine you have a web server running and you change a route and need to reload it. If you were just use a def, you can easily lose reference to that web server when reloading your namespace. Then you’d have a random server that you can’t shut down in the background bound to a port that you can’t use anymore.
  • Managing dependencies between components in your system. A web server component is going to need access to its handlers as a component. Some of those handlers are going to need access to a database component.
  • Forbids circular dependencies between components.
  • You can have multiple systems. Component, unlike mount, allows you to have as many systems as you want.
  • Components are reusable. You can create a database component one time that takes a config map and can connect to whatever database(s) you need.
  • Everything is a map. Your system-map is a map, and components are records that implement Lifecycle, which are also maps. It makes it easy to manipulate. For instance, you can have your application’s system map in your tests, and instead of using an actual database connection, you can assoc a mock database instead to use.

Component Weaknesses

  • You have to buy completely into component. If you already have an existing application, adding component to it will be tough because you needed to have designed your system in terms of components from the ground up.
  • You have to specify your components’ dependencies manually.
  • Stuart Sierra says it might be overkill for small applications. Based on my own experience, I say it depends.
  • Component makes it so you have to pass (inject) your dependencies everywhere. Every function that touches your database is going to need to be passed in that database component somewhere. In general, I believe this is a good practice. If you would rather reference a singleton instead, you’re not going to like component.

An Example Of Component

Let’s say you’re planning on building a new service for managing emails in your application. The work involved is taking messages off of a queue, doing some work, writing data to a database, and then putting things onto other queues. Everything will be asynchronous.

Imagine we had a queue that had requests for messages that needed to go out. An outgoing queue. You have other services that put things onto this queue when they need to send an email. You might have your website’s user authentication functionality queue up emails to go out when a user signs up or forgets their password. You might also have a background job that queues up product recommendations to go out to users every week.

The job of this service is to take those messages, send off a request to an email service provider, log the messages and their statuses in a database, and queue up messages for other services to consume.

How would you break this down into components?

These are the components I see us needing:

  • Database
  • Queue
  • Worker Process
  • Core async channels
  • Email service component

The database and queue should be super obvious candidates for componetizing. The worker process is basically a handler. The core async channels and email service are probably not so obvious components.

Let’s sketch out some code. I’ll explain as we go.

Note: This will just be snippets of code that I haven’t fully tested as I don’t plan on writing an entire microservice. If you find major errors, go ahead and send me an email.

;; Note again, I didn't actually run this code so there ~may~ WILL be
;; errors and mistakes. The point is to demonstrate the power of
;; component in an example!
(require '[clojure.core.async :as async]
         '[com.stuartsierra.component :as component])

;; Here's the system we'll implement
(defn ->system
  [config]
  (component/system-map
   :input-chan (async/chan 1024)
   :result-chan (async/chan 1024)
   ;; This is the queue of emails that need to go out, which is the
   ;; INPUT to our service. We're never going to queue anything here
   ;; so we give it a simple chan to satisfy dependencies
   :outgoing-emails (component/using (map->Queue (assoc (:outgoing-emails config)
                                                        :outgoing-messages-chan (async/chan)))
                                     {:incoming-messages-chan :input-chan})
   :db (map->Database (:database config))
   :mailgun (map->EmailService (:mailgun config))
   :worker (component/using (map->Worker {:work-fn log-and-send-emails})
                            {:input-chan :input-chan
                             :db :db
                             :email-service :mailgun
                             :result-chan :result-chan})
   :sent-emails (component/using (map->Queue (assoc (:sent-emails config)
                                                    :incoming-messages-chan (async/chan))
                                             {:outgoing-messages-chan :result-chan}))))


;; We're going to pretend we have some library for some queue that
;; allows us to connect, subscribe, and publish messages
(defrecord Queue
    [config conn incoming-messages-chan outgoing-messages-chan]
    component/Lifecycle
    (start [this]
      ;;Put whatever messages that come in onto our incoming channel
      (subscribe-to-queue config
                          (fn message-handler [msg]
                            (async/put! incoming-messages-chan msg)))
      (let [conn (connect-to-queue config)
            stop-chan (async/chan 1)]
        ;; Start a loop that goes on until we put something onto the
        ;; stop channel
        (async/go-loop []
          (async/alt!
            outgoing-messages-chan
            ([msg]
             (publish-to-queue config msg)
             (recur))
            stop-chan
            ([_]
             :no-op)))
        ;; Assoc the stop-chan onto the record so we have access to it
        ;; in stop function
        (assoc this
               :stop-chan stop-chan
               :conn conn)))
    (stop [this]
      ;; Stop the go-loop
      (async/put! (:stop-chan this) :stop)
      ;; Close the connection
      (close-connection (:conn this))
      (assoc this
             :config nil
             :incoming-messages-chan nil
             :outgoing-messages-chan nil
             :stop-chan nil)))

;; The database example is a common one so I'll be really brief. In
;; fact, Stuart Sierra has one in the readme of component
(defrecord Database
    [config conn]
  component/Lifecycle
  (start [this]
    (let [conn (connect-to-db config)]
      (assoc this :conn conn)))
  (stop [this]
    (close-db-conn (:conn this))
    (assoc this :conn nil)))

;; Here's an interesting example and use case. You can implement more
;; protocols on records than just the Lifecycle one, something I don't
;; see very many people doing.

;; For instance, we're going to have a generic MessageService protocol
;; that our EmailService can implement.
(defprotocol MessageService
  (send [this msg opts]))

;; Our EmailService can implement MessageService
(defrecord EmailService
    [config]
  ;; I explain at the bottom that all java objects implement
  ;; Lifecycle. Since this is going to be stateless here, this is
  ;; OPTIONAL
  component/Lifecycle
  (start [this] this)
  (stop [this] this)

  MessageService
  (send [this msg opts]
    (smtp-send! config msg opts)))

;; The good thing about implementing a message protocol for our
;; EmailService is that in testing we can mock it out by making a
;; fixture that implements it. This fixture just adds the emails to an
;; atom we can inspect later to see that our message has been "sent"
;; in testing. Also, with the component model, you can just take your
;; application's system map and replace your EmailService with the
;; MockEmailService and it'll just work.
(defrecord MockEmailService
    [sent-emails-atom]
  MessageService
  (send [this msg opts]
    (swap! sent-emails-atom conj msg)))

;; Next up is the Worker component

;; I want to note important something here. In the system below, we
;; give the Worker a db and email service on top of the input/result
;; chans and work-fn. Components are just records. You don't have to
;; explicity say in your defrecord that your record needs a db and an
;; email service. As long as you specify that in your system map,
;; component will assoc them onto the record for you. What does this
;; mean in practice? You can have a generic Worker component that
;; takes things off of a channel and puts the result onto another
;; channel. I provide the Worker component's work-fn with the worker
;; itself (work-fn this result). This way, the work-fn has access to
;; the worker's dependencies if it needed to, and our record
;; definition is general.
(defrecord Worker
    [input-chan result-chan work-fn]
  component/Lifecycle
  (start [this]
    (let [stop-chan (async/chan 1)]
      (async/go-loop []
        (async/alt!
          input-chan
          ([result]
           (async/put! result-chan (work-fn this result))
           (recur))
          stop-chan
          ([_]
           :no-op)))
      this))
  (stop [this]
    (a/put! stop-chan :stop)
    (assoc this
           :input-chan nil
           :stop-chan nil
           :result-chan nil
           :work-fn nil)))

;; Some work function we'll provide to our worker
(defn log-and-send-emails
  [worker email-msg]
  (smtp-send! (:email-service worker) email-msg {})
  (write-status-to-database (:db worker))
  ;; Return some message to be queued to sent-emails
  {:status :sent
   :email-msg email-msg})

;; I'm going to omit the sent-emails queue as it'll be the same as the
;; other queue (but with different config).

;; Here's our system again for reference
(defn ->system
  [config]
  (component/system-map
   :input-chan (async/chan 1024)
   :result-chan (async/chan 1024)
   ;; This is the queue of emails that need to go out, which is the
   ;; INPUT to our service. We're never going to queue anything here
   ;; so we give it a simple chan to satisfy dependencies
   :outgoing-emails (component/using (map->Queue (assoc (:outgoing-emails config)
                                                        :outgoing-messages-chan (async/chan)))
                                     {:incoming-messages-chan :input-chan})
   :db (map->Database (:database config))
   :mailgun (map->EmailService (:mailgun config))
   :worker (component/using (map->Worker {:work-fn log-and-send-emails})
                            {:input-chan :input-chan
                             :db :db
                             :email-service :mailgun
                             :result-chan :result-chan})
   :sent-emails (component/using (map->Queue (assoc (:sent-emails config)
                                                    :incoming-messages-chan (async/chan))
                                             {:outgoing-messages-chan :result-chan}))))

Tips and Tricks

  • I’ve seen projects with code like this:
(defrecord SomeRecord
    [config]

  component/Lifecycle
  (start [this] this)
  (stop [this] this))

Component provides a no-op implementation of the Lifecycle protocol on all java objects. What that means is, if you need to provide an atom, map, or something else as a dependency, there’s no need to make a record for it.

Just put it in your system map as-is.

(component/system-map
   ;; Position Tracking
   :position (atom 0)
   :position-tracker (component/using (map->Worker {})
                                      {:position :position}))
  • If you’re wondering why I assoc a nil instead of using dissoc, it’s because dissoc returns a map and assoc keeps our record type.
(start [this]
    (assoc this
           :config nil
           :incoming-messages-chan nil
           :outgoing-messages-chan nil))

Resources

Master GitHub Actions with a Senior Infrastructure Engineer

As a senior staff infrastructure engineer, I share exclusive, behind-the-scenes insights that you won't find anywhere else. Get the strategies and techniques I've used to save companies $500k in CI costs and transform teams with GitOps best practices—delivered straight to your inbox.

Not sure yet? Check out the archive.

Unsubscribe at any time.