How To Organize A Clojure Project And Its Dependencies

People coming from opinionated frameworks like Ruby on Rails often become disoriented in a Clojure project. Rails, being a framework, has the code already organized for you. You just had to learn where to put your models, views, and controllers.

Clojure doesn’t have a major opinionated framework like Ruby on Rails that forces you to follow its conventions.

A lot of beginners get confused and have questions like the following:

• How do you organize a Clojure project?
• What happens when you want to split a module from your application into a separate library?
• How do you work with your own libraries locally?

Let’s explore our options.

Project File Structure

---

Leiningen by default generates src and test directories for you. Clojure code can go in either directory and it’ll know where to find and load them.

You can customize this behavior and change it in project.clj by configuring the :source-paths and :test-paths vectors.

One useful thing I like to do is have a dev source directory only under my :dev leiningen profile. This folder would have repl utilities and maybe saved data for my tests.

Another use of :source-paths is when you add Clojurescript to your project and want to separate the code bases. You could make a src-cljs and test-cljs directory and then add those entries to your project.clj

Check out the handy dandy leiningen sample project.clj here.

Namespaces

---

At a namespace level, it’s preferable to have many small namespaces that do one specific thing and avoid having a single namespace that deals with multiple concerns.

Even with that in mind, it’s almost impossible to avoid this completely. Just about every Clojure project I’ve worked with end up with an “junk drawer” namespace named util that had a swiss army knife’s worth of functions in it.

Breaking Namespaces Up

An issue for people coming from object oriented languages is knowing when is it worth breaking something into a separate namespace. My advice is to just group functions logically by similar overall concerns. It’s probably not the answer you were looking for, but I feel people coming from OOP just need to know it’s okay to group functions arbitrarily.

Namespaces are “cheap” compared to objects so you shouldn’t be afraid to use them. Unlike OOP, you don’t have to manually re-balance an entire inheritance hierarchy tree if you end up needing to move functions around like you would with object methods. In Clojure, just move the function and refer to it in its new location.

Directory Structure

Another common question is on directory and file structure. How should you structure a project that does MVC and has models, views, controllers?

This question often comes from people that have worked with Rails and Django where there are best practices and conventions to your directory structure. In this situation, people would ask: Should I make a directory for models, views, and controllers? Or should I make a directory for each model, view, controller and group like that?

Similarly to the previous section on breaking up namespaces I’ll tell you it’s okay to organize your project in whatever way makes logical sense to you. You get to make the decision and choose the tradeoffs that make sense for you, without having a framework do it for you. There’s no established convention that a large group of people are using.

If you want to look for inspiration, look to how the Duct Framework organizes files.

Requires and Imports

Within a namespace, you should always require the functions you need or alias the namespace instead of fully qualifying the namspace/function.

Instead of clojure.set/intersection, require the set namespace and just do set/intersection.

Some of you may be wondering why I say to use require over use. Use has been deprecated and discouraged. It provided the exact same functionality as a require :refer :all and only confused new Clojure developers.

It’s also helpful to alphabetize your requires and imports if possible and if your IDE/editor supports it. It’ll make it easier for future readers to scan through quickly to see if something is already required.

Function order

Namespaces should flow well; the things further down are dependent on the things at top and just above it.

Public functions should be at the bottom.

Break your functions into logical sections and groupings within a namespace. For instance, keep all a web application’s handlers in one group separated by a comment block.

Managing State

---

For applications with state, I’m a big fan of Stuart Sierra’s component library over mount. It feels more idiomatic to me than mount because it doesn’t rely on global singletons. If you’re starting a new project from the ground up, it’s easy to integrate it into your application.

Component forces you to organize and architect your entire application to use it to its fullest potential.

What you end up doing is splitting your application into components that have internal state, for example, a web server or database component. Then you specify dependencies between them and it’ll figure out how to start, restart, and stop your application in the right order.

I have a tutorial to Component if you’d like to learn more.

Component also helps you make information that you feel should be a global singleton, like a database connection, into something that is just only accessible by the functions that need them.

Dependencies and Libraries

---

Often you would be developing an application and have a set of common functionality/utilities that would be useful in other projects at your organization.

Of course, the answer is the extract this set of functionality to its own library. The real question is how do you do this and how does it affect your workflow?

Extract The Functionality

---

Start by extracting the functionality you want and its associated tests into it’s own Clojure project.

You can install your project locally to your .m2 directory by running lein install while in the library’s directory. Then you can require this library like any other application by going to that application’s project.clj

Note: A common mistake I see developers make is constantly using lein install after making a change and restarting the repl of the project that depends on the library. There’s a better way with checkout dependencies. I talk about this below.

Setup a Remote Repository

Your library needs to be accessible somewhere in order for other developers to download it when they clone your project. If you have an open source library, then you can just deploy it to clojars.

However if you need the code to be private, I recommend using the s3-wagon-private plugin to create a remote repository to deploy your artifacts to.

The .m2 Folder

---

Let’s take a quick step back and talk about the .m2 directory.

Clojure puts a .m2 folder in your home directory that holds your dependencies (jar and pom files). This directory is where leiningen first looks for dependencies before downloading them.

The order leiningen looks for dependencies goes like this:

1. Local repository (.m2)
2. Remote repositories (defaults and based on remotes you’ve configured)
3. Optionally, if you have a checkouts directory in your project folder, leiningen will look for directories with project.clj files and load the code from there.

A lot of people seem learn about leiningen’s behavior for the local and remote repositories, but not about checkout dependencies.

Leiningen Checkout Dependencies

---

The key change to your workflow when you have a lot of libraries that you’d like to work on currently without running a lein install every time you make a change and restarting your REPL is to use checkout dependencies.

Essentially, checkout dependencies are a local, checked out version of the dependency from your project.

What you do is at the root level of your primary/application’s directory (the one with project.clj in it), you make a directory called checkouts. From there, you symlink the dependencies you have locally into this folder so that you end up with a checkouts/my-utils structure in which the my-utils directory contains its own project.clj.

Now you can start your REPL and work on your application and jump to the source of your checkout dependencies and edit, save, and reload the code into your REPL without ever stopping to lein install and reinstall your local changes.

Automate Library Deployment and Releases

---

On the master branch of the library you’re extracting, keep the version number suffixed with -SNAPSHOT. This really just indicates that the version of the library is latest from master.

You shouldn’t really touch the actual version number of the library in project.clj manually. Whenever you need to do a release and version bump, the lein release task will do it for you. You can find out more about lein release on the leiningen wiki.

The most ideal scenario is to have your continuous integration software do releases for you at the press of a button.

It turns out that if you don’t make releases of dependent libraries as quick and painless as possible, people are very hesitant to make trivial changes and instead make those changes within the application instead. I’ve seen this myself first hand and am in the process of creating Jenkins jobs for all my libraries to solve this problem.

If the default lein release task doesn’t suit you or your company’s workflow, you can customize it per project yourself. Read the docs I linked above to find out more.

---