Prerequisites

You will need a working installation of Nix, the package manager. Instructions are here. In addition to Nix, I also make use of niv to pin nixpkgs and other Nix sources³. To install niv through Nix and make it available in your user environment, run:

nix-env -iA nixpkgs.niv

Pinning Nixpkgs

I start with a fresh Git repo, and then I use niv to pin down nixpkgs:

mkdir repo && cd repo
git init
niv init && niv update nixpkgs -b 22.11

What are nixpkgs anyway? nixpkgs is a collection of build formulas, called Nix derivations, that are used to build⁴ various packages. These packages can be anything, but are usually binary artifacts – like libraries and executables. I guess you can think of Nixpkgs in terms of a repository of software that can be “installed”, similar to Debian sources lists or RPM package repositories.

There are two salient points concerning Nix that give it advantage over standard package repos:

1. Every package built with Nix contains in its name a unique hash that depends on all of its build-time dependencies, all the way down to specific glibc version and all the way up to configure flags used to build that package. In an ideal world, if you build 123deadbeef-x-1.0 on your machine, and I build 123deadbeef-x-1.0 on my machine, these two packages are absolutely identical. The real world is messy, so it’s not exactly like that, but it’s pretty close.
2. Nix installs packages and all their dependencies to Nix store, an isolated part of filesystem. These packages are not in your PATH, or LD_LIBRARY_PATH, and they do not interfere with the rest of your system in any way. In fact, many different versions of the same package can coexist peacefully in Nix store.

Nixpkgs gets major releases every 6 months, and this is reflected in the version (Nix calls it channels): nixpkgs-22.11 was released in November 2022. There’s also nixpkgs-unstable, which regularly gets updates, doesn’t have a fixed release schedule, and generally contains the most recent versions of packages. It’s worth noting that stable Nixpkgs channels, such as 22.11, do get security patches after being released. Therefore, stable doesn’t mean immutable.

Different channels contain different versions of packages. For example, version of bash in 22.11 is pinned to 5.1-p16, whereas 21.05 had slightly more dated bash-5.1-p4. On the other hand, nixpks-unstable currently has bleeding edge bash-5.2-p15. You can use search.nixos.org to check this, albeit only for latest stable channel and nixpkgs-unstable. If you can’t find a package in stable channels, try unstable, or even master branch – someone may have added it. Alternatively, you can be a good open source citizen and write a Nix derivation for the thing you need yourself, and submit a pull request.

Good thing about niv is that it doesn’t really care about channels. Instead, it records commit hash of the tip of specified channel at time of niv init execution. This guarantees that Nix will always use the same version of Nixpkgs, regardless of which channel you reference⁵. Note that you can update pinned commit hash to latest tip, or even change the channel you are pinning with niv update nixpkgs -b <branch>.

All right, let’s get back to setting our project up. I will also add my nixfiles to Nix sources, so I can reference Nix expressions of custom setups of some of my favorite development tools:

niv add dimitrijer/nixfiles -b main # niv will pin this to tip of main branch

At this point, there are a few files in my project:

.
└── nix
    ├── sources.json
    └── sources.nix

2 directories, 2 files

niv keeps source of truth in sources.json, which contains commit hashes and SHA-256 checksums of all Nix sources. sources.nix is a Nix expressions that is used to reference Nix derivations specified in Nix sources that were pinned through niv.

Nix shell

Next, I create shell.nix file. This Nix expression is evaluated when you invoke nix-shell from your project’s root directory. shell.nix describes your development environment by listing all your tools and dependencies. I’ll start out small, and add things along the way as I need them:

let
  sources = import ./nix/sources.nix; # sources.nix was generated by niv
  pkgs = import sources.nixpkgs { };
  nixfiles = import sources.nixfiles { };
  neovim = nixfiles.neovim {
    pkgs = pkgs;
    withOCaml = true;
    withWriting = true;
  };
in
pkgs.mkShell
{
  # nix-shell evaluates shellHooks at start
  shellHooks = ''
    alias vim='nvim' 
  '';

  # all packages in development environment are listed here
  buildInputs = [
    pkgs.bash
  ];
}

If the syntax looks weird, don’t worry about it – I will point out important bits and pieces. This is also the only Nix file we’re going to be looking at, and the only changes to this file will be adding more packages. That’s it.

let part contains some prep work and establishes bindings that will be used below. I first import auto-generated sources.nix file, and then I reference two Nix sources that I added through niv: nixpkgs and nixfiles. Remember, both of these are already pinned down. Next, I invoke a Nix function from nixfiles that builds my customized Neovim derivation.

Now, the important part: I invoke pkgs.mkShell function, which sets up the development shell. This function is defined in nixpkgs. I provide two arguments to this function: shellHooks is a shell expression that is evaluated when nix-shell runs, and buildInputs lists all packages that Nix should provide in our development shell. For starters, I just need bash. I will add in custom neovim that I defined above later on.

Let’s try it out by running nix-shell --pure:

these 38 paths will be fetched (66.44 MiB download, 303.90 MiB unpacked):
  /nix/store/026hln0aq1hyshaxsdvhg0kmcm6yf45r-zlib-1.2.13
  /nix/store/039g378vc3pc3dvi9dzdlrd0i4q93qwf-binutils-2.39
  /nix/store/1d6ian3r8kdzspw8hacjhl3xkp40g1lj-binutils-wrapper-2.39
  /nix/store/1dgws25664p544znpc6f1nh9xmjf4ykc-pcre-8.45
  /nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0
  /nix/store/34xlpp3j3vy7ksn09zh44f1c04w77khf-libunistring-1.0
  /nix/store/38db4p333ibll7r1v151yc5f6ms1fr00-bash-interactive-5.2-p15
  (... omitted for brevity ...)
copying path '/nix/store/38db4p333ibll7r1v151yc5f6ms1fr00-bash-interactive-5.2-p15' from 'https://cache.nixos.org'...
copying path '/nix/store/34xlpp3j3vy7ksn09zh44f1c04w77khf-libunistring-1.0' from 'https://cache.nixos.org'...
copying path '/nix/store/i38jcxrwa4fxk2b7acxircpi399kyixw-linux-headers-6.0' from 'https://cache.nixos.org'...
copying path '/nix/store/5mh5019jigj0k14rdnjam1xwk5avn1id-libidn2-2.3.2' from 'https://cache.nixos.org'...
copying path '/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163' from 'https://cache.nixos.org'...
copying path '/nix/store/cr5fmwri3601s7724ayjvckhsg6cz4rv-attr-2.5.1' from 'https://cache.nixos.org'...
copying path '/nix/store/dsd5gz46hdbdk2rfdimqddhq6m8m8fqs-bash-5.1-p16' from 'https://cache.nixos.org'...
  (... omitted for brevity ...)
[nix-shell:/home/dimitrije/git/repo]$ echo $PATH | tr ':' '\n'
/nix/store/38db4p333ibll7r1v151yc5f6ms1fr00-bash-interactive-5.2-p15/bin
/nix/store/pr5n59mb4jzmfx6kanwxly0l07p861fg-patchelf-0.15.0/bin
/nix/store/dq0xwmsk1g0i2ayg6pb7y87na2knzylh-gcc-wrapper-11.3.0/bin
/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/bin
/nix/store/57xv61c5zi8pphjbcwxxjlgc34p61ic9-glibc-2.35-163-bin/bin
/nix/store/a7gvj343m05j2s32xcnwr35v31ynlypr-coreutils-9.1/bin
/nix/store/1d6ian3r8kdzspw8hacjhl3xkp40g1lj-binutils-wrapper-2.39/bin
/nix/store/039g378vc3pc3dvi9dzdlrd0i4q93qwf-binutils-2.39/bin
/nix/store/dsd5gz46hdbdk2rfdimqddhq6m8m8fqs-bash-5.1-p16/bin
/nix/store/a7gvj343m05j2s32xcnwr35v31ynlypr-coreutils-9.1/bin
/nix/store/mydc6f4k2z73xlcz7ilif3v2lcaiqvza-findutils-4.9.0/bin
/nix/store/j9p3g8472iijd50vhdprx0nmk2fqn5gv-diffutils-3.8/bin
/nix/store/89zs7rms6x00xfq4dq6m7mjnhkr8a6r4-gnused-4.8/bin
/nix/store/86bp03jkmsl6f92w0yzg4s59g5mhxwmy-gnugrep-3.7/bin
/nix/store/hwcdqw4jrjnd37wxqgsd47hd0j8bnj09-gawk-5.1.1/bin
/nix/store/cfbhw8r8ags41vwqaz47r583d0p4h4a1-gnutar-1.34/bin
/nix/store/p3m1ndl1lapwrlh698bnb5lvvxh67378-gzip-1.12/bin
/nix/store/a8mhcagrsly7c7mpjrpsnaahk4aax056-bzip2-1.0.8-bin/bin
/nix/store/mblgz65m3zv9x548a3d5m96fj2pbwr09-gnumake-4.3/bin
/nix/store/dsd5gz46hdbdk2rfdimqddhq6m8m8fqs-bash-5.1-p16/bin
/nix/store/v7ljksji50mg3w61dykaa3n3y79n6nil-patch-2.7.6/bin
/nix/store/zlcnmqq14jz5x9439jf937mvayyl63da-xz-5.2.7-bin/bin
/nix/store/y6aj732zm9m87c82fpvf103a1xb22blp-file-5.43/bin
[nix-shell:/home/dimitrije/git/repo]$

Nix first builds all specified packages and their dependencies. All of them are fetched from Nix binary cache and copied to local Nix store. Then Nix drops me in a brand new shell. man nix-shell can tell us what --pure does:

 • --pure
   If  this  flag is specified, the environment is almost entirely cleared before
   the interactive shell is started, so you get an environment that more closely
   corresponds to the “real” Nix build. A few variables, in particular HOME, USER
   and DISPLAY, are retained.

Indeed, you can see that PATH does not list the usual binary paths like /usr/bin, but only lists a few essential binaries that are provided in the development environment, including bash. If you omit --pure, Nix will modify PATH such that Nix binaries are first, but will not get rid of your existing PATH. This is useful, but it means that you might inadvertently use something that is outside of your development environment, so take heed.

You can get out of Nix shell with ^D, or just type exit.

Let’s add neovim to buildInputs:

diff --git a/shell.nix b/shell.nix
index 2a58852..7e47567 100644
--- a/shell.nix
+++ b/shell.nix
@@ -18,5 +18,6 @@ pkgs.mkShell
   # lists all packages in development environment
   buildInputs = [
     pkgs.bash
+    neovim
   ];
 }

(Yes, getting new stuff in your development environment is as simple as adding it to buildInputs). Running nix-shell --pure again pulls in more packages, and drops me in a shell where I have my customized nvim available:

[nix-shell:/home/dimitrije/git/repo]$ echo $PATH | tr ':' '\n' | grep neovim
/nix/store/31hxsrr6akz91nxaf4zzlgh6wwijqvd1-neovim-0.8.1/bin

I could have simply specified pkgs.neovim instead of neovim to use Nixpkgs derivation of Neovim, but my derivation contains plugins and other customizations. Note that I didn’t have to install Neovim through package manager, or otherwise mutate the state of my system to get it to run. The binary is not even visible outside of Nix shell, and someone would have to figure out its Nix store path in order to find it.

In general, you don’t really need to specify bash as build input. By default, Nix will use bash from your global Nixpkgs, or system-wide bash if there’s no Nixpkgs one. And there are ways to override your existing shell environment with nix-shell environment without having to drop into a new shell. For that, I suggest you check out Xe’s blog post above, specifically the part about lorri and direnv.

Let’s wrap this part up:

git commit -m "Initial commit."

OCaml project setup

Now let’s add some OCaml-specific packages:

• First and foremost, I need ocaml package, which includes native OCaml compiler ocamlopt and bytecode OCaml compiler ocamlc, among other things.
• I’m going to be using Dune as my build tool. Dune can build stuff, setup project directory structure, run tests, generate docs etc.
• findlib is necessary for Dune to be able to find libraries in Nix shell environment.
• ocaml already includes eponymously named top-level (REPL) binary, but utop is pretty much ubiquitous these days, and it’s much easier on the eye.
• odoc is documentation generator that plays nicely with Dune.
• I will use ocamlformat to autoformat my OCaml sources.
• Finally, I need LSP implementation for OCaml ocaml-lsp, so I can make use of Neovim LSP-powered goodies.

diff --git a/shell.nix b/shell.nix
index 7e47567..077ed4b 100644
--- a/shell.nix
+++ b/shell.nix
@@ -16,8 +16,13 @@ pkgs.mkShell
   '';

   # lists all packages in development environment
-  buildInputs = [
-    pkgs.bash
-    neovim
-  ];
+  buildInputs = with pkgs; [
+    bash
+    ocamlPackages.ocaml
+    ocamlPackages.dune_3
+    ocamlPackages.findlib
+    ocamlPackages.utop
+    ocamlPackages.odoc
+    ocamlPackages.ocaml-lsp
+    ocamlformat
+  ] ++ [ neovim ];
 }

I used with keyword to reduce repetition a bit. As you can notice, sometimes packages are not to be found in nixpkgs root, but are nested. I usually use search.nixos.org to figure out where to find expression for specific package. If that’s not yielding any results, try grepping the output of nix-env -qaP. At times I even had to fetch a local clone of nixpkgs and grep sources. Not ideal, I know ¯\_(ツ)_/¯.

Next thing I need to do is to setup a standard OCaml project directory structure with dune (from my development shell):

# Dune always creates a project directory, so I run it in parent directory, and
# pass in project dir as project name
(cd .. && dune init proj repo)

This will result in the following layout:

.
├── bin
│   ├── dune
│   └── main.ml
├── dune-project
├── lib
│   └── dune
├── nix
│   ├── sources.json
│   └── sources.nix
├── repo.opam
├── shell.nix
└── test
    ├── dune
    └── repo.ml

5 directories, 10 files

Stating the obvious here: source for binaries should end up in bin, libraries are in lib and tests should be in test, although dune runtest will also trigger inline and expect tests added through PPX rewriters. Dune stores outputs in _build, and it also automatically generates repo.opam based on contents of dune-project.

I should be able to build and execute the main binary now:

dune exec --display quiet repo

Which, unsurprisingly, yields:

Hello, World!

Next, I add _build and repo.opam to .gitignore. In addition, you would probably want to modify the generated dune-project file to specify project description, author, homepage, license etc. Time for another commit:

git commit -am "Set up Dune project structure."

Adding OCaml libraries

Many languages have tools that handle installation and management of multiple toolchains, and/or different versions of libraries on a single system in order to avoid what’s colloquially known as “dependency hell”. Haskell has Stack, OCaml has opam, Python has pyenv etc. One of the best things about Nix is that it solves this problem for any language. When we talked about pinning above, I mentioned that pinning Nixpkgs effectively nails down versions of all packages in Nixpkgs. In other words, all OCaml libraries are already pinned to predefined versions, along with their dependencies. This is also true for OCaml compiler – running ocamlc --version gives me 4.14.0, and this will be the same for anyone who recreates this development environment. Furthermore, this environment is local to this directory/project, meaning I can easily move between different projects that use different versions of OCaml compiler, libraries, tools etc. on the same system.

There is one nasty side to this global pinning mechanism: you need to use Nix overlays or other weird tricks if you want to use a different version of a library than what’s available in pinned Nixpkgs. The easiest thing for me oftentimes is to simply roll Nixpkgs forward to newer channel, or even nixpkgs-unstable, assuming it has the version I need. This is far from ideal, given that changing pinned Nixpkgs effectively changes all versions of all packages at once.

Anyway, let’s get back to my barebones OCaml project. For purpose of using external libraries, I stole the idea of a simple TCP echo server powered by Async library from Real World OCaml:

open Core
open Async

let command =
  let%map_open.Command port =
    flag "-port" (required int) ~doc:"port on which to listen"
  in
  fun () ->
    Echo.Server.create ~port (fun input ->
      match input with
      | `Ok "bye!" | `Eof -> `Disconnect
      | `Ok line -> `Ok (String.uppercase line))
    >>= Echo.Server.close_finished
;;

let () =
  Command.async ~summary:"A simple echo server that shouts back at you." command
  |> Command_unix.run
;;

The rest of the code is on Github. I use local Echo library within the same project, but I also refer to Core, Async and Command_unix in this code. In addition, I use some PPX rewriters to write monadic code more concisely. If I were to try and build this right now, Dune would complain about missing libraries. So what do we do about it? Add missing libraries to shell.nix, of course:

diff --git a/shell.nix b/shell.nix
index b3a940a..d797380 100644
--- a/shell.nix
+++ b/shell.nix
@@ -26,5 +26,10 @@ pkgs.mkShell
     ocamlPackages.odoc
     ocamlPackages.ocaml-lsp
     ocamlformat
+
+    ocamlPackages.janeStreet.base
+    ocamlPackages.janeStreet.async
+    ocamlPackages.janeStreet.core_unix
+    ocamlPackages.janeStreet.ppx_let
   ] ++ [ neovim ];
 }

Note that names of libraries that need to be added in dune files are sometimes different than this, or are more granular (e.g. core_unix.command_unix). You can list all OCaml libraries available to Dune in your development environment with ocamlfind list.

Now Dune will happily build and start the server:

dune exec echo -- -port 12345

Feel free to fire up a couple of instances of netcat so the server can holler back at you simultaneously.

Closing remarks

At this point, I have a fully functional local development environment that I can replicate on any machine running Linux and Nix. My usual approach to writing code is: open a new tmux session with one window and two panes, both inside nix-shell. I fire up Vim in one pane and dune build -w in the other. My editor supports autocomplete, hover, jumping to declaration, refactoring and other LSP-powered features. My code is autoformatted on save. I get compilation failures directly in the editor and in the Dune pane, which displays more verbose messages. In addition to that, versions of tools, packages and dependencies are implicitly pinned.

I feel like there are more benefits to be reaped with Nix and NixOS. I already mentioned declarative host management, but there’s also writing Nix derivations for your packages. This gets you very close to hermetic, reproducible builds (did I mention Nix builds packages in sandboxed environments on Linux?). Furthermore, Nix has facilities for building minimal Docker images containing the package and its dependencies, which opens up possibility of using Nix as a full-fledged build system for your production images.

OTOH there’s the steep learning curve. I feel like things are getting better, but the fact remains that you need to sink a fair amount of time learning Nix the language, Nix the package manager, NixOS the operating system and all their peculiarities.

And that’s why I think the approach I described here, or any approach that targets reproducible dev envs with Nix for that matter, is the sweet spot in terms of effort vs. gain. And it seems it’s not only me – projects like devenv.sh are popping up all over the place.

Hope this has been useful. Have something to say? Send me an email, I’d love to have a chat.