When we first started work on Devbox, we relied on the “stable” nix commands, but the documentation was full of these experimental commands that were dependent on something called Flakes. Our customers would also reach out for ways of integrating custom packages into their projects. Upon investigating different approaches, we realized that Flakes provided a flexible solution.
But just what do Flakes do, and why were they introduced in the first place? Piecing this together from various sources took me some time. Hopefully, this first-principles approach helps you understand their motivation faster than I could.
First, we’ll need to learn about two Nix concepts and how they lead to two different problems that Flakes solve:
Nix derivations, which lead to composability problems
Nix channels, which lead to composability and package version pinning problems
Nix Derivations
To build a package with Nix, we start by defining a derivation. Think of a derivation as a build task, and we’ll gloss over the details for now.
Defining Derivations in default.nix
Most projects will define their derivations using a default.nix file, which Nix uses as the default instructions for building a package.
As an example, let’s look at the hello project’s default.nix here:
bash
{ stdenv, fetchurl }:
stdenv.mkDerivation rec {
name = "hello-2.10";
src = fetchurl {
url = "mirror://gnu/hello/${name}.tar.gz";
sha256 = "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i";
};
doCheck = true;
meta = {
description = "A program that produces a familiar, friendly greeting";
longDescription = ''
GNU Hello is a program that prints "Hello, world!" when you run it.
It is fully customizable.
'';
homepage = http://www.gnu.org/software/hello/manual/;
license = stdenv.lib.licenses.gpl3Plus;
maintainers = [ stdenv.lib.maintainers.eelco ];
platforms = stdenv.lib.platforms.all;
};
}To build this derivation, Nix will do:
nix-instantiate. This step translates a high-level nix expression into nix-store expressions.nix-store --realizewhere “realized” means “built”. It will build the store derivation and produce an output path like/nix/store/giwwxmq37cvfanwlhz6k38793qmhxq76-hello-2.12.1.
The problem is that default.nix allows too much freedom for users. It can produce a derivation, a set, a function or maybe something else. This harms composability, since the build outputs from the default.nix are not standardized or deterministic. Another nix derivation will not know exactly how to consume an arbitrary default.nix that some external package has defined.
Nix Channels and Version Pinning
The traditional way that users install Nix packages is through channels. Channels can be thought of as “the CI-tested branch of NixOS/Nixpkgs”.
Nix Channels exist to distribute a coherent set of nix packages. By coherent, I mean that every package that depends on openssl has been updated to depend on the specific version of openssl in that nix channel distribution. The packages in a channel are expected to work together.
But Nix Channels have a few problems:
Channels do not compose well. While one can have multiple channels active at the same time, doing so requires the user to download and evaluate the index of each channel they use. This can be tricky to manage, and often leads to poor performance and excessive disk usage. In other words, there isn’t an easy way to piecemeal update certain packages from the channel, while keeping the other packages stable.
Pinning a channel can lead to skew for packages that update frequently. For example, many users may choose to pin the
nixpkgsstable channel, but this is infrequently updated and so may miss out on updates from packages that are faster moving.
So, why Nix Flakes?
Summarizing from above, Flakes solve two critical problems:
Nix flakes define a schema with standardized inputs and outputs to solve the composability problem that Nix Derivations have.
Nix flakes also introduce pinning for all dependencies by having a lockfile to solve the problem that Nix Channels have.
Lets look at an example flake.nix file. This flake has a single input which is the latest nixpkgs . It also has two programs hello and cowsay which it exports as outputs.
bash
{
inputs = { nixpkgs.url = "github:nixos/nixpkgs"; };
outputs = { self, nixpkgs }:
let pkgs = nixpkgs.legacyPackages.x86_64-darwin;
in {
packages.x86_64-darwin.hello = pkgs.hello;
packages.x86_64-darwin.cowsay = pkgs.cowsay;
packages.x86_64-darwin.default = self.packages.x86_64-darwin.hello;
};
}To exercise this:
bash
> nix run .#hello
Hello, world!
> nix run .#cowsay -- flakes are neat
_________________
< flakes are neat >
-----------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
# oh why not:
> nix run .#cowsay -- $(nix run .#hello)
_______________
< Hello, world! >
---------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||The flake.nix above can be imported into another consumer flake.nix which wants to compose its functionality of hello and cowsay with its own custom logic.
Exercising this flake also generates a flake.lock . This pins the exact version of nixpkgs that will run everytime this flake is subsequently exercised. As a result, hello and cowsay will also resolve to the very same binaries every time.
bash
> cat flake.lock
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1686862348,
"narHash": "sha256-XcxE6PJ6OqtsC73J+D/N2qttW7x69zoKt9QbMQy3jSk=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "6d402731689fbeb403b03a0f486b8c8a26530d5a",
"type": "github"
},
"original": {
"owner": "nixos",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}If a flake.nix imports a package using that package's flake, then the package's flake metadata will get added to the flake.lock. This provides a path beyond Nix Channels to track packages that update more frequently.
There’s a lot more to Flakes, if you look under the hood. The awesome nix maintainers have exploited a lot of these properties for better performance, reproducibility, and composition. Thank you!
More Reading
There are several references on using Flakes out there. The references below are focussed on helping understand the motivation behind Flakes, Channels and related features of nix.
Flakes MVP: https://gist.github.com/edolstra/40da6e3a4d4ee8fd019395365e0772e7
https://www.reddit.com/r/Nix/comments/u1psl5/differences_between_channels_and_flakes/
https://github.com/NixOS/nixpkgs/issues/93327 ← user was very confused about what channels are and what pinning is, but posted a good summary at the bottom.
https://zimbatm.com/notes/summary-of-nix-flakes-vs-original-nix
Nix Flakes at Devbox
Flakes have been transformational to the internals of how Devbox operates. Some of our prior posts delve into this:
We have more changes in the pipeline to make Devbox the most delightful package manager to use, while preserving the important properties of Nix. Stay tuned!
Stay up to Date with Jetify
If you're reading this, we'd love to hear from you about how you've been using Devbox for your projects. You can follow us on Twitter, or chat with our developers live on our Discord Server. We also welcome issues and pull requests on our Github Repo.
