Published 3 Aug 2020, last modified 15 Sep 2020
After 15+ years of using various Linux-based operating systems (called distributions or “distros” in the jargon), I’ve finally found my favorite. It’s called NixOS.
At a very basic, technical level, most Linux distributions are pretty much the same. They share a common kernel (the core part of the operating system that runs and manages applications and provides them with a generic interface to the computer’s hardware), a common way of organizing files and command line interfaces based on the POSIX standard, and a common set of core system utilities, many of them originating in the GNU project that began in 1983. What really sets the distributions apart from each other from a user perspective is mostly:
A brief word on package managers for anyone unfamiliar with them: a package manager is a piece of software that installs, updates, and uninstalls software on the system, allowing the user to manage updates for everything from their image editor and web browser to the kernel itself from one place. Most Linux distributions have always had some kind of package manager, and there are several popular options, including the Debian project’s APT, Red Hat’s RPM, and Arch’s Pacman. A real centralized package manager like this is a relatively new concept in Microsoft Windows, but will be familiar to users of the Play Store or F-Droid on Android phones, or the App Store on iOS devices.
Package managers in desktop and server Linux distributions face an extra challenge that these popular mobile package managers don’t; Linux applications are frequently packaged and distributed separately from the libraries on which they depend. A library, in software, is a set of instructions for doing something that a lot of applications might need to do, like playing a sound file or displaying clickable buttons. The Linux kernel doesn’t contain instructions for playing sounds files or displaying clickable buttons; after all, a Linux web server doesn’t need to do those things. So the library allows an application to do these things in a consistent way on various machines without having to reinvent the metaphorical wheel. On Linux systems, these libraries are typically shared amongst different applications; one copy of a library can be used by a web browser, a media player, and an image editor. Doing things this way saves disk space and means that when when the library is upgraded, all the applications using it could benefit right away.
The problem with shared libraries is that updates to a library often require changes in the way that applications use them, and not every application is developed or maintained at the same place, so often package managers wind up in a situation where the latest versions of different applications require different versions of the same shared library. In the worst cases this can trap the user in a “dependency hell” where different applications cannot coexist because they require different versions of the shared libraries. Dependency hell has accelerated in recent years as the growth of Linux systems in corporate settings has driven rapid changes in common libraries to accommodate their security and “scalability” needs.
There are several approaches to preventing dependency hell. Systems like AppImage and Zero Install include copies of the libraries bundled up with the application itself so that the application has exactly the library versions it expects and doesn’t have to share. Increasingly popular options like Snap and Flatpak use a container approach that takes things a step further by also including various system files—pretty much a little chunk of the operating system, everything the application needs to run except the kernel itself. But all of these systems are meant to supplement, rather than replace, the systems primary package manager, to handle a few complex applications and not all the system software.
Nix is a relatively new package manager designed to solve the dependency hell problem with a more middle-of-the-road approach that can efficiently manage all system software while allowing multiple versions of shared libraries to coexist. While traditional system package managers keep applications all jumbled up together in the same few folders (for example, most of the programs themselves are in a folder called
/bin/, while lots of shared libraries are all kept together in
/usr/local/lib/), Nix instead keeps every individual package, whether it’s an application, a library, or a font, in its own folder identified by a cryptographic hash, a unique number mathematically derived from its contents, and uses symlinks (basically shortcuts between folders) to connect each package with the folders containing the exact dependency versions it was built to use. If the user upgrades a shared library but has an application that still requires the older version of it, the application simply remains connected to the older version, while applications that have been updated to take advantage of the upgraded library can use that, and each application is only aware of the presence of the exact version of the library it expects.
Packaging software for Nix is surprisingly easy compared to other software packaging approaches I’ve tried. It often requires the package maintainer to write just a single file, a “derivation” that typically describes how to download the source code for some software from the internet, compile it, and install the compiled files to the correct location. Nix derivations are written in the Nix programming language, a pure functional language designed just for defining software packages and system configurations. Like any pure functional language, Nix enforces a philosophy of reproducibility—that the same inputs (in this case, the same source files downloaded from the internet) should produce the same outputs (the same working, compiled piece of software) every time.
So one thing I love about NixOS is its package manager, the Nix package manager, and I’ve chosen to explain this first because (to refer back to my earlier list of three things that differentiate Linux distributions from each other) NixOS extends the same principles of reproducibility and efficient modularity to the default set of applications and the installation and system configuration experience.
When you prepare to download an install disk for NixOS, you don’t have the same array of options to choose from that you have with some other popular systems like Ubuntu or Fedora. With those distributions, you can choose from different Ubuntu “flavors” or Fedora “spins” with different sets of default packages included. Maybe you want one with an XFCE desktop that runs efficiently on older, slower graphics hardware, or maybe you want one that installs a bunch of multimedia creation tools by default, or maybe you want one designed for a server, that doesn’t come with any graphical tools at all. NixOS doesn’t come in “flavors” or “spins” like this, though of course you have a choice of install disks for different architectures, and you can choose between a smaller disk with just a basic set of command line tools and a larger one with a KDE desktop, Firefox browser, and the GParted graphical disk partition editor. You can still choose between different desktop environments and default applications, but you declare those choices during the installation process.
Here’s where I should note one big caveat about NixOS and the Nix package manager that will deter a lot of users: neither the NixOS installation process itself nor the Nix package manager has a graphical interface. Right now (as of August 2020) both are command-line-only tools, whereas the Ubuntu, Fedora, and Manjaro Linux distributions (for example) provide friendly graphical interfaces for these tasks. But the basic technical underpinnings of NixOS make its command-line installation experience very different from installing, say, Arch from the command line, or from the process I used to install Debian through a series of textual dialog boxes ca. 2005.
The part of the NixOS installation process where you’re most on your own is in partitioning your hard disk to allocate and format the space that will contain NixOS. This is something that I hope can eventually be smoothed out for beginners with a graphical tool that can automatically configure sensible defaults. As it currently stands, the NixOS Manual provides step-by-step instructions for this task that will probably be sufficient for users already familiar with the Bash command line in Linux environments, and the GParted disk partition editor provides a graphical interface for this step that will be self-explanatory to those who already understand how hard disks are usually partitioned to accommodate a Linux-based operating system. You can probably see how this limits the current audience for NixOS.
But we haven’t got to the good part yet. Next the NixOS manual provides command line instructions for mounting the freshly-partitioned disk (again, a technical step that could be automated away behind a friendly graphical installer) and a single command for generating a special configuration file called
/etc/nixos/configuration.nix. This file is where the real magic happens.
Today’s software engineering field is keen on configuration as code. If you have an important server running somewhere, say in a virtual machine (a sort of simulated computer which could be one of several running on one actual, physical computer) and doing something that other people rely on (e.g. hosting videos online, or securely transmitting patient records between medical institutions), you need a way to recover this server in the event that it abruptly disappears. Natural disasters could happen wherever your server is physically located, newly discovered security flaws could require you to start over with a patched version of your server’s operating system, or someone could just mistakenly delete an important virtual machine; it happens all the time! When that disaster comes, or if demand suddenly increases and you need more copies of that server to keep up, you need to have a predefined set of steps for setting up a new server that works just like the old one. The more traditional old way of doing it is to write these steps as an instruction manual for a human operator: press this button, then choose that menu option, then run the commands… This method is prone to being misunderstood, or becoming obsolete because the tools that the human operator uses have been changed. Even when it works, it takes time. If you want to create a bunch of temporary servers all at once to handle a surge in demand (something businesses these days often do) having a human operator follow an instruction manual to create each one is not a viable approach. A more robust approach is configuration as code, where all those manual steps are somehow replaced with code that can be saved somewhere and describes the configuration of the whole system in a computer-readable way, usually something understood by a special tool like Puppet, Chef, or Ansible that can use that code to reproduce that configuration in one go, taking care of all the individual steps that would otherwise have to be done manually, and ensuring that things turn out the same way every time.
The NixOS configuration file is configuration as code built right into the operating system itself. Like Nix package derivations, it’s written in the Nix language. This is where really all of the basic system configuration is stored. Settings that, in other Linux distributions, are spread across a bunch of different system tools and components, like GRUB (which loads the operating system when the computer is powered on), systemd (which starts and controls system and background processes), the display manager (which provides a graphical login screen and opens the user’s desktop), and the user/group management utilities, can all be described right in
/etc/nixos/configuration.nix. you can list all the applications you want installed by default here too—LibreOffice, VirtualBox with or without its Extension Pack, VSCodium, nginx, you name it. By default this file created with the right configuration to properly boot the operating system when you start your computer, and with comments showing the lines of code you would need to set up a few commonly desired features like printing, WiFi, and a graphical desktop (the KDE desktop). But all of this is optional, and online you can find examples for all kinds of different configurations you can mix and match to create something like the different “flavors” and “spins” of popular Linux distributions, fit to run anything from a small web server to an office-and-gaming desktop.
This level of customization isn’t new. Fifteen years ago I could achieve that through the seemingly endless dialog boxes of the Debian network installation. What is new is that it’s captured in this one file, which I can change at any time and even copy over to another machine if I ever have to replace the laptop I’m using. And once everything in the file is right all I have to do is run one command to tell NixOS to finish the system installation based on that file. If I’ve done something—if, say, I forgot a semicolon in the file where there should have been one—i just fix the file and run that command again; the NixOS installation command picks up where it left of and accommodates any changes I’ve made. Later, when I’m actually running the system I installed, if I want to change this configuration file again, I can apply the new changes—even something as drastic as replacing the whole desktop environment—with a single command. And if I don’t like the way it turned out, I can easily go back to an earlier version without even directly editing the file; old configurations are saved by default.
Maybe the magic of NixOS will be lost on most everyday computer users. I do a lot of hobbyist tinkering, and what I love about NixOS is that it makes that tinkering less tedious and fragile. I can have my weird, very specialized, very customized, exactly-right-for-me system on any new machine without having to put in all this time trying to get things to be the way they were before. And I can try new ways of using my Linux desktop without worrying about whether I’ll be able to get back to how it was before, if the new setup doesn’t work out. I like it so much that I’ve only been using ita couple weeks and already wrote this long article about it and submitted three Nix derivations to the NixOS community to update or add packages for some of my favorite niche programs.
There’s another factor that distinguishes Linux distributions from each other that I didn’t list before: community. This is why I hope NixOS will introduce graphical installation and package management tools that provide sensible defaults to smooth the learning experience for beginners, while still enabling the fully customizable configuration as code approach for tinkerers like me. Perhaps there could be a set of default configuration files for the beginner to choose from, like the flavors and spins of other distributions. Fostering beginner curiosity is key to creating free and open-source software communities that grow, adapt, and thrive.
One final note that didn’t fit in the rest of the article: the GNU project has created its own operating system inspired by NixOS, called Guix, which replaces the Nix language with a pre-existing GNU-created language called Guile and replaces the mainline Linux kernel with Linux-libre, a version with all proprietary device drivers removed. Much of what I have said about NixOS could apply equally well to Guix.
Tags: software development technical