Our first derivation
Welcome to the sixth pill. In the previous fifth pill, we got acquainted with functions and imports. Functions and imports are very simple concepts that allow you to build complex abstractions and module composition to build a flexible Nix system.
In this post, we finally get to writing a derivation. Derivations, from a file system perspective, are the building blocks of the Nix system. The Nix language is used to describe derivations.
Let me remind you how to enter the Nix environment: source ~/.nix-profile/etc/profile.d/nix.sh.
(Address of this page on the original translation site).
“derivation” function
Used to create derivations built-in function `derivation“. Before moving further, please follow the link and check out what is written in the official guide. In Nix language terms, a derivation is just a collection with a few attributes, so you can store it in a variable and pass it to other functions like any other value.
This is where the real power comes in.
Function derivation takes a set as its first argument. It requires at least the following three attributes:
name: name of derivation. In Nix storage, derivations are stored in the format/hash-nameand partnameis taken from this attribute.system: name of the system in which the derivation can be collected. For example,x86_64-linux.builder: a program that assembles a derivation – a binary image or script.
First of all, we need to find out what, from Nix’s point of view, is the name of our system?
nix-repl> builtins.currentSystem
"x86_64-linux"
What happens if you use a non-existent system name?
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> d
«derivation /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv»
So, so, what is this? The script collected the derivation? No, he didn't collect it, he just created a .drv file. nix repl does not collect derivations unless you explicitly ask for it.
A digression about .drv files
What is a file .drv? This is a specification on how to assemble a derivation, without the unnecessary language noise that appears when working with any high-level languages, including Nix.
To understand this, we can draw several analogies with the C language:
Files
.nixsimilar to files.c.Files
.drvare intermediate files like files.o. File.drvdescribes how to collect a derivation. Here is the bare minimum of information.The final result of the assembly is the output path.
As you can see from the example, both the output path and the file path .drv lead to the storage room.
So inside the file .drv? It can be read, but it is better to print out its contents in an understandable form:
ℹ️ If your version of Nix does not have the command
nix derivation showuse it insteadnix show-derivation.
$ nix derivation show /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
{
"/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv": {
"outputs": {
"out": {
"path": "/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
}
},
"inputSrcs": [],
"inputDrvs": {},
"platform": "mysystem",
"builder": "mybuilder",
"args": [],
"env": {
"builder": "mybuilder",
"name": "myname",
"out": "/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname",
"system": "mysystem"
}
}
}
So, we see the output path here, but it's not on the disk yet. Even before the build begins, we know exactly where the files will be written. Why is this done?
Large derivatives such as Firefox take a long time to build. If Nix had to build them every time, even if we didn't want to run them but just test something, we would have to wait for the build to complete. Therefore, Nix calculates the path in advance, but does not write anything there.
Comment: The hash in the output path in the current version of Nix is calculated solely based on the input derivations, without taking into account the content. However, as we will see later, you can also create derivations content dependentsay, from archives .tar.
In file .drv Not all fields are filled in, so I will briefly describe their purpose:
Output paths (there may be several of them). By default, Nix creates one output path called “out”.
List of input derivations. It is empty because we have not referenced other derivations. Otherwise there would be a list of other files here
.drv.The name of the system (system) and the path to the collector (yes, now this is not a real program).
Finally, a list of environment variables passed to the build program.
This is the minimum information needed to assemble a derivation.
Important Note: The environment variables passed to the collector are not just the ones you see in the file .drv, but also some others related to the Nix configuration (number of cores, temporary directory, etc.). The builder will not inherit any variables from your shell, otherwise the builds would suffer from non-determinism.
But let's return to our educational derivation.
Let's try to collect it, despite the fact that it is almost unreal.
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> :b d
[...]
these derivations will be built:
/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
building path(s) `/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname'
error: a `mysystem' is required to build `/nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv', but I am a `x86_64-linux'
Team :b only available in nix repl, it is used to assemble the derivation. You can get command help by typing :?. On the screen we see the path to the file .drv, which describes how to collect derivations. Next it says which path will be our exit route. Finally, we see the error we should have seen: the derivation cannot be collected in our system.
We build using nix repl, but this is not the only way. You can implement file .drv using the command:
\$ nix-store -r /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv
The same will be printed on the screen.
Let's fix the `system“ attribute:
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = builtins.currentSystem; }
nix-repl> :b d
[...]
build error: invalid file name `mybuilder'
The build program is listed as mybuilder, which doesn't really exist. What will running such a file lead to? .drv?
What's in the derivation set?
First, let's take a look at the result of the function derivation. This is a simple set:
nix-repl> d = derivation { name = "myname"; builder = "mybuilder"; system = "mysystem"; }
nix-repl> builtins.isAttrs d
true
nix-repl> builtins.attrNames d
[ "all" "builder" "drvAttrs" "drvPath" "name" "out" "outPath" "outputName" "system" "type" ]
You can guess what does builtins.isAttrs: she returns trueif the argument is a set. Second function builtins.attrNames, returns a list of keys from the given set. You could say that this is a kind of reflection.
Let's start with drvAttrs:
nix-repl> d.drvAttrs
{ builder = "mybuilder"; name = "myname"; system = "mysystem"; }
Essentially this is the input we passed to the function derivation. Specifically d.name, d.system And d.builder — those attributes that we specified when calling.
nix-repl> (d == d.out)
true
And here we see that the attribute value out – this is a derivation, which seems strange. The reason is that a derivation can only have one output path.
For the same reason d.all is a singleton. Later we will learn how to create multiple output paths for one derivation.
d.drvPath – let's go to the file .drv: /nix/store/z3hhlxbckx4g3n9sw91nnvlkjvyw754p-myname.drv.
Something interesting about the attribute type. Its meaning "derivation". Nix adds a little magic when working with derivation type sets, but really not much. To figure it out, create a set with type “derivation”:
nix-repl> { type = "derivation"; }
«derivation ???»
Of course, there is no information in it, so Nix doesn't know what to print. As you understand, type = "derivation" – just an agreement for Nix and for us so that we understand that a set is a derivation.
When creating packages, we are interested in the end result – files written to disk.
All other metadata is needed by Nix to know how to determine the path to the file .drv and exit path out.
Attribute outPath is the output path to the Nix repository: /nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname.
Links to other derivations
Other package managers allow packages to reference each other. How to refer to other derivations in Nix, what paths to indicate? The attribute is used for this outPath, which contains the path to the files of the desired derivation. For convenience, Nix can convert a set of derivations into a string.
nix-repl> d.outPath
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
nix-repl> builtins.toString d
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
Nix does a “set to string conversion” if the set has an attribute outPath (it's something like a method toString in other languages):
nix-repl> builtins.toString { outPath = "foo"; }
"foo"
nix-repl> builtins.toString { a = "b"; }
error: cannot coerce a set to a string, at (string):1:1
Let's say we want to use binary programs from coreutils:
nix-repl> :l <nixpkgs>
Added 3950 variables.
nix-repl> coreutils
«derivation /nix/store/1zcs1y4n27lqs0gw4v038i303pb89rw6-coreutils-8.21.drv»
nix-repl> builtins.toString coreutils
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21"
Whatever nixpkgsimagine if we added variables to the scope and one of them is coreutils. This is a derivation of the package coreutils, which you may be familiar with from other Linux distributions, contains core programs for GNU/Linux systems. Your repository may have multiple derivations coreutils.
$ ls /nix/store/*coreutils*/bin
[...]
As a reminder, you can interpolate Nix expressions within strings using ${...}:
nix-repl> "${d}"
"/nix/store/40s0qmrfb45vlh6610rk29ym318dswdr-myname"
nix-repl> "${coreutils}"
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21"
This is very convenient because we can link to, say, a binary bin/true like this:
nix-repl> "${coreutils}/bin/true"
"/nix/store/8w4cbiy7wqvaqsnsnb3zvabq1cp2zhyz-coreutils-8.21/bin/true"
Almost working derivation
In the previous example we used a fake build program, mybuilder, which obviously doesn't exist. For experiments we can call the program bin/truewhich always exits with code 0 (success).
nix-repl> :l <nixpkgs>
nix-repl> d = derivation { name = "myname"; builder = "${coreutils}/bin/true"; system = builtins.currentSystem; }
nix-repl> :b d
[...]
builder for `/nix/store/qyfrcd53wmc0v22ymhhd5r6sz5xmdc8a-myname.drv' failed to produce output path `/nix/store/ly2k1vswbfmswr33hw0kf0ccilrpisnk-myname'
As we can see, the build program is running bin/truewhich does not create anything, neither a file nor a directory in the output path, it simply exits with code 0.
Obvious note: Every time we change the derivation, a new hash is calculated.
Let's check the new file .drvafter we have referenced another derivation:
$ nix derivation show /nix/store/qyfrcd53wmc0v22ymhhd5r6sz5xmdc8a-myname.drv
{
"/nix/store/qyfrcd53wmc0v22ymhhd5r6sz5xmdc8a-myname.drv": {
"outputs": {
"out": {
"path": "/nix/store/ly2k1vswbfmswr33hw0kf0ccilrpisnk-myname"
}
},
"inputSrcs": [],
"inputDrvs": {
"/nix/store/hixdnzz2wp75x1jy65cysq06yl74vx7q-coreutils-8.29.drv": [
"out"
]
},
"platform": "x86_64-linux",
"builder": "/nix/store/qrxs7sabhqcr3j9ai0j0cp58zfnny0jz-coreutils-8.29/bin/true",
"args": [],
"env": {
"builder": "/nix/store/qrxs7sabhqcr3j9ai0j0cp58zfnny0jz-coreutils-8.29/bin/true",
"name": "myname",
"out": "/nix/store/ly2k1vswbfmswr33hw0kf0ccilrpisnk-myname",
"system": "x86_64-linux"
}
}
}
Yeah! Nix added a dependency to our myname.drvThis coreutils.drv. Before we can build our derivation, Nix must build coreutils.drv. But, since coreutils is already in our storage, there is no need to build anything, so we just apply along the way /nix/store/qrxs7sabhqcr3j9ai0j0cp58zfnny0jz-coreutils-8.29.
When is derivation actually going to happen?
Nix does not build derivations in the process of calculation Nix expressions. In particular, this is why we must call “:b drv” in nix repl or use nix-store -r.
There is an important division in Nix:
Instantiation/Computation Time: Nix expressions are parsed, interpreted and turned into a derivation set. When calculating, you can reference other derivations because Nix will create the files
.drv, for which we will know their output paths. For this we use the command nix-instantiate. — Implementation/Building Time: derivation is collected from a file.drvhaving previously collected derivations from files.drv, on which it depends. For this we use the command nix-store -r.
The process is similar to compiling and linking in C/C++ projects. First you compile all the source files into object files. And then you link the object files into one executable file.
On Nix, first the expression (usually from a file .nix) compiles to .drvand then every .drv collected and written to storage along the output paths.
Conclusion
Is it difficult to create a package in Nix? No, it's not difficult.
We covered the basics of Nix derivations to understand how they work and how they are stored. Real packages in Nix tend to be simpler, but we had a learning curve, so we went through all the nooks and crannies. New information awaits us in the next pills.
Using the function derivation we describe the construction of the package and accept the finished file .drv as a result. Nix converts set to string when set has attribute outPath. Thanks to this convenient feature, it is easy to create entries between derivations.
When Nix collects a derivation, it first creates a file from the expression .drvand then uses this file to build the output files. It does this recursively for all dependencies and “executes” the files .drv one after another. Not much magic if you think about it.
In the next pill
…we'll eventually write our first working derivation. Yes, this post is also about “our first derivation,” but I never claimed that it would work.
