Copyright © 2019 Ricardo Wurmus

Copyright © 2019 Efraim Flashner

Copyright © 2019 Pierre Neidhardt

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license is included in the section entitled ``GNU Free Documentation License''.

GNU Guix Cookbook
*****************

This document presents tutorials and detailed examples for GNU Guix, a functional package management tool written for the GNU system.  Please see GNU Guix reference manual for details about the system, its API, and related concepts.

If you would like to translate this document in your native language, consider joining the Translation Project (https://translationproject.org/domain/guix-cookbook.html).

Scheme tutorials
****************

GNU Guix is written in the general purpose programming language Scheme, and many of its features can be accessed and manipulated programmatically. You can use Scheme to generate package definitions, to modify them, to build them, to deploy whole operating systems, etc.

Knowing the basics of how to program in Scheme will unlock many of the advanced features Guix provides --- and you don't even need to be an experienced programmer to use them!

Let's get started!

A Scheme Crash Course
=====================

Guix uses the Guile implementation of Scheme.  To start playing with the language, install it with guix install guile and start a REPL (https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) by running guile from the command line.

Alternatively you can also run guix environment --ad-hoc guile -- guile if you'd rather not have Guile installed in your user profile.

In the following examples we use the > symbol to denote the REPL prompt, that is, the line reserved for user input.  See Using Guile Interactively (GNU Guile Reference Manual)) for more details on the REPL.

  * Scheme syntax boils down to a tree of expressions (or s-expression in Lisp lingo).  An expression can be a literal such as numbers and strings, or a compound which is a parenthesized list of compounds and literals.  #t and #f stand for the booleans "true" and "false", respectively.

     Examples of valid expressions:

          > "Hello World!"
          "Hello World!"
          > 17
          17
          > (display (string-append "Hello " "Guix" "\n"))
          "Hello Guix!"

  * This last example is a function call nested in another function call.  When a parenthesized expression is evaluated, the first term is the function and the rest are the arguments passed to the function.  Every function returns the last evaluated expression as its return value.

  * Anonymous functions are declared with the lambda term:

          > (lambda (x) (* x x))
          #<procedure 120e348 at <unknown port>:24:0 (x)>

     The above procedure returns the square of its argument.  Since everything is an expression, the lambda expression returns an anonymous procedure, which can in turn be applied to an argument:

          > ((lambda (x) (* x x)) 3)
          9

  * Anything can be assigned a global name with define:

          > (define a 3)
          > (define square (lambda (x) (* x x)))
          > (square a)
          9

  * Procedures can be defined more concisely with the following syntax:

          (define (square x) (* x x))

  * A list structure can be created with the list procedure:

          > (list 2 a 5 7)
          (2 3 5 7)

  * The quote disables evaluation of a parenthesized expression: the first term is not called over the other terms.  Thus it effectively returns a list of terms.

          > '(display (string-append "Hello " "Guix" "\n"))
          (display (string-append "Hello " "Guix" "\n"))
          > '(2 a 5 7)
          (2 a 5 7)

  * The quasiquote disables evaluation of a parenthesized expression until a comma re-enables it.  Thus it provides us with fine-grained control over what is evaluated and what is not.

          > `(2 a 5 7 (2 ,a 5 ,(+ a 4)))
          (2 a 5 7 (2 3 5 7))

     Note that the above result is a list of mixed elements: numbers, symbols (here a) and the last element is a list itself.

  * Multiple variables can be named locally with let:

          > (define x 10)
          > (let ((x 2)
                  (y 3))
              (list x y))
          (2 3)
          > x
          10
          > y
          ERROR: In procedure module-lookup: Unbound variable: y

     Use let* to allow later variable declarations to refer to earlier definitions.

          > (let* ((x 2)
                   (y (* x 3)))
              (list x y))
          (2 6)

  * The keyword syntax is #:; it is used to create unique identifiers. see Keywords (GNU Guile Reference Manual).

  * The percentage % is typically used for read-only global variables in the build stage.  Note that it is merely a convention, like _ in C. Scheme treats % exactly the same as any other letter.

  * Modules are created with define-module.  For instance

          (define-module (guix build-system ruby)
            #:use-module (guix store)
            #:export (ruby-build
                      ruby-build-system))

     defines the module guix build-system ruby which must be located in guix/build-system/ruby.scm somewhere in the Guile load path.  It depends on the (guix store) module and it exports two variables, ruby-build and ruby-build-system.

For a more detailed introduction, check out Scheme at a Glance (http://www.troubleshooters.com/codecorn/scheme_guile/hello.htm), by Steve Litt.

One of the reference Scheme books is the seminal ``Structure and Interpretation of Computer Programs'', by Harold Abelson and Gerald Jay Sussman, with Julie Sussman.  You'll find a free copy online (https://mitpress.mit.edu/sites/default/files/sicp/index.html), together with videos of the lectures by the authors (https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/).  The book is available in Texinfo format as the sicp Guix package.  Go ahead, run guix install sicp and start reading with info sicp (or with the Emacs Info reader). An unofficial ebook is also available (https://sarabander.github.io/sicp/).

You'll find more books, tutorials and other resources at https://schemers.org/.

Packaging
*********

This chapter is dedicated to teaching you how to add packages to the collection of packages that come with GNU Guix.  This involves writing package definitions in Guile Scheme, organizing them in package modules, and building them.

Packaging Tutorial
==================

GNU Guix stands out as the hackable package manager, mostly because it uses GNU Guile (https://www.gnu.org/software/guile/), a powerful high-level programming language, one of the Scheme (https://en.wikipedia.org/wiki/Scheme_%28programming_language%29) dialects from the Lisp family (https://en.wikipedia.org/wiki/Lisp_%28programming_language%29).

Package definitions are also written in Scheme, which empowers Guix in some very unique ways, unlike most other package managers that use shell scripts or simple languages.

  * Use functions, structures, macros and all of Scheme expressiveness for your package definitions.

  * Inheritance makes it easy to customize a package by inheriting from it and modifying only what is needed.

  * Batch processing: the whole package collection can be parsed, filtered and processed.  Building a headless server with all graphical interfaces stripped out?  It's possible.  Want to rebuild everything from source using specific compiler optimization flags?  Pass the #:make-flags "..." argument to the list of packages.  It wouldn't be a stretch to think Gentoo USE flags (https://wiki.gentoo.org/wiki/USE_flag) here, but this goes even further: the changes don't have to be thought out beforehand by the packager, they can be programmed by the user!

The following tutorial covers all the basics around package creation with Guix. It does not assume much knowledge of the Guix system nor of the Lisp language. The reader is only expected to be familiar with the command line and to have some basic programming knowledge.

A "Hello World" package
-----------------------

The “Defining Packages” section of the manual introduces the basics of Guix packaging (see Defining Packages (GNU Guix Reference Manual)).  In the following section, we will partly go over those basics again.

``GNU hello'' is a dummy project that serves as an idiomatic example for packaging.  It uses the GNU build system (./configure && make && make install).  Guix already provides a package definition which is a perfect example to start with.  You can look up its declaration with guix edit hello from the command line.  Let's see how it looks:

     (define-public hello
       (package
         (name "hello")
         (version "2.10")
         (source (origin
                   (method url-fetch)
                   (uri (string-append "mirror://gnu/hello/hello-" version
                                       ".tar.gz"))
                   (sha256
                    (base32
                     "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
         (build-system gnu-build-system)
         (synopsis "Hello, GNU world: An example GNU package")
         (description
          "GNU Hello prints the message \"Hello, world!\" and then exits.  It
     serves as an example of standard GNU coding practices.  As such, it supports
     command-line arguments, multiple languages, and so on.")
         (home-page "https://www.gnu.org/software/hello/")
         (license gpl3+)))

As you can see, most of it is rather straightforward.  But let's review the fields together:

name
     The project name.  Using Scheme conventions, we prefer to keep it lower case, without underscore and using dash-separated words.

source
     This field contains a description of the source code origin.  The origin record contains these fields:

      1. The method, here url-fetch to download via HTTP/FTP, but other methods exist, such as git-fetch for Git repositories.

      2. The URI, which is typically some https:// location for url-fetch.  Here the special `mirror://gnu` refers to a set of well known locations, all of which can be used by Guix to fetch the source, should some of them fail.

      3. The sha256 checksum of the requested file.  This is essential to ensure the source is not corrupted.  Note that Guix works with base32 strings, hence the call to the base32 function.

build-system

     This is where the power of abstraction provided by the Scheme language really shines: in this case, the gnu-build-system abstracts away the famous ./configure && make && make install shell invocations.  Other build systems include the trivial-build-system which does not do anything and requires from the packager to program all the build steps, the python-build-system, the emacs-build-system, and many more (see Build Systems (GNU Guix Reference Manual)).

synopsis
     It should be a concise summary of what the package does.  For many packages a tagline from the project's home page can be used as the synopsis.

description
     Same as for the synopsis, it's fine to re-use the project description from the homepage.  Note that Guix uses Texinfo syntax.

home-page
     Use HTTPS if available.

license
     See guix/licenses.scm in the project source for a full list of available licenses.

Time to build our first package!  Nothing fancy here for now: we will stick to a dummy "my-hello", a copy of the above declaration.

As with the ritualistic "Hello World" taught with most programming languages, this will possibly be the most "manual" approach.  We will work out an ideal setup later; for now we will go the simplest route.

Save the following to a file my-hello.scm.

     (use-modules (guix packages)
                  (guix download)
                  (guix build-system gnu)
                  (guix licenses))

     (package
       (name "my-hello")
       (version "2.10")
       (source (origin
                 (method url-fetch)
                 (uri (string-append "mirror://gnu/hello/hello-" version
                                     ".tar.gz"))
                 (sha256
                  (base32
                   "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
       (build-system gnu-build-system)
       (synopsis "Hello, Guix world: An example custom Guix package")
       (description
        "GNU Hello prints the message \"Hello, world!\" and then exits.  It
     serves as an example of standard GNU coding practices.  As such, it supports
     command-line arguments, multiple languages, and so on.")
       (home-page "https://www.gnu.org/software/hello/")
       (license gpl3+))

We will explain the extra code in a moment.

Feel free to play with the different values of the various fields.  If you change the source, you'll need to update the checksum.  Indeed, Guix refuses to build anything if the given checksum does not match the computed checksum of the source code.  To obtain the correct checksum of the package declaration, we need to download the source, compute the sha256 checksum and convert it to base32.

Thankfully, Guix can automate this task for us; all we need is to provide the URI:

     $ guix download mirror://gnu/hello/hello-2.10.tar.gz

     Starting download of /tmp/guix-file.JLYgL7
     From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
     following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
      …10.tar.gz  709KiB                                 2.5MiB/s 00:00 [##################] 100.0%
     /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
     0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i

In this specific case the output tells us which mirror was chosen. If the result of the above command is not the same as in the above snippet, update your my-hello declaration accordingly.

Note that GNU package tarballs come with an OpenPGP signature, so you should definitely check the signature of this tarball with `gpg` to authenticate it before going further:

     $ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig

     Starting download of /tmp/guix-file.03tFfb
     From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
     following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
      ….tar.gz.sig  819B                                                                                                                       1.2MiB/s 00:00 [##################] 100.0%
     /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
     0q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
     $ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
     gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
     gpg:                using RSA key A9553245FDE9B739
     gpg: Good signature from "Sami Kerola <kerolasa@iki.fi>" [unknown]
     gpg:                 aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@iki.fi>" [unknown]
     gpg: WARNING: This key is not certified with a trusted signature!
     gpg:          There is no indication that the signature belongs to the owner.
     Primary key fingerprint: 8ED3 96E3 7E38 D471 A005  30D3 A955 3245 FDE9 B739

You can then happily run

     $ guix package --install-from-file=my-hello.scm

You should now have my-hello in your profile!

     $ guix package --list-installed=my-hello
     my-hello	2.10	out
     /gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10

We've gone as far as we could without any knowledge of Scheme.  Before moving on to more complex packages, now is the right time to brush up on your Scheme knowledge.  see A Scheme Crash Course to get up to speed.

System Configuration
********************

Guix offers a flexible language for declaratively configuring your Guix System.  This flexibility can at times be overwhelming.  The purpose of this chapter is to demonstrate some advanced configuration concepts.

see System Configuration (GNU Guix Reference Manual) for a complete reference.

Customizing the Kernel
======================

Guix is, at its core, a source based distribution with substitutes (see Substitutes (GNU Guix Reference Manual)), and as such building packages from their source code is an expected part of regular package installations and upgrades.  Given this starting point, it makes sense that efforts are made to reduce the amount of time spent compiling packages, and recent changes and upgrades to the building and distribution of substitutes continues to be a topic of discussion within Guix.

The kernel, while not requiring an overabundance of RAM to build, does take a rather long time on an average machine.  The official kernel configuration, as is the case with many GNU/Linux distributions, errs on the side of inclusiveness, and this is really what causes the build to take such a long time when the kernel is built from source.

The Linux kernel, however, can also just be described as a regular old package, and as such can be customized just like any other package.  The procedure is a little bit different, although this is primarily due to the nature of how the package definition is written.

The linux-libre kernel package definition is actually a procedure which creates a package.

     (define* (make-linux-libre version hash supported-systems
                                #:key
                                ;; A function that takes an arch and a variant.
                                ;; See kernel-config for an example.
                                (extra-version #f)
                                (configuration-file #f)
                                (defconfig "defconfig")
                                (extra-options %default-extra-linux-options)
                                (patches (list %boot-logo-patch)))
       ...)

The current linux-libre package is for the 5.1.x series, and is declared like this:

     (define-public linux-libre
       (make-linux-libre %linux-libre-version
                         %linux-libre-hash
                         '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux")
                         #:patches %linux-libre-5.1-patches
                         #:configuration-file kernel-config))

Any keys which are not assigned values inherit their default value from the make-linux-libre definition.  When comparing the two snippets above, you may notice that the code comment in the first doesn't actually refer to the #:extra-version keyword; it is actually for #:configuration-file.  Because of this, it is not actually easy to include a custom kernel configuration from the definition, but don't worry, there are other ways to work with what we do have.

There are two ways to create a kernel with a custom kernel configuration.  The first is to provide a standard .config file during the build process by including an actual .config file as a native input to our custom kernel.  The following is a snippet from the custom 'configure phase of the make-linux-libre package definition:

     (let ((build  (assoc-ref %standard-phases 'build))
           (config (assoc-ref (or native-inputs inputs) "kconfig")))

       ;; Use a custom kernel configuration file or a default
       ;; configuration file.
       (if config
           (begin
             (copy-file config ".config")
             (chmod ".config" #o666))
           (invoke "make" ,defconfig))

Below is a sample kernel package.  The linux-libre package is nothing special and can be inherited from and have its fields overridden like any other package:

     (define-public linux-libre/E2140
       (package
         (inherit linux-libre)
         (native-inputs
          `(("kconfig" ,(local-file "E2140.config"))
           ,@(alist-delete "kconfig"
                           (package-native-inputs linux-libre))))))

In the same directory as the file defining linux-libre-E2140 is a file named E2140.config, which is an actual kernel configuration file.  The defconfig keyword of make-linux-libre is left blank here, so the only kernel configuration in the package is the one which was included in the native-inputs field.

The second way to create a custom kernel is to pass a new value to the extra-options keyword of the make-linux-libre procedure.  The extra-options keyword works with another function defined right below it:

     (define %default-extra-linux-options
       `(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
        ("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #t)
        ;; Modules required for initrd:
        ("CONFIG_NET_9P" . m)
        ("CONFIG_NET_9P_VIRTIO" . m)
        ("CONFIG_VIRTIO_BLK" . m)
        ("CONFIG_VIRTIO_NET" . m)
        ("CONFIG_VIRTIO_PCI" . m)
        ("CONFIG_VIRTIO_BALLOON" . m)
        ("CONFIG_VIRTIO_MMIO" . m)
        ("CONFIG_FUSE_FS" . m)
        ("CONFIG_CIFS" . m)
        ("CONFIG_9P_FS" . m)))

     (define (config->string options)
       (string-join (map (match-lambda
                           ((option . 'm)
                            (string-append option "=m"))
                           ((option . #t)
                            (string-append option "=y"))
                           ((option . #f)
                            (string-append option "=n")))
                         options)
                    "\n"))

And in the custom configure script from the `make-linux-libre` package:

     ;; Appending works even when the option wasn't in the
     ;; file.  The last one prevails if duplicated.
     (let ((port (open-file ".config" "a"))
           (extra-configuration ,(config->string extra-options)))
       (display extra-configuration port)
       (close-port port))

     (invoke "make" "oldconfig"))))

So by not providing a configuration-file the .config starts blank, and then we write into it the collection of flags that we want.  Here's another custom kernel:

     (define %macbook41-full-config
       (append %macbook41-config-options
               %filesystems
               %efi-support
               %emulation
               (@@ (gnu packages linux) %default-extra-linux-options)))

     (define-public linux-libre-macbook41
       ;; XXX: Access the internal 'make-linux-libre' procedure, which is
       ;; private and unexported, and is liable to change in the future.
       ((@@ (gnu packages linux) make-linux-libre) (@@ (gnu packages linux) %linux-libre-version)
                           (@@ (gnu packages linux) %linux-libre-hash)
                           '("x86_64-linux")
                           #:extra-version "macbook41"
                           #:patches (@@ (gnu packages linux) %linux-libre-5.1-patches)
                           #:extra-options %macbook41-config-options))

In the above example %filesystems is a collection of flags enabling different filesystem support, %efi-support enables EFI support and %emulation enables a x86_64-linux machine to act in 32-bit mode also. %default-extra-linux-options are the ones quoted above, which had to be added in since they were replaced in the extra-options keyword.

This all sounds like it should be doable, but how does one even know which modules are required for a particular system?  Two places that can be helpful in trying to answer this question is the Gentoo Handbook (https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel) and the documentation from the kernel itself (https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig).  From the kernel documentation, it seems that make localmodconfig is the command we want.

In order to actually run make localmodconfig we first need to get and unpack the kernel source code:

     tar xf $(guix build linux-libre --source)

Once inside the directory containing the source code run touch .config to create an initial, empty .config to start with.  make localmodconfig works by seeing what you already have in .config and letting you know what you're missing.  If the file is blank then you're missing everything.  The next step is to run:

     guix environment linux-libre -- make localmodconfig

and note the output.  Do note that the .config file is still empty. The output generally contains two types of warnings.  The first start with "WARNING" and can actually be ignored in our case.  The second read:

     module pcspkr did not have configs CONFIG_INPUT_PCSPKR

For each of these lines, copy the CONFIG_XXXX_XXXX portion into the .config in the directory, and append =m, so in the end it looks like this:

     CONFIG_INPUT_PCSPKR=m
     CONFIG_VIRTIO=m

After copying all the configuration options, run make localmodconfig again to make sure that you don't have any output starting with "module". After all of these machine specific modules there are a couple more left that are also needed.  CONFIG_MODULES is necessary so that you can build and load modules separately and not have everything built into the kernel. CONFIG_BLK_DEV_SD is required for reading from hard drives.  It is possible that there are other modules which you will need.

This post does not aim to be a guide to configuring your own kernel however, so if you do decide to build a custom kernel you'll have to seek out other guides to create a kernel which is just right for your needs.

The second way to setup the kernel configuration makes more use of Guix's features and allows you to share configuration segments between different kernels.  For example, all machines using EFI to boot have a number of EFI configuration flags that they need.  It is likely that all the kernels will share a list of filesystems to support.  By using variables it is easier to see at a glance what features are enabled and to make sure you don't have features in one kernel but missing in another.

Left undiscussed however, is Guix's initrd and its customization.  It is likely that you'll need to modify the initrd on a machine using a custom kernel, since certain modules which are expected to be built may not be available for inclusion into the initrd.

Acknowledgments
***************

Guix is based on the Nix package manager (https://nixos.org/nix/), which was designed and implemented by Eelco Dolstra, with contributions from other people (see the nix/AUTHORS file in Guix.)  Nix pioneered functional package management, and promoted unprecedented features, such as transactional package upgrades and rollbacks, per-user profiles, and referentially transparent build processes.  Without this work, Guix would not exist.

The Nix-based software distributions, Nixpkgs and NixOS, have also been an inspiration for Guix.

GNU Guix itself is a collective work with contributions from a number of people.  See the AUTHORS file in Guix for more information on these fine people.  The THANKS file lists people who have helped by reporting bugs, taking care of the infrastructure, providing artwork and themes, making suggestions, and more---thank you!

This document includes adapted sections from articles that have previously been published on the Guix blog at https://guix.gnu.org/blog.

GNU Free Documentation License
******************************

     [file fdl-1.3.texi not included]