Copyright @copyright{} 2016 Alex ter Weele@*
Copyright @copyright{} 2016, 2017, 2018, 2019 Christopher Baines@*
Copyright @copyright{} 2017, 2018, 2019 Clément Lassieur@*
-Copyright @copyright{} 2017, 2018 Mathieu Othacehe@*
+Copyright @copyright{} 2017, 2018, 2020 Mathieu Othacehe@*
Copyright @copyright{} 2017 Federico Beffa@*
Copyright @copyright{} 2017, 2018 Carlo Zancanaro@*
Copyright @copyright{} 2017 Thomas Danckaert@*
Copyright @copyright{} 2020 Pierre Langlois@*
Copyright @copyright{} 2020 pinoaffe@*
Copyright @copyright{} 2020 André Batista@*
+Copyright @copyright{} 2020 Alexandru-Sergiu Marton@*
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
* Official Substitute Server:: One particular source of substitutes.
* Substitute Server Authorization:: How to enable or disable substitutes.
+* Getting Substitutes from Other Servers:: Substitute diversity.
* Substitute Authentication:: How Guix verifies substitutes.
* Proxy Settings:: How to get substitutes via proxy.
* Substitution Failure:: What happens when substitution fails.
* Using a Custom Guix Channel:: Using a customized Guix.
* Replicating Guix:: Running the @emph{exact same} Guix.
* Channel Authentication:: How Guix verifies what it fetches.
-* Primary URL:: Distinguishing mirror to original.
* Creating a Channel:: How to write your custom channel.
* Package Modules in a Sub-directory:: Specifying the channel's package modules location.
* Declaring Channel Dependencies:: How to depend on other channels.
* Specifying Channel Authorizations:: Defining channel authors authorizations.
+* Primary URL:: Distinguishing mirror to original.
* Writing Channel News:: Communicating information to channel's users.
Development
* Package Modules:: Packages from the programmer's viewpoint.
* Defining Packages:: Defining new packages.
+* Defining Package Variants:: Customizing packages.
* Build Systems:: Specifying how packages are built.
+* Build Phases:: Phases of the build process of a package.
+* Build Utilities:: Helpers for your package definitions and more.
* The Store:: Manipulating the package store.
* Derivations:: Low-level interface to package derivations.
* The Store Monad:: Purely functional interface to the store.
* G-Expressions:: Manipulating build expressions.
-* Invoking guix repl:: Programming Guix in Guile
+* Invoking guix repl:: Programming Guix in Guile.
Defining Packages
* Service Reference:: API reference.
* Shepherd Services:: A particular type of service.
+Installing Debugging Files
+
+* Separate Debug Info:: Installing 'debug' outputs.
+* Rebuilding Debug Info:: Building missing debug info.
+
Bootstrapping
* Reduced Binary Seed Bootstrap:: A Bootstrap worthy of GNU.
@table @code
@item x86_64-linux
-Intel/AMD @code{x86_64} architecture, Linux-Libre kernel;
+Intel/AMD @code{x86_64} architecture, Linux-Libre kernel.
@item i686-linux
-Intel 32-bit architecture (IA32), Linux-Libre kernel;
+Intel 32-bit architecture (IA32), Linux-Libre kernel.
@item armhf-linux
ARMv7-A architecture with hard float, Thumb-2 and NEON,
@item aarch64-linux
little-endian 64-bit ARMv8-A processors, Linux-Libre kernel.
+@item i586-gnu
+@uref{https://hurd.gnu.org, GNU/Hurd} on the Intel 32-bit architecture
+(IA32).
+
+This configuration is experimental and under development. The easiest
+way for you to give it a try is by setting up an instance of
+@code{hurd-vm-service-type} on your GNU/Linux machine
+(@pxref{transparent-emulation-qemu, @code{hurd-vm-service-type}}).
+@xref{Contributing}, on how to help!
+
@item mips64el-linux (deprecated)
little-endian 64-bit MIPS processors, specifically the Loongson series,
n32 ABI, and Linux-Libre kernel. This configuration is no longer fully
warnings were triggered by GNU@tie{}tar 1.26 and older; recent
versions are fine).
They stem from the fact that all the
-files in the archive have their modification time set to zero (which
+files in the archive have their modification time set to 1 (which
means January 1st, 1970). This is done on purpose to make sure the
archive content is independent of its creation time, thus making it
reproducible.
@item @uref{https://notabug.org/guile-lzlib/guile-lzlib, Guile-lzlib};
@item
@c FIXME: Specify a version number once a release has been made.
-@uref{https://gitlab.com/guile-git/guile-git, Guile-Git}, from August
-2017 or later;
+@uref{https://gitlab.com/guile-git/guile-git, Guile-Git}, version 0.3.0
+or later;
@item @uref{https://savannah.nongnu.org/projects/guile-json/, Guile-JSON}
4.3.0 or later;
@item @url{https://www.gnu.org/software/make/, GNU Make}.
offload it to one of the machines that satisfy the constraints of the
derivation, in particular its system types---e.g., @code{x86_64-linux}.
A single machine can have multiple system types, either because its
-architecture natively supports it, via emulation (@pxref{Transparent
-Emulation with QEMU}), or both. Missing prerequisites for the build are
+architecture natively supports it, via emulation
+(@pxref{transparent-emulation-qemu, Transparent Emulation with QEMU}),
+or both. Missing prerequisites for the build are
copied over SSH to the target machine, which then proceeds with the
build; upon success the output(s) of the build are copied back to the
-initial machine.
+initial machine. The offload facility comes with a basic scheduler that
+attempts to select the best machine. The best machine is chosen among
+the available machines based on criteria such as:
+
+@enumerate
+@item
+The availability of a build slot. A build machine can have as many
+build slots (connections) as the value of the @code{parallel-builds}
+field of its @code{build-machine} object.
+
+@item
+Its relative speed, as defined via the @code{speed} field of its
+@code{build-machine} object.
+
+@item
+Its load. The normalized machine load must be lower than a threshold
+value, configurable via the @code{overload-threshold} field of its
+@code{build-machine} object.
+
+@item
+Disk space availability. More than a 100 MiB must be available.
+@end enumerate
The @file{/etc/guix/machines.scm} file typically looks like this:
File name of the Unix-domain socket @command{guix-daemon} is listening
to on that machine.
+@item @code{overload-threshold} (default: @code{0.6})
+The load threshold above which a potential offload machine is
+disregarded by the offload scheduler. The value roughly translates to
+the total processor usage of the build machine, ranging from 0.0 (0%) to
+1.0 (100%). It can also be disabled by setting
+@code{overload-threshold} to @code{#f}.
+
@item @code{parallel-builds} (default: @code{1})
The number of builds that may run in parallel on the machine.
explicitly enable substitution @i{via} the @code{set-build-options}
remote procedure call (@pxref{The Store}).
-@item --substitute-urls=@var{urls}
@anchor{daemon-substitute-urls}
+@item --substitute-urls=@var{urls}
Consider @var{urls} the default whitespace-separated list of substitute
source URLs. When this option is omitted,
@indicateurl{https://@value{SUBSTITUTE-SERVER}} is used.
This means that substitutes may be downloaded from @var{urls}, as long
as they are signed by a trusted signature (@pxref{Substitutes}).
+@xref{Getting Substitutes from Other Servers}, for more information on
+how to configure the daemon to get substitutes from other servers.
+
@cindex offloading
@item --no-offload
Do not use offload builds to other machines (@pxref{Daemon Offload
system} command, specifically:
@example
-guix system disk-image --file-system-type=iso9660 \
- gnu/system/install.scm
+guix system disk-image -t iso9660 gnu/system/install.scm
@end example
Have a look at @file{gnu/system/install.scm} in the source tree,
you should regularly run @command{guix pull} (@pxref{Invoking guix
pull}).
+@cindex package transformations, upgrades
+When upgrading, package transformations that were originally applied
+when creating the profile are automatically re-applied (@pxref{Package
+Transformation Options}). For example, assume you first installed Emacs
+from the tip of its development branch with:
+
+@example
+guix install emacs-next --with-branch=emacs-next=master
+@end example
+
+Next time you run @command{guix upgrade}, Guix will again pull the tip
+of the Emacs development branch and build @code{emacs-next} from that
+checkout.
+
+Note that transformation options such as @option{--with-branch} and
+@option{--with-source} depend on external state; it is up to you to
+ensure that they work as expected. You can also discard a
+transformations that apply to a package by running:
+
+@example
+guix install @var{package}
+@end example
+
@item --do-not-upgrade[=@var{regexp} @dots{}]
When used together with the @option{--upgrade} option, do @emph{not}
upgrade any packages whose name matches a @var{regexp}. For example, to
@menu
* Official Substitute Server:: One particular source of substitutes.
* Substitute Server Authorization:: How to enable or disable substitutes.
+* Getting Substitutes from Other Servers:: Substitute diversity.
* Substitute Authentication:: How Guix verifies substitutes.
* Proxy Settings:: How to get substitutes via proxy.
* Substitution Failure:: What happens when substitution fails.
archive}). Doing so implies that you trust @code{@value{SUBSTITUTE-SERVER}} to not
be compromised and to serve genuine substitutes.
+@quotation Note
+If you are using Guix System, you can skip this section: Guix System
+authorizes substitutes from @code{@value{SUBSTITUTE-SERVER}} by default.
+@end quotation
+
The public key for @code{@value{SUBSTITUTE-SERVER}} is installed along with Guix, in
@code{@var{prefix}/share/guix/@value{SUBSTITUTE-SERVER}.pub}, where @var{prefix} is
the installation prefix of Guix. If you installed Guix from source,
@option{--no-substitutes} option to @command{guix package},
@command{guix build}, and other command-line tools.
+@node Getting Substitutes from Other Servers
+@subsection Getting Substitutes from Other Servers
+
+@cindex substitute servers, adding more
+Guix can look up and fetch substitutes from several servers. This is
+useful when you are using packages from additional channels for which
+the official server does not have substitutes but another server
+provides them. Another situation where this is useful is when you would
+prefer to download from your organization's substitute server, resorting
+to the official server only as a fallback or dismissing it altogether.
+
+You can give Guix a list of substitute server URLs and it will check
+them in the specified order. You also need to explicitly authorize the
+public keys of substitute servers to instruct Guix to accept the
+substitutes they sign.
+
+On Guix System, this is achieved by modifying the configuration of the
+@code{guix} service. Since the @code{guix} service is part of the
+default lists of services, @code{%base-services} and
+@code{%desktop-services}, you can use @code{modify-services} to change
+its configuration and add the URLs and substitute keys that you want
+(@pxref{Service Reference, @code{modify-services}}).
+
+As an example, suppose you want to fetch substitutes from
+@code{guix.example.org} and to authorize the signing key of that server,
+in addition to the default @code{@value{SUBSTITUTE-SERVER}}. The
+resulting operating system configuration will look something like:
+
+@lisp
+(operating-system
+ ;; @dots{}
+ (services
+ ;; Assume we're starting from '%desktop-services'. Replace it
+ ;; with the list of services you're actually using.
+ (modify-services %desktop-services
+ (guix-service-type config =>
+ (guix-configuration
+ (inherit config)
+ (substitute-urls
+ (append (list "https://guix.example.org")
+ %default-substitute-urls))
+ (authorized-keys
+ (append (list (local-file "./key.pub"))
+ %default-authorized-guix-keys)))))))
+@end lisp
+
+This assumes that the file @file{key.pub} contains the signing key of
+@code{guix.example.org}. With this change in place in your operating
+system configuration file (say @file{/etc/config.scm}), you can
+reconfigure and restart the @code{guix-daemon} service or reboot so the
+changes take effect:
+
+@example
+$ sudo guix system reconfigure /etc/config.scm
+$ sudo herd restart guix-daemon
+@end example
+
+If you're running Guix on a ``foreign distro'', you would instead take
+the following steps to get substitutes from additional servers:
+
+@enumerate
+@item
+Edit the service configuration file for @code{guix-daemon}; when using
+systemd, this is normally
+@file{/etc/systemd/system/guix-daemon.service}. Add the
+@option{--substitute-urls} option on the @command{guix-daemon} command
+line and list the URLs of interest (@pxref{daemon-substitute-urls,
+@code{guix-daemon --substitute-urls}}):
+
+@example
+@dots{} --substitute-urls='https://guix.example.org https://@value{SUBSTITUTE-SERVER}'
+@end example
+
+@item
+Restart the daemon. For systemd, it goes like this:
+
+@example
+systemctl daemon-reload
+systemctl restart guix-daemon.service
+@end example
+
+@item
+Authorize the key of the new server (@pxref{Invoking guix archive}):
+
+@example
+guix archive --authorize < key.pub
+@end example
+
+Again this assumes @file{key.pub} contains the public key that
+@code{guix.example.org} uses to sign substitutes.
+@end enumerate
+
+Now you're all set! Substitutes will be preferably taken from
+@code{https://guix.example.org}, using @code{@value{SUBSTITUTE-SERVER}}
+as a fallback. Of course you can list as many substitute servers as you
+like, with the caveat that substitute lookup can be slowed down if too
+many servers need to be contacted.
+
+Note that there are also situations where one may want to add the URL of
+a substitute server @emph{without} authorizing its key.
+@xref{Substitute Authentication}, to understand this fine point.
+
@node Substitute Authentication
@subsection Substitute Authentication
@end table
As for @command{guix pull}, the absence of any options means that the
-the latest commit on the master branch will be used. The command
+latest commit on the master branch will be used. The command
@example
guix time-machine -- build hello
@item --generate-key[=@var{parameters}]
@cindex signing, archives
Generate a new key pair for the daemon. This is a prerequisite before
-archives can be exported with @option{--export}. Note that this
-operation usually takes time, because it needs to gather enough entropy
-to generate the key pair.
+archives can be exported with @option{--export}. This
+operation is usually instantaneous but it can take time if the system's
+entropy pool needs to be refilled. On Guix System,
+@code{guix-service-type} takes care of generating this key pair the
+first boot.
The generated key pair is typically stored under @file{/etc/guix}, in
@file{signing-key.pub} (public key) and @file{signing-key.sec} (private
* Using a Custom Guix Channel:: Using a customized Guix.
* Replicating Guix:: Running the @emph{exact same} Guix.
* Channel Authentication:: How Guix verifies what it fetches.
-* Primary URL:: Distinguishing mirror to original.
* Creating a Channel:: How to write your custom channel.
* Package Modules in a Sub-directory:: Specifying the channel's package modules location.
* Declaring Channel Dependencies:: How to depend on other channels.
* Specifying Channel Authorizations:: Defining channel authors authorizations.
+* Primary URL:: Distinguishing mirror to original.
* Writing Channel News:: Communicating information to channel's users.
@end menu
If you're curious about the authentication mechanics, read on!
-@cindex primary URL, channels
-@node Primary URL
-@section Primary URL
-
-Channel authors can indicate the primary URL of their channel's Git
-repository in the @file{.guix-channel} file, like so:
-
-@lisp
-(channel
- (version 0)
- (url "https://example.org/guix.git"))
-@end lisp
-
-This allows @command{guix pull} to determine whether it is pulling code
-from a mirror of the channel; when that is the case, it warns the user
-that the mirror might be stale and displays the primary URL. That way,
-users cannot be tricked into fetching code from a stale mirror that does
-not receive security updates.
-
-This feature only makes sense for authenticated repositories, such as
-the official @code{guix} channel, for which @command{guix pull} ensures
-the code it fetches is authentic.
-
-@cindex personal packages (channels)
-@cindex channels, for personal packages
@node Creating a Channel
@section Creating a Channel
+@cindex personal packages (channels)
+@cindex channels, for personal packages
Let's say you have a bunch of custom package variants or personal packages
that you think would make little sense to contribute to the Guix project, but
would like to have these packages transparently available to you at the
on how to do it.
-@cindex subdirectory, channels
@node Package Modules in a Sub-directory
@section Package Modules in a Sub-directory
+@cindex subdirectory, channels
As a channel author, you may want to keep your channel modules in a
sub-directory. If your modules are in the sub-directory @file{guix}, you must
add a meta-data file @file{.guix-channel} that contains:
(directory "guix"))
@end lisp
-@cindex dependencies, channels
-@cindex meta-data, channels
@node Declaring Channel Dependencies
@section Declaring Channel Dependencies
+@cindex dependencies, channels
+@cindex meta-data, channels
Channel authors may decide to augment a package collection provided by other
channels. They can declare their channel to be dependent on other channels in
a meta-data file @file{.guix-channel}, which is to be placed in the root of
on channels that you don't control, and you should aim to keep the number of
dependencies to a minimum.
-@cindex channel authorizations
@node Specifying Channel Authorizations
@section Specifying Channel Authorizations
+@cindex channel authorizations
@anchor{channel-authorizations}
As we saw above, Guix ensures the source code it pulls from channels
comes from authorized developers. As a channel author, you need to
are considered authentic if and only if they are signed by a key present
in the @file{.guix-authorizations} file of @emph{both} branches.
-@cindex news, for channels
+@node Primary URL
+@section Primary URL
+
+@cindex primary URL, channels
+Channel authors can indicate the primary URL of their channel's Git
+repository in the @file{.guix-channel} file, like so:
+
+@lisp
+(channel
+ (version 0)
+ (url "https://example.org/guix.git"))
+@end lisp
+
+This allows @command{guix pull} to determine whether it is pulling code
+from a mirror of the channel; when that is the case, it warns the user
+that the mirror might be stale and displays the primary URL. That way,
+users cannot be tricked into fetching code from a stale mirror that does
+not receive security updates.
+
+This feature only makes sense for authenticated repositories, such as
+the official @code{guix} channel, for which @command{guix pull} ensures
+the code it fetches is authentic.
+
@node Writing Channel News
@section Writing Channel News
+@cindex news, for channels
Channel authors may occasionally want to communicate to their users
information about important changes in the channel. You'd send them all
an email, but that's not convenient.
applications such as a web browser. To run Eolie, we must expose and
share some files and directories; we include @code{nss-certs} and expose
@file{/etc/ssl/certs/} for HTTPS authentication; finally we preserve the
-the @env{DISPLAY} environment variable since containerized graphical
+@env{DISPLAY} environment variable since containerized graphical
applications won't display without it.
@example
@vindex GUIX_EXECUTION_ENGINE
When running a wrapped program, you can explicitly request one of the
execution engines listed above by setting the
-@code{GUIX_EXECUTION_ENGINE} environment variable accordingly.
+@env{GUIX_EXECUTION_ENGINE} environment variable accordingly.
@end quotation
@cindex entry point, for Docker images
@menu
* Package Modules:: Packages from the programmer's viewpoint.
* Defining Packages:: Defining new packages.
+* Defining Package Variants:: Customizing packages.
* Build Systems:: Specifying how packages are built.
+* Build Phases:: Phases of the build process of a package.
+* Build Utilities:: Helpers for your package definitions and more.
* The Store:: Manipulating the package store.
* Derivations:: Low-level interface to package derivations.
* The Store Monad:: Purely functional interface to the store.
the package you are interested in from another repository, using the
@code{guix import} command (@pxref{Invoking guix import}).
-In the example above, @var{hello} is defined in a module of its own,
+In the example above, @code{hello} is defined in a module of its own,
@code{(gnu packages hello)}. Technically, this is not strictly
necessary, but it is convenient to do so: all the packages defined in
modules under @code{(gnu packages @dots{})} are automatically known to
@item
@cindex GNU Build System
The @code{build-system} field specifies the procedure to build the
-package (@pxref{Build Systems}). Here, @var{gnu-build-system}
+package (@pxref{Build Systems}). Here, @code{gnu-build-system}
represents the familiar GNU Build System, where packages may be
configured, built, and installed with the usual @code{./configure &&
make && make check && make install} command sequence.
+When you start packaging non-trivial software, you may need tools to
+manipulate those build phases, manipulate files, and so on. @xref{Build
+Utilities}, for more on this.
+
@item
The @code{arguments} field specifies options for the build system
(@pxref{Build Systems}). Here it is interpreted by
-@var{gnu-build-system} as a request run @file{configure} with the
+@code{gnu-build-system} as a request run @file{configure} with the
@option{--enable-silent-rules} flag.
@cindex quote
@item
The @code{inputs} field specifies inputs to the build process---i.e.,
build-time or run-time dependencies of the package. Here, we define an
-input called @code{"gawk"} whose value is that of the @var{gawk}
-variable; @
var{gawk} is itself bound to a @code{<package>} object.
+input called @code{"gawk"} whose value is that of the @code{gawk}
+variable; @
code{gawk} is itself bound to a @code{<package>} object.
@cindex backquote (quasiquote)
@findex `
Reference Manual}).
Note that GCC, Coreutils, Bash, and other essential tools do not need to
-be specified as inputs here. Instead, @var{gnu-build-system} takes care
+be specified as inputs here. Instead, @code{gnu-build-system} takes care
of ensuring that they are present (@pxref{Build Systems}).
However, any other dependencies need to be specified in the
(@pxref{Specifying Target Triplets,,, autoconf, Autoconf}).
@end deffn
-@cindex package transformations
-@cindex input rewriting
-@cindex dependency tree rewriting
-Packages can be manipulated in arbitrary ways. An example of a useful
-transformation is @dfn{input rewriting}, whereby the dependency tree of
-a package is rewritten by replacing specific inputs by others:
-
-@deffn {Scheme Procedure} package-input-rewriting @var{replacements} @
- [@var{rewrite-name}] [#:deep? #t]
-Return a procedure that, when passed a package, replaces its direct and
-indirect dependencies, including implicit inputs when @var{deep?} is
-true, according to @var{replacements}. @var{replacements} is a list of
-package pairs; the first element of each pair is the package to replace,
-and the second one is the replacement.
-
-Optionally, @var{rewrite-name} is a one-argument procedure that takes
-the name of a package and returns its new name after rewrite.
-@end deffn
-
-@noindent
-Consider this example:
-
-@lisp
-(define libressl-instead-of-openssl
- ;; This is a procedure to replace OPENSSL by LIBRESSL,
- ;; recursively.
- (package-input-rewriting `((,openssl . ,libressl))))
-
-(define git-with-libressl
- (libressl-instead-of-openssl git))
-@end lisp
-
-@noindent
-Here we first define a rewriting procedure that replaces @var{openssl}
-with @var{libressl}. Then we use it to define a @dfn{variant} of the
-@var{git} package that uses @var{libressl} instead of @var{openssl}.
-This is exactly what the @option{--with-input} command-line option does
-(@pxref{Package Transformation Options, @option{--with-input}}).
-
-The following variant of @code{package-input-rewriting} can match packages to
-be replaced by name rather than by identity.
-
-@deffn {Scheme Procedure} package-input-rewriting/spec @var{replacements} [#:deep? #t]
-Return a procedure that, given a package, applies the given
-@var{replacements} to all the package graph, including implicit inputs
-unless @var{deep?} is false. @var{replacements} is a list of
-spec/procedures pair; each spec is a package specification such as
-@code{"gcc"} or @code{"guile@@2"}, and each procedure takes a matching
-package and returns a replacement for that package.
-@end deffn
-
-The example above could be rewritten this way:
-
-@lisp
-(define libressl-instead-of-openssl
- ;; Replace all the packages called "openssl" with LibreSSL.
- (package-input-rewriting/spec `(("openssl" . ,(const libressl)))))
-@end lisp
-
-The key difference here is that, this time, packages are matched by spec and
-not by identity. In other words, any package in the graph that is called
-@code{openssl} will be replaced.
-
-A more generic procedure to rewrite a package dependency graph is
-@code{package-mapping}: it supports arbitrary changes to nodes in the
-graph.
-
-@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] [#:deep? #f]
-Return a procedure that, given a package, applies @var{proc} to all the packages
-depended on and returns the resulting package. The procedure stops recursion
-when @var{cut?} returns true for a given package. When @var{deep?} is true, @var{proc} is
-applied to implicit inputs as well.
-@end deffn
+Once you have package definitions, you can easily define @emph{variants}
+of those packages. @xref{Defining Package Variants}, for more on that.
@menu
* package Reference:: The package data type.
It is an error to refer to @code{this-package} outside a package definition.
@end deffn
+Because packages are regular Scheme objects that capture a complete
+dependency graph and associated build procedures, it is often useful to
+write procedures that take a package and return a modified version
+thereof according to some parameters. Below are a few examples.
+
+@cindex tool chain, choosing a package's tool chain
+@deffn {Scheme Procedure} package-with-c-toolchain @var{package} @var{toolchain}
+Return a variant of @var{package} that uses @var{toolchain} instead of
+the default GNU C/C++ toolchain. @var{toolchain} must be a list of
+inputs (label/package tuples) providing equivalent functionality, such
+as the @code{gcc-toolchain} package.
+
+The example below returns a variant of the @code{hello} package built
+with GCC@tie{}10.x and the rest of the GNU tool chain (Binutils and the
+GNU C Library) instead of the default tool chain:
+
+@lisp
+(let ((toolchain (specification->package "gcc-toolchain@@10")))
+ (package-with-c-toolchain hello `(("toolchain" ,toolchain))))
+@end lisp
+
+The build tool chain is part of the @dfn{implicit inputs} of
+packages---it's usually not listed as part of the various ``inputs''
+fields and is instead pulled in by the build system. Consequently, this
+procedure works by changing the build system of @var{package} so that it
+pulls in @var{toolchain} instead of the defaults. @ref{Build Systems},
+for more on build systems.
+@end deffn
+
@node origin Reference
@subsection @code{origin} Reference
-This section summarizes all the options available in @code{origin}
-declarations (@pxref{Defining Packages}).
+This section documents @dfn{origins}. An @code{origin} declaration
+specifies data that must be ``produced''---downloaded, usually---and
+whose content hash is known in advance. Origins are primarily used to
+represent the source code of packages (@pxref{Defining Packages}). For
+that reason, the @code{origin} form allows you to declare patches to
+apply to the original source code as well as code snippets to modify it.
@deftp {Data Type} origin
This is the data type representing a source code origin.
@var{url-fetch} method of @code{(guix download)}, the valid @code{uri}
values are: a URL represented as a string, or a list thereof.
+@cindex fixed-output derivations, for download
@item @code{method}
-A procedure that handles the URI.
-
-Examples include:
-
-@table @asis
-@item @var{url-fetch} from @code{(guix download)}
-download a file from the HTTP, HTTPS, or FTP URL specified in the
-@code{uri} field;
-
-@vindex git-fetch
-@item @var{git-fetch} from @code{(guix git-download)}
-clone the Git version control repository, and check out the revision
-specified in the @code{uri} field as a @code{git-reference} object; a
-@code{git-reference} looks like this:
+A monadic procedure that handles the given URI. The procedure must
+accept at least three arguments: the value of the @code{uri} field and
+the hash algorithm and hash value specified by the @code{hash} field.
+It must return a store item or a derivation in the store monad
+(@pxref{The Store Monad}); most methods return a fixed-output derivation
+(@pxref{Derivations}).
-@lisp
-(git-reference
- (url "https://git.savannah.gnu.org/git/hello.git")
- (commit "v2.10"))
-@end lisp
-@end table
+Commonly used methods include @code{url-fetch}, which fetches data from
+a URL, and @code{git-fetch}, which fetches data from a Git repository
+(see below).
@item @code{sha256}
A bytevector containing the SHA-256 hash of the source. This is
depend on the value of @code{%current-system} or
@code{%current-target-system}.
-@item @code{snippet} (default: @code{#f})
-A G-expression (@pxref{G-Expressions}) or S-expression that will be run
-in the source directory. This is a convenient way to modify the source,
-sometimes more convenient than a patch.
+@item @code{snippet} (default: @code{#f})
+A G-expression (@pxref{G-Expressions}) or S-expression that will be run
+in the source directory. This is a convenient way to modify the source,
+sometimes more convenient than a patch.
+
+@item @code{patch-flags} (default: @code{'("-p1")})
+A list of command-line flags that should be passed to the @code{patch}
+command.
+
+@item @code{patch-inputs} (default: @code{#f})
+Input packages or derivations to the patching process. When this is
+@code{#f}, the usual set of inputs necessary for patching are provided,
+such as GNU@tie{}Patch.
+
+@item @code{modules} (default: @code{'()})
+A list of Guile modules that should be loaded during the patching
+process and while running the code in the @code{snippet} field.
+
+@item @code{patch-guile} (default: @code{#f})
+The Guile package that should be used in the patching process. When
+this is @code{#f}, a sensible default is used.
+@end table
+@end deftp
+
+@deftp {Data Type} content-hash @var{value} [@var{algorithm}]
+Construct a content hash object for the given @var{algorithm}, and with
+@var{value} as its hash value. When @var{algorithm} is omitted, assume
+it is @code{sha256}.
+
+@var{value} can be a literal string, in which case it is base32-decoded,
+or it can be a bytevector.
+
+The following forms are all equivalent:
+
+@lisp
+(content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj")
+(content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj"
+ sha256)
+(content-hash (base32
+ "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj"))
+(content-hash (base64 "kkb+RPaP7uyMZmu4eXPVkM4BN8yhRd8BTHLslb6f/Rc=")
+ sha256)
+@end lisp
+
+Technically, @code{content-hash} is currently implemented as a macro.
+It performs sanity checks at macro-expansion time, when possible, such
+as ensuring that @var{value} has the right size for @var{algorithm}.
+@end deftp
+
+As we have seen above, how exactly the data an origin refers to is
+retrieved is determined by its @code{method} field. The @code{(guix
+download)} module provides the most common method, @code{url-fetch},
+described below.
+
+@deffn {Scheme Procedure} url-fetch @var{url} @var{hash-algo} @var{hash} @
+ [name] [#:executable? #f]
+Return a fixed-output derivation that fetches data from @var{url} (a
+string, or a list of strings denoting alternate URLs), which is expected
+to have hash @var{hash} of type @var{hash-algo} (a symbol). By default,
+the file name is the base name of URL; optionally, @var{name} can
+specify a different file name. When @var{executable?} is true, make the
+downloaded file executable.
+
+When one of the URL starts with @code{mirror://}, then its host part is
+interpreted as the name of a mirror scheme, taken from @file{%mirror-file}.
+
+Alternatively, when URL starts with @code{file://}, return the
+corresponding file name in the store.
+@end deffn
+
+Likewise, the @code{(guix git-download)} module defines the
+@code{git-fetch} origin method, which fetches data from a Git version
+control repository, and the @code{git-reference} data type to describe
+the repository and revision to fetch.
+
+@deffn {Scheme Procedure} git-fetch @var{ref} @var{hash-algo} @var{hash}
+Return a fixed-output derivation that fetches @var{ref}, a
+@code{<git-reference>} object. The output is expected to have recursive
+hash @var{hash} of type @var{hash-algo} (a symbol). Use @var{name} as
+the file name, or a generic name if @code{#f}.
+@end deffn
+
+@deftp {Data Type} git-reference
+This data type represents a Git reference for @code{git-fetch} to
+retrieve.
+
+@table @asis
+@item @code{url}
+The URL of the Git repository to clone.
+
+@item @code{commit}
+This string denotes either the commit to fetch (a hexadecimal string,
+either the full SHA1 commit or a ``short'' commit string; the latter is
+not recommended) or the tag to fetch.
+
+@item @code{recursive?} (default: @code{#f})
+This Boolean indicates whether to recursively fetch Git sub-modules.
+@end table
+
+The example below denotes the @code{v2.10} tag of the GNU@tie{}Hello
+repository:
+
+@lisp
+(git-reference
+ (url "https://git.savannah.gnu.org/git/hello.git")
+ (commit "v2.10"))
+@end lisp
+
+This is equivalent to the reference below, which explicitly names the
+commit:
+
+@lisp
+(git-reference
+ (url "https://git.savannah.gnu.org/git/hello.git")
+ (commit "dc7dc56a00e48fe6f231a58f6537139fe2908fb9"))
+@end lisp
+@end deftp
+
+@node Defining Package Variants
+@section Defining Package Variants
+
+@cindex customizing packages
+@cindex variants, of packages
+One of the nice things with Guix is that, given a package definition,
+you can easily @emph{derive} variants of that package---for a different
+upstream version, with different dependencies, different compilation
+options, and so on. Some of these custom packages can be defined
+straight from the command line (@pxref{Package Transformation Options}).
+This section describes how to define package variants in code. This can
+be useful in ``manifests'' (@pxref{profile-manifest,
+@option{--manifest}}) and in your own package collection
+(@pxref{Creating a Channel}), among others!
+
+@cindex inherit, for package definitions
+As discussed earlier, packages are first-class objects in the Scheme
+language. The @code{(guix packages)} module provides the @code{package}
+construct to define new package objects (@pxref{package Reference}).
+The easiest way to define a package variant is using the @code{inherit}
+keyword together with @code{package}. This allows you to inherit from a
+package definition while overriding the fields you want.
+
+For example, given the @code{hello} variable, which contains a
+definition for the current version of GNU@tie{}Hello, here's how you
+would define a variant for version 2.2 (released in 2006, it's
+vintage!):
+
+@lisp
+(use-modules (gnu packages base)) ;for 'hello'
+
+(define hello-2.2
+ (package
+ (inherit hello)
+ (version "2.2")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "mirror://gnu/hello/hello-" version
+ ".tar.gz"))
+ (sha256
+ (base32
+ "0lappv4slgb5spyqbh6yl5r013zv72yqg2pcl30mginf3wdqd8k9"))))))
+@end lisp
+
+The example above corresponds to what the @option{--with-source} package
+transformation option does. Essentially @code{hello-2.2} preserves all
+the fields of @code{hello}, except @code{version} and @code{source},
+which it overrides. Note that the original @code{hello} variable is
+still there, in the @code{(gnu packages base)} module, unchanged. When
+you define a custom package like this, you are really @emph{adding} a
+new package definition; the original one remains available.
+
+You can just as well define variants with a different set of
+dependencies than the original package. For example, the default
+@code{gdb} package depends on @code{guile}, but since that is an
+optional dependency, you can define a variant that removes that
+dependency like so:
+
+@lisp
+(use-modules (gnu packages gdb) ;for 'gdb'
+ (srfi srfi-1)) ;for 'alist-delete'
+
+(define gdb-sans-guile
+ (package
+ (inherit gdb)
+ (inputs (alist-delete "guile"
+ (package-inputs gdb)))))
+@end lisp
+
+The @code{alist-delete} call above removes the tuple from the
+@code{inputs} field that has @code{"guile"} as its first element
+(@pxref{SRFI-1 Association Lists,,, guile, GNU Guile Reference
+Manual}).
+
+@cindex package transformations
+These are pretty simple package variants. As a convenience, the
+@code{(guix transformations)} module provides a high-level interface
+that directly maps to package transformation options (@pxref{Package
+Transformation Options}):
+
+@deffn {Scheme Procedure} options->transformation @var{opts}
+Return a procedure that, when passed an object to build (package,
+derivation, etc.), applies the transformations specified by @var{opts} and returns
+the resulting objects. @var{opts} must be a list of symbol/string pairs such as:
+
+@example
+((with-branch . "guile-gcrypt=master")
+ (without-tests . "libgcrypt"))
+@end example
+
+Each symbol names a transformation and the corresponding string is an argument
+to that transformation.
+@end deffn
+
+For instance, a manifest equivalent to this command:
+
+@example
+guix build guix \
+ --with-branch=guile-gcrypt=master \
+ --with-debug-info=zlib
+@end example
+
+@noindent
+... would look like this:
+
+@lisp
+(use-modules (guix transformations))
+
+(define transform
+ ;; The package transformation procedure.
+ (options->transformation
+ '((with-branch . "guile-gcrypt=master")
+ (with-debug-info . "zlib"))))
+
+(packages->manifest
+ (list (transform (specification->package "guix"))))
+@end lisp
+
+@cindex input rewriting
+@cindex dependency graph rewriting
+The @code{options->transformation} procedure is convenient, but it's
+perhaps also not as flexible as you may like. How is it implemented?
+The astute reader probably noticed that most package transformation
+options go beyond the superficial changes shown in the first examples of
+this section: they involve @dfn{input rewriting}, whereby the dependency
+graph of a package is rewritten by replacing specific inputs by others.
+
+Dependency graph rewriting, for the purposes of swapping packages in the
+graph, is what the @code{package-input-rewriting} procedure in
+@code{(guix packages)} implements.
+
+@deffn {Scheme Procedure} package-input-rewriting @var{replacements} @
+ [@var{rewrite-name}] [#:deep? #t]
+Return a procedure that, when passed a package, replaces its direct and
+indirect dependencies, including implicit inputs when @var{deep?} is
+true, according to @var{replacements}. @var{replacements} is a list of
+package pairs; the first element of each pair is the package to replace,
+and the second one is the replacement.
+
+Optionally, @var{rewrite-name} is a one-argument procedure that takes
+the name of a package and returns its new name after rewrite.
+@end deffn
-@item @code{patch-flags} (default: @code{'("-p1")})
-A list of command-line flags that should be passed to the @code{patch}
-command.
+@noindent
+Consider this example:
-@item @code{patch-inputs} (default: @code{#f})
-Input packages or derivations to the patching process. When this is
-@code{#f}, the usual set of inputs necessary for patching are provided,
-such as GNU@tie{}Patch.
+@lisp
+(define libressl-instead-of-openssl
+ ;; This is a procedure to replace OPENSSL by LIBRESSL,
+ ;; recursively.
+ (package-input-rewriting `((,openssl . ,libressl))))
-@item @code{modules} (default: @code{'()})
-A list of Guile modules that should be loaded during the patching
-process and while running the code in the @code{snippet} field.
+(define git-with-libressl
+ (libressl-instead-of-openssl git))
+@end lisp
-@item @code{patch-guile} (default: @code{#f})
-The Guile package that should be used in the patching process. When
-this is @code{#f}, a sensible default is used.
-@end table
-@end deftp
+@noindent
+Here we first define a rewriting procedure that replaces @var{openssl}
+with @var{libressl}. Then we use it to define a @dfn{variant} of the
+@var{git} package that uses @var{libressl} instead of @var{openssl}.
+This is exactly what the @option{--with-input} command-line option does
+(@pxref{Package Transformation Options, @option{--with-input}}).
-@deftp {Data Type} content-hash @var{value} [@var{algorithm}]
-Construct a content hash object for the given @var{algorithm}, and with
-@var{value} as its hash value. When @var{algorithm} is omitted, assume
-it is @code{sha256}.
+The following variant of @code{package-input-rewriting} can match packages to
+be replaced by name rather than by identity.
-@var{value} can be a literal string, in which case it is base32-decoded,
-or it can be a bytevector.
+@deffn {Scheme Procedure} package-input-rewriting/spec @var{replacements} [#:deep? #t]
+Return a procedure that, given a package, applies the given
+@var{replacements} to all the package graph, including implicit inputs
+unless @var{deep?} is false. @var{replacements} is a list of
+spec/procedures pair; each spec is a package specification such as
+@code{"gcc"} or @code{"guile@@2"}, and each procedure takes a matching
+package and returns a replacement for that package.
+@end deffn
-The following forms are all equivalent:
+The example above could be rewritten this way:
@lisp
-(content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj")
-(content-hash "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj"
- sha256)
-(content-hash (base32
- "05zxkyz9bv3j9h0xyid1rhvh3klhsmrpkf3bcs6frvlgyr2gwilj"))
-(content-hash (base64 "kkb+RPaP7uyMZmu4eXPVkM4BN8yhRd8BTHLslb6f/Rc=")
- sha256)
+(define libressl-instead-of-openssl
+ ;; Replace all the packages called "openssl" with LibreSSL.
+ (package-input-rewriting/spec `(("openssl" . ,(const libressl)))))
@end lisp
-Technically, @code{content-hash} is currently implemented as a macro.
-It performs sanity checks at macro-expansion time, when possible, such
-as ensuring that @var{value} has the right size for @var{algorithm}.
-@end deftp
+The key difference here is that, this time, packages are matched by spec and
+not by identity. In other words, any package in the graph that is called
+@code{openssl} will be replaced.
+
+A more generic procedure to rewrite a package dependency graph is
+@code{package-mapping}: it supports arbitrary changes to nodes in the
+graph.
+
+@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] [#:deep? #f]
+Return a procedure that, given a package, applies @var{proc} to all the packages
+depended on and returns the resulting package. The procedure stops recursion
+when @var{cut?} returns true for a given package. When @var{deep?} is true, @var{proc} is
+applied to implicit inputs as well.
+@end deffn
+
@node Build Systems
@section Build Systems
a package, which includes all the inputs of that package, including some
that were implicitly added by the build system. This intermediate
representation is then compiled to a derivation (@pxref{Derivations}).
+The @code{package-with-c-toolchain} is an example of a way to change the
+implicit inputs that a package's build system pulls in (@pxref{package
+Reference, @code{package-with-c-toolchain}}).
Build systems accept an optional list of @dfn{arguments}. In package
definitions, these are passed @i{via} the @code{arguments} field
@code{%standard-phases} is a list of symbol/procedure pairs, where the
procedure implements the actual phase.
-The list of phases used for a particular package can be changed with the
-@code{#:phases} parameter. For instance, passing:
-
-@example
-#:phases (modify-phases %standard-phases (delete 'configure))
-@end example
-
-means that all the phases described above will be used, except the
-@code{configure} phase.
+@xref{Build Phases}, for more info on build phases and ways to customize
+them.
In addition, this build system ensures that the ``standard'' environment
for GNU packages is available. This includes tools such as GCC, libc,
specified in the @code{#:cargo-inputs} and @code{#:cargo-development-inputs}
parameters available to cargo. It will also remove an included
@code{Cargo.lock} file to be recreated by @code{cargo} during the
-@code{build} phase. The @code{install} phase installs any crate the binaries
-if they are defined by the crate.
+@code{build} phase. The @code{install} phase installs the binaries
+defined by the crate.
@end defvr
julia} packages, which essentially is similar to running @samp{julia -e
'using Pkg; Pkg.add(package)'} in an environment where
@env{JULIA_LOAD_PATH} contains the paths to all Julia package inputs.
-Tests are run not run.
+Tests are run with @code{Pkg.test}.
Julia packages require the source @code{file-name} to be the real name of the
package, correctly capitalized.
@code{build-expression->derivation}}).
@end defvr
+@node Build Phases
+@section Build Phases
+
+@cindex build phases, for packages
+Almost all package build systems implement a notion @dfn{build phases}:
+a sequence of actions that the build system executes, when you build the
+package, leading to the installed byproducts in the store. A notable
+exception is the ``bare-bones'' @code{trivial-build-system}
+(@pxref{Build Systems}).
+
+As discussed in the previous section, those build systems provide a
+standard list of phases. For @code{gnu-build-system}, the standard
+phases include an @code{unpack} phase to unpack the source code tarball,
+a @command{configure} phase to run @code{./configure}, a @code{build}
+phase to run @command{make}, and (among others) an @code{install} phase
+to run @command{make install}; @pxref{Build Systems}, for a more
+detailed view of these phases. Likewise, @code{cmake-build-system}
+inherits these phases, but its @code{configure} phase runs
+@command{cmake} instead of @command{./configure}. Other build systems,
+such as @code{python-build-system}, have a wholly different list of
+standard phases. All this code runs on the @dfn{build side}: it is
+evaluated when you actually build the package, in a dedicated build
+process spawned by the build daemon (@pxref{Invoking guix-daemon}).
+
+Build phases are represented as association lists or ``alists''
+(@pxref{Association Lists,,, guile, GNU Guile Reference Manual}) where
+each key is a symbol for the name of the phase and the associated value
+is a procedure that accepts an arbitrary number of arguments. By
+convention, those procedures receive information about the build in the
+form of @dfn{keyword parameters}, which they can use or ignore.
+
+For example, here is how @code{(guix build gnu-build-system)} defines
+@code{%standard-phases}, the variable holding its alist of build
+phases@footnote{We present a simplified view of those build phases, but
+do take a look at @code{(guix build gnu-build-system)} to see all the
+details!}:
+
+@lisp
+;; The build phases of 'gnu-build-system'.
+
+(define* (unpack #:key source #:allow-other-keys)
+ ;; Extract the source tarball.
+ (invoke "tar" "xvf" source))
+
+(define* (configure #:key outputs #:allow-other-keys)
+ ;; Run the 'configure' script. Install to output "out".
+ (let ((out (assoc-ref outputs "out")))
+ (invoke "./configure"
+ (string-append "--prefix=" out))))
+
+(define* (build #:allow-other-keys)
+ ;; Compile.
+ (invoke "make"))
+
+(define* (check #:key (test-target "check") (tests? #true)
+ #:allow-other-keys)
+ ;; Run the test suite.
+ (if tests?
+ (invoke "make" test-target)
+ (display "test suite not run\n")))
+
+(define* (install #:allow-other-keys)
+ ;; Install files to the prefix 'configure' specified.
+ (invoke "make" "install"))
+
+(define %standard-phases
+ ;; The list of standard phases (quite a few are omitted
+ ;; for brevity). Each element is a symbol/procedure pair.
+ (list (cons 'unpack unpack)
+ (cons 'configure configure)
+ (cons 'build build)
+ (cons 'check check)
+ (cons 'install install)))
+@end lisp
+
+This shows how @code{%standard-phases} is defined as a list of
+symbol/procedure pairs (@pxref{Pairs,,, guile, GNU Guile Reference
+Manual}). The first pair associates the @code{unpack} procedure with
+the @code{unpack} symbol---a name; the second pair defines the
+@code{configure} phase similarly, and so on. When building a package
+that uses @code{gnu-build-system} with its default list of phases, those
+phases are executed sequentially. You can see the name of each phase
+started and completed in the build log of packages that you build.
+
+Let's now look at the procedures themselves. Each one is defined with
+@code{define*}: @code{#:key} lists keyword parameters the procedure
+accepts, possibly with a default value, and @code{#:allow-other-keys}
+specifies that other keyword parameters are ignored (@pxref{Optional
+Arguments,,, guile, GNU Guile Reference Manual}).
+
+The @code{unpack} procedure honors the @code{source} parameter, which
+the build system uses to pass the file name of the source tarball (or
+version control checkout), and it ignores other parameters. The
+@code{configure} phase only cares about the @code{outputs} parameter, an
+alist mapping package output names to their store file name
+(@pxref{Packages with Multiple Outputs}). It extracts the file name of
+for @code{out}, the default output, and passes it to
+@command{./configure} as the installation prefix, meaning that
+@command{make install} will eventually copy all the files in that
+directory (@pxref{Configuration, configuration and makefile
+conventions,, standards, GNU Coding Standards}). @code{build} and
+@code{install} ignore all their arguments. @code{check} honors the
+@code{test-target} argument, which specifies the name of the Makefile
+target to run tests; it prints a message and skips tests when
+@code{tests?} is false.
+
+@cindex build phases, customizing
+The list of phases used for a particular package can be changed with the
+@code{#:phases} parameter of the build system. Changing the set of
+build phases boils down to building a new alist of phases based on the
+@code{%standard-phases} alist described above. This can be done with
+standard alist procedures such as @code{alist-delete} (@pxref{SRFI-1
+Association Lists,,, guile, GNU Guile Reference Manual}); however, it is
+more convenient to do so with @code{modify-phases} (@pxref{Build
+Utilities, @code{modify-phases}}).
+
+Here is an example of a package definition that removes the
+@code{configure} phase of @code{%standard-phases} and inserts a new
+phase before the @code{build} phase, called
+@code{set-prefix-in-makefile}:
+
+@lisp
+(define-public example
+ (package
+ (name "example")
+ ;; other fields omitted
+ (build-system gnu-build-system)
+ (arguments
+ '(#:phases (modify-phases %standard-phases
+ (delete 'configure)
+ (add-before 'build 'set-prefix-in-makefile
+ (lambda* (#:key outputs #:allow-other-keys)
+ ;; Modify the makefile so that its
+ ;; 'PREFIX' variable points to "out".
+ (let ((out (assoc-ref outputs "out")))
+ (substitute* "Makefile"
+ (("PREFIX =.*")
+ (string-append "PREFIX = "
+ out "\n")))
+ #true))))))))
+@end lisp
+
+The new phase that is inserted is written as an anonymous procedure,
+introduced with @code{lambda*}; it honors the @code{outputs} parameter
+we have seen before. @xref{Build Utilities}, for more about the helpers
+used by this phase, and for more examples of @code{modify-phases}.
+
+@cindex code staging
+@cindex staging, of code
+Keep in mind that build phases are code evaluated at the time the
+package is actually built. This explains why the whole
+@code{modify-phases} expression above is quoted (it comes after the
+@code{'} or apostrophe): it is @dfn{staged} for later execution.
+@xref{G-Expressions}, for an explanation of code staging and the
+@dfn{code strata} involved.
+
+@node Build Utilities
+@section Build Utilities
+
+As soon as you start writing non-trivial package definitions
+(@pxref{Defining Packages}) or other build actions
+(@pxref{G-Expressions}), you will likely start looking for helpers for
+``shell-like'' actions---creating directories, copying and deleting
+files recursively, manipulating build phases, and so on. The
+@code{(guix build utils)} module provides such utility procedures.
+
+Most build systems load @code{(guix build utils)} (@pxref{Build
+Systems}). Thus, when writing custom build phases for your package
+definitions, you can usually assume those procedures are in scope.
+
+When writing G-expressions, you can import @code{(guix build utils)} on
+the ``build side'' using @code{with-imported-modules} and then put it in
+scope with the @code{use-modules} form (@pxref{Using Guile Modules,,,
+guile, GNU Guile Reference Manual}):
+
+@lisp
+(with-imported-modules '((guix build utils)) ;import it
+ (computed-file "empty-tree"
+ #~(begin
+ ;; Put it in scope.
+ (use-modules (guix build utils))
+
+ ;; Happily use its 'mkdir-p' procedure.
+ (mkdir-p (string-append #$output "/a/b/c")))))
+@end lisp
+
+The remainder of this section is the reference for most of the utility
+procedures provided by @code{(guix build utils)}.
+
+@c TODO Document what's missing.
+
+@subsection Dealing with Store File Names
+
+This section documents procedures that deal with store file names.
+
+@deffn {Scheme Procedure} %store-directory
+Return the directory name of the store.
+@end deffn
+
+@deffn {Scheme Procedure} store-file-name? @var{file}
+Return true if @var{file} is in the store.
+@end deffn
+
+@deffn {Scheme Procedure} strip-store-file-name @var{file}
+Strip the @file{/gnu/store} and hash from @var{file}, a store file name.
+The result is typically a @code{"@var{package}-@var{version}"} string.
+@end deffn
+
+@deffn {Scheme Procedure} package-name->name+version @var{name}
+Given @var{name}, a package name like @code{"foo-0.9.1b"}, return two
+values: @code{"foo"} and @code{"0.9.1b"}. When the version part is
+unavailable, @var{name} and @code{#f} are returned. The first hyphen
+followed by a digit is considered to introduce the version part.
+@end deffn
+
+@subsection File Types
+
+The procedures below deal with files and file types.
+
+@deffn {Scheme Procedure} directory-exists? @var{dir}
+Return @code{#t} if @var{dir} exists and is a directory.
+@end deffn
+
+@deffn {Scheme Procedure} executable-file? @var{file}
+Return @code{#t} if @var{file} exists and is executable.
+@end deffn
+
+@deffn {Scheme Procedure} symbolic-link? @var{file}
+Return @code{#t} if @var{file} is a symbolic link (aka. a ``symlink'').
+@end deffn
+
+@deffn {Scheme Procedure} elf-file? @var{file}
+@deffnx {Scheme Procedure} ar-file? @var{file}
+@deffnx {Scheme Procedure} gzip-file? @var{file}
+Return @code{#t} if @var{file} is, respectively, an ELF file, an
+@code{ar} archive (such as a @file{.a} static library), or a gzip file.
+@end deffn
+
+@deffn {Scheme Procedure} reset-gzip-timestamp @var{file} [#:keep-mtime? #t]
+If @var{file} is a gzip file, reset its embedded timestamp (as with
+@command{gzip --no-name}) and return true. Otherwise return @code{#f}.
+When @var{keep-mtime?} is true, preserve @var{file}'s modification time.
+@end deffn
+
+@subsection File Manipulation
+
+The following procedures and macros help create, modify, and delete
+files. They provide functionality comparable to common shell utilities
+such as @command{mkdir -p}, @command{cp -r}, @command{rm -r}, and
+@command{sed}. They complement Guile's extensive, but low-level, file
+system interface (@pxref{POSIX,,, guile, GNU Guile Reference Manual}).
+
+@deffn {Scheme Syntax} with-directory-excursion @var{directory} @var{body}@dots{}
+Run @var{body} with @var{directory} as the process's current directory.
+
+Essentially, this macro changes the current directory to @var{directory}
+before evaluating @var{body}, using @code{chdir} (@pxref{Processes,,,
+guile, GNU Guile Reference Manual}). It changes back to the initial
+directory when the dynamic extent of @var{body} is left, be it @i{via}
+normal procedure return or @i{via} a non-local exit such as an
+exception.
+@end deffn
+
+@deffn {Scheme Procedure} mkdir-p @var{dir}
+Create directory @var{dir} and all its ancestors.
+@end deffn
+
+@deffn {Scheme Procedure} install-file @var{file} @var{directory}
+Create @var{directory} if it does not exist and copy @var{file} in there
+under the same name.
+@end deffn
+
+@deffn {Scheme Procedure} make-file-writable @var{file}
+Make @var{file} writable for its owner.
+@end deffn
+
+@deffn {Scheme Procedure} copy-recursively @var{source} @var{destination} @
+ [#:log (current-output-port)] [#:follow-symlinks? #f] [#:keep-mtime? #f]
+Copy @var{source} directory to @var{destination}. Follow symlinks if
+@var{follow-symlinks?} is true; otherwise, just preserve them. When
+@var{keep-mtime?} is true, keep the modification time of the files in
+@var{source} on those of @var{destination}. Write verbose output to the
+@var{log} port.
+@end deffn
+
+@deffn {Scheme Procedure} delete-file-recursively @var{dir} @
+ [#:follow-mounts? #f]
+Delete @var{dir} recursively, like @command{rm -rf}, without following
+symlinks. Don't follow mount points either, unless @var{follow-mounts?}
+is true. Report but ignore errors.
+@end deffn
+
+@deffn {Scheme Syntax} substitute* @var{file} @
+ ((@var{regexp} @var{match-var}@dots{}) @var{body}@dots{}) @dots{}
+Substitute @var{regexp} in @var{file} by the string returned by
+@var{body}. @var{body} is evaluated with each @var{match-var} bound to
+the corresponding positional regexp sub-expression. For example:
+
+@lisp
+(substitute* file
+ (("hello")
+ "good morning\n")
+ (("foo([a-z]+)bar(.*)$" all letters end)
+ (string-append "baz" letter end)))
+@end lisp
+
+Here, anytime a line of @var{file} contains @code{hello}, it is replaced
+by @code{good morning}. Anytime a line of @var{file} matches the second
+regexp, @code{all} is bound to the complete match, @code{letters} is bound
+to the first sub-expression, and @code{end} is bound to the last one.
+
+When one of the @var{match-var} is @code{_}, no variable is bound to the
+corresponding match substring.
+
+Alternatively, @var{file} may be a list of file names, in which case
+they are all subject to the substitutions.
+
+Be careful about using @code{$} to match the end of a line; by itself it
+won't match the terminating newline of a line.
+@end deffn
+
+@subsection File Search
+
+@cindex file, searching
+This section documents procedures to search and filter files.
+
+@deffn {Scheme Procedure} file-name-predicate @var{regexp}
+Return a predicate that returns true when passed a file name whose base
+name matches @var{regexp}.
+@end deffn
+
+@deffn {Scheme Procedure} find-files @var{dir} [@var{pred}] @
+ [#:stat lstat] [#:directories? #f] [#:fail-on-error? #f]
+Return the lexicographically sorted list of files under @var{dir} for
+which @var{pred} returns true. @var{pred} is passed two arguments: the
+absolute file name, and its stat buffer; the default predicate always
+returns true. @var{pred} can also be a regular expression, in which
+case it is equivalent to @code{(file-name-predicate @var{pred})}.
+@var{stat} is used to obtain file information; using @code{lstat} means
+that symlinks are not followed. If @var{directories?} is true, then
+directories will also be included. If @var{fail-on-error?} is true,
+raise an exception upon error.
+@end deffn
+
+Here are a few examples where we assume that the current directory is
+the root of the Guix source tree:
+
+@lisp
+;; List all the regular files in the current directory.
+(find-files ".")
+@result{} ("./.dir-locals.el" "./.gitignore" @dots{})
+
+;; List all the .scm files under gnu/services.
+(find-files "gnu/services" "\\.scm$")
+@result{} ("gnu/services/admin.scm" "gnu/services/audio.scm" @dots{})
+
+;; List ar files in the current directory.
+(find-files "." (lambda (file stat) (ar-file? file)))
+@result{} ("./libformat.a" "./libstore.a" @dots{})
+@end lisp
+
+@deffn {Scheme Procedure} which @var{program}
+Return the complete file name for @var{program} as found in
+@code{$PATH}, or @code{#f} if @var{program} could not be found.
+@end deffn
+
+@subsection Build Phases
+
+@cindex build phases
+The @code{(guix build utils)} also contains tools to manipulate build
+phases as used by build systems (@pxref{Build Systems}). Build phases
+are represented as association lists or ``alists'' (@pxref{Association
+Lists,,, guile, GNU Guile Reference Manual}) where each key is a symbol
+naming the phase and the associated value is a procedure (@pxref{Build
+Phases}).
+
+Guile core and the @code{(srfi srfi-1)} module both provide tools to
+manipulate alists. The @code{(guix build utils)} module complements
+those with tools written with build phases in mind.
+
+@cindex build phases, modifying
+@deffn {Scheme Syntax} modify-phases @var{phases} @var{clause}@dots{}
+Modify @var{phases} sequentially as per each @var{clause}, which may
+have one of the following forms:
+
+@lisp
+(delete @var{old-phase-name})
+(replace @var{old-phase-name} @var{new-phase})
+(add-before @var{old-phase-name} @var{new-phase-name} @var{new-phase})
+(add-after @var{old-phase-name} @var{new-phase-name} @var{new-phase})
+@end lisp
+
+Where every @var{phase-name} above is an expression evaluating to a
+symbol, and @var{new-phase} an expression evaluating to a procedure.
+@end deffn
+
+The example below is taken from the definition of the @code{grep}
+package. It adds a phase to run after the @code{install} phase, called
+@code{fix-egrep-and-fgrep}. That phase is a procedure (@code{lambda*}
+is for anonymous procedures) that takes a @code{#:outputs} keyword
+argument and ignores extra keyword arguments (@pxref{Optional
+Arguments,,, guile, GNU Guile Reference Manual}, for more on
+@code{lambda*} and optional and keyword arguments.) The phase uses
+@code{substitute*} to modify the installed @file{egrep} and @file{fgrep}
+scripts so that they refer to @code{grep} by its absolute file name:
+
+@lisp
+(modify-phases %standard-phases
+ (add-after 'install 'fix-egrep-and-fgrep
+ ;; Patch 'egrep' and 'fgrep' to execute 'grep' via its
+ ;; absolute file name instead of searching for it in $PATH.
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin")))
+ (substitute* (list (string-append bin "/egrep")
+ (string-append bin "/fgrep"))
+ (("^exec grep")
+ (string-append "exec " bin "/grep")))
+ #t))))
+@end lisp
+
+In the example below, phases are modified in two ways: the standard
+@code{configure} phase is deleted, presumably because the package does
+not have a @file{configure} script or anything similar, and the default
+@code{install} phase is replaced by one that manually copies the
+executable files to be installed:
+
+@lisp
+(modify-phases %standard-phases
+ (delete 'configure) ;no 'configure' script
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ ;; The package's Makefile doesn't provide an "install"
+ ;; rule so do it by ourselves.
+ (let ((bin (string-append (assoc-ref outputs "out")
+ "/bin")))
+ (install-file "footswitch" bin)
+ (install-file "scythe" bin)
+ #t))))
+@end lisp
+
+@c TODO: Add more examples.
+
@node The Store
@section The Store
build the derivations; they are run by the daemon in a container
(@pxref{Invoking guix-daemon}).
+@cindex code staging
+@cindex staging, of code
@cindex strata of code
It should come as no surprise that we like to write these build actions
in Scheme. When we do that, we end up with two @dfn{strata} of Scheme
@dfn{staging}.}: the ``host code''---code that defines packages, talks
to the daemon, etc.---and the ``build code''---code that actually
performs build actions, such as making directories, invoking
-@command{make}, etc.
+@command{make}, and so on (@pxref{Build Phases}).
To describe a derivation and its build actions, one typically needs to
embed build code inside host code. It boils down to manipulating build
@code{let-system} is useful in the occasional case where the object
spliced into the gexp depends on the target system, as in this example:
-@example
+@lisp
#~(system*
#+(let-system system
(cond ((string-prefix? "armhf-" system)
(else
(error "dunno!"))))
"-net" "user" #$image)
-@end example
+@end lisp
@end deffn
@deffn {Scheme Syntax} with-parameters ((@var{parameter} @var{value}) @dots{}) @var{exp}
practically impossible for users to @emph{verify} whether third-party
binaries are genuine. @xref{Invoking guix challenge}, for more.
-Note that, currently, the differing build results are not kept around,
-so you will have to manually investigate in case of an error---e.g., by
-stashing one of the build results with @code{guix archive --export}
-(@pxref{Invoking guix archive}), then rebuilding, and finally comparing
-the two results.
+When used in conjunction with @option{--keep-failed}, the differing
+output is kept in the store, under @file{/gnu/store/@dots{}-check}.
+This makes it easy to look for differences between the two results.
@item --no-offload
Do not use offload builds to other machines (@pxref{Daemon Offload
without having to type in the definitions of package variants
(@pxref{Defining Packages}).
+Package transformation options are preserved across upgrades:
+@command{guix upgrade} attempts to apply transformation options
+initially used when creating the profile to the upgraded packages.
+
+The available options are listed below. Most commands support them and
+also support a @option{--help-transform} option that lists all the
+available options and a synopsis (these options are not shown in the
+@option{--help} output for brevity).
+
@table @code
@item --with-source=@var{source}
@var{package}, then the resulting package may be unusable. Use with
care!
+@cindex debugging info, rebuilding
+@item --with-debug-info=@var{package}
+Build @var{package} in a way that preserves its debugging info and graft
+it onto packages that depend on it. This is useful if @var{package}
+does not already provide debugging info as a @code{debug} output
+(@pxref{Installing Debugging Files}).
+
+For example, suppose you're experiencing a crash in Inkscape and would
+like to see what's up in GLib, a library deep down in Inkscape's
+dependency graph. GLib lacks a @code{debug} output, so debugging is
+tough. Fortunately, you rebuild GLib with debugging info and tack it on
+Inkscape:
+
+@example
+guix install inkscape --with-debug-info=glib
+@end example
+
+Only GLib needs to be recompiled so this takes a reasonable amount of
+time. @xref{Installing Debugging Files}, for more info.
+
+@quotation Note
+Under the hood, this option works by passing the @samp{#:strip-binaries?
+#f} to the build system of the package of interest (@pxref{Build
+Systems}). Most build systems support that option but some do not. In
+that case, an error is raised.
+
+Likewise, if a C/C++ package is built without @code{-g} (which is rarely
+the case), debugging info will remain unavailable even when
+@code{#:strip-binaries?} is false.
+@end quotation
+
+@cindex tool chain, changing the build tool chain of a package
+@item --with-c-toolchain=@var{package}=@var{toolchain}
+This option changes the compilation of @var{package} and everything that
+depends on it so that they get built with @var{toolchain} instead of the
+default GNU tool chain for C/C++.
+
+Consider this example:
+
+@example
+guix build octave-cli \
+ --with-c-toolchain=fftw=gcc-toolchain@@10 \
+ --with-c-toolchain=fftwf=gcc-toolchain@@10
+@end example
+
+The command above builds a variant of the @code{fftw} and @code{fftwf}
+packages using version 10 of @code{gcc-toolchain} instead of the default
+tool chain, and then builds a variant of the GNU@tie{}Octave
+command-line interface using them. GNU@tie{}Octave itself is also built
+with @code{gcc-toolchain@@10}.
+
+This other example builds the Hardware Locality (@code{hwloc}) library
+and its dependents up to @code{intel-mpi-benchmarks} with the Clang C
+compiler:
+
+@example
+guix build --with-c-toolchain=hwloc=clang-toolchain \
+ intel-mpi-benchmarks
+@end example
+
+@quotation Note
+There can be application binary interface (ABI) incompatibilities among
+tool chains. This is particularly true of the C++ standard library and
+run-time support libraries such as that of OpenMP. By rebuilding all
+dependents with the same tool chain, @option{--with-c-toolchain} minimizes
+the risks of incompatibility but cannot entirely eliminate them. Choose
+@var{package} wisely.
+@end quotation
+
@item --with-git-url=@var{package}=@var{url}
@cindex Git, using the latest commit
@cindex latest commit, building
rebuilds everything that depends on @code{python}, including
@code{python-notebook} itself.
+Internally, @option{--without-tests} relies on changing the
+@code{#:tests?} option of a package's @code{check} phase (@pxref{Build
+Systems}). Note that some packages use a customized @code{check} phase
+that does not respect a @code{#:tests? #f} setting. Therefore,
+@option{--without-tests} has no effect on these packages.
+
@end table
+Wondering how to achieve the same effect using Scheme code, for example
+in your manifest, or how to write your own package transformation?
+@xref{Defining Package Variants}, for an overview of the programming
+interfaces available.
+
@node Additional Build Options
@subsection Additional Build Options
When @option{--archive=bioconductor} is added, metadata is imported from
@uref{https://www.bioconductor.org/, Bioconductor}, a repository of R
-packages for for the analysis and comprehension of high-throughput
+packages for the analysis and comprehension of high-throughput
genomic data in bioinformatics.
Information is extracted from the @file{DESCRIPTION} file contained in the
Only enable the checkers specified in a comma-separated list using the
names returned by @option{--list-checkers}.
+@item --exclude
+@itemx -x
+Only disable the checkers specified in a comma-separated list using the
+names returned by @option{--list-checkers}.
+
+@item --no-network
+@itemx -n
+Only enable the checkers that do not depend on Internet access.
+
@item --load-path=@var{directory}
@itemx -L @var{directory}
Add @var{directory} to the front of the package module search path
guix publish
@end example
-Once a publishing server has been authorized (@pxref{Invoking guix
-archive}), the daemon may download substitutes from it:
-
-@example
-guix-daemon --substitute-urls=http://example.org:8080
-@end example
+Once a publishing server has been authorized, the daemon may download
+substitutes from it. @xref{Getting Substitutes from Other Servers}.
By default, @command{guix publish} compresses archives on the fly as it
serves them. This ``on-the-fly'' mode is convenient in that it requires
prevents clients from knowing the amount of data being downloaded.
Conversely, when @option{--cache} is used, the first request for a store
-item (@i{via} a @code{.narinfo} URL) returns 404 and triggers a
+item (@i{via} a @code{.narinfo} URL) triggers a
background process to @dfn{bake} the archive---computing its
@code{.narinfo} and compressing the archive, if needed. Once the
archive is cached in @var{directory}, subsequent requests succeed and
are served directly from the cache, which guarantees that clients get
the best possible bandwidth.
+That first @code{.narinfo} request nonetheless returns 200, provided the
+requested store item is ``small enough'', below the cache bypass
+threshold---see @option{--cache-bypass-threshold} below. That way,
+clients do not have to wait until the archive is baked. For larger
+store items, the first @code{.narinfo} request returns 404, meaning that
+clients have to wait until the archive is baked.
+
The ``baking'' process is performed by worker threads. By default, one
thread per CPU core is created, but this can be customized. See
@option{--workers} below.
not been accessed for @var{ttl} and that no longer have a corresponding
item in the store, may be deleted.
+@item --cache-bypass-threshold=@var{size}
+When used in conjunction with @option{--cache}, store items smaller than
+@var{size} are immediately available, even when they are not yet in
+cache. @var{size} is a size in bytes, or it can be prefixed by @code{M}
+for megabytes and so on. The default is @code{10M}.
+
+``Cache bypass'' allows you to reduce the publication delay for clients
+at the expense of possibly additional I/O and CPU use on the server
+side: depending on the client access patterns, those store items can end
+up being baked several times until a copy is available in cache.
+
+Increasing the threshold may be useful for sites that have few users, or
+to guarantee that users get substitutes even for store items that are
+not popular.
+
@item --nar-path=@var{path}
Use @var{path} as the prefix for the URLs of ``nar'' files
(@pxref{Invoking guix archive, normalized archives}).
(guix-service-type config =>
(guix-configuration
(inherit config)
- (use-substitutes? #f)
- (extra-options '("--gc-keep-derivations"))))
+ ;; Fetch substitutes from example.org.
+ (substitute-urls
+ (list "https://example.org/guix"
+ "https://ci.guix.gnu.org"))))
(mingetty-service-type config =>
(mingetty-configuration
- (inherit config)))))
+ (inherit config)
+ ;; Automatially log in as "guest".
+ (auto-login "guest")))))
(operating-system
;; @dots{}
@cindex hurd
@item @code{hurd} (default: @code{#f})
-The package object of the hurd to be started by the kernel. When this
+The package object of the Hurd to be started by the kernel. When this
field is set, produce a GNU/Hurd operating system. In that case,
@code{kernel} must also be set to the @code{gnumach} package---the
microkernel the Hurd runs on.
@item @code{file-systems}
A list of file systems. @xref{File Systems}.
-@item @code{swap-devices} (default: @code{'()})
@cindex swap devices
-A list of strings identifying devices or files to be used for ``swap
+@cindex swap space
+@item @code{swap-devices} (default: @code{'()})
+A list of UUIDs, file system labels, or strings identifying devices or
+files to be used for ``swap
space'' (@pxref{Memory Concepts,,, libc, The GNU C Library Reference
-Manual}). For example, @code{'("/dev/sda3")} or @code{'("/swapfile")}.
+Manual}). Here are some examples:
+
+@table @code
+@item (list (uuid "4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
+Use the swap partition with the given UUID. You can learn the UUID of a
+Linux swap partition by running @command{swaplabel @var{device}}, where
+@var{device} is the @file{/dev} file name of that partition.
+
+@item (list (file-system-label "swap"))
+Use the partition with label @code{swap}. Again, the
+@command{swaplabel} command allows you to view and change the label of a
+Linux swap partition.
+
+@item (list "/swapfile")
+Use the file @file{/swapfile} as swap space.
+
+@item (list "/dev/sda3" "/dev/sdb2")
+Use the @file{/dev/sda3} and @file{/dev/sdb2} partitions as swap space.
+We recommend referring to swap devices by UUIDs or labels as shown above
+instead.
+@end table
+
It is possible to specify a swap file in a file system on a mapped
-device, provided that the necessary device mapping and file system are
-also specified. @xref{Mapped Devices} and @ref{File Systems}.
+device (under @file{/dev/mapper}), provided that the necessary device
+mapping and file system are also specified. @xref{Mapped Devices} and
+@ref{File Systems}.
@item @code{users} (default: @code{%base-user-accounts})
@itemx @code{groups} (default: @code{%base-groups})
"audio" ;sound card
"video" ;video devices such as webcams
"cdrom")) ;the good ol' CD-ROM
- (comment "Bob's sister")
- (home-directory "/home/alice"))
+ (comment "Bob's sister"))
+@end lisp
+
+Here's a user account that uses a different shell and a custom home
+directory (the default would be @file{"/home/bob"}):
+
+@lisp
+(user-account
+ (name "bob")
+ (group "users")
+ (comment "Alice's bro")
+ (shell (file-append zsh "/bin/zsh"))
+ (home-directory "/home/robert"))
@end lisp
When booting or upon completion of @command{guix system reconfigure},
@item @code{shell} (default: Bash)
This is a G-expression denoting the file name of a program to be used as
-the shell (@pxref{G-Expressions}).
+the shell (@pxref{G-Expressions}). For example, you would refer to the
+Bash executable like this:
+
+@lisp
+(file-append bash "/bin/bash")
+@end lisp
+
+@noindent
+... and to the Zsh executable like that:
+
+@lisp
+(file-append zsh "/bin/zsh")
+@end lisp
@item @code{system?} (default: @code{#f})
This Boolean value indicates whether the account is a ``system''
a language code such as ``fi'' for Finnish or ``jp'' for Japanese), an
optional variant name, an optional keyboard model name, and a possibly empty
list of additional options. In most cases the layout name is all you care
-about. Here are a few example:
+about.
+
+@deffn {Scheme Procedure} keyboard-layout @var{name} [@var{variant}] @
+ [#:model] [#:options '()]
+Return a new keyboard layout with the given @var{name} and @var{variant}.
+
+@var{name} must be a string such as @code{"fr"}; @var{variant} must be a
+string such as @code{"bepo"} or @code{"nodeadkeys"}. See the
+@code{xkeyboard-config} package for valid options.
+@end deffn
+
+Here are a few examples:
@lisp
;; The German QWERTZ layout. Here we assume a standard
@code{authorized-keys}---by default that of @code{@value{SUBSTITUTE-SERVER}}
(@pxref{Substitutes}).
+When @code{authorize-key?} is true, @file{/etc/guix/acl} cannot be
+changed by invoking @command{guix archive --authorize}. You must
+instead adjust @code{guix-configuration} as you wish and reconfigure the
+system. This ensures that your operating system configuration file is
+self-contained.
+
+@quotation Note
+When booting or reconfiguring to a system where @code{authorize-key?}
+is true, the existing @file{/etc/guix/acl} file is backed up as
+@file{/etc/guix/acl.bak} if it was determined to be a manually modified
+file. This is to facilitate migration from earlier versions, which
+allowed for in-place modifications to @file{/etc/guix/acl}.
+@end quotation
+
@vindex %default-authorized-guix-keys
@item @code{authorized-keys} (default: @code{%default-authorized-guix-keys})
The list of authorized key files for archive imports, as a list of
string-valued gexps (@pxref{Invoking guix archive}). By default, it
contains that of @code{@value{SUBSTITUTE-SERVER}} (@pxref{Substitutes}).
+See @code{substitute-urls} below for an example on how to change it.
@item @code{use-substitutes?} (default: @code{#t})
Whether to use substitutes.
@item @code{substitute-urls} (default: @code{%default-substitute-urls})
The list of URLs where to look for substitutes by default.
+Suppose you would like to fetch substitutes from @code{guix.example.org}
+in addition to @code{@value{SUBSTITUTE-SERVER}}. You will need to do
+two things: (1) add @code{guix.example.org} to @code{substitute-urls},
+and (2) authorize its signing key, having done appropriate checks
+(@pxref{Substitute Server Authorization}). The configuration below does
+exactly that:
+
+@lisp
+(guix-configuration
+ (substitute-urls
+ (append (list "https://guix.example.org")
+ %default-substitute-urls))
+ (authorized-keys
+ (append (list (local-file "./guix.example.org-key.pub"))
+ %default-authorized-guix-keys)))
+@end lisp
+
+This example assumes that the file @file{./guix.example.org-key.pub}
+contains the public key that @code{guix.example.org} uses to sign
+substitutes.
+
@item @code{max-silent-time} (default: @code{0})
@itemx @code{timeout} (default: @code{0})
The number of seconds of silence and the number of seconds of activity,
caching; when @code{#f}, the number of processors is used.
@xref{Invoking guix publish, @option{--workers}}, for more information.
+@item @code{cache-bypass-threshold} (default: 10 MiB)
+When @code{cache} is true, this is the maximum size in bytes of a store
+item for which @command{guix publish} may bypass its cache in case of a
+cache miss. @xref{Invoking guix publish,
+@option{--cache-bypass-threshold}}, for more information.
+
@item @code{ttl} (default: @code{#f})
When it is an integer, this denotes the @dfn{time-to-live} in seconds
of the published archives. @xref{Invoking guix publish, @option{--ttl}},
This is the configuration record for OpenSSH's @command{sshd}.
@table @asis
+@item @code{openssh} (default @var{openssh})
+The Openssh package to use.
+
@item @code{pid-file} (default: @code{"/var/run/sshd.pid"})
Name of the file where @command{sshd} writes its PID.
@end table
@end deftp
+@cindex WebSSH
+@deffn {Scheme Variable} webssh-service-type
+This is the type for the @uref{https://webssh.huashengdun.org/, WebSSH}
+program that runs a web SSH client. WebSSH can be run manually from the
+command-line by passing arguments to the binary @command{wssh} from the
+package @code{webssh}, but it can also be run as a Guix service. This
+latter use case is documented here.
+
+For example, to specify a service running WebSSH on loopback interface
+on port @code{8888} with reject policy with a list of allowed to
+connection hosts, and NGINX as a reverse-proxy to this service listening
+for HTTPS connection, add this call to the operating system's
+@code{services} field:
+
+@lisp
+(service webssh-service-type
+ (webssh-configuration (address "127.0.0.1")
+ (port 8888)
+ (policy 'reject)
+ (known-hosts '("localhost ecdsa-sha2-nistp256 AAAA…"
+ "127.0.0.1 ecdsa-sha2-nistp256 AAAA…"))))
+
+(service nginx-service-type
+ (nginx-configuration
+ (server-blocks
+ (list
+ (nginx-server-configuration
+ (inherit %webssh-configuration-nginx)
+ (server-name '("webssh.example.com"))
+ (listen '("443 ssl"))
+ (ssl-certificate (letsencrypt-certificate "webssh.example.com"))
+ (ssl-certificate-key (letsencrypt-key "webssh.example.com"))
+ (locations
+ (cons (nginx-location-configuration
+ (uri "/.well-known")
+ (body '("root /var/www;")))
+ (nginx-server-configuration-locations %webssh-configuration-nginx))))))))
+@end lisp
+@end deffn
+
+@deftp {Data Type} webssh-configuration
+Data type representing the configuration for @code{webssh-service}.
+
+@table @asis
+@item @code{package} (default: @var{webssh})
+@code{webssh} package to use.
+
+@item @code{user-name} (default: @var{"webssh"})
+User name or user ID that file transfers to and from that module should take
+place.
+
+@item @code{group-name} (default: @var{"webssh"})
+Group name or group ID that will be used when accessing the module.
+
+@item @code{address} (default: @var{#f})
+IP address on which @command{webssh} listens for incoming connections.
+
+@item @code{port} (default: @var{8888})
+TCP port on which @command{webssh} listens for incoming connections.
+
+@item @code{policy} (default: @var{#f})
+Connection policy. @var{reject} policy requires to specify @var{known-hosts}.
+
+@item @code{known-hosts} (default: @var{'()})
+List of hosts which allowed for SSH connection from @command{webssh}.
+
+@item @code{log-file} (default: @file{"/var/log/webssh.log"})
+Name of the file where @command{rsync} writes its log file.
+
+@item @code{log-level} (default: @var{#f})
+Logging level.
+
+@end table
+@end deftp
+
@defvr {Scheme Variable} %facebook-host-aliases
This variable contains a string for use in @file{/etc/hosts}
(@pxref{Host Names,,, libc, The GNU C Library Reference Manual}). Each
This is the service that runs @command{avahi-daemon}, a system-wide
mDNS/DNS-SD responder that allows for service discovery and
``zero-configuration'' host name lookups (see @uref{https://avahi.org/}).
-Its value must be a @code{zero-configuration} record---see below.
+Its value must be an @code{avahi-configuration} record---see below.
This service extends the name service cache daemon (nscd) so that it can
resolve @code{.local} host names using
Reference, @code{services}}).
Additionally, the @code{gnome-desktop-service-type},
-@code{xfce-desktop-service}, @code{mate-desktop-service-type} and
-@code{enlightenment-desktop-service-type} procedures can add GNOME, Xfce, MATE
-and/or Enlightenment to a system. To ``add GNOME'' means that system-level
-services like the backlight adjustment helpers and the power management
-utilities are added to the system, extending @code{polkit} and @code{dbus}
-appropriately, allowing GNOME to operate with elevated privileges on a
-limited number of special-purpose system interfaces. Additionally,
-adding a service made by @code{gnome-desktop-service-type} adds the GNOME
-metapackage to the system profile. Likewise, adding the Xfce service
-not only adds the @code{xfce} metapackage to the system profile, but it
-also gives the Thunar file manager the ability to open a ``root-mode''
-file management window, if the user authenticates using the
-administrator's password via the standard polkit graphical interface.
-To ``add MATE'' means that @code{polkit} and @code{dbus} are extended
-appropriately, allowing MATE to operate with elevated privileges on a
-limited number of special-purpose system interfaces. Additionally,
-adding a service of type @code{mate-desktop-service-type} adds the MATE
-metapackage to the system profile. ``Adding Enlightenment'' means that
-@code{dbus} is extended appropriately, and several of Enlightenment's binaries
-are set as setuid, allowing Enlightenment's screen locker and other
-functionality to work as expected.
+@code{xfce-desktop-service}, @code{mate-desktop-service-type},
+@code{lxqt-desktop-service-type} and @code{enlightenment-desktop-service-type}
+procedures can add GNOME, Xfce, MATE and/or Enlightenment to a system. To
+``add GNOME'' means that system-level services like the backlight adjustment
+helpers and the power management utilities are added to the system, extending
+@code{polkit} and @code{dbus} appropriately, allowing GNOME to operate with
+elevated privileges on a limited number of special-purpose system interfaces.
+Additionally, adding a service made by @code{gnome-desktop-service-type} adds
+the GNOME metapackage to the system profile. Likewise, adding the Xfce
+service not only adds the @code{xfce} metapackage to the system profile, but
+it also gives the Thunar file manager the ability to open a ``root-mode'' file
+management window, if the user authenticates using the administrator's
+password via the standard polkit graphical interface. To ``add MATE'' means
+that @code{polkit} and @code{dbus} are extended appropriately, allowing MATE
+to operate with elevated privileges on a limited number of special-purpose
+system interfaces. Additionally, adding a service of type
+@code{mate-desktop-service-type} adds the MATE metapackage to the system
+profile. ``Adding Enlightenment'' means that @code{dbus} is extended
+appropriately, and several of Enlightenment's binaries are set as setuid,
+allowing Enlightenment's screen locker and other functionality to work as
+expected.
The desktop environments in Guix use the Xorg display server by
default. If you'd like to use the newer display server protocol
@end table
@end deftp
+@deffn {Scheme Variable} lxqt-desktop-service-type
+This is the type of the service that runs the @uref{https://lxqt.github.io,
+LXQt desktop environment}. Its value is a @code{lxqt-desktop-configuration}
+object (see below).
+
+This service adds the @code{lxqt} package to the system
+profile.
+@end deffn
+
+@deftp {Data Type} lxqt-desktop-configuration
+Configuration record for the LXQt desktop environment.
+
+@table @asis
+@item @code{lxqt} (default: @code{lxqt})
+The LXQT package to use.
+@end table
+@end deftp
+
@deffn {Scheme Variable} enlightenment-desktop-service-type
Return a service that adds the @code{enlightenment} package to the system
profile, and extends dbus with actions from @code{efl}.
@code{suspend}
@item handle-lid-switch-docked
@code{ignore}
+@item handle-lid-switch-external-power
+@code{ignore}
@item power-key-ignore-inhibited?
@code{#f}
@item suspend-key-ignore-inhibited?
@quotation Warning
This service overrides per-user configuration files. If you want
-PulseAudio to honor configuraton files in @file{~/.config/pulse} you
+PulseAudio to honor configuration files in @file{~/.config/pulse} you
have to unset the environment variables @env{PULSE_CONFIG} and
@env{PULSE_CLIENTCONFIG} in your @file{~/.bash_profile}.
@end quotation
Data type representing the configuration for @code{pulseaudio-service}.
@table @asis
-@item @var{client-conf} (default: @code{'()})
+@item @code{client-conf} (default: @code{'()})
List of settings to set in @file{client.conf}.
Accepts a list of strings or a symbol-value pairs. A string will be
inserted as-is with a newline added. A pair will be formatted as
``key = value'', again with a newline added.
-@item @var{daemon-conf} (default: @code{'((flat-volumes . no))})
+@item @code{daemon-conf} (default: @code{'((flat-volumes . no))})
List of settings to set in @file{daemon.conf}, formatted just like
@var{client-conf}.
-@item @var{script-file} (default: @code{(file-append pulseaudio "/etc/pulse/default.pa")})
-Script file to use as as @file{default.pa}.
+@item @code{script-file} (default: @code{(file-append pulseaudio "/etc/pulse/default.pa")})
+Script file to use as @file{default.pa}.
-@item @var{system-script-file} (default: @code{(file-append pulseaudio "/etc/pulse/system.pa")})
-Script file to use as as @file{system.pa}.
+@item @code{system-script-file} (default: @code{(file-append pulseaudio "/etc/pulse/system.pa")})
+Script file to use as @file{system.pa}.
@end table
@end deftp
@cindex SQL
The @code{(gnu services databases)} module provides the following services.
-@deffn {Scheme Procedure} postgresql-service [#:postgresql postgresql] @
- [#:config-file] [#:data-directory ``/var/lib/postgresql/data''] @
- [#:port 5432] [#:locale ``en_US.utf8''] [#:extension-packages '()]
-Return a service that runs @var{postgresql}, the PostgreSQL database
-server.
+@subsubheading PostgreSQL
+
+The following example describes a PostgreSQL service with the default
+configuration.
-The PostgreSQL daemon loads its runtime configuration from @var{config-file},
-creates a database cluster with @var{locale} as the default
-locale, stored in @var{data-directory}. It then listens on @var{port}.
+@lisp
+(service postgresql-service-type
+ (postgresql-configuration
+ (postgresql postgresql-10)))
+@end lisp
If the services fails to start, it may be due to an incompatible
cluster already present in @var{data-directory}. Adjust it (or, if you
createdb $MY_USER_LOGIN # Replace appropriately.
@end example
+@deftp {Data Type} postgresql-configuration
+Data type representing the configuration for the
+@code{postgresql-service-type}.
+
+@table @asis
+@item @code{postgresql}
+PostgreSQL package to use for the service.
+
+@item @code{port} (default: @code{5432})
+Port on which PostgreSQL should listen.
+
+@item @code{locale} (default: @code{"en_US.utf8"})
+Locale to use as the default when creating the database cluster.
+
+@item @code{config-file} (default: @code{(postgresql-config-file)})
+The configuration file to use when running PostgreSQL. The default
+behaviour uses the postgresql-config-file record with the default values
+for the fields.
+
+@item @code{data-directory} (default: @code{"/var/lib/postgresql/data"})
+Directory in which to store the data.
+
+@item @code{extension-packages} (default: @code{'()})
@cindex postgresql extension-packages
Additional extensions are loaded from packages listed in
@var{extension-packages}. Extensions are available at runtime. For instance,
(packages (cons* postgresql %base-packages))
(services
(cons*
- (postgresql-service #:extension-packages (list postgis))
+ (service postgresql-service-type
+ (postgresql-configuration
+ (postgresql postgresql-10)
+ (extension-packages (list postgis))))
%base-services)))
@end lisp
There is no need to add this field for contrib extensions such as hstore or
dblink as they are already loadable by postgresql. This field is only
required to add extensions provided by other packages.
-@end deffn
+
+@end table
+@end deftp
+
+@deftp {Data Type} postgresql-config-file
+Data type representing the PostgreSQL configuration file. As shown in
+the following example, this can be used to customize the configuration
+of PostgreSQL. Note that you can use any G-expression or filename in
+place of this record, if you already have a configuration file you'd
+like to use for example.
+
+@lisp
+(service postgresql-service-type
+ (postgresql-configuration
+ (config-file
+ (postgresql-config-file
+ (log-destination "stderr")
+ (hba-file
+ (plain-file "pg_hba.conf"
+ "
+local all all trust
+host all all 127.0.0.1/32 md5
+host all all ::1/128 md5"))
+ (extra-config
+ '(("session_preload_libraries" "'auto_explain'")
+ ("random_page_cost" "2")
+ ("auto_explain.log_min_duration" "'100ms'")
+ ("work_mem" "'500MB'")
+ ("logging_collector" "on")
+ ("log_directory" "'/var/log/postgresql'")))))))
+@end lisp
+
+@table @asis
+@item @code{log-destination} (default: @code{"syslog"})
+The logging method to use for PostgreSQL. Multiple values are accepted,
+separated by commas.
+
+@item @code{hba-file} (default: @code{%default-postgres-hba})
+Filename or G-expression for the host-based authentication
+configuration.
+
+@item @code{ident-file} (default: @code{%default-postgres-ident})
+Filename or G-expression for the user name mapping configuration.
+
+@item @code{extra-config} (default: @code{'()})
+List of additional keys and values to include in the PostgreSQL config
+file. Each entry in the list should be a list where the first element
+is the key, and the remaining elements are the values.
+
+@end table
+@end deftp
+
+@subsubheading MariaDB/MySQL
@deffn {Scheme Procedure} mysql-service [#:config (mysql-configuration)]
Return a service that runs @command{mysqld}, the MySQL or MariaDB
@end table
@end deftp
+@subsubheading Memcached
+
@defvr {Scheme Variable} memcached-service-type
This is the service type for the @uref{https://memcached.org/,
Memcached} service, which provides a distributed in memory cache. The
@end table
@end deftp
+@subsubheading MongoDB
+
@defvr {Scheme Variable} mongodb-service-type
This is the service type for @uref{https://www.mongodb.com/, MongoDB}.
The value for the service type is a @code{mongodb-configuration} object.
@end table
@end deftp
+@subsubheading Redis
+
@defvr {Scheme Variable} redis-service-type
This is the service type for the @uref{https://redis.io/, Redis}
key/value store, whose value is a @code{redis-configuration} object.
(modules
(list
(file-append nginx-accept-language-module "\
-/etc/nginx/modules/ngx_http_accept_language_module.so")))
+/etc/nginx/modules/ngx_http_accept_language_module.so")
+ (file-append nginx-lua-module "\
+/etc/nginx/modules/ngx_http_lua_module.so")))
+@end lisp
+
+@item @code{lua-package-path} (default: @code{'()})
+List of nginx lua packages to load. This should be a list of package
+names of loadable lua modules, as in this example:
+
+@lisp
+(lua-package-path (list lua-resty-core
+ lua-resty-lrucache
+ lua-resty-signal
+ lua-tablepool
+ lua-resty-shell))
+@end lisp
+
+@item @code{lua-package-cpath} (default: @code{'()})
+List of nginx lua C packages to load. This should be a list of package
+names of loadable lua C modules, as in this example:
+
+@lisp
+(lua-package-cpath (list lua-resty-signal))
@end lisp
@item @code{global-directives} (default: @code{'((events . ()))})
more information on X.509 certificates.
@end quotation
+@subsubheading gmnisrv
+
+@cindex gmnisrv
+The @uref{https://git.sr.ht/~sircmpwn/gmnisrv, gmnisrv} program is a
+simple @uref{https://gemini.circumlunar.space/, Gemini} protocol server.
+
+@deffn {Scheme Variable} gmnisrv-service-type
+This is the type of the gmnisrv service, whose value should be a
+@code{gmnisrv-configuration} object, as in this example:
+
+@lisp
+(service gmnisrv-service-type
+ (gmnisrv-configuration
+ (config-file (local-file "./my-gmnisrv.ini"))))
+@end lisp
+@end deffn
+
+@deftp {Data Type} gmnisrv-configuration
+Data type representing the configuration of gmnisrv.
+
+@table @asis
+@item @code{package} (default: @var{gmnisrv})
+Package object of the gmnisrv server.
+
+@item @code{config-file} (default: @code{%default-gmnisrv-config-file})
+File-like object of the gmnisrv configuration file to use. The default
+configuration listens on port 1965 and serves files from
+@file{/srv/gemini}. Certificates are stored in
+@file{/var/lib/gemini/certs}. For more information, run @command{man
+gmnisrv} and @command{man gmnisrv.ini}.
+
+@end table
+@end deftp
+
@node Certificate Services
@subsection Certificate Services
@table @asis
@item @code{id} (default: @code{""})
-An identifier for ether configuration fields to refer to this key. IDs must be
+An identifier for other configuration fields to refer to this key. IDs must be
unique and must not be empty.
@item @code{address} (default: @code{'()})
@item @code{negative-cache?} (default: @code{#t})
When false, disable negative caching.
+@item @code{tftp-enable?} (default: @code{#f})
+Whether to enable the built-in TFTP server.
+
+@item @code{tftp-no-fail?} (default: @code{#f})
+If true, does not fail dnsmasq if the TFTP server could not start up.
+
+@item @code{tftp-single-port?} (default: @code{#f})
+Whether to use only one single port for TFTP.
+
+@item @code{tftp-secure?} (default: @code{#f})
+If true, only files owned by the user running the dnsmasq process are accessible.
+
+If dnsmasq is being run as root, different rules apply:
+@code{tftp-secure?} has no effect, but only files which have the
+world-readable bit set are accessible.
+
+@item @code{tftp-max} (default: @code{#f})
+If set, sets the maximal number of concurrent connections allowed.
+
+@item @code{tftp-mtu} (default: @code{#f})
+If set, sets the MTU for TFTP packets to that value.
+
+@item @code{tftp-no-blocksize?} (default: @code{#f})
+If true, stops the TFTP server from negotiating the blocksize with a client.
+
+@item @code{tftp-lowercase?} (default: @code{#f})
+Whether to convert all filenames in TFTP requests to lowercase.
+
+@item @code{tftp-port-range} (default: @code{#f})
+If set, fixes the dynamical ports (one per client) to the given range
+(@code{"<start>,<end>"}).
+
+@item @code{tftp-root} (default: @code{/var/empty,lo})
+Look for files to transfer using TFTP relative to the given directory.
+When this is set, TFTP paths which include ".." are rejected, to stop clients
+getting outside the specified root. Absolute paths (starting with /) are
+allowed, but they must be within the tftp-root. If the optional interface
+argument is given, the directory is only used for TFTP requests via that
+interface.
+
+@item @code{tftp-unique-root} (default: @code{#f})
+If set, add the IP or hardware address of the TFTP client as a path component
+on the end of the TFTP-root. Only valid if a TFTP root is set and the
+directory exists. Defaults to adding IP address (in standard dotted-quad
+format).
+
+For instance, if --tftp-root is "/tftp" and client 1.2.3.4 requests file
+"myfile" then the effective path will be "/tftp/1.2.3.4/myfile" if
+/tftp/1.2.3.4 exists or /tftp/myfile otherwise. When "=mac" is specified
+it will append the MAC address instead, using lowercase zero padded digits
+separated by dashes, e.g.: 01-02-03-04-aa-bb Note that resolving MAC
+addresses is only possible if the client is in the local network or obtained
+a DHCP lease from dnsmasq.
+
@end table
@end deftp
effect; this can be used as a trick to implement an external mixer
External Mixer) or no mixer (@code{none}).
-@item @code{extra-options} (default: @code{'()"})
+@item @code{extra-options} (default: @code{'()})
An association list of option symbols to string values to be appended to
the audio output configuration.
@node Virtualization Services
-@subsection Virtualization services
+@subsection Virtualization Services
The @code{(gnu services virtualization)} module provides services for
the libvirt and virtlog daemons, as well as other virtualization-related
services.
@subsubheading Libvirt daemon
+
@code{libvirtd} is the server side daemon component of the libvirt
virtualization management system. This daemon runs on host servers
and performs required management tasks for virtualized guests.
@deftypevr {@code{libvirt-configuration} parameter} boolean listen-tls?
Flag listening for secure TLS connections on the public TCP/IP port.
-must set @code{listen} for this to have any effect.
+You must set @code{listen} for this to have any effect.
It is necessary to setup a CA and issue server certificates before using
this capability.
@end deftypevr
@deftypevr {@code{libvirt-configuration} parameter} boolean listen-tcp?
-Listen for unencrypted TCP connections on the public TCP/IP port. must
+Listen for unencrypted TCP connections on the public TCP/IP port. You must
set @code{listen} for this to have any effect.
Using the TCP socket requires SASL authentication by default. Only SASL
mechanisms which support data encryption are allowed. This is
-DIGEST_MD5 and GSSAPI (Kerberos5)
+DIGEST_MD5 and GSSAPI (Kerberos5).
Defaults to @samp{#f}.
@end deftypevr
@deftypevr {@code{libvirt-configuration} parameter} string tls-port
-Port for accepting secure TLS connections This can be a port number, or
-service name
+Port for accepting secure TLS connections. This can be a port number,
+or service name.
Defaults to @samp{"16514"}.
@end deftypevr
@deftypevr {@code{libvirt-configuration} parameter} string tcp-port
-Port for accepting insecure TCP connections This can be a port number,
-or service name
+Port for accepting insecure TCP connections. This can be a port number,
+or service name.
Defaults to @samp{"16509"}.
Logging filters.
A filter allows to select a different logging level for a given category
-of logs The format for a filter is one of:
+of logs. The format for a filter is one of:
@itemize @bullet
@item
Defaults to @samp{3}
@end deftypevr
-@node Transparent Emulation with QEMU
+
+@anchor{transparent-emulation-qemu}
@subsubheading Transparent Emulation with QEMU
@cindex emulation
it allows you to transparently execute an ARMv7 program on an x86_64
machine. It achieves this by combining the @uref{https://www.qemu.org,
QEMU} emulator and the @code{binfmt_misc} feature of the kernel Linux.
+This feature only allows you to emulate GNU/Linux on a different
+architecture, but see below for GNU/Hurd support.
@defvr {Scheme Variable} qemu-binfmt-service-type
This is the type of the QEMU/binfmt service for transparent emulation.
with forwarded ports:
@example
-@var{ssh-port}: @code{(+ 11004 (* 1000 @var{ID}))}
+@var{secrets-port}: @code{(+ 11004 (* 1000 @var{ID}))}
@var{ssh-port}: @code{(+ 10022 (* 1000 @var{ID}))}
@var{vnc-port}: @code{(+ 15900 (* 1000 @var{ID}))}
@end example
@code{secret-service} running in the Childhurd will be sent an empty
list of secrets.
-Typical use to populate @file{"/etc/childhurd"} with a tree of
-non-volatile secrets, like so
+By default, the service automatically populates @file{/etc/childhurd}
+with the following non-volatile secrets, unless they already exist:
@example
+/etc/childhurd/etc/guix/acl
/etc/childhurd/etc/guix/signing-key.pub
/etc/childhurd/etc/guix/signing-key.sec
/etc/childhurd/etc/ssh/ssh_host_ed25519_key
/etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
@end example
-to be sent to the Childhurd, including permissions.
+These files are automatically sent to the guest Hurd VM when it boots,
+including permissions.
+
+@cindex childhurd, offloading
+@cindex Hurd, offloading
+Having these files in place means that only a couple of things are
+missing to allow the host to offload @code{i586-gnu} builds to the
+childhurd:
+
+@enumerate
+@item
+Authorizing the childhurd's key on the host so that the host accepts
+build results coming from the childhurd, which can be done like so:
+
+@example
+guix archive --authorize < \
+ /etc/childhurd/etc/guix/signing-key.pub
+@end example
+
+@item
+Adding the childhurd to @file{/etc/guix/machines.scm} (@pxref{Daemon
+Offload Setup}).
+@end enumerate
+We're working towards making that happen automatically---get in touch
+with us at @email{guix-devel@@gnu.org} to discuss it!
@end table
@end deftp
@defvr {Scheme Variable} ganeti-watcher-service-type
@command{ganeti-watcher} is a script designed to run periodically and ensure
the health of a cluster. It will automatically restart instances that have
-stopped without Ganetis consent, and repairs DRBD links in case a node has
+stopped without Ganeti's consent, and repairs DRBD links in case a node has
rebooted. It also archives old cluster jobs and restarts Ganeti daemons
that are not running. If the cluster parameter @code{ensure_node_health}
is set, the watcher will also shutdown instances and DRBD devices if the
@file{/etc/security/pam_mount.conf.xml}.
The configuration rules are SXML elements (@pxref{SXML,,, guile, GNU
-Guile Reference Manual}), and the the default ones don't mount anything
-for anyone at login:
+Guile Reference Manual}), and the default ones don't mount anything for
+anyone at login:
@lisp
`((debug (@@ (enable "0")))
@node Guix Services
@subsection Guix Services
+@subsubheading Guix Build Coordinator
+The @uref{https://git.cbaines.net/guix/build-coordinator/,Guix Build
+Coordinator} aids in distributing derivation builds among machines
+running an @dfn{agent}. The build daemon is still used to build the
+derivations, but the Guix Build Coordinator manages allocating builds
+and working with the results.
+
+@quotation Note
+This service is considered experimental. Configuration options may be
+changed in a backwards-incompatible manner, and not all features have
+been thorougly tested.
+@end quotation
+
+The Guix Build Coordinator consists of one @dfn{coordinator}, and one or
+more connected @dfn{agent} processes. The coordinator process handles
+clients submitting builds, and allocating builds to agents. The agent
+processes talk to a build daemon to actually perform the builds, then
+send the results back to the coordinator.
+
+There is a script to run the coordinator component of the Guix Build
+Coordinator, but the Guix service uses a custom Guile script instead, to
+provide better integration with G-expressions used in the configuration.
+
+@defvar {Scheme Variable} guix-build-coordinator-service-type
+Service type for the Guix Build Coordinator. Its value must be a
+@code{guix-build-coordinator-configuration} object.
+@end defvar
+
+@deftp {Data Type} guix-build-coordinator-configuration
+Data type representing the configuration of the Guix Build Coordinator.
+
+@table @asis
+@item @code{package} (default: @code{guix-build-coordinator})
+The Guix Build Coordinator package to use.
+
+@item @code{user} (default: @code{"guix-build-coordinator"})
+The system user to run the service as.
+
+@item @code{group} (default: @code{"guix-build-coordinator"})
+The system group to run the service as.
+
+@item @code{database-uri-string} (default: @code{"sqlite:///var/lib/guix-build-coordinator/guix_build_coordinator.db"})
+The URI to use for the database.
+
+@item @code{agent-communication-uri} (default: @code{"http://0.0.0.0:8745"})
+The URI describing how to listen to requests from agent processes.
+
+@item @code{client-communication-uri} (default: @code{"http://127.0.0.1:8746"})
+The URI describing how to listen to requests from clients. The client
+API allows submitting builds and currently isn't authenticated, so take
+care when configuring this value.
+
+@item @code{allocation-strategy} (default: @code{#~basic-build-allocation-strategy})
+A G-expression for the allocation strategy to be used. This is a
+procedure that takes the datastore as an argument and populates the
+allocation plan in the database.
+
+@item @code{hooks} (default: @var{'()})
+An association list of hooks. These provide a way to execute arbitrary
+code upon certain events, like a build result being processed.
+
+@item @code{guile} (default: @code{guile-3.0-latest})
+The Guile package with which to run the Guix Build Coordinator.
+
+@end table
+@end deftp
+
+@defvar {Scheme Variable} guix-build-coordinator-agent-service-type
+Service type for a Guix Build Coordinator agent. Its value must be a
+@code{guix-build-coordinator-agent-configuration} object.
+@end defvar
+
+@deftp {Data Type} guix-build-coordinator-agent-configuration
+Data type representing the configuration a Guix Build Coordinator agent.
+
+@table @asis
+@item @code{package} (default: @code{guix-build-coordinator})
+The Guix Build Coordinator package to use.
+
+@item @code{user} (default: @code{"guix-build-coordinator-agent"})
+The system user to run the service as.
+
+@item @code{coordinator} (default: @code{"http://localhost:8745"})
+The URI to use when connecting to the coordinator.
+
+@item @code{uuid}
+The UUID of the agent. This should be generated by the coordinator
+process, stored in the coordinator database, and used by the intended
+agent.
+
+@item @code{password} (default: @code{#f})
+The password to use when connecting to the coordinator. A file to read
+the password from can also be specified, and this is more secure.
+
+@item @code{password-file} (default: @code{#f})
+A file containing the password to use when connecting to the
+coordinator.
+
+@item @code{systems} (default: @var{#f})
+The systems for which this agent should fetch builds. The agent process
+will use the current system it's running on as the default.
+
+@item @code{max-parallel-builds} (default: @code{1})
+The number of builds to perform in parallel.
+
+@item @code{derivation-substitute-urls} (default: @code{1})
+URLs from which to attempt to fetch substitutes for derivations, if the
+derivations aren't already available.
+
+@item @code{non-derivation-substitute-urls} (default: @code{1})
+URLs from which to attempt to fetch substitutes for build inputs, if the
+input store items aren't already available.
+
+@end table
+@end deftp
+
+The Guix Build Coordinator package contains a script to query an
+instance of the Guix Data Service for derivations to build, and then
+submit builds for those derivations to the coordinator. The service
+type below assists in running this script. This is an additional tool
+that may be useful when building derivations contained within an
+instance of the Guix Data Service.
+
+@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-type
+Service type for the
+guix-build-coordinator-queue-builds-from-guix-data-service script. Its
+value must be a @code{guix-build-coordinator-queue-builds-configuration}
+object.
+@end defvar
+
+@deftp {Data Type} guix-build-coordinator-queue-builds-configuration
+Data type representing the options to the queue builds from guix data
+service script.
+
+@table @asis
+@item @code{package} (default: @code{guix-build-coordinator})
+The Guix Build Coordinator package to use.
+
+@item @code{user} (default: @code{"guix-build-coordinator-queue-builds"})
+The system user to run the service as.
+
+@item @code{coordinator} (default: @code{"http://localhost:8745"})
+The URI to use when connecting to the coordinator.
+
+@item @code{systems} (default: @code{#f})
+The systems for which to fetch derivations to build.
+
+@item @code{systems-and-targets} (default: @code{#f})
+An association list of system and target pairs for which to fetch
+derivations to build.
+
+@item @code{guix-data-service} (default: @code{"https://data.guix.gnu.org"})
+The Guix Data Service instance from which to query to find out about
+derivations to build.
+
+@item @code{processed-commits-file} (default: @code{"/var/cache/guix-build-coordinator-queue-builds/processed-commits"})
+A file to record which commits have been processed, to avoid needlessly
+processing them again if the service is restarted.
+
+@end table
+@end deftp
+
@subsubheading Guix Data Service
The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores
and provides data about GNU Guix. This includes information about
service.
@table @asis
-@item @code{size} (default @var{"1G"})
+@item @code{size} (default @code{"1G"})
This is the amount of space you wish to provide for the zram device. It
accepts a string and can be a number of bytes or use a suffix, eg.:
-@var{"512M"} or @var{1024000}.
-@item @code{compression-algorithm} (default @var{'lzo})
+@code{"512M"} or @code{1024000}.
+@item @code{compression-algorithm} (default @code{'lzo})
This is the compression algorithm you wish to use. It is difficult to
list all the possible compression options, but common ones supported by
-Guix's Linux Libre Kernel include @var{'lzo}, @var{'lz4} and @var{'zstd}.
-@item @code{memory-limit} (default @var{0})
+Guix's Linux Libre Kernel include @code{'lzo}, @code{'lz4} and @code{'zstd}.
+@item @code{memory-limit} (default @code{0})
This is the maximum amount of memory which the zram device can use.
Setting it to '0' disables the limit. While it is generally expected
that compression will be 2:1, it is possible that uncompressable data
can be written to swap and this is a method to limit how much memory can
be used. It accepts a string and can be a number of bytes or use a
-suffix, eg.: @var{"2G"}.
-@item @code{priority} (default @var{-1})
+suffix, eg.: @code{"2G"}.
+@item @code{priority} (default @code{-1})
This is the priority of the swap device created from the zram device.
@code{swapon} accepts values between -1 and 32767, with higher values
indicating higher priority. Higher priority swap will generally be used
@end table
@end deftp
+@cindex Singularity, container service
+@defvr {Scheme Variable} singularity-service-type
+This is the type of the service that allows you to run
+@url{https://www.sylabs.io/singularity/, Singularity}, a Docker-style tool to
+create and run application bundles (aka. ``containers''). The value for this
+service is the Singularity package to use.
+
+The service does not install a daemon; instead, it installs helper programs as
+setuid-root (@pxref{Setuid Programs}) such that unprivileged users can invoke
+@command{singularity run} and similar commands.
+@end defvr
+
@cindex Audit
@subsubheading Auditd Service
@end table
@end deftp
-@defvr {Scheme Variable} singularity-service-type
-This is the type of the service that allows you to run
-@url{https://www.sylabs.io/singularity/, Singularity}, a Docker-style tool to
-create and run application bundles (aka. ``containers''). The value for this
-service is the Singularity package to use.
-
-The service does not install a daemon; instead, it installs helper programs as
-setuid-root (@pxref{Setuid Programs}) such that unprivileged users can invoke
-@command{singularity run} and similar commands.
-@end defvr
-
@cindex rshiny
@subsubheading R-Shiny service
@defvr {Scheme Variable} rshiny-service-type
This is a type of service which is used to run a webapp created with
-@code{r-shiny}. This service sets the @code{R_LIBS_USER} environment
+@code{r-shiny}. This service sets the @env{R_LIBS_USER} environment
variable and runs the provided script to call @code{runApp}.
@deftp {Data Type} rshiny-configuration
@table @asis
@item @code{gfxmode} (default: @code{'("auto")})
-The GRUB @code{gfxmode} to set (a list of screen resolution strings, see
+The GRUB @code{gfxmode} to set (a list of screen resolution strings,
@pxref{gfxmode,,, grub, GNU GRUB manual}).
@end table
@end deftp
the @option{--image-size} option is ignored in the case of
@code{docker-image}.
-You can specify the root file system type by using the
-@option{--file-system-type} option. It defaults to @code{ext4}. When its
-value is @code{iso9660}, the @option{--label} option can be used to specify
-a volume ID with @code{disk-image}.
-
-When using @code{vm-image}, the returned image is in qcow2 format, which
-the QEMU emulator can efficiently use. @xref{Running Guix in a VM},
-for more information on how to run the image in a virtual machine.
+The @code{disk-image} command can produce various image types. The
+image type can be selected using the @command{--image-type} option. It
+defaults to @code{raw}. When its value is @code{iso9660}, the
+@option{--label} option can be used to specify a volume ID with
+@code{disk-image}.
-When using @code{disk-image}, a raw disk image is produced; it can be
-copied as is to a USB stick, for instance. Assuming @code{/dev/sdc} is
-the device corresponding to a USB stick, one can copy the image to it
-using the following command:
+When using the @code{raw} image type, a raw disk image is produced; it
+can be copied as is to a USB stick, for instance. Assuming
+@code{/dev/sdc} is the device corresponding to a USB stick, one can copy
+the image to it using the following command:
@example
# dd if=$(guix system disk-image my-os.scm) of=/dev/sdc status=progress
@end example
+The @code{--list-image-types} command lists all the available image
+types.
+
+When using @code{vm-image}, the returned image is in qcow2 format, which
+the QEMU emulator can efficiently use. @xref{Running Guix in a VM},
+for more information on how to run the image in a virtual machine.
+
When using @code{docker-image}, a Docker image is produced. Guix builds
the image from scratch, not from a pre-existing Docker base image. As a
result, it contains @emph{exactly} what you define in the operating
what it pretends to contain; or they could use that to derive a variant
of the image.
-@item --file-system-type=@var{type}
+@item --image-type=@var{type}
@itemx -t @var{type}
-For the @code{disk-image} action, create a file system of the given
-@var{type} on the image.
+For the @code{disk-image} action, create an image with given @var{type}.
-When this option is omitted, @command{guix system} uses @code{ext4}.
+When this option is omitted, @command{guix system} uses the @code{raw}
+image type.
@cindex ISO-9660 format
@cindex CD image format
@cindex DVD image format
-@option{--file-system-type=iso9660} produces an ISO-9660 image, suitable
+@option{--image-type=iso9660} produces an ISO-9660 image, suitable
for burning on CDs and DVDs.
@item --image-size=@var{size}
Do note that you first need to generate a key pair on the coordinator machine
to allow the daemon to export signed archives of files from the store
-(@pxref{Invoking guix archive}).
+(@pxref{Invoking guix archive}), though this step is automatic on Guix
+System:
@example
# guix archive --generate-key
as QEMU (see below for details).
This image boots the Xfce graphical environment and it contains some
-commonly-used tools. You can install more software in the image by running
+commonly used tools. You can install more software in the image by running
@command{guix package} in a terminal (@pxref{Invoking guix package}). You can
also reconfigure the system based on its initial configuration file available
as @file{/run/current-system/configuration.scm} (@pxref{Using the
QEMU and KVM documentation for more info.
@item -drive if=none,file=/tmp/qemu-image,id=myhd
-Use our QCOW image, the @file{/tmp/qemu-image} file, as the backing store the
-the ``myhd'' drive.
+Use our QCOW image, the @file{/tmp/qemu-image} file, as the backing
+store of the ``myhd'' drive.
@end table
The default @command{run-vm.sh} script that is returned by an invocation of
(service guix-service-type
(guix-configuration
(build-accounts 5)
- (use-substitutes? #f)))
+ (extra-options '("--gc-keep-derivations"))))
@end lisp
The second argument to the @code{service} form is a value representing
debugger, GDB, to map binary code to source code; it is required to
debug a compiled program in good conditions.
+This chapter explains how to use separate debug info when packages
+provide it, and how to rebuild packages with debug info when it's
+missing.
+
+@menu
+* Separate Debug Info:: Installing 'debug' outputs.
+* Rebuilding Debug Info:: Building missing debug info.
+@end menu
+
+@node Separate Debug Info
+@section Separate Debug Info
+
The problem with debugging information is that is takes up a fair amount
of disk space. For example, debugging information for the GNU C Library
weighs in at more than 60 MiB. Thus, as a user, keeping all the
@c XXX: keep me up-to-date
The @code{debug} output mechanism in Guix is implemented by the
@code{gnu-build-system} (@pxref{Build Systems}). Currently, it is
-opt-in---debugging information is available only for the packages
-with definitions explicitly declaring a @code{debug} output. This may be
-changed to opt-out in the future if our build farm servers can handle
-the load. To check whether a package has a @code{debug} output, use
-@command{guix package --list-available} (@pxref{Invoking guix package}).
+opt-in---debugging information is available only for the packages with
+definitions explicitly declaring a @code{debug} output. To check
+whether a package has a @code{debug} output, use @command{guix package
+--list-available} (@pxref{Invoking guix package}).
+
+Read on for how to deal with packages lacking a @code{debug} output.
+
+@node Rebuilding Debug Info
+@section Rebuilding Debug Info
+
+@cindex debugging info, rebuilding
+As we saw above, some packages, but not all, provide debugging info in a
+@code{debug} output. What can you do when debugging info is missing?
+The @option{--with-debug-info} option provides a solution to that: it
+allows you to rebuild the package(s) for which debugging info is
+missing---and only those---and to graft those onto the application
+you're debugging. Thus, while it's not as fast as installing a
+@code{debug} output, it is relatively inexpensive.
+
+Let's illustrate that. Suppose you're experiencing a bug in Inkscape
+and would like to see what's going on in GLib, a library that's deep
+down in its dependency graph. As it turns out, GLib does not have a
+@code{debug} output and the backtrace GDB shows is all sadness:
+
+@example
+(gdb) bt
+#0 0x00007ffff5f92190 in g_getenv ()
+ from /gnu/store/@dots{}-glib-2.62.6/lib/libglib-2.0.so.0
+#1 0x00007ffff608a7d6 in gobject_init_ctor ()
+ from /gnu/store/@dots{}-glib-2.62.6/lib/libgobject-2.0.so.0
+#2 0x00007ffff7fe275a in call_init (l=<optimized out>, argc=argc@@entry=1, argv=argv@@entry=0x7fffffffcfd8,
+ env=env@@entry=0x7fffffffcfe8) at dl-init.c:72
+#3 0x00007ffff7fe2866 in call_init (env=0x7fffffffcfe8, argv=0x7fffffffcfd8, argc=1, l=<optimized out>)
+ at dl-init.c:118
+@end example
+
+To address that, you install Inkscape linked against a variant GLib that
+contains debug info:
+
+@example
+guix install inkscape --with-debug-info=glib
+@end example
+
+This time, debugging will be a whole lot nicer:
+
+@example
+$ gdb --args sh -c 'exec inkscape'
+@dots{}
+(gdb) b g_getenv
+Function "g_getenv" not defined.
+Make breakpoint pending on future shared library load? (y or [n]) y
+Breakpoint 1 (g_getenv) pending.
+(gdb) r
+Starting program: /gnu/store/@dots{}-profile/bin/sh -c exec\ inkscape
+@dots{}
+(gdb) bt
+#0 g_getenv (variable=variable@@entry=0x7ffff60c7a2e "GOBJECT_DEBUG") at ../glib-2.62.6/glib/genviron.c:252
+#1 0x00007ffff608a7d6 in gobject_init () at ../glib-2.62.6/gobject/gtype.c:4380
+#2 gobject_init_ctor () at ../glib-2.62.6/gobject/gtype.c:4493
+#3 0x00007ffff7fe275a in call_init (l=<optimized out>, argc=argc@@entry=3, argv=argv@@entry=0x7fffffffd088,
+ env=env@@entry=0x7fffffffd0a8) at dl-init.c:72
+@dots{}
+@end example
+
+Much better!
+Note that there can be packages for which @option{--with-debug-info}
+will not have the desired effect. @xref{Package Transformation Options,
+@option{--with-debug-info}}, for more information.
@node Security Updates
@chapter Security Updates
HCoop / jackhill / guix / guix.git / blobdiff