gnu: Add LAPACK.
[jackhill/guix/guix.git] / doc / guix.texi
CommitLineData
568717fd
LC
1\input texinfo
2@c -*-texinfo-*-
3
4@c %**start of header
5@setfilename guix.info
6@documentencoding UTF-8
f8348b91 7@settitle GNU Guix Reference Manual
568717fd
LC
8@c %**end of header
9
10@include version.texi
7df7a74e
NK
11
12@copying
13Copyright @copyright{} 2012, 2013 Ludovic Courtès@*
14Copyright @copyright{} 2013 Andreas Enge@*
15Copyright @copyright{} 2013 Nikita Karetnikov
16
17Permission is granted to copy, distribute and/or modify this document
18under the terms of the GNU Free Documentation License, Version 1.3 or
19any later version published by the Free Software Foundation; with no
20Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
21copy of the license is included in the section entitled ``GNU Free
22Documentation License''.
23@end copying
568717fd 24
eeaf4427 25@dircategory Package management
568717fd
LC
26@direntry
27* guix: (guix). Guix, the functional package manager.
e49951eb 28* guix package: (guix)Invoking guix package
eeaf4427 29 Managing packages with Guix.
e49951eb 30* guix build: (guix)Invoking guix build
568717fd
LC
31 Building packages with Guix.
32@end direntry
568717fd
LC
33
34@titlepage
7730d112
LC
35@title GNU Guix Reference Manual
36@subtitle Using the GNU Guix Functional Package Manager
568717fd 37@author Ludovic Courtès
da7cabd4 38@author Andreas Enge
acc08466 39@author Nikita Karetnikov
568717fd
LC
40
41@page
42@vskip 0pt plus 1filll
43Edition @value{EDITION} @*
44@value{UPDATED} @*
45
7df7a74e 46@insertcopying
568717fd
LC
47@end titlepage
48
568717fd
LC
49@contents
50
51@c *********************************************************************
52@node Top
f8348b91 53@top GNU Guix
568717fd 54
f8348b91
LC
55This document describes GNU Guix version @value{VERSION}, a functional
56package management tool written for the GNU system.
568717fd
LC
57
58@menu
59* Introduction:: What is Guix about?
bd5e766b 60* Installation:: Installing Guix.
eeaf4427 61* Package Management:: Package installation, upgrade, etc.
568717fd
LC
62* Programming Interface:: Using Guix in Scheme.
63* Utilities:: Package management commands.
a1ba8475 64* GNU Distribution:: Software for your friendly GNU system.
9bf3c1a7 65* Contributing:: Your help needed!
568717fd
LC
66
67* Acknowledgments:: Thanks!
68* GNU Free Documentation License:: The license of this manual.
69* Concept Index:: Concepts.
70* Function Index:: Functions.
71@end menu
72
73@c *********************************************************************
74@node Introduction
75@chapter Introduction
76
c80e7e55
LC
77GNU Guix@footnote{``Guix'' is pronounced like ``geeks'', or ``ɡiːks''
78using the international phonetic alphabet (IPA).} is a functional
79package management tool for the GNU system. Package management consists
4bfc4ea3
NK
80of all activities that relate to building packages from sources,
81honoring their build-time and run-time dependencies,
c80e7e55
LC
82installing packages in user environments, upgrading installed packages
83to new versions or rolling back to a previous set, removing unused
84software packages, etc.
568717fd
LC
85
86@cindex functional package management
87The term @dfn{functional} refers to a specific package management
88discipline. In Guix, the package build and installation process is seen
4bfc4ea3
NK
89as a function, in the mathematical sense. That function takes inputs,
90such as build scripts, a compiler, and libraries, and
91returns an installed package. As a pure function, its result depends
568717fd
LC
92solely on its inputs---for instance, it cannot refer to software or
93scripts that were not explicitly passed as inputs. A build function
4bfc4ea3
NK
94always produces the same result when passed a given set of inputs. It
95cannot alter the system's environment in
568717fd
LC
96any way; for instance, it cannot create, modify, or delete files outside
97of its build and installation directories. This is achieved by running
4bfc4ea3
NK
98build processes in isolated environments (or @dfn{chroots}), where only their
99explicit inputs are visible.
568717fd 100
e531ac2a 101@cindex store
568717fd 102The result of package build functions is @dfn{cached} in the file
e531ac2a
LC
103system, in a special directory called @dfn{the store} (@pxref{The
104Store}). Each package is installed in a directory of its own, in the
568717fd
LC
105store---by default under @file{/nix/store}. The directory name contains
106a hash of all the inputs used to build that package; thus, changing an
107input yields a different directory name.
108
109This approach is the foundation of Guix's salient features: support for
4bfc4ea3 110transactional package upgrade and rollback, per-user installation, and
eeaf4427 111garbage collection of packages (@pxref{Features}).
568717fd 112
4bfc4ea3 113Guix has a command-line interface, which allows users to build, install,
568717fd 114upgrade, and remove packages, as well as a Scheme programming interface.
568717fd 115
a1ba8475
LC
116Last but not least, Guix is used to build a distribution of the GNU
117system, with many GNU and non-GNU free software packages. @xref{GNU
118Distribution}.
119
bd5e766b
LC
120@c *********************************************************************
121@node Installation
122@chapter Installation
123
48febeb8
LC
124GNU Guix is available for download from its website at
125@url{http://www.gnu.org/software/guix/}. This section describes the
126software requirements of Guix, as well as how to install it and get
127ready to use it.
bd5e766b 128
b22a12fd 129The build procedure for Guix is the same as for other GNU software, and
1da983b9 130is not covered here. Please see the files @file{README} and
b22a12fd
LC
131@file{INSTALL} in the Guix source tree for additional details.
132
bd5e766b
LC
133@menu
134* Requirements:: Software needed to build and run Guix.
135* Setting Up the Daemon:: Preparing the build daemon's environment.
136* Invoking guix-daemon:: Running the build daemon.
137@end menu
138
139@node Requirements
140@section Requirements
141
142GNU Guix depends on the following packages:
143
144@itemize
4a328f73 145@item @url{http://gnu.org/software/guile/, GNU Guile}, version 2.0.5 or later;
bd5e766b
LC
146@item @url{http://gnupg.org/, GNU libgcrypt}
147@end itemize
148
149Unless @code{--disable-daemon} was passed to @command{configure}, the
150following packages are also needed:
151
152@itemize
153@item @url{http://sqlite.org, SQLite 3}
154@item @url{http://www.bzip.org, libbz2}
155@item @url{http://gcc.gnu.org, GCC's g++}
156@end itemize
157
4bfc4ea3
NK
158When a working installation of @url{http://nixos.org/nix/, the Nix package
159manager} is available, you
bd5e766b 160can instead configure Guix with @code{--disable-daemon}. In that case,
4bfc4ea3 161Nix replaces the three dependencies above.
bd5e766b 162
b22a12fd
LC
163Guix is compatible with Nix, so it is possible to share the same store
164between both. To do so, you must pass @command{configure} not only the
165same @code{--with-store-dir} value, but also the same
4bfc4ea3
NK
166@code{--localstatedir} value. The latter is essential because it
167specifies where the database that stores metadata about the store is
168located, among other things. The default values are
b22a12fd 169@code{--with-store-dir=/nix/store} and @code{--localstatedir=/nix/var}.
4bfc4ea3
NK
170Note that @code{--disable-daemon} is not required if
171your goal is to share the store with Nix.
b22a12fd 172
bd5e766b
LC
173@node Setting Up the Daemon
174@section Setting Up the Daemon
175
176@cindex daemon
177Operations such as building a package or running the garbage collector
178are all performed by a specialized process, the @dfn{Guix daemon}, on
179behalf of clients. Only the daemon may access the store and its
180associated database. Thus, any operation that manipulates the store
181goes through the daemon. For instance, command-line tools such as
e49951eb 182@command{guix package} and @command{guix build} communicate with the
bd5e766b
LC
183daemon (@i{via} remote procedure calls) to instruct it what to do.
184
185In a standard multi-user setup, Guix and its daemon---the
186@command{guix-daemon} program---are installed by the system
187administrator; @file{/nix/store} is owned by @code{root} and
188@command{guix-daemon} runs as @code{root}. Unprivileged users may use
189Guix tools to build packages or otherwise access the store, and the
190daemon will do it on their behalf, ensuring that the store is kept in a
191consistent state, and allowing built packages to be shared among users.
192
193@cindex build users
194When @command{guix-daemon} runs as @code{root}, you may not want package
195build processes themselves to run as @code{root} too, for obvious
196security reasons. To avoid that, a special pool of @dfn{build users}
197should be created for use by build processes started by the daemon.
198These build users need not have a shell and a home directory: they will
199just be used when the daemon drops @code{root} privileges in build
200processes. Having several such users allows the daemon to launch
201distinct build processes under separate UIDs, which guarantees that they
202do not interfere with each other---an essential feature since builds are
203regarded as pure functions (@pxref{Introduction}).
204
205On a GNU/Linux system, a build user pool may be created like this (using
206Bash syntax and the @code{shadow} commands):
207
091196b3
LC
208@c See http://lists.gnu.org/archive/html/bug-guix/2013-01/msg00239.html
209@c for why `-G' is needed.
bd5e766b
LC
210@example
211# groupadd guix-builder
212# for i in `seq 1 10`;
213 do
091196b3
LC
214 useradd -g guix-builder -G guix-builder \
215 -d /var/empty -s `which nologin` \
80ba8cc0 216 -c "Guix build user $i" guix-builder$i;
bd5e766b
LC
217 done
218@end example
219
220@noindent
221The @code{guix-daemon} program may then be run as @code{root} with:
222
223@example
224# guix-daemon --build-users-group=guix-builder
225@end example
226
b095792f
LC
227@noindent
228This way, the daemon starts build processes in a chroot, under one of
229the @code{guix-builder} users. On GNU/Linux, by default, the chroot
230environment contains nothing but the @code{/dev} and @code{/proc}
231directories@footnote{On some systems @code{/dev/shm}, which supports
232shared memory, is a symlink to another directory such as
233@code{/run/shm}, that is @emph{not} is the chroot. When that is the
234case, shared memory support is unavailable in the chroot environment.
235The workaround is to make sure that @file{/dev/shm} is directly a
236@code{tmpfs} mount point.}.
237
bd5e766b 238Guix may also be used in a single-user setup, with @command{guix-daemon}
1da983b9 239running as an unprivileged user. However, to maximize non-interference
bd5e766b
LC
240of build processes, the daemon still needs to perform certain operations
241that are restricted to @code{root} on GNU/Linux: it should be able to
242run build processes in a chroot, and to run them under different UIDs.
243To that end, the @command{nix-setuid-helper} program is provided; it is
244a small C program (less than 300 lines) that, if it is made setuid
245@code{root}, can be executed by the daemon to perform these operations
246on its behalf. The @code{root}-owned @file{/etc/nix-setuid.conf} file
247is read by @command{nix-setuid-helper}; it should contain exactly two
248words: the user name under which the authorized @command{guix-daemon}
249runs, and the name of the build users group.
250
251If you are installing Guix as an unprivileged user and do not have the
252ability to make @file{nix-setuid-helper} setuid-@code{root}, it is still
253possible to run @command{guix-daemon}. However, build processes will
254not be isolated from one another, and not from the rest of the system.
255Thus, build processes may interfere with each other, and may access
256programs, libraries, and other files available on the system---making it
257much harder to view them as @emph{pure} functions.
258
259@node Invoking guix-daemon
260@section Invoking @command{guix-daemon}
261
262The @command{guix-daemon} program implements all the functionality to
263access the store. This includes launching build processes, running the
264garbage collector, querying the availability of a build result, etc. It
265is normally run as @code{root} like this:
266
267@example
268# guix-daemon --build-users-group=guix-builder
269@end example
270
271@noindent
272For details on how to set it up, @ref{Setting Up the Daemon}.
273
274By default, @command{guix-daemon} launches build processes under
275different UIDs, taken from the build group specified with
276@code{--build-users-group}. In addition, each build process is run in a
277chroot environment that only contains the subset of the store that the
278build process depends on, as specified by its derivation
279(@pxref{Programming Interface, derivation}), plus a set of specific
280system directories. By default, the latter contains @file{/dev} and
281@file{/dev/pts}.
282
283The following command-line options are supported:
284
285@table @code
286@item --build-users-group=@var{group}
287Take users from @var{group} to run build processes (@pxref{Setting Up
288the Daemon, build users}).
289
6858f9d1
LC
290@item --no-substitutes
291Do not use substitutes for build products. That is, always build things
292locally instead of allowing downloads of pre-built binaries.
293
bd5e766b
LC
294@item --cache-failures
295Cache build failures. By default, only successful builds are cached.
296
297@item --cores=@var{n}
298@itemx -c @var{n}
299Use @var{n} CPU cores to build each derivation; @code{0} means as many
300as available.
301
302The default value is @code{1}, but it may be overridden by clients, such
e49951eb
MW
303as the @code{--cores} option of @command{guix build} (@pxref{Invoking
304guix build}).
bd5e766b
LC
305
306The effect is to define the @code{NIX_BUILD_CORES} environment variable
307in the build process, which can then use it to exploit internal
308parallelism---for instance, by running @code{make -j$NIX_BUILD_CORES}.
309
310@item --max-jobs=@var{n}
311@itemx -M @var{n}
312Allow at most @var{n} build jobs in parallel. The default value is
313@code{1}.
314
315@item --debug
316Produce debugging output.
317
318This is useful to debug daemon start-up issues, but then it may be
319overridden by clients, for example the @code{--verbosity} option of
e49951eb 320@command{guix build} (@pxref{Invoking guix build}).
bd5e766b
LC
321
322@item --chroot-directory=@var{dir}
323Add @var{dir} to the build chroot.
324
325Doing this may change the result of build processes---for instance if
326they use optional dependencies found in @var{dir} when it is available,
327and not otherwise. For that reason, it is not recommended to do so.
328Instead, make sure that each derivation declares all the inputs that it
329needs.
330
331@item --disable-chroot
332Disable chroot builds.
333
334Using this option is not recommended since, again, it would allow build
335processes to gain access to undeclared dependencies.
336
337@item --disable-log-compression
338Disable compression of the build logs.
339
1da983b9
LC
340Unless @code{--lose-logs} is used, all the build logs are kept in the
341@var{localstatedir}. To save space, the daemon automatically compresses
342them with bzip2 by default. This option disables that.
343
bd5e766b
LC
344@item --disable-store-optimization
345Disable automatic file ``deduplication'' in the store.
346
1da983b9
LC
347By default, files added to the store are automatically ``deduplicated'':
348if a newly added file is identical as another one found in the store,
349the daemon makes the new file a hard link to the other file. This
350slightly increases the input/output load at the end of a build process.
351This option disables this.
352
bd5e766b
LC
353@item --impersonate-linux-2.6
354On Linux-based systems, impersonate Linux 2.6. This means that the
355kernel's @code{uname} system call will report 2.6 as the release number.
356
357This might be helpful to build programs that (usually wrongfully) depend
358on the kernel version number.
359
360@item --lose-logs
361Do not keep build logs. By default they are kept under
362@code{@var{localstatedir}/nix/log}.
363
364@item --system=@var{system}
365Assume @var{system} as the current system type. By default it is the
366architecture/kernel pair found at configure time, such as
367@code{x86_64-linux}.
b8d2aa26
LC
368
369@item --listen=@var{socket}
370Listen for connections on @var{socket}, the file name of a Unix-domain
371socket. The default socket is
372@file{@var{localstatedir}/daemon-socket/socket}. This option is only
373useful in exceptional circumstances, such as if you need to run several
374daemons on the same machine.
bd5e766b
LC
375@end table
376
377
eeaf4427
LC
378@c *********************************************************************
379@node Package Management
380@chapter Package Management
381
f8348b91 382The purpose of GNU Guix is to allow users to easily install, upgrade, and
eeaf4427
LC
383remove software packages, without having to know about their build
384procedure or dependencies. Guix also goes beyond this obvious set of
385features.
386
387This chapter describes the main features of Guix, as well as the package
388management tools it provides.
389
390@menu
391* Features:: How Guix will make your life brighter.
e49951eb 392* Invoking guix package:: Package installation, removal, etc.
6e721c4d 393* Packages with Multiple Outputs:: Single source package, multiple outputs.
e49951eb 394* Invoking guix gc:: Running the garbage collector.
f651b477 395* Invoking guix pull:: Fetching the latest Guix and distribution.
eeaf4427
LC
396@end menu
397
398@node Features
399@section Features
400
401When using Guix, each package ends up in the @dfn{package store}, in its
402own directory---something that resembles
403@file{/nix/store/xxx-package-1.2}, where @code{xxx} is a base32 string.
404
405Instead of referring to these directories, users have their own
406@dfn{profile}, which points to the packages that they actually want to
821b0015
LC
407use. These profiles are stored within each user's home directory, at
408@code{$HOME/.guix-profile}.
eeaf4427 409
821b0015 410For example, @code{alice} installs GCC 4.7.2. As a result,
eeaf4427 411@file{/home/alice/.guix-profile/bin/gcc} points to
821b0015
LC
412@file{/nix/store/@dots{}-gcc-4.7.2/bin/gcc}. Now, on the same machine,
413@code{bob} had already installed GCC 4.8.0. The profile of @code{bob}
414simply continues to point to
415@file{/nix/store/@dots{}-gcc-4.8.0/bin/gcc}---i.e., both versions of GCC
416coexist on the same system without any interference.
eeaf4427 417
e49951eb
MW
418The @command{guix package} command is the central tool to manage
419packages (@pxref{Invoking guix package}). It operates on those per-user
821b0015 420profiles, and can be used @emph{with normal user privileges}.
eeaf4427
LC
421
422The command provides the obvious install, remove, and upgrade
423operations. Each invocation is actually a @emph{transaction}: either
ba55b1cb 424the specified operation succeeds, or nothing happens. Thus, if the
e49951eb 425@command{guix package} process is terminated during the transaction,
eeaf4427
LC
426or if a power outage occurs during the transaction, then the user's
427profile remains in its previous state, and remains usable.
428
429In addition, any package transaction may be @emph{rolled back}. So, if,
430for example, an upgrade installs a new version of a package that turns
431out to have a serious bug, users may roll back to the previous instance
432of their profile, which was known to work well.
433
434All those packages in the package store may be @emph{garbage-collected}.
435Guix can determine which packages are still referenced by the user
fe8ff028 436profiles, and remove those that are provably no longer referenced
e49951eb 437(@pxref{Invoking guix gc}). Users may also explicitly remove old
fe8ff028
LC
438generations of their profile so that the packages they refer to can be
439collected.
eeaf4427
LC
440
441Finally, Guix takes a @dfn{purely functional} approach to package
442management, as described in the introduction (@pxref{Introduction}).
443Each @file{/nix/store} package directory name contains a hash of all the
444inputs that were used to build that package---compiler, libraries, build
445scripts, etc. This direct correspondence allows users to make sure a
446given package installation matches the current state of their
a1ba8475 447distribution, and helps maximize @dfn{reproducibility}.
eeaf4427
LC
448
449This foundation allows Guix to support @dfn{transparent binary/source
3b78d1ea 450deployment}. When a pre-built binary for a @file{/nix/store} path is
eeaf4427
LC
451available from an external source, Guix just downloads it; otherwise, it
452builds the package from source, locally.
453
e49951eb
MW
454@node Invoking guix package
455@section Invoking @command{guix package}
eeaf4427 456
e49951eb 457The @command{guix package} command is the tool that allows users to
eeaf4427
LC
458install, upgrade, and remove packages, as well as rolling back to
459previous configurations. It operates only on the user's own profile,
460and works with normal user privileges (@pxref{Features}). Its syntax
461is:
462
463@example
e49951eb 464guix package @var{options}
eeaf4427
LC
465@end example
466
ba55b1cb 467Primarily, @var{options} specifies the operations to be performed during
eeaf4427
LC
468the transaction. Upon completion, a new profile is created, but
469previous generations of the profile remain available, should the user
470want to roll back.
471
b9e5c0a9 472For each user, a symlink to the user's default profile is automatically
0ec1af59 473created in @file{$HOME/.guix-profile}. This symlink always points to the
b9e5c0a9
LC
474current generation of the user's default profile. Thus, users can add
475@file{$HOME/.guix-profile/bin} to their @code{PATH} environment
476variable, and so on.
477
0ec1af59
LC
478In a multi-user setup, user profiles must be stored in a place
479registered as a @dfn{garbage-collector root}, which
e49951eb 480@file{$HOME/.guix-profile} points to (@pxref{Invoking guix gc}). That
0ec1af59
LC
481directory is normally
482@code{@var{localstatedir}/profiles/per-user/@var{user}}, where
483@var{localstatedir} is the value passed to @code{configure} as
484@code{--localstatedir}, and @var{user} is the user name. It must be
485created by @code{root}, with @var{user} as the owner. When it does not
70c43291
LC
486exist, or is not owned by @var{user}, @command{guix package} emits an
487error about it.
0ec1af59
LC
488
489The @var{options} can be among the following:
490
eeaf4427
LC
491@table @code
492
493@item --install=@var{package}
51c8d790 494@itemx -i @var{package}
eeaf4427
LC
495Install @var{package}.
496
497@var{package} may specify either a simple package name, such as
498@code{guile}, or a package name followed by a hyphen and version number,
dc5669cd
MW
499such as @code{guile-1.8.8}. If no version number is specified, the
500newest available version will be selected. In addition, @var{package}
501may contain a colon, followed by the name of one of the outputs of the
6e721c4d
LC
502package, as in @code{gcc:doc} or @code{binutils-2.22:lib}
503(@pxref{Packages with Multiple Outputs}).
eeaf4427 504
461572cc
LC
505@cindex propagated inputs
506Sometimes packages have @dfn{propagated inputs}: these are dependencies
507that automatically get installed along with the required package.
508
509An example is the GNU MPC library: its C header files refer to those of
510the GNU MPFR library, which in turn refer to those of the GMP library.
511Thus, when installing MPC, the MPFR and GMP libraries also get installed
512in the profile; removing MPC also removes MPFR and GMP---unless they had
513also been explicitly installed independently.
514
ba7ea5ce 515Besides, packages sometimes rely on the definition of environment
5924080d 516variables for their search paths (see explanation of
ba7ea5ce 517@code{--search-paths} below). Any missing or possibly incorrect
5924080d
LC
518environment variable definitions are reported here.
519
ef010c0f 520@c XXX: keep me up-to-date
5924080d 521Finally, when installing a GNU package, the tool reports the
ef010c0f
LC
522availability of a newer upstream version. In the future, it may provide
523the option of installing directly from the upstream version, even if
524that version is not yet in the distribution.
525
5d4b411f
LC
526@item --install-from-expression=@var{exp}
527@itemx -e @var{exp}
528Install the package @var{exp} evaluates to.
529
530@var{exp} must be a Scheme expression that evaluates to a
531@code{<package>} object. This option is notably useful to disambiguate
532between same-named variants of a package, with expressions such as
533@code{(@@ (gnu packages base) guile-final)}.
534
535Note that this option installs the first output of the specified
536package, which may be insufficient when needing a specific output of a
537multiple-output package.
538
eeaf4427
LC
539@item --remove=@var{package}
540@itemx -r @var{package}
541Remove @var{package}.
542
13ed095c
LC
543As for @code{--install}, @var{package} may specify a version number
544and/or output name in addition to the package name. For instance,
545@code{-r glibc:debug} would remove the @code{debug} output of
546@code{glibc}.
547
acb6ba25
CR
548@item --upgrade[=@var{regexp}]
549@itemx -u [@var{regexp}]
550Upgrade all the installed packages. When @var{regexp} is specified, upgrade
551only installed packages whose name matches @var{regexp}.
eeaf4427 552
f651b477
LC
553Note that this upgrades package to the latest version of packages found
554in the distribution currently installed. To update your distribution,
555you should regularly run @command{guix pull} (@pxref{Invoking guix
556pull}).
557
24e262f0
LC
558@item --roll-back
559Roll back to the previous @dfn{generation} of the profile---i.e., undo
560the last transaction.
561
562When combined with options such as @code{--install}, roll back occurs
563before any other actions.
564
d9307267 565When rolling back from the first generation that actually contains
4b2bc804
NK
566installed packages, the profile is made to point to the @dfn{zeroth
567generation}, which contains no files apart from its own meta-data.
d9307267 568
82fe08ed
LC
569Installing, removing, or upgrading packages from a generation that has
570been rolled back to overwrites previous future generations. Thus, the
571history of a profile's generations is always linear.
572
5924080d
LC
573@item --search-paths
574@cindex search paths
575Report environment variable definitions, in Bash syntax, that may be
576needed in order to use the set of installed packages. These environment
577variables are used to specify @dfn{search paths} for files used by some
578of the installed packages.
579
580For example, GCC needs the @code{CPATH} and @code{LIBRARY_PATH}
581environment variables to be defined so it can look for headers and
582libraries in the user's profile (@pxref{Environment Variables,,, gcc,
583Using the GNU Compiler Collection (GCC)}). If GCC and, say, the C
584library are installed in the profile, then @code{--search-paths} will
585suggest setting these variables to @code{@var{profile}/include} and
586@code{@var{profile}/lib}, respectively.
587
eeaf4427
LC
588@item --profile=@var{profile}
589@itemx -p @var{profile}
590Use @var{profile} instead of the user's default profile.
591
592@item --dry-run
593@itemx -n
594Show what would be done without actually doing it.
595
56b1f4b7
LC
596@item --fallback
597When substituting a pre-built binary fails, fall back to building
598packages locally.
599
3b824605 600@item --no-substitutes
969e678e
LC
601@itemx --max-silent-time=@var{seconds}
602Same as for @command{guix build} (@pxref{Invoking guix build}).
3b824605 603
70915c1a
LC
604@item --verbose
605Produce verbose output. In particular, emit the environment's build log
606on the standard error port.
607
eeaf4427
LC
608@item --bootstrap
609Use the bootstrap Guile to build the profile. This option is only
610useful to distribution developers.
611
612@end table
613
e49951eb 614In addition to these actions @command{guix package} supports the
733b4130
LC
615following options to query the current state of a profile, or the
616availability of packages:
eeaf4427 617
733b4130
LC
618@table @option
619
acc08466
NK
620@item --search=@var{regexp}
621@itemx -s @var{regexp}
622List the available packages whose synopsis or description matches
299112d3
LC
623@var{regexp}. Print all the meta-data of matching packages in
624@code{recutils} format (@pxref{Top, GNU recutils databases,, recutils,
625GNU recutils manual}).
acc08466 626
299112d3
LC
627This allows specific fields to be extracted using the @command{recsel}
628command, for instance:
629
630@example
e49951eb 631$ guix package -s malloc | recsel -p name,version
299112d3
LC
632name: glibc
633version: 2.17
634
635name: libgc
636version: 7.2alpha6
637@end example
acc08466 638
733b4130
LC
639@item --list-installed[=@var{regexp}]
640@itemx -I [@var{regexp}]
bd9bde1c
LC
641List the currently installed packages in the specified profile, with the
642most recently installed packages shown last. When @var{regexp} is
643specified, list only installed packages whose name matches @var{regexp}.
733b4130
LC
644
645For each installed package, print the following items, separated by
646tabs: the package name, its version string, the part of the package that
647is installed (for instance, @code{out} for the default output,
648@code{include} for its headers, etc.), and the path of this package in
649the store.
650
64fc89b6
LC
651@item --list-available[=@var{regexp}]
652@itemx -A [@var{regexp}]
a1ba8475
LC
653List packages currently available in the software distribution
654(@pxref{GNU Distribution}). When @var{regexp} is specified, list only
655installed packages whose name matches @var{regexp}.
64fc89b6
LC
656
657For each package, print the following items separated by tabs: its name,
6e721c4d
LC
658its version string, the parts of the package (@pxref{Packages with
659Multiple Outputs}), and the source location of its definition.
64fc89b6 660
f566d765
LC
661@item --list-generations[=@var{pattern}]
662@itemx -l [@var{pattern}]
663Return a list of generations along with their creation dates; for each
664generation, show the installed packages, with the most recently
4b2bc804
NK
665installed packages shown last. Note that the zeroth generation is never
666shown.
f566d765
LC
667
668For each installed package, print the following items, separated by
669tabs: the name of a package, its version string, the part of the package
670that is installed (@pxref{Packages with Multiple Outputs}), and the
671location of this package in the store.
672
673When @var{pattern} is used, the command returns only matching
674generations. Valid patterns include:
675
676@itemize
677@item @emph{Integers and comma-separated integers}. Both patterns denote
678generation numbers. For instance, @code{--list-generations=1} returns
679the first one.
680
681And @code{--list-generations=1,8,2} outputs three generations in the
682specified order. Neither spaces nor trailing commas are allowed.
683
684@item @emph{Ranges}. @code{--list-generations=2..9} prints the
685specified generations and everything in between. Note that the start of
686a range must be lesser than its end.
687
688It is also possible to omit the endpoint. For example,
689@code{--list-generations=2..}, returns all generations starting from the
690second one.
691
692@item @emph{Durations}. You can also get the last @emph{N}@tie{}days, weeks,
693or months by passing an integer along with the first letter of the
d7ddb257
LC
694duration. For example, @code{--list-generations=20d} lists generations
695that are up to 20 days old.
f566d765
LC
696@end itemize
697
b7884ca3
NK
698@item --delete-generations[=@var{pattern}]
699@itemx -d [@var{pattern}]
d7ddb257
LC
700When @var{pattern} is omitted, delete all generations except the current
701one.
b7884ca3
NK
702
703This command accepts the same patterns as @option{--list-generations}.
d7ddb257
LC
704When @var{pattern} is specified, delete the matching generations. When
705@var{pattern} specifies a duration, generations @emph{older} than the
706specified duration match. For instance, @code{--delete-generations=1m}
707deletes generations that are more than one month old.
708
709If the current generation matches, it is deleted atomically---i.e., by
710switching to the previous available generation. Note that the zeroth
711generation is never deleted.
b7884ca3 712
1bb9900a
LC
713Note that deleting generations prevents roll-back to them.
714Consequently, this command must be used with care.
715
733b4130 716@end table
eeaf4427 717
6e721c4d
LC
718@node Packages with Multiple Outputs
719@section Packages with Multiple Outputs
720
721@cindex multiple-output packages
722@cindex package outputs
723
724Often, packages defined in Guix have a single @dfn{output}---i.e., the
725source package leads exactly one directory in the store. When running
726@command{guix package -i glibc}, one installs the default output of the
727GNU libc package; the default output is called @code{out}, but its name
728can be omitted as shown in this command. In this particular case, the
729default output of @code{glibc} contains all the C header files, shared
730libraries, static libraries, Info documentation, and other supporting
731files.
732
733Sometimes it is more appropriate to separate the various types of files
734produced from a single source package into separate outputs. For
735instance, the GLib C library (used by GTK+ and related packages)
736installs more than 20 MiB of reference documentation as HTML pages.
737To save space for users who do not need it, the documentation goes to a
738separate output, called @code{doc}. To install the main GLib output,
739which contains everything but the documentation, one would run:
740
741@example
742guix package -i glib
743@end example
744
745The command to install its documentation is:
746
747@example
748guix package -i glib:doc
749@end example
750
751Some packages install programs with different ``dependency footprints''.
752For instance, the WordNet package install both command-line tools and
753graphical user interfaces (GUIs). The former depend solely on the C
754library, whereas the latter depend on Tcl/Tk and the underlying X
755libraries. In this case, we leave the command-line tools in the default
756output, whereas the GUIs are in a separate output. This allows users
757who do not need the GUIs to save space.
758
759There are several such multiple-output packages in the GNU distribution.
91ef73d4
LC
760Other conventional output names include @code{lib} for libraries and
761possibly header files, @code{bin} for stand-alone programs, and
762@code{debug} for debugging information (@pxref{Installing Debugging
763Files}). The outputs of a packages are listed in the third column of
764the output of @command{guix package --list-available} (@pxref{Invoking
765guix package}).
6e721c4d 766
eeaf4427 767
e49951eb
MW
768@node Invoking guix gc
769@section Invoking @command{guix gc}
fe8ff028
LC
770
771@cindex garbage collector
772Packages that are installed but not used may be @dfn{garbage-collected}.
e49951eb 773The @command{guix gc} command allows users to explicitly run the garbage
fe8ff028
LC
774collector to reclaim space from the @file{/nix/store} directory.
775
776The garbage collector has a set of known @dfn{roots}: any file under
777@file{/nix/store} reachable from a root is considered @dfn{live} and
778cannot be deleted; any other file is considered @dfn{dead} and may be
779deleted. The set of garbage collector roots includes default user
e49951eb
MW
780profiles, and may be augmented with @command{guix build --root}, for
781example (@pxref{Invoking guix build}).
fe8ff028 782
1bb9900a
LC
783Prior to running @code{guix gc --collect-garbage} to make space, it is
784often useful to remove old generations from user profiles; that way, old
785package builds referenced by those generations can be reclaimed. This
786is achieved by running @code{guix package --delete-generations}
787(@pxref{Invoking guix package}).
788
e49951eb 789The @command{guix gc} command has three modes of operation: it can be
fe8ff028
LC
790used to garbage-collect any dead files (the default), to delete specific
791files (the @code{--delete} option), or to print garbage-collector
792information. The available options are listed below:
793
794@table @code
795@item --collect-garbage[=@var{min}]
796@itemx -C [@var{min}]
797Collect garbage---i.e., unreachable @file{/nix/store} files and
798sub-directories. This is the default operation when no option is
799specified.
800
801When @var{min} is given, stop once @var{min} bytes have been collected.
802@var{min} may be a number of bytes, or it may include a unit as a
803suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes.
804
805When @var{min} is omitted, collect all the garbage.
806
807@item --delete
808@itemx -d
809Attempt to delete all the store files and directories specified as
810arguments. This fails if some of the files are not in the store, or if
811they are still live.
812
813@item --list-dead
814Show the list of dead files and directories still present in the
815store---i.e., files and directories no longer reachable from any root.
816
817@item --list-live
818Show the list of live store files and directories.
ba8b732d
LC
819
820@end table
821
822In addition, the references among existing store files can be queried:
823
824@table @code
825
826@item --references
827@itemx --referrers
828List the references (respectively, the referrers) of store files given
829as arguments.
830
8e59fdd5
LC
831@item --requisites
832@itemx -R
833List the requisites of the store files passed as arguments. Requisites
834include the store files themselves, their references, and the references
835of these, recursively. In other words, the returned list is the
836@dfn{transitive closure} of the store files.
837
fe8ff028
LC
838@end table
839
eeaf4427 840
f651b477
LC
841@node Invoking guix pull
842@section Invoking @command{guix pull}
843
844Packages are installed or upgraded to the latest version available in
845the distribution currently available on your local machine. To update
846that distribution, along with the Guix tools, you must run @command{guix
847pull}: the command downloads the latest Guix source code and package
848descriptions, and deploys it.
849
850On completion, @command{guix package} will use packages and package
851versions from this just-retrieved copy of Guix. Not only that, but all
852the Guix commands and Scheme modules will also be taken from that latest
853version. New @command{guix} sub-commands added by the update also
854become available.
855
856The @command{guix pull} command is usually invoked with no arguments,
857but it supports the following options:
858
859@table @code
860@item --verbose
861Produce verbose output, writing build logs to the standard error output.
862
ab5d72ad
LC
863@item --url=@var{url}
864Download the source tarball of Guix from @var{url}.
865
866By default, the tarball is taken from its canonical address at
867@code{gnu.org}, for the stable branch of Guix.
868
f651b477
LC
869@item --bootstrap
870Use the bootstrap Guile to build the latest Guix. This option is only
871useful to Guix developers.
872@end table
873
568717fd
LC
874@c *********************************************************************
875@node Programming Interface
876@chapter Programming Interface
877
3dc1970d
LC
878GNU Guix provides several Scheme programming interfaces (APIs) to
879define, build, and query packages. The first interface allows users to
880write high-level package definitions. These definitions refer to
881familiar packaging concepts, such as the name and version of a package,
882its build system, and its dependencies. These definitions can then be
883turned into concrete build actions.
884
ba55b1cb 885Build actions are performed by the Guix daemon, on behalf of users. In a
3dc1970d
LC
886standard setup, the daemon has write access to the store---the
887@file{/nix/store} directory---whereas users do not. The recommended
888setup also has the daemon perform builds in chroots, under a specific
889build users, to minimize interference with the rest of the system.
890
891@cindex derivation
892Lower-level APIs are available to interact with the daemon and the
893store. To instruct the daemon to perform a build action, users actually
894provide it with a @dfn{derivation}. A derivation is a low-level
895representation of the build actions to be taken, and the environment in
896which they should occur---derivations are to package definitions what
897assembly is to C programs.
898
899This chapter describes all these APIs in turn, starting from high-level
900package definitions.
901
568717fd 902@menu
b860f382
LC
903* Defining Packages:: Defining new packages.
904* The Store:: Manipulating the package store.
905* Derivations:: Low-level interface to package derivations.
906* The Store Monad:: Purely functional interface to the store.
568717fd
LC
907@end menu
908
909@node Defining Packages
910@section Defining Packages
911
3dc1970d
LC
912The high-level interface to package definitions is implemented in the
913@code{(guix packages)} and @code{(guix build-system)} modules. As an
914example, the package definition, or @dfn{recipe}, for the GNU Hello
915package looks like this:
916
917@example
b22a12fd
LC
918(use-modules (guix packages)
919 (guix download)
920 (guix build-system gnu)
921 (guix licenses))
922
3dc1970d
LC
923(define hello
924 (package
925 (name "hello")
926 (version "2.8")
927 (source (origin
928 (method url-fetch)
929 (uri (string-append "mirror://gnu/hello/hello-" version
930 ".tar.gz"))
931 (sha256
932 (base32 "0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6"))))
933 (build-system gnu-build-system)
934 (inputs `(("gawk" ,gawk)))
935 (synopsis "GNU Hello")
936 (description "Yeah...")
937 (home-page "http://www.gnu.org/software/hello/")
b22a12fd 938 (license gpl3+)))
3dc1970d
LC
939@end example
940
941@noindent
942Without being a Scheme expert, the reader may have guessed the meaning
943of the various fields here. This expression binds variable @var{hello}
944to a @code{<package>} object, which is essentially a record
945(@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}).
946This package object can be inspected using procedures found in the
947@code{(guix packages)} module; for instance, @code{(package-name hello)}
948returns---surprise!---@code{"hello"}.
949
950There are a few points worth noting in the above package definition:
951
952@itemize
953@item
954The @code{source} field of the package is an @code{<origin>} object.
955Here, the @code{url-fetch} method from @code{(guix download)} is used,
956meaning that the source is a file to be downloaded over FTP or HTTP.
957
958The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of
959the GNU mirrors defined in @code{(guix download)}.
960
961The @code{sha256} field specifies the expected SHA256 hash of the file
962being downloaded. It is mandatory, and allows Guix to check the
963integrity of the file. The @code{(base32 @dots{})} form introduces the
6c365eca 964base32 representation of the hash. You can obtain this information with
210cc920
LC
965@code{guix download} (@pxref{Invoking guix download}) and @code{guix
966hash} (@pxref{Invoking guix hash}).
3dc1970d
LC
967
968@item
969@cindex GNU Build System
970The @code{build-system} field is set to @var{gnu-build-system}. The
971@var{gnu-build-system} variable is defined in the @code{(guix
972build-system gnu)} module, and is bound to a @code{<build-system>}
973object.
974
975Naturally, @var{gnu-build-system} represents the familiar GNU Build
976System, and variants thereof (@pxref{Configuration, configuration and
977makefile conventions,, standards, GNU Coding Standards}). In a
ba55b1cb 978nutshell, packages using the GNU Build System may be configured, built,
3dc1970d
LC
979and installed with the usual @code{./configure && make && make check &&
980make install} command sequence. This is what @var{gnu-build-system}
981does.
982
983In addition, @var{gnu-build-system} ensures that the ``standard''
984environment for GNU packages is available. This includes tools such as
985GCC, Coreutils, Bash, Make, Diffutils, and Patch.
986
987@item
988The @code{inputs} field specifies inputs to the build process---i.e.,
989build-time or run-time dependencies of the package. Here, we define an
990input called @code{"gawk"} whose value is that of the @var{gawk}
991variable; @var{gawk} is itself bound to a @code{<package>} object.
992
993Note that GCC, Coreutils, Bash, and other essential tools do not need to
994be specified as inputs here. Instead, @var{gnu-build-system} takes care
995of ensuring that they are present.
996
997However, any other dependencies need to be specified in the
998@code{inputs} field. Any dependency not specified here will simply be
999unavailable to the build process, possibly leading to a build failure.
1000@end itemize
1001
1002There are other fields that package definitions may provide. Of
1003particular interest is the @code{arguments} field. When specified, it
1004must be bound to a list of additional arguments to be passed to the
1005build system. For instance, the above definition could be augmented
1006with the following field initializer:
1007
1008@example
1009 (arguments `(#:tests? #f
1010 #:configure-flags '("--enable-silent-rules")))
1011@end example
1012
1013@noindent
1014These are keyword arguments (@pxref{Optional Arguments, keyword
1015arguments in Guile,, guile, GNU Guile Reference Manual}). They are
1016passed to @var{gnu-build-system}, which interprets them as meaning ``do
1017not run @code{make check}'', and ``run @file{configure} with the
874e6874
LC
1018@code{--enable-silent-rules} flag''. The value of these keyword
1019parameters is actually evaluated in the @dfn{build stratum}---i.e., by a
1020Guile process launched by the daemon (@pxref{Derivations}).
3dc1970d
LC
1021
1022Once a package definition is in place@footnote{Simple package
1023definitions like the one above may be automatically converted from the
e49951eb
MW
1024Nixpkgs distribution using the @command{guix import} command.}, the
1025package may actually be built using the @code{guix build} command-line
37166310
LC
1026tool (@pxref{Invoking guix build}). Eventually, updating the package
1027definition to a new upstream version can be partly automated by the
1028@command{guix refresh} command (@pxref{Invoking guix refresh}).
3dc1970d
LC
1029
1030Behind the scenes, a derivation corresponding to the @code{<package>}
1031object is first computed by the @code{package-derivation} procedure.
1032That derivation is stored in a @code{.drv} file under @file{/nix/store}.
ba55b1cb 1033The build actions it prescribes may then be realized by using the
3dc1970d
LC
1034@code{build-derivations} procedure (@pxref{The Store}).
1035
1036@deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}]
59688fc4
LC
1037Return the @code{<derivation>} object of @var{package} for @var{system}
1038(@pxref{Derivations}).
3dc1970d
LC
1039
1040@var{package} must be a valid @code{<package>} object, and @var{system}
1041must be a string denoting the target system type---e.g.,
1042@code{"x86_64-linux"} for an x86_64 Linux-based GNU system. @var{store}
1043must be a connection to the daemon, which operates on the store
1044(@pxref{The Store}).
1045@end deffn
568717fd 1046
9c1edabd
LC
1047@noindent
1048@cindex cross-compilation
1049Similarly, it is possible to compute a derivation that cross-builds a
1050package for some other system:
1051
1052@deffn {Scheme Procedure} package-cross-derivation @var{store} @
1053 @var{package} @var{target} [@var{system}]
59688fc4
LC
1054Return the @code{<derivation>} object of @var{package} cross-built from
1055@var{system} to @var{target}.
9c1edabd
LC
1056
1057@var{target} must be a valid GNU triplet denoting the target hardware
1058and operating system, such as @code{"mips64el-linux-gnu"}
1059(@pxref{Configuration Names, GNU configuration triplets,, configure, GNU
1060Configure and Build System}).
1061@end deffn
1062
1063
568717fd
LC
1064@node The Store
1065@section The Store
1066
e531ac2a
LC
1067@cindex store
1068@cindex store paths
1069
1070Conceptually, the @dfn{store} is where derivations that have been
1071successfully built are stored---by default, under @file{/nix/store}.
1072Sub-directories in the store are referred to as @dfn{store paths}. The
1073store has an associated database that contains information such has the
1074store paths referred to by each store path, and the list of @emph{valid}
1075store paths---paths that result from a successful build.
1076
1077The store is always accessed by the daemon on behalf of its clients
1078(@pxref{Invoking guix-daemon}). To manipulate the store, clients
1079connect to the daemon over a Unix-domain socket, send it requests, and
1080read the result---these are remote procedure calls, or RPCs.
1081
1082The @code{(guix store)} module provides procedures to connect to the
1083daemon, and to perform RPCs. These are described below.
1084
1085@deffn {Scheme Procedure} open-connection [@var{file}] [#:reserve-space? #t]
1086Connect to the daemon over the Unix-domain socket at @var{file}. When
1087@var{reserve-space?} is true, instruct it to reserve a little bit of
1088extra space on the file system so that the garbage collector can still
1089operate, should the disk become full. Return a server object.
1090
1091@var{file} defaults to @var{%default-socket-path}, which is the normal
1092location given the options that were passed to @command{configure}.
1093@end deffn
1094
1095@deffn {Scheme Procedure} close-connection @var{server}
1096Close the connection to @var{server}.
1097@end deffn
1098
1099@defvr {Scheme Variable} current-build-output-port
1100This variable is bound to a SRFI-39 parameter, which refers to the port
1101where build and error logs sent by the daemon should be written.
1102@end defvr
1103
1104Procedures that make RPCs all take a server object as their first
1105argument.
1106
1107@deffn {Scheme Procedure} valid-path? @var{server} @var{path}
1108Return @code{#t} when @var{path} is a valid store path.
1109@end deffn
1110
cfbf9160 1111@deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}]
e531ac2a
LC
1112Add @var{text} under file @var{name} in the store, and return its store
1113path. @var{references} is the list of store paths referred to by the
1114resulting store path.
1115@end deffn
1116
874e6874 1117@deffn {Scheme Procedure} build-derivations @var{server} @var{derivations}
59688fc4
LC
1118Build @var{derivations} (a list of @code{<derivation>} objects or
1119derivation paths), and return when the worker is done building them.
1120Return @code{#t} on success.
874e6874
LC
1121@end deffn
1122
b860f382
LC
1123Note that the @code{(guix monads)} module provides a monad as well as
1124monadic versions of the above procedures, with the goal of making it
1125more convenient to work with code that accesses the store (@pxref{The
1126Store Monad}).
1127
e531ac2a
LC
1128@c FIXME
1129@i{This section is currently incomplete.}
568717fd
LC
1130
1131@node Derivations
1132@section Derivations
1133
874e6874
LC
1134@cindex derivations
1135Low-level build actions and the environment in which they are performed
1136are represented by @dfn{derivations}. A derivation contain the
1137following pieces of information:
1138
1139@itemize
1140@item
1141The outputs of the derivation---derivations produce at least one file or
1142directory in the store, but may produce more.
1143
1144@item
1145The inputs of the derivations, which may be other derivations or plain
1146files in the store (patches, build scripts, etc.)
1147
1148@item
1149The system type targeted by the derivation---e.g., @code{x86_64-linux}.
1150
1151@item
1152The file name of a build script in the store, along with the arguments
1153to be passed.
1154
1155@item
1156A list of environment variables to be defined.
1157
1158@end itemize
1159
1160@cindex derivation path
1161Derivations allow clients of the daemon to communicate build actions to
1162the store. They exist in two forms: as an in-memory representation,
1163both on the client- and daemon-side, and as files in the store whose
1164name end in @code{.drv}---these files are referred to as @dfn{derivation
1165paths}. Derivations paths can be passed to the @code{build-derivations}
1166procedure to perform the build actions they prescribe (@pxref{The
1167Store}).
1168
1169The @code{(guix derivations)} module provides a representation of
1170derivations as Scheme objects, along with procedures to create and
1171otherwise manipulate derivations. The lowest-level primitive to create
1172a derivation is the @code{derivation} procedure:
1173
858e9282 1174@deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] [#:hash-mode #f] [#:inputs '()] [#:env-vars '()] [#:system (%current-system)] [#:references-graphs #f]
59688fc4
LC
1175Build a derivation with the given arguments, and return the resulting
1176@code{<derivation>} object.
874e6874
LC
1177
1178When @var{hash}, @var{hash-algo}, and @var{hash-mode} are given, a
1179@dfn{fixed-output derivation} is created---i.e., one whose result is
1180known in advance, such as a file download.
5b0c9d16 1181
858e9282 1182When @var{references-graphs} is true, it must be a list of file
5b0c9d16
LC
1183name/store path pairs. In that case, the reference graph of each store
1184path is exported in the build environment in the corresponding file, in
1185a simple text format.
874e6874
LC
1186@end deffn
1187
1188@noindent
1189Here's an example with a shell script as its builder, assuming
1190@var{store} is an open connection to the daemon, and @var{bash} points
1191to a Bash executable in the store:
1192
1193@lisp
1194(use-modules (guix utils)
1195 (guix store)
1196 (guix derivations))
1197
59688fc4
LC
1198(let ((builder ; add the Bash script to the store
1199 (add-text-to-store store "my-builder.sh"
1200 "echo hello world > $out\n" '())))
1201 (derivation store "foo"
1202 bash `("-e" ,builder)
1203 #:env-vars '(("HOME" . "/homeless"))))
1204@result{} #<derivation /nix/store/@dots{}-foo.drv => /nix/store/@dots{}-foo>
874e6874
LC
1205@end lisp
1206
1207As can be guessed, this primitive is cumbersome to use directly. An
1208improved variant is @code{build-expression->derivation}, which allows
1209the caller to directly pass a Guile expression as the build script:
1210
858e9282 1211@deffn {Scheme Procedure} build-expression->derivation @var{store} @var{name} @var{system} @var{exp} @var{inputs} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] [#:env-vars '()] [#:modules '()] [#:references-graphs #f] [#:guile-for-build #f]
874e6874
LC
1212Return a derivation that executes Scheme expression @var{exp} as a
1213builder for derivation @var{name}. @var{inputs} must be a list of
1214@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
1215@code{"out"} is assumed. @var{modules} is a list of names of Guile
1216modules from the current search path to be copied in the store,
1217compiled, and made available in the load path during the execution of
1218@var{exp}---e.g., @code{((guix build utils) (guix build
1219gnu-build-system))}.
1220
1221@var{exp} is evaluated in an environment where @code{%outputs} is bound
1222to a list of output/path pairs, and where @code{%build-inputs} is bound
1223to a list of string/output-path pairs made from @var{inputs}.
1224Optionally, @var{env-vars} is a list of string pairs specifying the name
1225and value of environment variables visible to the builder. The builder
1226terminates by passing the result of @var{exp} to @code{exit}; thus, when
1227@var{exp} returns @code{#f}, the build is considered to have failed.
1228
1229@var{exp} is built using @var{guile-for-build} (a derivation). When
1230@var{guile-for-build} is omitted or is @code{#f}, the value of the
1231@code{%guile-for-build} fluid is used instead.
9c629a27 1232
858e9282 1233See the @code{derivation} procedure for the meaning of @var{references-graphs}.
874e6874
LC
1234@end deffn
1235
1236@noindent
1237Here's an example of a single-output derivation that creates a directory
1238containing one file:
1239
1240@lisp
1241(let ((builder '(let ((out (assoc-ref %outputs "out")))
1242 (mkdir out) ; create /nix/store/@dots{}-goo
1243 (call-with-output-file (string-append out "/test")
1244 (lambda (p)
1245 (display '(hello guix) p))))))
1246 (build-expression->derivation store "goo" (%current-system)
1247 builder '()))
1248
59688fc4 1249@result{} #<derivation /nix/store/@dots{}-goo.drv => @dots{}>
874e6874
LC
1250@end lisp
1251
1252@cindex strata of code
1253Remember that the build expression passed to
1254@code{build-expression->derivation} is run by a separate Guile process
1255than the one that calls @code{build-expression->derivation}: it is run
1256by a Guile process launched by the daemon, typically in a chroot. So,
1257while there is a single language for both the @dfn{host} and the build
1258side, there are really two @dfn{strata} of code: the host-side, and the
1259build-side code@footnote{The term @dfn{stratum} in this context was
1260coined by Manuel Serrano et al. in the context of their work on Hop.}.
1261This distinction is important to keep in mind, notably when using
1262higher-level constructs such as @var{gnu-build-system} (@pxref{Defining
1263Packages}). For this reason, Guix modules that are meant to be used in
1264the build stratum are kept in the @code{(guix build @dots{})} name
1265space.
568717fd 1266
b860f382
LC
1267@node The Store Monad
1268@section The Store Monad
1269
1270@cindex monad
1271
1272The procedures that operate on the store described in the previous
1273sections all take an open connection to the build daemon as their first
1274argument. Although the underlying model is functional, they either have
1275side effects or depend on the current state of the store.
1276
1277The former is inconvenient: the connection to the build daemon has to be
1278carried around in all those functions, making it impossible to compose
1279functions that do not take that parameter with functions that do. The
1280latter can be problematic: since store operations have side effects
1281and/or depend on external state, they have to be properly sequenced.
1282
1283@cindex monadic values
1284@cindex monadic functions
1285This is where the @code{(guix monads)} module comes in. This module
1286provides a framework for working with @dfn{monads}, and a particularly
1287useful monad for our uses, the @dfn{store monad}. Monads are a
1288construct that allows two things: associating ``context'' with values
1289(in our case, the context is the store), and building sequences of
1290computations (here computations includes accesses to the store.) Values
1291in a monad---values that carry this additional context---are called
1292@dfn{monadic values}; procedures that return such values are called
1293@dfn{monadic procedures}.
1294
1295Consider this ``normal'' procedure:
1296
1297@example
1298(define (profile.sh store)
1299 ;; Return the name of a shell script in the store that
1300 ;; initializes the 'PATH' environment variable.
1301 (let* ((drv (package-derivation store coreutils))
1302 (out (derivation->output-path drv)))
1303 (add-text-to-store store "profile.sh"
1304 (format #f "export PATH=~a/bin" out))))
1305@end example
1306
1307Using @code{(guix monads)}, it may be rewritten as a monadic function:
1308
1309@example
1310(define (profile.sh)
1311 ;; Same, but return a monadic value.
1312 (mlet %store-monad ((bin (package-file coreutils "bin")))
1313 (text-file "profile.sh"
1314 (string-append "export PATH=" bin))))
1315@end example
1316
1317There are two things to note in the second version: the @code{store}
1318parameter is now implicit, and the monadic value returned by
1319@code{package-file}---a wrapper around @code{package-derivation} and
1320@code{derivation->output-path}---is @dfn{bound} using @code{mlet}
1321instead of plain @code{let}.
1322
1323Calling the monadic @code{profile.sh} has no effect. To get the desired
1324effect, one must use @code{run-with-store}:
1325
1326@example
1327(run-with-store (open-connection) (profile.sh))
1328@result{} /nix/store/...-profile.sh
1329@end example
1330
1331The main syntactic forms to deal with monads in general are described
1332below.
1333
1334@deffn {Scheme Syntax} with-monad @var{monad} @var{body} ...
1335Evaluate any @code{>>=} or @code{return} forms in @var{body} as being
1336in @var{monad}.
1337@end deffn
1338
1339@deffn {Scheme Syntax} return @var{val}
1340Return a monadic value that encapsulates @var{val}.
1341@end deffn
1342
1343@deffn {Scheme Syntax} >>= @var{mval} @var{mproc}
1344@dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic
1345procedure @var{mproc}@footnote{This operation is commonly referred to as
1346``bind'', but that name denotes an unrelated procedure in Guile. Thus
1347we use this somewhat cryptic symbol inherited from the Haskell
1348language.}.
1349@end deffn
1350
1351@deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @
1352 @var{body} ...
1353@deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @
1354 @var{body} ...
1355Bind the variables @var{var} to the monadic values @var{mval} in
1356@var{body}. The form (@var{var} -> @var{val}) binds @var{var} to the
1357``normal'' value @var{val}, as per @code{let}.
1358
1359@code{mlet*} is to @code{mlet} what @code{let*} is to @code{let}
1360(@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}).
1361@end deffn
1362
1363The interface to the store monad provided by @code{(guix monads)} is as
1364follows.
1365
1366@defvr {Scheme Variable} %store-monad
1367The store monad. Values in the store monad encapsulate accesses to the
1368store. When its effect is needed, a value of the store monad must be
1369``evaluated'' by passing it to the @code{run-with-store} procedure (see
1370below.)
1371@end defvr
1372
1373@deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)]
1374Run @var{mval}, a monadic value in the store monad, in @var{store}, an
1375open store connection.
1376@end deffn
1377
1378@deffn {Monadic Procedure} text-file @var{name} @var{text}
1379Return as a monadic value the absolute file name in the store of the file
1380containing @var{text}.
1381@end deffn
1382
1383@deffn {Monadic Procedure} package-file @var{package} [@var{file}] @
1384 [#:system (%current-system)] [#:output "out"] Return as a monadic
1385value in the absolute file name of @var{file} within the @var{output}
1386directory of @var{package}. When @var{file} is omitted, return the name
1387of the @var{output} directory of @var{package}.
1388@end deffn
1389
1390@deffn {Monadic Procedure} derivation-expression @var{name} @var{system} @
1391 @var{exp} @var{inputs} [#:outputs '("out")] [#:hash #f] @
1392 [#:hash-algo #f] [#:env-vars '()] [#:modules '()] @
1393 [#:references-graphs #f] [#:guile-for-build #f]
1394Monadic version of @code{build-expression->derivation}
1395(@pxref{Derivations}).
1396@end deffn
1397
1398@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}]
1399Monadic version of @code{package-derivation} (@pxref{Defining
1400Packages}).
1401@end deffn
1402
1403
568717fd
LC
1404@c *********************************************************************
1405@node Utilities
1406@chapter Utilities
1407
210cc920
LC
1408This section describes tools primarily targeted at developers and users
1409who write new package definitions. They complement the Scheme
1410programming interface of Guix in a convenient way.
1411
568717fd 1412@menu
37166310 1413* Invoking guix build:: Building packages from the command line.
210cc920 1414* Invoking guix download:: Downloading a file and printing its hash.
37166310
LC
1415* Invoking guix hash:: Computing the cryptographic hash of a file.
1416* Invoking guix refresh:: Updating package definitions.
568717fd
LC
1417@end menu
1418
e49951eb
MW
1419@node Invoking guix build
1420@section Invoking @command{guix build}
568717fd 1421
e49951eb 1422The @command{guix build} command builds packages or derivations and
6798a8e4
LC
1423their dependencies, and prints the resulting store paths. Note that it
1424does not modify the user's profile---this is the job of the
e49951eb 1425@command{guix package} command (@pxref{Invoking guix package}). Thus,
6798a8e4
LC
1426it is mainly useful for distribution developers.
1427
1428The general syntax is:
c78bd12b
LC
1429
1430@example
e49951eb 1431guix build @var{options} @var{package-or-derivation}@dots{}
c78bd12b
LC
1432@end example
1433
1434@var{package-or-derivation} may be either the name of a package found in
5401dd75
LC
1435the software distribution such as @code{coreutils} or
1436@code{coreutils-8.20}, or a derivation such as
1437@file{/nix/store/@dots{}-coreutils-8.19.drv}. Alternatively, the
c78bd12b
LC
1438@code{--expression} option may be used to specify a Scheme expression
1439that evaluates to a package; this is useful when disambiguation among
1440several same-named packages or package variants is needed.
1441
1442The @var{options} may be zero or more of the following:
1443
1444@table @code
1445
1446@item --expression=@var{expr}
1447@itemx -e @var{expr}
1448Build the package @var{expr} evaluates to.
1449
5401dd75 1450For example, @var{expr} may be @code{(@@ (gnu packages guile)
c78bd12b
LC
1451guile-1.8)}, which unambiguously designates this specific variant of
1452version 1.8 of Guile.
1453
1454@item --source
1455@itemx -S
1456Build the packages' source derivations, rather than the packages
1457themselves.
1458
e49951eb 1459For instance, @code{guix build -S gcc} returns something like
6798a8e4 1460@file{/nix/store/@dots{}-gcc-4.7.2.tar.bz2}, which is GCC's source tarball.
c78bd12b
LC
1461
1462@item --system=@var{system}
1463@itemx -s @var{system}
1464Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of
1465the host's system type.
1466
1467An example use of this is on Linux-based systems, which can emulate
1468different personalities. For instance, passing
1469@code{--system=i686-linux} on an @code{x86_64-linux} system allows users
1470to build packages in a complete 32-bit environment.
1471
e55ec43d
LC
1472@item --target=@var{triplet}
1473@cindex cross-compilation
1474Cross-build for @var{triplet}, which must be a valid GNU triplet, such
1475as @code{"mips64el-linux-gnu"} (@pxref{Configuration Names, GNU
1476configuration triplets,, configure, GNU Configure and Build System}).
1477
c78bd12b
LC
1478@item --derivations
1479@itemx -d
1480Return the derivation paths, not the output paths, of the given
1481packages.
1482
1483@item --keep-failed
1484@itemx -K
1485Keep the build tree of failed builds. Thus, if a build fail, its build
1486tree is kept under @file{/tmp}, in a directory whose name is shown at
1487the end of the build log. This is useful when debugging build issues.
1488
1489@item --dry-run
1490@itemx -n
1491Do not build the derivations.
1492
56b1f4b7
LC
1493@item --fallback
1494When substituting a pre-built binary fails, fall back to building
1495packages locally.
1496
c78bd12b
LC
1497@item --no-substitutes
1498Build instead of resorting to pre-built substitutes.
1499
969e678e
LC
1500@item --max-silent-time=@var{seconds}
1501When the build or substitution process remains silent for more than
1502@var{seconds}, terminate it and report a build failure.
1503
c78bd12b
LC
1504@item --cores=@var{n}
1505@itemx -c @var{n}
1506Allow the use of up to @var{n} CPU cores for the build. The special
1507value @code{0} means to use as many CPU cores as available.
1508
1509@item --root=@var{file}
1510@itemx -r @var{file}
1511Make @var{file} a symlink to the result, and register it as a garbage
1512collector root.
07ab4bf1
LC
1513
1514@item --verbosity=@var{level}
1515Use the given verbosity level. @var{level} must be an integer between 0
1516and 5; higher means more verbose output. Setting a level of 4 or more
1517may be helpful when debugging setup issues with the build daemon.
1518
c78bd12b
LC
1519@end table
1520
e49951eb 1521Behind the scenes, @command{guix build} is essentially an interface to
c78bd12b
LC
1522the @code{package-derivation} procedure of the @code{(guix packages)}
1523module, and to the @code{build-derivations} procedure of the @code{(guix
1524store)} module.
1525
210cc920
LC
1526@node Invoking guix download
1527@section Invoking @command{guix download}
1528
1529When writing a package definition, developers typically need to download
1530the package's source tarball, compute its SHA256 hash, and write that
1531hash in the package definition (@pxref{Defining Packages}). The
1532@command{guix download} tool helps with this task: it downloads a file
1533from the given URI, adds it to the store, and prints both its file name
1534in the store and its SHA256 hash.
1535
1536The fact that the downloaded file is added to the store saves bandwidth:
1537when the developer eventually tries to build the newly defined package
1538with @command{guix build}, the source tarball will not have to be
1539downloaded again because it is already in the store. It is also a
1540convenient way to temporarily stash files, which may be deleted
1541eventually (@pxref{Invoking guix gc}).
1542
1543The @command{guix download} command supports the same URIs as used in
1544package definitions. In particular, it supports @code{mirror://} URIs.
1545@code{https} URIs (HTTP over TLS) are supported @emph{provided} the
1546Guile bindings for GnuTLS are available in the user's environment; when
1547they are not available, an error is raised.
1548
1549The following option is available:
1550
1551@table @code
1552@item --format=@var{fmt}
1553@itemx -f @var{fmt}
1554Write the hash in the format specified by @var{fmt}. For more
1555information on the valid values for @var{fmt}, @ref{Invoking guix hash}.
1556@end table
1557
6c365eca
NK
1558@node Invoking guix hash
1559@section Invoking @command{guix hash}
1560
210cc920 1561The @command{guix hash} command computes the SHA256 hash of a file.
6c365eca
NK
1562It is primarily a convenience tool for anyone contributing to the
1563distribution: it computes the cryptographic hash of a file, which can be
1564used in the definition of a package (@pxref{Defining Packages}).
1565
1566The general syntax is:
1567
1568@example
1569guix hash @var{option} @var{file}
1570@end example
1571
1572@command{guix hash} has the following option:
1573
1574@table @code
1575
1576@item --format=@var{fmt}
1577@itemx -f @var{fmt}
210cc920 1578Write the hash in the format specified by @var{fmt}.
6c365eca
NK
1579
1580Supported formats: @code{nix-base32}, @code{base32}, @code{base16}
1581(@code{hex} and @code{hexadecimal} can be used as well).
1582
1583If the @option{--format} option is not specified, @command{guix hash}
1584will output the hash in @code{nix-base32}. This representation is used
1585in the definitions of packages.
1586
1587@end table
1588
37166310
LC
1589@node Invoking guix refresh
1590@section Invoking @command{guix refresh}
1591
1592The primary audience of the @command{guix refresh} command is developers
1593of the GNU software distribution. By default, it reports any packages
1594provided by the distribution that are outdated compared to the latest
1595upstream version, like this:
1596
1597@example
1598$ guix refresh
1599gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1
1600gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0
1601@end example
1602
1603It does so by browsing each package's FTP directory and determining the
1604highest version number of the source tarballs
1605therein@footnote{Currently, this only works for GNU packages.}.
1606
1607When passed @code{--update}, it modifies distribution source files to
1608update the version numbers and source tarball hashes of those packages'
1609recipes (@pxref{Defining Packages}). This is achieved by downloading
1610each package's latest source tarball and its associated OpenPGP
1611signature, authenticating the downloaded tarball against its signature
1612using @command{gpg}, and finally computing its hash. When the public
1613key used to sign the tarball is missing from the user's keyring, an
1614attempt is made to automatically retrieve it from a public key server;
1615when it's successful, the key is added to the user's keyring; otherwise,
1616@command{guix refresh} reports an error.
1617
1618The following options are supported:
1619
1620@table @code
1621
1622@item --update
1623@itemx -u
1624Update distribution source files (package recipes) in place.
1625@ref{Defining Packages}, for more information on package definitions.
1626
1627@item --select=[@var{subset}]
1628@itemx -s @var{subset}
1629Select all the packages in @var{subset}, one of @code{core} or
1630@code{non-core}.
1631
1632The @code{core} subset refers to all the packages at the core of the
1633distribution---i.e., packages that are used to build ``everything
1634else''. This includes GCC, libc, Binutils, Bash, etc. Usually,
1635changing one of these packages in the distribution entails a rebuild of
1636all the others. Thus, such updates are an inconvenience to users in
1637terms of build time or bandwidth used to achieve the upgrade.
1638
1639The @code{non-core} subset refers to the remaining packages. It is
1640typically useful in cases where an update of the core packages would be
1641inconvenient.
1642
1643@end table
1644
1645In addition, @command{guix refresh} can be passed one or more package
1646names, as in this example:
1647
1648@example
1649guix refresh -u emacs idutils
1650@end example
1651
1652@noindent
1653The command above specifically updates the @code{emacs} and
1654@code{idutils} packages. The @code{--select} option would have no
1655effect in this case.
1656
f9230085
LC
1657The following options can be used to customize GnuPG operation:
1658
1659@table @code
1660
1661@item --key-server=@var{host}
1662Use @var{host} as the OpenPGP key server when importing a public key.
1663
1664@item --gpg=@var{command}
1665Use @var{command} as the GnuPG 2.x command. @var{command} is searched
1666for in @code{$PATH}.
1667
1668@end table
1669
37166310 1670
a1ba8475
LC
1671@c *********************************************************************
1672@node GNU Distribution
1673@chapter GNU Distribution
1674
1675Guix comes with a distribution of free software@footnote{The term
1676``free'' here refers to the
1677@url{http://www.gnu.org/philosophy/free-sw.html,freedom provided to
1678users of that software}.} that form the basis of the GNU system. This
1679includes core GNU packages such as GNU libc, GCC, and Binutils, as well
1680as many GNU and non-GNU applications. The complete list of available
e49951eb
MW
1681packages can be seen by running @command{guix package} (@pxref{Invoking
1682guix package}):
a1ba8475
LC
1683
1684@example
e49951eb 1685guix package --list-available
a1ba8475
LC
1686@end example
1687
401c53c4
LC
1688Our goal is to build a practical 100% free software distribution of
1689Linux-based and other variants of GNU, with a focus on the promotion and
1690tight integration of GNU components, and an emphasis on programs and
1691tools that help users exert that freedom.
1692
1693@menu
91ef73d4 1694* Installing Debugging Files:: Feeding the debugger.
401c53c4 1695* Package Modules:: Packages from the programmer's viewpoint.
da7cabd4 1696* Packaging Guidelines:: Growing the distribution.
401c53c4 1697* Bootstrapping:: GNU/Linux built from scratch.
8b315a6d 1698* Porting:: Targeting another platform or kernel.
401c53c4
LC
1699@end menu
1700
1701Building this distribution is a cooperative effort, and you are invited
1702to join! @ref{Contributing}, for information about how you can help.
1703
b208a005 1704
91ef73d4
LC
1705@node Installing Debugging Files
1706@section Installing Debugging Files
1707
1708Program binaries, as produced by the GCC compilers for instance, are
1709typically written in the ELF format, with a section containing
1710@dfn{debugging information}. Debugging information is what allows the
1711debugger, GDB, to map binary code to source code; it is required to
1712debug a compiled program in good conditions.
1713
1714The problem with debugging information is that is takes up a fair amount
1715of disk space. For example, debugging information for the GNU C Library
1716weighs in at more than 60 MiB. Thus, as a user, keeping all the
1717debugging info of all the installed programs is usually not an option.
1718Yet, space savings should not come at the cost of an impediment to
1719debugging---especially in the GNU system, which should make it easier
1720for users to exert their computing freedom (@pxref{GNU Distribution}).
1721
1722Thankfully, the GNU Binary Utilities (Binutils) and GDB provide a
1723mechanism that allows users to get the best of both worlds: debugging
1724information can be stripped from the binaries and stored in separate
1725files. GDB is then able to load debugging information from those files,
1726when they are available (@pxref{Separate Debug Files,,, gdb, Debugging
1727with GDB}).
1728
1729The GNU distribution takes advantage of this by storing debugging
1730information in the @code{lib/debug} sub-directory of a separate package
1731output unimaginatively called @code{debug} (@pxref{Packages with
1732Multiple Outputs}). Users can choose to install the @code{debug} output
1733of a package when they need it. For instance, the following command
1734installs the debugging information for the GNU C Library and for GNU
1735Guile:
1736
1737@example
1738guix package -i glibc:debug -i guile:debug
1739@end example
1740
1741GDB must then be told to look for debug files in the user's profile, by
1742setting the @code{debug-file-directory} variable (consider setting it
1743from the @file{~/.gdbinit} file, @pxref{Startup,,, gdb, Debugging with
1744GDB}):
1745
1746@example
1747(gdb) set debug-file-directory ~/.guix-profile/lib/debug
1748@end example
1749
1750From there on, GDB will pick up debugging information from the
1751@code{.debug} files under @file{~/.guix-profile/lib/debug}.
1752
1753@c XXX: keep me up-to-date
1754The @code{debug} output mechanism in Guix is implemented by the
1755@code{gnu-build-system} (@pxref{Defining Packages}). Currently, it is
1756opt-in---debugging information is available only for those packages
1757whose definition explicitly declares a @code{debug} output. This may be
1758changed to opt-out in the future, if our build farm servers can handle
1759the load. To check whether a package has a @code{debug} output, use
1760@command{guix package --list-available} (@pxref{Invoking guix package}).
1761
1762
401c53c4
LC
1763@node Package Modules
1764@section Package Modules
1765
1766From a programming viewpoint, the package definitions of the
1767distribution are provided by Guile modules in the @code{(gnu packages
b81e1947
LC
1768...)} name space (@pxref{Modules, Guile modules,, guile, GNU Guile
1769Reference Manual}). For instance, the @code{(gnu packages emacs)}
1770module exports a variable named @code{emacs}, which is bound to a
401c53c4
LC
1771@code{<package>} object (@pxref{Defining Packages}). The @code{(gnu
1772packages)} module provides facilities for searching for packages.
a1ba8475
LC
1773
1774The distribution is fully @dfn{bootstrapped} and @dfn{self-contained}:
1775each package is built based solely on other packages in the
1776distribution. The root of this dependency graph is a small set of
1777@dfn{bootstrap binaries}, provided by the @code{(gnu packages
b81e1947
LC
1778bootstrap)} module. For more information on bootstrapping,
1779@ref{Bootstrapping}.
1780
da7cabd4
AE
1781@node Packaging Guidelines
1782@section Packaging Guidelines
b81e1947
LC
1783
1784The GNU distribution is nascent and may well lack some of your favorite
1785packages. This section describes how you can help make the distribution
c8c871d1 1786grow. @xref{Contributing}, for additional information on how you can
b81e1947
LC
1787help.
1788
b81e1947
LC
1789Free software packages are usually distributed in the form of
1790@dfn{source code tarballs}---typically @file{tar.gz} files that contain
1791all the source files. Adding a package to the distribution means
1792essentially two things: adding a @dfn{recipe} that describes how to
1793build the package, including a list of other packages required to build
1794it, and adding @dfn{package meta-data} along with that recipe, such as a
1795description and licensing information.
1796
1797In Guix all this information is embodied in @dfn{package definitions}.
1798Package definitions provide a high-level view of the package. They are
1799written using the syntax of the Scheme programming language; in fact,
1800for each package we define a variable bound to the package definition,
1801and export that variable from a module (@pxref{Package Modules}).
1802However, in-depth Scheme knowledge is @emph{not} a prerequisite for
1803creating packages. For more information on package definitions,
1804@ref{Defining Packages}.
1805
1806Once a package definition is in place, stored in a file in the Guix
1807source tree, it can be tested using the @command{guix build} command
1808(@pxref{Invoking guix build}). For example, assuming the new package is
1809called @code{gnew}, you may run this command from the Guix build tree:
1810
1811@example
1812./pre-inst-env guix build gnew --keep-failed
1813@end example
1814
1815Using @code{--keep-failed} makes it easier to debug build failures since
1816it provides access to the failed build tree.
1817
1818Once your package builds correctly, please send us a patch
1819(@pxref{Contributing}). Well, if you need help, we will be happy to
1820help you too. Once the patch is committed in the Guix repository, the
1821new package automatically gets built on the supported platforms by
1822@url{http://hydra.gnu.org/gnu/master, our continuous integration
1823system}.
1824
1825@cindex substituter
1826Users can obtain the new package definition simply by running
1827@command{guix pull} (@pxref{Invoking guix pull}). When
1828@code{hydra.gnu.org} is done building the package, installing the
1829package automatically downloads binaries from there (except when using
1830@code{--no-substitutes}). The only place where human intervention is
1831needed is to review and apply the patch.
401c53c4
LC
1832
1833
da7cabd4 1834@menu
ee85f3db
AE
1835* Software Freedom:: What may go into the distribution.
1836* Package Naming:: What's in a name?
1837* Version Numbers:: When the name is not enough.
1838* Python Modules:: Taming the snake.
da7cabd4
AE
1839@end menu
1840
1841@node Software Freedom
1842@subsection Software Freedom
1843
1844@c Adapted from http://www.gnu.org/philosophy/philosophy.html.
1845
1846The GNU operating system has been developed so that users can have
1847freedom in their computing. GNU is @dfn{free software}, meaning that
1848users have the @url{http://www.gnu.org/philosophy/free-sw.html,four
1849essential freedoms}: to run the program, to study and change the program
1850in source code form, to redistribute exact copies, and to distribute
1851modified versions. Packages found in the GNU distribution provide only
1852software that conveys these four freedoms.
1853
1854In addition, the GNU distribution follow the
1855@url{http://www.gnu.org/distros/free-system-distribution-guidelines.html,free
1856software distribution guidelines}. Among other things, these guidelines
1857reject non-free firmware, recommendations of non-free software, and
1858discuss ways to deal with trademarks and patents.
1859
1860
ee85f3db
AE
1861@node Package Naming
1862@subsection Package Naming
1863
c8c871d1 1864A package has actually two names associated with it:
ee85f3db 1865First, there is the name of the @emph{Scheme variable}, the one following
c8c871d1
AE
1866@code{define-public}. By this name, the package can be made known in the
1867Scheme code, for instance as input to another package. Second, there is
1868the string in the @code{name} field of a package definition. This name
1869is used by package management commands such as
1870@command{guix package} and @command{guix build}.
ee85f3db
AE
1871
1872Both are usually the same and correspond to the lowercase conversion of the
c8c871d1
AE
1873project name chosen upstream. For instance, the GNUnet project is packaged
1874as @code{gnunet}. We do not add @code{lib} prefixes for library packages,
1875unless these are already part of the official project name. But see
1876@ref{Python Modules} for special rules concerning modules for
ee85f3db
AE
1877the Python language.
1878
1879
1880@node Version Numbers
1881@subsection Version Numbers
1882
1883We usually package only the latest version of a given free software
c8c871d1
AE
1884project. But sometimes, for instance for incompatible library versions,
1885two (or more) versions of the same package are needed. These require
1886different Scheme variable names. We use the name as defined
1887in @ref{Package Naming}
ee85f3db
AE
1888for the most recent version; previous versions use the same name, suffixed
1889by @code{-} and the smallest prefix of the version number that may
1890distinguish the two versions.
1891
1892The name inside the package definition is the same for all versions of a
1893package and does not contain any version number.
1894
1895For instance, the versions 2.24.20 and 3.9.12 of GTK+ may be packaged as follows:
c8c871d1 1896
ee85f3db
AE
1897@example
1898(define-public gtk+
1899 (package
1900 (name "gtk+")
1901 (version "3.9.12")
1902 ...))
1903(define-public gtk+-2
1904 (package
1905 (name "gtk+")
1906 (version "2.24.20")
1907 ...))
1908@end example
1909If we also wanted GTK+ 3.8.2, this would be packaged as
1910@example
1911(define-public gtk+-3.8
1912 (package
1913 (name "gtk+")
1914 (version "3.8.2")
1915 ...))
1916@end example
1917
1918
1919@node Python Modules
1920@subsection Python Modules
1921
1922We currently package Python 2 and Python 3, under the Scheme variable names
1923@code{python-2} and @code{python} as explained in @ref{Version Numbers}.
1924To avoid confusion and naming clashes with other programming languages, it
1925seems desirable that the name of a package for a Python module contains
1926the word @code{python}.
c8c871d1 1927
ee85f3db
AE
1928Some modules are compatible with only one version of Python, others with both.
1929If the package Foo compiles only with Python 3, we name it
1930@code{python-foo}; if it compiles only with Python 2, we name it
1931@code{python2-foo}. If it is compatible with both versions, we create two
1932packages with the corresponding names.
1933
1934If a project already contains the word @code{python}, we drop this;
1935for instance, the module python-dateutil is packaged under the names
1936@code{python-dateutil} and @code{python2-dateutil}.
1937
1938
1939
1940
1941
401c53c4
LC
1942@node Bootstrapping
1943@section Bootstrapping
1944
1945@c Adapted from the ELS 2013 paper.
1946
1947@cindex bootstrapping
1948
1949Bootstrapping in our context refers to how the distribution gets built
1950``from nothing''. Remember that the build environment of a derivation
1951contains nothing but its declared inputs (@pxref{Introduction}). So
1952there's an obvious chicken-and-egg problem: how does the first package
1953get built? How does the first compiler get compiled? Note that this is
1954a question of interest only to the curious hacker, not to the regular
1955user, so you can shamelessly skip this section if you consider yourself
1956a ``regular user''.
1957
1958@cindex bootstrap binaries
1959The GNU system is primarily made of C code, with libc at its core. The
1960GNU build system itself assumes the availability of a Bourne shell and
1961command-line tools provided by GNU Coreutils, Awk, Findutils, `sed', and
1962`grep'. Furthermore, build programs---programs that run
1963@code{./configure}, @code{make}, etc.---are written in Guile Scheme
1964(@pxref{Derivations}). Consequently, to be able to build anything at
1965all, from scratch, Guix relies on pre-built binaries of Guile, GCC,
1966Binutils, libc, and the other packages mentioned above---the
1967@dfn{bootstrap binaries}.
1968
1969These bootstrap binaries are ``taken for granted'', though we can also
ba7ea5ce 1970re-create them if needed (more on that later).
401c53c4
LC
1971
1972@unnumberedsubsec Preparing to Use the Bootstrap Binaries
1973
1974@c As of Emacs 24.3, Info-mode displays the image, but since it's a
1975@c large image, it's hard to scroll. Oh well.
29f66ddd 1976@image{images/bootstrap-graph,6in,,Dependency graph of the early bootstrap derivations}
401c53c4
LC
1977
1978The figure above shows the very beginning of the dependency graph of the
1979distribution, corresponding to the package definitions of the @code{(gnu
1980packages bootstrap)} module. At this level of detail, things are
1981slightly complex. First, Guile itself consists of an ELF executable,
1982along with many source and compiled Scheme files that are dynamically
1983loaded when it runs. This gets stored in the @file{guile-2.0.7.tar.xz}
1984tarball shown in this graph. This tarball is part of Guix's ``source''
1985distribution, and gets inserted into the store with @code{add-to-store}
1986(@pxref{The Store}).
a1ba8475 1987
401c53c4
LC
1988But how do we write a derivation that unpacks this tarball and adds it
1989to the store? To solve this problem, the @code{guile-bootstrap-2.0.drv}
1990derivation---the first one that gets built---uses @code{bash} as its
1991builder, which runs @code{build-bootstrap-guile.sh}, which in turn calls
1992@code{tar} to unpack the tarball. Thus, @file{bash}, @file{tar},
1993@file{xz}, and @file{mkdir} are statically-linked binaries, also part of
1994the Guix source distribution, whose sole purpose is to allow the Guile
1995tarball to be unpacked.
1996
1997Once @code{guile-bootstrap-2.0.drv} is built, we have a functioning
1998Guile that can be used to run subsequent build programs. Its first task
1999is to download tarballs containing the other pre-built binaries---this
2000is what the @code{.tar.xz.drv} derivations do. Guix modules such as
2001@code{ftp-client.scm} are used for this purpose. The
2002@code{module-import.drv} derivations import those modules in a directory
2003in the store, using the original layout. The
2004@code{module-import-compiled.drv} derivations compile those modules, and
2005write them in an output directory with the right layout. This
2006corresponds to the @code{#:modules} argument of
2007@code{build-expression->derivation} (@pxref{Derivations}).
2008
2009Finally, the various tarballs are unpacked by the
2010derivations @code{gcc-bootstrap-0.drv}, @code{glibc-bootstrap-0.drv},
2011etc., at which point we have a working C tool chain.
2012
2013
2014@unnumberedsubsec Building the Build Tools
2015
2016@c TODO: Add a package-level dependency graph generated from (gnu
2017@c packages base).
2018
2019Bootstrapping is complete when we have a full tool chain that does not
2020depend on the pre-built bootstrap tools discussed above. This
2021no-dependency requirement is verified by checking whether the files of
2022the final tool chain contain references to the @file{/nix/store}
2023directories of the bootstrap inputs. The process that leads to this
2024``final'' tool chain is described by the package definitions found in
2025the @code{(gnu packages base)} module.
2026
2027@c See <http://lists.gnu.org/archive/html/gnu-system-discuss/2012-10/msg00000.html>.
2028The first tool that gets built with the bootstrap binaries is
2029GNU Make, which is a prerequisite for all the following packages.
2030From there Findutils and Diffutils get built.
2031
2032Then come the first-stage Binutils and GCC, built as pseudo cross
2033tools---i.e., with @code{--target} equal to @code{--host}. They are
2034used to build libc. Thanks to this cross-build trick, this libc is
2035guaranteed not to hold any reference to the initial tool chain.
2036
2037From there the final Binutils and GCC are built. GCC uses @code{ld}
2038from the final Binutils, and links programs against the just-built libc.
2039This tool chain is used to build the other packages used by Guix and by
2040the GNU Build System: Guile, Bash, Coreutils, etc.
2041
2042And voilà! At this point we have the complete set of build tools that
2043the GNU Build System expects. These are in the @code{%final-inputs}
2044variables of the @code{(gnu packages base)} module, and are implicitly
2045used by any package that uses @code{gnu-build-system} (@pxref{Defining
2046Packages}).
2047
2048
2049@unnumberedsubsec Building the Bootstrap Binaries
2050
2051Because the final tool chain does not depend on the bootstrap binaries,
2052those rarely need to be updated. Nevertheless, it is useful to have an
2053automated way to produce them, should an update occur, and this is what
2054the @code{(gnu packages make-bootstrap)} module provides.
2055
2056The following command builds the tarballs containing the bootstrap
2057binaries (Guile, Binutils, GCC, libc, and a tarball containing a mixture
2058of Coreutils and other basic command-line tools):
a1ba8475 2059
401c53c4
LC
2060@example
2061guix build bootstrap-tarballs
2062@end example
2063
2064The generated tarballs are those that should be referred to in the
2065@code{(gnu packages bootstrap)} module mentioned at the beginning of
2066this section.
2067
2068Still here? Then perhaps by now you've started to wonder: when do we
2069reach a fixed point? That is an interesting question! The answer is
2070unknown, but if you would like to investigate further (and have
2071significant computational and storage resources to do so), then let us
2072know.
a1ba8475 2073
8b315a6d
LC
2074@node Porting
2075@section Porting to a New Platform
2076
2077As discussed above, the GNU distribution is self-contained, and
2078self-containment is achieved by relying on pre-built ``bootstrap
2079binaries'' (@pxref{Bootstrapping}). These binaries are specific to an
2080operating system kernel, CPU architecture, and application binary
2081interface (ABI). Thus, to port the distribution to a platform that is
2082not yet supported, one must build those bootstrap binaries, and update
2083the @code{(gnu packages bootstrap)} module to use them on that platform.
2084
2085Fortunately, Guix can @emph{cross compile} those bootstrap binaries.
2086When everything goes well, and assuming the GNU tool chain supports the
2087target platform, this can be as simple as running a command like this
2088one:
2089
2090@example
2091guix build --target=armv5tel-linux-gnueabi bootstrap-tarballs
2092@end example
2093
72e25e35
LC
2094Once these are built, the @code{(gnu packages bootstrap)} module needs
2095to be updated to refer to these binaries on the target platform. In
2096addition, the @code{glibc-dynamic-linker} procedure in that module must
2097be augmented to return the right file name for libc's dynamic linker on
2098that platform; likewise, @code{system->linux-architecture} in @code{(gnu
2099packages linux)} must be taught about the new platform.
2100
8b315a6d
LC
2101In practice, there may be some complications. First, it may be that the
2102extended GNU triplet that specifies an ABI (like the @code{eabi} suffix
2103above) is not recognized by all the GNU tools. Typically, glibc
2104recognizes some of these, whereas GCC uses an extra @code{--with-abi}
ba7ea5ce 2105configure flag (see @code{gcc.scm} for examples of how to handle this).
8b315a6d
LC
2106Second, some of the required packages could fail to build for that
2107platform. Lastly, the generated binaries could be broken for some
2108reason.
2109
9bf3c1a7
LC
2110
2111@c *********************************************************************
2112@node Contributing
2113@chapter Contributing
2114
2115This project is a cooperative effort, and we need your help to make it
63f6004b 2116grow! Please get in touch with us on @email{guix-devel@@gnu.org}. We
a1ba8475 2117welcome ideas, bug reports, patches, and anything that may be helpful to
da7cabd4
AE
2118the project. We particularly welcome help on packaging
2119(@pxref{Packaging Guidelines}).
a1ba8475 2120
9bf3c1a7
LC
2121Please see the
2122@url{http://git.savannah.gnu.org/cgit/guix.git/tree/HACKING,
2123@file{HACKING} file} that comes with the Guix source code for practical
2124details about contributions.
2125
c78bd12b 2126
568717fd
LC
2127@c *********************************************************************
2128@node Acknowledgments
2129@chapter Acknowledgments
2130
2131Guix is based on the Nix package manager, which was designed and
2132implemented by Eelco Dolstra. Nix pioneered functional package
2133management, and promoted unprecedented features, such as transactional
2134package upgrades and rollbacks, per-user profiles, and referentially
2135transparent build processes. Without this work, Guix would not exist.
2136
2137The Nix-based software distributions, Nixpkgs and NixOS, have also been
2138an inspiration for Guix.
2139
2140@c *********************************************************************
2141@node GNU Free Documentation License
2142@appendix GNU Free Documentation License
2143
2144@include fdl-1.3.texi
2145
2146@c *********************************************************************
2147@node Concept Index
2148@unnumbered Concept Index
2149@printindex cp
2150
2151@node Function Index
2152@unnumbered Function Index
2153@printindex fn
2154
2155@bye
2156
2157@c Local Variables:
2158@c ispell-local-dictionary: "american";
2159@c End: