cfa5aac3269086ea205872769b4701831ebffebe
[jackhill/guix/guix.git] / doc / guix.texi
1 \input texinfo
2 @c -*-texinfo-*-
3
4 @c %**start of header
5 @setfilename guix.info
6 @documentencoding UTF-8
7 @settitle GNU Guix Reference Manual
8 @c %**end of header
9
10 @include version.texi
11
12 @copying
13 Copyright @copyright{} 2012, 2013 Ludovic Courtès@*
14 Copyright @copyright{} 2013 Andreas Enge@*
15 Copyright @copyright{} 2013 Nikita Karetnikov
16
17 Permission is granted to copy, distribute and/or modify this document
18 under the terms of the GNU Free Documentation License, Version 1.3 or
19 any later version published by the Free Software Foundation; with no
20 Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
21 copy of the license is included in the section entitled ``GNU Free
22 Documentation License''.
23 @end copying
24
25 @dircategory Package management
26 @direntry
27 * guix: (guix). Guix, the functional package manager.
28 * guix package: (guix)Invoking guix package
29 Managing packages with Guix.
30 * guix build: (guix)Invoking guix build
31 Building packages with Guix.
32 @end direntry
33
34 @titlepage
35 @title GNU Guix Reference Manual
36 @subtitle Using the GNU Guix Functional Package Manager
37 @author Ludovic Courtès
38 @author Andreas Enge
39 @author Nikita Karetnikov
40
41 @page
42 @vskip 0pt plus 1filll
43 Edition @value{EDITION} @*
44 @value{UPDATED} @*
45
46 @insertcopying
47 @end titlepage
48
49 @contents
50
51 @c *********************************************************************
52 @node Top
53 @top GNU Guix
54
55 This document describes GNU Guix version @value{VERSION}, a functional
56 package management tool written for the GNU system.
57
58 @menu
59 * Introduction:: What is Guix about?
60 * Installation:: Installing Guix.
61 * Package Management:: Package installation, upgrade, etc.
62 * Programming Interface:: Using Guix in Scheme.
63 * Utilities:: Package management commands.
64 * GNU Distribution:: Software for your friendly GNU system.
65 * Contributing:: Your help needed!
66
67 * Acknowledgments:: Thanks!
68 * GNU Free Documentation License:: The license of this manual.
69 * Concept Index:: Concepts.
70 * Function Index:: Functions.
71 @end menu
72
73 @c *********************************************************************
74 @node Introduction
75 @chapter Introduction
76
77 GNU Guix@footnote{``Guix'' is pronounced like ``geeks'', or ``ɡiːks''
78 using the international phonetic alphabet (IPA).} is a functional
79 package management tool for the GNU system. Package management consists
80 of all activities that relate to building packages from sources,
81 honoring their build-time and run-time dependencies,
82 installing packages in user environments, upgrading installed packages
83 to new versions or rolling back to a previous set, removing unused
84 software packages, etc.
85
86 @cindex functional package management
87 The term @dfn{functional} refers to a specific package management
88 discipline. In Guix, the package build and installation process is seen
89 as a function, in the mathematical sense. That function takes inputs,
90 such as build scripts, a compiler, and libraries, and
91 returns an installed package. As a pure function, its result depends
92 solely on its inputs---for instance, it cannot refer to software or
93 scripts that were not explicitly passed as inputs. A build function
94 always produces the same result when passed a given set of inputs. It
95 cannot alter the system's environment in
96 any way; for instance, it cannot create, modify, or delete files outside
97 of its build and installation directories. This is achieved by running
98 build processes in isolated environments (or @dfn{containers}), where only their
99 explicit inputs are visible.
100
101 @cindex store
102 The result of package build functions is @dfn{cached} in the file
103 system, in a special directory called @dfn{the store} (@pxref{The
104 Store}). Each package is installed in a directory of its own, in the
105 store---by default under @file{/nix/store}. The directory name contains
106 a hash of all the inputs used to build that package; thus, changing an
107 input yields a different directory name.
108
109 This approach is the foundation of Guix's salient features: support for
110 transactional package upgrade and rollback, per-user installation, and
111 garbage collection of packages (@pxref{Features}).
112
113 Guix has a command-line interface, which allows users to build, install,
114 upgrade, and remove packages, as well as a Scheme programming interface.
115
116 Last but not least, Guix is used to build a distribution of the GNU
117 system, with many GNU and non-GNU free software packages. @xref{GNU
118 Distribution}.
119
120 @c *********************************************************************
121 @node Installation
122 @chapter Installation
123
124 GNU Guix is available for download from its website at
125 @url{http://www.gnu.org/software/guix/}. This section describes the
126 software requirements of Guix, as well as how to install it and get
127 ready to use it.
128
129 The build procedure for Guix is the same as for other GNU software, and
130 is not covered here. Please see the files @file{README} and
131 @file{INSTALL} in the Guix source tree for additional details.
132
133 @menu
134 * Requirements:: Software needed to build and run Guix.
135 * Setting Up the Daemon:: Preparing the build daemon's environment.
136 * Invoking guix-daemon:: Running the build daemon.
137 @end menu
138
139 @node Requirements
140 @section Requirements
141
142 GNU Guix depends on the following packages:
143
144 @itemize
145 @item @url{http://gnu.org/software/guile/, GNU Guile}, version 2.0.5 or later;
146 @item @url{http://gnupg.org/, GNU libgcrypt}
147 @end itemize
148
149 Unless @code{--disable-daemon} was passed to @command{configure}, the
150 following packages are also needed:
151
152 @itemize
153 @item @url{http://sqlite.org, SQLite 3}
154 @item @url{http://www.bzip.org, libbz2}
155 @item @url{http://gcc.gnu.org, GCC's g++}
156 @end itemize
157
158 When a working installation of @url{http://nixos.org/nix/, the Nix package
159 manager} is available, you
160 can instead configure Guix with @code{--disable-daemon}. In that case,
161 Nix replaces the three dependencies above.
162
163 Guix is compatible with Nix, so it is possible to share the same store
164 between both. To do so, you must pass @command{configure} not only the
165 same @code{--with-store-dir} value, but also the same
166 @code{--localstatedir} value. The latter is essential because it
167 specifies where the database that stores metadata about the store is
168 located, among other things. The default values are
169 @code{--with-store-dir=/nix/store} and @code{--localstatedir=/nix/var}.
170 Note that @code{--disable-daemon} is not required if
171 your goal is to share the store with Nix.
172
173 @node Setting Up the Daemon
174 @section Setting Up the Daemon
175
176 @cindex daemon
177 Operations such as building a package or running the garbage collector
178 are all performed by a specialized process, the @dfn{Guix daemon}, on
179 behalf of clients. Only the daemon may access the store and its
180 associated database. Thus, any operation that manipulates the store
181 goes through the daemon. For instance, command-line tools such as
182 @command{guix package} and @command{guix build} communicate with the
183 daemon (@i{via} remote procedure calls) to instruct it what to do.
184
185 In a standard multi-user setup, Guix and its daemon---the
186 @command{guix-daemon} program---are installed by the system
187 administrator; @file{/nix/store} is owned by @code{root} and
188 @command{guix-daemon} runs as @code{root}. Unprivileged users may use
189 Guix tools to build packages or otherwise access the store, and the
190 daemon will do it on their behalf, ensuring that the store is kept in a
191 consistent state, and allowing built packages to be shared among users.
192
193 @cindex build users
194 When @command{guix-daemon} runs as @code{root}, you may not want package
195 build processes themselves to run as @code{root} too, for obvious
196 security reasons. To avoid that, a special pool of @dfn{build users}
197 should be created for use by build processes started by the daemon.
198 These build users need not have a shell and a home directory: they will
199 just be used when the daemon drops @code{root} privileges in build
200 processes. Having several such users allows the daemon to launch
201 distinct build processes under separate UIDs, which guarantees that they
202 do not interfere with each other---an essential feature since builds are
203 regarded as pure functions (@pxref{Introduction}).
204
205 On a GNU/Linux system, a build user pool may be created like this (using
206 Bash syntax and the @code{shadow} commands):
207
208 @c See http://lists.gnu.org/archive/html/bug-guix/2013-01/msg00239.html
209 @c for why `-G' is needed.
210 @example
211 # groupadd guix-builder
212 # for i in `seq 1 10`;
213 do
214 useradd -g guix-builder -G guix-builder \
215 -d /var/empty -s `which nologin` \
216 -c "Guix build user $i" guix-builder$i;
217 done
218 @end example
219
220 @noindent
221 The @code{guix-daemon} program may then be run as @code{root} with:
222
223 @example
224 # guix-daemon --build-users-group=guix-builder
225 @end example
226
227 @cindex chroot
228 @noindent
229 This way, the daemon starts build processes in a chroot, under one of
230 the @code{guix-builder} users. On GNU/Linux, by default, the chroot
231 environment contains nothing but the @code{/dev} and @code{/proc}
232 directories@footnote{On some systems @code{/dev/shm}, which supports
233 shared memory, is a symlink to another directory such as
234 @code{/run/shm}, that is @emph{not} is the chroot. When that is the
235 case, shared memory support is unavailable in the chroot environment.
236 The workaround is to make sure that @file{/dev/shm} is directly a
237 @code{tmpfs} mount point.}.
238
239 Guix may also be used in a single-user setup, with @command{guix-daemon}
240 running as an unprivileged user. However, to maximize non-interference
241 of build processes, the daemon still needs to perform certain operations
242 that are restricted to @code{root} on GNU/Linux: it should be able to
243 run build processes in a chroot, and to run them under different UIDs.
244 To that end, the @command{nix-setuid-helper} program is provided; it is
245 a small C program (less than 300 lines) that, if it is made setuid
246 @code{root}, can be executed by the daemon to perform these operations
247 on its behalf. The @code{root}-owned @file{/etc/nix-setuid.conf} file
248 is read by @command{nix-setuid-helper}; it should contain exactly two
249 words: the user name under which the authorized @command{guix-daemon}
250 runs, and the name of the build users group.
251
252 If you are installing Guix as an unprivileged user and do not have the
253 ability to make @file{nix-setuid-helper} setuid-@code{root}, it is still
254 possible to run @command{guix-daemon}. However, build processes will
255 not be isolated from one another, and not from the rest of the system.
256 Thus, build processes may interfere with each other, and may access
257 programs, libraries, and other files available on the system---making it
258 much harder to view them as @emph{pure} functions.
259
260 @node Invoking guix-daemon
261 @section Invoking @command{guix-daemon}
262
263 The @command{guix-daemon} program implements all the functionality to
264 access the store. This includes launching build processes, running the
265 garbage collector, querying the availability of a build result, etc. It
266 is normally run as @code{root} like this:
267
268 @example
269 # guix-daemon --build-users-group=guix-builder
270 @end example
271
272 @noindent
273 For details on how to set it up, @ref{Setting Up the Daemon}.
274
275 @cindex chroot
276 @cindex container, build environment
277 @cindex build environment
278 @cindex reproducible builds
279 By default, @command{guix-daemon} launches build processes under
280 different UIDs, taken from the build group specified with
281 @code{--build-users-group}. In addition, each build process is run in a
282 chroot environment that only contains the subset of the store that the
283 build process depends on, as specified by its derivation
284 (@pxref{Programming Interface, derivation}), plus a set of specific
285 system directories. By default, the latter contains @file{/dev} and
286 @file{/dev/pts}. Furthermore, on GNU/Linux, the build environment is a
287 @dfn{container}: in addition to having its own file system tree, it has
288 a separate mount name space, its own PID name space, network name space,
289 etc. This helps achieve reproducible builds (@pxref{Features}).
290
291 The following command-line options are supported:
292
293 @table @code
294 @item --build-users-group=@var{group}
295 Take users from @var{group} to run build processes (@pxref{Setting Up
296 the Daemon, build users}).
297
298 @item --no-substitutes
299 @cindex substitutes
300 Do not use substitutes for build products. That is, always build things
301 locally instead of allowing downloads of pre-built binaries.
302
303 By default substitutes are used, unless the client---such as the
304 @command{guix package} command---is explicitly invoked with
305 @code{--no-substitutes}.
306
307 When the daemon runs with @code{--no-substitutes}, clients can still
308 explicitly enable substitution @i{via} the @code{set-build-options}
309 remote procedure call (@pxref{The Store}).
310
311 @item --cache-failures
312 Cache build failures. By default, only successful builds are cached.
313
314 @item --cores=@var{n}
315 @itemx -c @var{n}
316 Use @var{n} CPU cores to build each derivation; @code{0} means as many
317 as available.
318
319 The default value is @code{1}, but it may be overridden by clients, such
320 as the @code{--cores} option of @command{guix build} (@pxref{Invoking
321 guix build}).
322
323 The effect is to define the @code{NIX_BUILD_CORES} environment variable
324 in the build process, which can then use it to exploit internal
325 parallelism---for instance, by running @code{make -j$NIX_BUILD_CORES}.
326
327 @item --max-jobs=@var{n}
328 @itemx -M @var{n}
329 Allow at most @var{n} build jobs in parallel. The default value is
330 @code{1}.
331
332 @item --debug
333 Produce debugging output.
334
335 This is useful to debug daemon start-up issues, but then it may be
336 overridden by clients, for example the @code{--verbosity} option of
337 @command{guix build} (@pxref{Invoking guix build}).
338
339 @item --chroot-directory=@var{dir}
340 Add @var{dir} to the build chroot.
341
342 Doing this may change the result of build processes---for instance if
343 they use optional dependencies found in @var{dir} when it is available,
344 and not otherwise. For that reason, it is not recommended to do so.
345 Instead, make sure that each derivation declares all the inputs that it
346 needs.
347
348 @item --disable-chroot
349 Disable chroot builds.
350
351 Using this option is not recommended since, again, it would allow build
352 processes to gain access to undeclared dependencies.
353
354 @item --disable-log-compression
355 Disable compression of the build logs.
356
357 Unless @code{--lose-logs} is used, all the build logs are kept in the
358 @var{localstatedir}. To save space, the daemon automatically compresses
359 them with bzip2 by default. This option disables that.
360
361 @item --disable-store-optimization
362 Disable automatic file ``deduplication'' in the store.
363
364 By default, files added to the store are automatically ``deduplicated'':
365 if a newly added file is identical as another one found in the store,
366 the daemon makes the new file a hard link to the other file. This
367 slightly increases the input/output load at the end of a build process.
368 This option disables this.
369
370 @item --impersonate-linux-2.6
371 On Linux-based systems, impersonate Linux 2.6. This means that the
372 kernel's @code{uname} system call will report 2.6 as the release number.
373
374 This might be helpful to build programs that (usually wrongfully) depend
375 on the kernel version number.
376
377 @item --lose-logs
378 Do not keep build logs. By default they are kept under
379 @code{@var{localstatedir}/nix/log}.
380
381 @item --system=@var{system}
382 Assume @var{system} as the current system type. By default it is the
383 architecture/kernel pair found at configure time, such as
384 @code{x86_64-linux}.
385
386 @item --listen=@var{socket}
387 Listen for connections on @var{socket}, the file name of a Unix-domain
388 socket. The default socket is
389 @file{@var{localstatedir}/daemon-socket/socket}. This option is only
390 useful in exceptional circumstances, such as if you need to run several
391 daemons on the same machine.
392 @end table
393
394
395 @c *********************************************************************
396 @node Package Management
397 @chapter Package Management
398
399 The purpose of GNU Guix is to allow users to easily install, upgrade, and
400 remove software packages, without having to know about their build
401 procedure or dependencies. Guix also goes beyond this obvious set of
402 features.
403
404 This chapter describes the main features of Guix, as well as the package
405 management tools it provides.
406
407 @menu
408 * Features:: How Guix will make your life brighter.
409 * Invoking guix package:: Package installation, removal, etc.
410 * Packages with Multiple Outputs:: Single source package, multiple outputs.
411 * Invoking guix gc:: Running the garbage collector.
412 * Invoking guix pull:: Fetching the latest Guix and distribution.
413 @end menu
414
415 @node Features
416 @section Features
417
418 When using Guix, each package ends up in the @dfn{package store}, in its
419 own directory---something that resembles
420 @file{/nix/store/xxx-package-1.2}, where @code{xxx} is a base32 string.
421
422 Instead of referring to these directories, users have their own
423 @dfn{profile}, which points to the packages that they actually want to
424 use. These profiles are stored within each user's home directory, at
425 @code{$HOME/.guix-profile}.
426
427 For example, @code{alice} installs GCC 4.7.2. As a result,
428 @file{/home/alice/.guix-profile/bin/gcc} points to
429 @file{/nix/store/@dots{}-gcc-4.7.2/bin/gcc}. Now, on the same machine,
430 @code{bob} had already installed GCC 4.8.0. The profile of @code{bob}
431 simply continues to point to
432 @file{/nix/store/@dots{}-gcc-4.8.0/bin/gcc}---i.e., both versions of GCC
433 coexist on the same system without any interference.
434
435 The @command{guix package} command is the central tool to manage
436 packages (@pxref{Invoking guix package}). It operates on those per-user
437 profiles, and can be used @emph{with normal user privileges}.
438
439 The command provides the obvious install, remove, and upgrade
440 operations. Each invocation is actually a @emph{transaction}: either
441 the specified operation succeeds, or nothing happens. Thus, if the
442 @command{guix package} process is terminated during the transaction,
443 or if a power outage occurs during the transaction, then the user's
444 profile remains in its previous state, and remains usable.
445
446 In addition, any package transaction may be @emph{rolled back}. So, if,
447 for example, an upgrade installs a new version of a package that turns
448 out to have a serious bug, users may roll back to the previous instance
449 of their profile, which was known to work well.
450
451 All those packages in the package store may be @emph{garbage-collected}.
452 Guix can determine which packages are still referenced by the user
453 profiles, and remove those that are provably no longer referenced
454 (@pxref{Invoking guix gc}). Users may also explicitly remove old
455 generations of their profile so that the packages they refer to can be
456 collected.
457
458 @cindex reproducibility
459 @cindex reproducible builds
460 Finally, Guix takes a @dfn{purely functional} approach to package
461 management, as described in the introduction (@pxref{Introduction}).
462 Each @file{/nix/store} package directory name contains a hash of all the
463 inputs that were used to build that package---compiler, libraries, build
464 scripts, etc. This direct correspondence allows users to make sure a
465 given package installation matches the current state of their
466 distribution. It also helps maximize @dfn{build reproducibility}:
467 thanks to the isolated build environments that are used, a given build
468 is likely to yield bit-identical files when performed on different
469 machines (@pxref{Invoking guix-daemon, container}).
470
471 @cindex substitute
472 This foundation allows Guix to support @dfn{transparent binary/source
473 deployment}. When a pre-built binary for a @file{/nix/store} path is
474 available from an external source---a @dfn{substitute}, Guix just
475 downloads it@footnote{@c XXX: Remove me when outdated.
476 As of version @value{VERSION}, substitutes are downloaded from
477 @url{http://hydra.gnu.org/} but are @emph{not} authenticated---i.e.,
478 Guix cannot tell whether binaries it downloaded have been tampered with,
479 nor whether they come from the genuine @code{gnu.org} build farm. This
480 will be fixed in future versions. In the meantime, concerned users can
481 opt for @code{--no-substitutes} (@pxref{Invoking guix-daemon}).};
482 otherwise, it builds the package from source, locally.
483
484 @node Invoking guix package
485 @section Invoking @command{guix package}
486
487 The @command{guix package} command is the tool that allows users to
488 install, upgrade, and remove packages, as well as rolling back to
489 previous configurations. It operates only on the user's own profile,
490 and works with normal user privileges (@pxref{Features}). Its syntax
491 is:
492
493 @example
494 guix package @var{options}
495 @end example
496
497 Primarily, @var{options} specifies the operations to be performed during
498 the transaction. Upon completion, a new profile is created, but
499 previous generations of the profile remain available, should the user
500 want to roll back.
501
502 For each user, a symlink to the user's default profile is automatically
503 created in @file{$HOME/.guix-profile}. This symlink always points to the
504 current generation of the user's default profile. Thus, users can add
505 @file{$HOME/.guix-profile/bin} to their @code{PATH} environment
506 variable, and so on.
507
508 In a multi-user setup, user profiles must be stored in a place
509 registered as a @dfn{garbage-collector root}, which
510 @file{$HOME/.guix-profile} points to (@pxref{Invoking guix gc}). That
511 directory is normally
512 @code{@var{localstatedir}/profiles/per-user/@var{user}}, where
513 @var{localstatedir} is the value passed to @code{configure} as
514 @code{--localstatedir}, and @var{user} is the user name. It must be
515 created by @code{root}, with @var{user} as the owner. When it does not
516 exist, or is not owned by @var{user}, @command{guix package} emits an
517 error about it.
518
519 The @var{options} can be among the following:
520
521 @table @code
522
523 @item --install=@var{package}
524 @itemx -i @var{package}
525 Install @var{package}.
526
527 @var{package} may specify either a simple package name, such as
528 @code{guile}, or a package name followed by a hyphen and version number,
529 such as @code{guile-1.8.8}. If no version number is specified, the
530 newest available version will be selected. In addition, @var{package}
531 may contain a colon, followed by the name of one of the outputs of the
532 package, as in @code{gcc:doc} or @code{binutils-2.22:lib}
533 (@pxref{Packages with Multiple Outputs}).
534
535 @cindex propagated inputs
536 Sometimes packages have @dfn{propagated inputs}: these are dependencies
537 that automatically get installed along with the required package.
538
539 An example is the GNU MPC library: its C header files refer to those of
540 the GNU MPFR library, which in turn refer to those of the GMP library.
541 Thus, when installing MPC, the MPFR and GMP libraries also get installed
542 in the profile; removing MPC also removes MPFR and GMP---unless they had
543 also been explicitly installed independently.
544
545 Besides, packages sometimes rely on the definition of environment
546 variables for their search paths (see explanation of
547 @code{--search-paths} below). Any missing or possibly incorrect
548 environment variable definitions are reported here.
549
550 @c XXX: keep me up-to-date
551 Finally, when installing a GNU package, the tool reports the
552 availability of a newer upstream version. In the future, it may provide
553 the option of installing directly from the upstream version, even if
554 that version is not yet in the distribution.
555
556 @item --install-from-expression=@var{exp}
557 @itemx -e @var{exp}
558 Install the package @var{exp} evaluates to.
559
560 @var{exp} must be a Scheme expression that evaluates to a
561 @code{<package>} object. This option is notably useful to disambiguate
562 between same-named variants of a package, with expressions such as
563 @code{(@@ (gnu packages base) guile-final)}.
564
565 Note that this option installs the first output of the specified
566 package, which may be insufficient when needing a specific output of a
567 multiple-output package.
568
569 @item --remove=@var{package}
570 @itemx -r @var{package}
571 Remove @var{package}.
572
573 As for @code{--install}, @var{package} may specify a version number
574 and/or output name in addition to the package name. For instance,
575 @code{-r glibc:debug} would remove the @code{debug} output of
576 @code{glibc}.
577
578 @item --upgrade[=@var{regexp}]
579 @itemx -u [@var{regexp}]
580 Upgrade all the installed packages. When @var{regexp} is specified, upgrade
581 only installed packages whose name matches @var{regexp}.
582
583 Note that this upgrades package to the latest version of packages found
584 in the distribution currently installed. To update your distribution,
585 you should regularly run @command{guix pull} (@pxref{Invoking guix
586 pull}).
587
588 @item --roll-back
589 Roll back to the previous @dfn{generation} of the profile---i.e., undo
590 the last transaction.
591
592 When combined with options such as @code{--install}, roll back occurs
593 before any other actions.
594
595 When rolling back from the first generation that actually contains
596 installed packages, the profile is made to point to the @dfn{zeroth
597 generation}, which contains no files apart from its own meta-data.
598
599 Installing, removing, or upgrading packages from a generation that has
600 been rolled back to overwrites previous future generations. Thus, the
601 history of a profile's generations is always linear.
602
603 @item --search-paths
604 @cindex search paths
605 Report environment variable definitions, in Bash syntax, that may be
606 needed in order to use the set of installed packages. These environment
607 variables are used to specify @dfn{search paths} for files used by some
608 of the installed packages.
609
610 For example, GCC needs the @code{CPATH} and @code{LIBRARY_PATH}
611 environment variables to be defined so it can look for headers and
612 libraries in the user's profile (@pxref{Environment Variables,,, gcc,
613 Using the GNU Compiler Collection (GCC)}). If GCC and, say, the C
614 library are installed in the profile, then @code{--search-paths} will
615 suggest setting these variables to @code{@var{profile}/include} and
616 @code{@var{profile}/lib}, respectively.
617
618 @item --profile=@var{profile}
619 @itemx -p @var{profile}
620 Use @var{profile} instead of the user's default profile.
621
622 @item --dry-run
623 @itemx -n
624 Show what would be done without actually doing it.
625
626 @item --fallback
627 When substituting a pre-built binary fails, fall back to building
628 packages locally.
629
630 @item --no-substitutes
631 Do not use substitutes for build products. That is, always build things
632 locally instead of allowing downloads of pre-built binaries.
633
634 @item --max-silent-time=@var{seconds}
635 Same as for @command{guix build} (@pxref{Invoking guix build}).
636
637 @item --verbose
638 Produce verbose output. In particular, emit the environment's build log
639 on the standard error port.
640
641 @item --bootstrap
642 Use the bootstrap Guile to build the profile. This option is only
643 useful to distribution developers.
644
645 @end table
646
647 In addition to these actions @command{guix package} supports the
648 following options to query the current state of a profile, or the
649 availability of packages:
650
651 @table @option
652
653 @item --search=@var{regexp}
654 @itemx -s @var{regexp}
655 List the available packages whose synopsis or description matches
656 @var{regexp}. Print all the meta-data of matching packages in
657 @code{recutils} format (@pxref{Top, GNU recutils databases,, recutils,
658 GNU recutils manual}).
659
660 This allows specific fields to be extracted using the @command{recsel}
661 command, for instance:
662
663 @example
664 $ guix package -s malloc | recsel -p name,version
665 name: glibc
666 version: 2.17
667
668 name: libgc
669 version: 7.2alpha6
670 @end example
671
672 @item --list-installed[=@var{regexp}]
673 @itemx -I [@var{regexp}]
674 List the currently installed packages in the specified profile, with the
675 most recently installed packages shown last. When @var{regexp} is
676 specified, list only installed packages whose name matches @var{regexp}.
677
678 For each installed package, print the following items, separated by
679 tabs: the package name, its version string, the part of the package that
680 is installed (for instance, @code{out} for the default output,
681 @code{include} for its headers, etc.), and the path of this package in
682 the store.
683
684 @item --list-available[=@var{regexp}]
685 @itemx -A [@var{regexp}]
686 List packages currently available in the software distribution
687 (@pxref{GNU Distribution}). When @var{regexp} is specified, list only
688 installed packages whose name matches @var{regexp}.
689
690 For each package, print the following items separated by tabs: its name,
691 its version string, the parts of the package (@pxref{Packages with
692 Multiple Outputs}), and the source location of its definition.
693
694 @item --list-generations[=@var{pattern}]
695 @itemx -l [@var{pattern}]
696 Return a list of generations along with their creation dates; for each
697 generation, show the installed packages, with the most recently
698 installed packages shown last. Note that the zeroth generation is never
699 shown.
700
701 For each installed package, print the following items, separated by
702 tabs: the name of a package, its version string, the part of the package
703 that is installed (@pxref{Packages with Multiple Outputs}), and the
704 location of this package in the store.
705
706 When @var{pattern} is used, the command returns only matching
707 generations. Valid patterns include:
708
709 @itemize
710 @item @emph{Integers and comma-separated integers}. Both patterns denote
711 generation numbers. For instance, @code{--list-generations=1} returns
712 the first one.
713
714 And @code{--list-generations=1,8,2} outputs three generations in the
715 specified order. Neither spaces nor trailing commas are allowed.
716
717 @item @emph{Ranges}. @code{--list-generations=2..9} prints the
718 specified generations and everything in between. Note that the start of
719 a range must be lesser than its end.
720
721 It is also possible to omit the endpoint. For example,
722 @code{--list-generations=2..}, returns all generations starting from the
723 second one.
724
725 @item @emph{Durations}. You can also get the last @emph{N}@tie{}days, weeks,
726 or months by passing an integer along with the first letter of the
727 duration. For example, @code{--list-generations=20d} lists generations
728 that are up to 20 days old.
729 @end itemize
730
731 @item --delete-generations[=@var{pattern}]
732 @itemx -d [@var{pattern}]
733 When @var{pattern} is omitted, delete all generations except the current
734 one.
735
736 This command accepts the same patterns as @option{--list-generations}.
737 When @var{pattern} is specified, delete the matching generations. When
738 @var{pattern} specifies a duration, generations @emph{older} than the
739 specified duration match. For instance, @code{--delete-generations=1m}
740 deletes generations that are more than one month old.
741
742 If the current generation matches, it is deleted atomically---i.e., by
743 switching to the previous available generation. Note that the zeroth
744 generation is never deleted.
745
746 Note that deleting generations prevents roll-back to them.
747 Consequently, this command must be used with care.
748
749 @end table
750
751 @node Packages with Multiple Outputs
752 @section Packages with Multiple Outputs
753
754 @cindex multiple-output packages
755 @cindex package outputs
756
757 Often, packages defined in Guix have a single @dfn{output}---i.e., the
758 source package leads exactly one directory in the store. When running
759 @command{guix package -i glibc}, one installs the default output of the
760 GNU libc package; the default output is called @code{out}, but its name
761 can be omitted as shown in this command. In this particular case, the
762 default output of @code{glibc} contains all the C header files, shared
763 libraries, static libraries, Info documentation, and other supporting
764 files.
765
766 Sometimes it is more appropriate to separate the various types of files
767 produced from a single source package into separate outputs. For
768 instance, the GLib C library (used by GTK+ and related packages)
769 installs more than 20 MiB of reference documentation as HTML pages.
770 To save space for users who do not need it, the documentation goes to a
771 separate output, called @code{doc}. To install the main GLib output,
772 which contains everything but the documentation, one would run:
773
774 @example
775 guix package -i glib
776 @end example
777
778 The command to install its documentation is:
779
780 @example
781 guix package -i glib:doc
782 @end example
783
784 Some packages install programs with different ``dependency footprints''.
785 For instance, the WordNet package install both command-line tools and
786 graphical user interfaces (GUIs). The former depend solely on the C
787 library, whereas the latter depend on Tcl/Tk and the underlying X
788 libraries. In this case, we leave the command-line tools in the default
789 output, whereas the GUIs are in a separate output. This allows users
790 who do not need the GUIs to save space.
791
792 There are several such multiple-output packages in the GNU distribution.
793 Other conventional output names include @code{lib} for libraries and
794 possibly header files, @code{bin} for stand-alone programs, and
795 @code{debug} for debugging information (@pxref{Installing Debugging
796 Files}). The outputs of a packages are listed in the third column of
797 the output of @command{guix package --list-available} (@pxref{Invoking
798 guix package}).
799
800
801 @node Invoking guix gc
802 @section Invoking @command{guix gc}
803
804 @cindex garbage collector
805 Packages that are installed but not used may be @dfn{garbage-collected}.
806 The @command{guix gc} command allows users to explicitly run the garbage
807 collector to reclaim space from the @file{/nix/store} directory.
808
809 The garbage collector has a set of known @dfn{roots}: any file under
810 @file{/nix/store} reachable from a root is considered @dfn{live} and
811 cannot be deleted; any other file is considered @dfn{dead} and may be
812 deleted. The set of garbage collector roots includes default user
813 profiles, and may be augmented with @command{guix build --root}, for
814 example (@pxref{Invoking guix build}).
815
816 Prior to running @code{guix gc --collect-garbage} to make space, it is
817 often useful to remove old generations from user profiles; that way, old
818 package builds referenced by those generations can be reclaimed. This
819 is achieved by running @code{guix package --delete-generations}
820 (@pxref{Invoking guix package}).
821
822 The @command{guix gc} command has three modes of operation: it can be
823 used to garbage-collect any dead files (the default), to delete specific
824 files (the @code{--delete} option), or to print garbage-collector
825 information. The available options are listed below:
826
827 @table @code
828 @item --collect-garbage[=@var{min}]
829 @itemx -C [@var{min}]
830 Collect garbage---i.e., unreachable @file{/nix/store} files and
831 sub-directories. This is the default operation when no option is
832 specified.
833
834 When @var{min} is given, stop once @var{min} bytes have been collected.
835 @var{min} may be a number of bytes, or it may include a unit as a
836 suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes.
837
838 When @var{min} is omitted, collect all the garbage.
839
840 @item --delete
841 @itemx -d
842 Attempt to delete all the store files and directories specified as
843 arguments. This fails if some of the files are not in the store, or if
844 they are still live.
845
846 @item --list-dead
847 Show the list of dead files and directories still present in the
848 store---i.e., files and directories no longer reachable from any root.
849
850 @item --list-live
851 Show the list of live store files and directories.
852
853 @end table
854
855 In addition, the references among existing store files can be queried:
856
857 @table @code
858
859 @item --references
860 @itemx --referrers
861 List the references (respectively, the referrers) of store files given
862 as arguments.
863
864 @item --requisites
865 @itemx -R
866 List the requisites of the store files passed as arguments. Requisites
867 include the store files themselves, their references, and the references
868 of these, recursively. In other words, the returned list is the
869 @dfn{transitive closure} of the store files.
870
871 @end table
872
873
874 @node Invoking guix pull
875 @section Invoking @command{guix pull}
876
877 Packages are installed or upgraded to the latest version available in
878 the distribution currently available on your local machine. To update
879 that distribution, along with the Guix tools, you must run @command{guix
880 pull}: the command downloads the latest Guix source code and package
881 descriptions, and deploys it.
882
883 On completion, @command{guix package} will use packages and package
884 versions from this just-retrieved copy of Guix. Not only that, but all
885 the Guix commands and Scheme modules will also be taken from that latest
886 version. New @command{guix} sub-commands added by the update also
887 become available.
888
889 The @command{guix pull} command is usually invoked with no arguments,
890 but it supports the following options:
891
892 @table @code
893 @item --verbose
894 Produce verbose output, writing build logs to the standard error output.
895
896 @item --url=@var{url}
897 Download the source tarball of Guix from @var{url}.
898
899 By default, the tarball is taken from its canonical address at
900 @code{gnu.org}, for the stable branch of Guix.
901
902 @item --bootstrap
903 Use the bootstrap Guile to build the latest Guix. This option is only
904 useful to Guix developers.
905 @end table
906
907 @c *********************************************************************
908 @node Programming Interface
909 @chapter Programming Interface
910
911 GNU Guix provides several Scheme programming interfaces (APIs) to
912 define, build, and query packages. The first interface allows users to
913 write high-level package definitions. These definitions refer to
914 familiar packaging concepts, such as the name and version of a package,
915 its build system, and its dependencies. These definitions can then be
916 turned into concrete build actions.
917
918 Build actions are performed by the Guix daemon, on behalf of users. In a
919 standard setup, the daemon has write access to the store---the
920 @file{/nix/store} directory---whereas users do not. The recommended
921 setup also has the daemon perform builds in chroots, under a specific
922 build users, to minimize interference with the rest of the system.
923
924 @cindex derivation
925 Lower-level APIs are available to interact with the daemon and the
926 store. To instruct the daemon to perform a build action, users actually
927 provide it with a @dfn{derivation}. A derivation is a low-level
928 representation of the build actions to be taken, and the environment in
929 which they should occur---derivations are to package definitions what
930 assembly is to C programs.
931
932 This chapter describes all these APIs in turn, starting from high-level
933 package definitions.
934
935 @menu
936 * Defining Packages:: Defining new packages.
937 * The Store:: Manipulating the package store.
938 * Derivations:: Low-level interface to package derivations.
939 * The Store Monad:: Purely functional interface to the store.
940 @end menu
941
942 @node Defining Packages
943 @section Defining Packages
944
945 The high-level interface to package definitions is implemented in the
946 @code{(guix packages)} and @code{(guix build-system)} modules. As an
947 example, the package definition, or @dfn{recipe}, for the GNU Hello
948 package looks like this:
949
950 @example
951 (use-modules (guix packages)
952 (guix download)
953 (guix build-system gnu)
954 (guix licenses))
955
956 (define hello
957 (package
958 (name "hello")
959 (version "2.8")
960 (source (origin
961 (method url-fetch)
962 (uri (string-append "mirror://gnu/hello/hello-" version
963 ".tar.gz"))
964 (sha256
965 (base32 "0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6"))))
966 (build-system gnu-build-system)
967 (inputs `(("gawk" ,gawk)))
968 (synopsis "GNU Hello")
969 (description "Yeah...")
970 (home-page "http://www.gnu.org/software/hello/")
971 (license gpl3+)))
972 @end example
973
974 @noindent
975 Without being a Scheme expert, the reader may have guessed the meaning
976 of the various fields here. This expression binds variable @var{hello}
977 to a @code{<package>} object, which is essentially a record
978 (@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}).
979 This package object can be inspected using procedures found in the
980 @code{(guix packages)} module; for instance, @code{(package-name hello)}
981 returns---surprise!---@code{"hello"}.
982
983 There are a few points worth noting in the above package definition:
984
985 @itemize
986 @item
987 The @code{source} field of the package is an @code{<origin>} object.
988 Here, the @code{url-fetch} method from @code{(guix download)} is used,
989 meaning that the source is a file to be downloaded over FTP or HTTP.
990
991 The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of
992 the GNU mirrors defined in @code{(guix download)}.
993
994 The @code{sha256} field specifies the expected SHA256 hash of the file
995 being downloaded. It is mandatory, and allows Guix to check the
996 integrity of the file. The @code{(base32 @dots{})} form introduces the
997 base32 representation of the hash. You can obtain this information with
998 @code{guix download} (@pxref{Invoking guix download}) and @code{guix
999 hash} (@pxref{Invoking guix hash}).
1000
1001 @cindex patches
1002 When needed, the @code{origin} form can also have a @code{patches} field
1003 listing patches to be applied, and a @code{snippet} field giving a
1004 Scheme expression to modify the source code.
1005
1006 @item
1007 @cindex GNU Build System
1008 The @code{build-system} field is set to @var{gnu-build-system}. The
1009 @var{gnu-build-system} variable is defined in the @code{(guix
1010 build-system gnu)} module, and is bound to a @code{<build-system>}
1011 object.
1012
1013 Naturally, @var{gnu-build-system} represents the familiar GNU Build
1014 System, and variants thereof (@pxref{Configuration, configuration and
1015 makefile conventions,, standards, GNU Coding Standards}). In a
1016 nutshell, packages using the GNU Build System may be configured, built,
1017 and installed with the usual @code{./configure && make && make check &&
1018 make install} command sequence. This is what @var{gnu-build-system}
1019 does.
1020
1021 In addition, @var{gnu-build-system} ensures that the ``standard''
1022 environment for GNU packages is available. This includes tools such as
1023 GCC, Coreutils, Bash, Make, Diffutils, and Patch.
1024
1025 @item
1026 The @code{inputs} field specifies inputs to the build process---i.e.,
1027 build-time or run-time dependencies of the package. Here, we define an
1028 input called @code{"gawk"} whose value is that of the @var{gawk}
1029 variable; @var{gawk} is itself bound to a @code{<package>} object.
1030
1031 Note that GCC, Coreutils, Bash, and other essential tools do not need to
1032 be specified as inputs here. Instead, @var{gnu-build-system} takes care
1033 of ensuring that they are present.
1034
1035 However, any other dependencies need to be specified in the
1036 @code{inputs} field. Any dependency not specified here will simply be
1037 unavailable to the build process, possibly leading to a build failure.
1038 @end itemize
1039
1040 There are other fields that package definitions may provide. Of
1041 particular interest is the @code{arguments} field. When specified, it
1042 must be bound to a list of additional arguments to be passed to the
1043 build system. For instance, the above definition could be augmented
1044 with the following field initializer:
1045
1046 @example
1047 (arguments `(#:tests? #f
1048 #:configure-flags '("--enable-silent-rules")))
1049 @end example
1050
1051 @noindent
1052 These are keyword arguments (@pxref{Optional Arguments, keyword
1053 arguments in Guile,, guile, GNU Guile Reference Manual}). They are
1054 passed to @var{gnu-build-system}, which interprets them as meaning ``do
1055 not run @code{make check}'', and ``run @file{configure} with the
1056 @code{--enable-silent-rules} flag''. The value of these keyword
1057 parameters is actually evaluated in the @dfn{build stratum}---i.e., by a
1058 Guile process launched by the daemon (@pxref{Derivations}).
1059
1060 Once a package definition is in place@footnote{Simple package
1061 definitions like the one above may be automatically converted from the
1062 Nixpkgs distribution using the @command{guix import} command.}, the
1063 package may actually be built using the @code{guix build} command-line
1064 tool (@pxref{Invoking guix build}). Eventually, updating the package
1065 definition to a new upstream version can be partly automated by the
1066 @command{guix refresh} command (@pxref{Invoking guix refresh}).
1067
1068 Behind the scenes, a derivation corresponding to the @code{<package>}
1069 object is first computed by the @code{package-derivation} procedure.
1070 That derivation is stored in a @code{.drv} file under @file{/nix/store}.
1071 The build actions it prescribes may then be realized by using the
1072 @code{build-derivations} procedure (@pxref{The Store}).
1073
1074 @deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}]
1075 Return the @code{<derivation>} object of @var{package} for @var{system}
1076 (@pxref{Derivations}).
1077
1078 @var{package} must be a valid @code{<package>} object, and @var{system}
1079 must be a string denoting the target system type---e.g.,
1080 @code{"x86_64-linux"} for an x86_64 Linux-based GNU system. @var{store}
1081 must be a connection to the daemon, which operates on the store
1082 (@pxref{The Store}).
1083 @end deffn
1084
1085 @noindent
1086 @cindex cross-compilation
1087 Similarly, it is possible to compute a derivation that cross-builds a
1088 package for some other system:
1089
1090 @deffn {Scheme Procedure} package-cross-derivation @var{store} @
1091 @var{package} @var{target} [@var{system}]
1092 Return the @code{<derivation>} object of @var{package} cross-built from
1093 @var{system} to @var{target}.
1094
1095 @var{target} must be a valid GNU triplet denoting the target hardware
1096 and operating system, such as @code{"mips64el-linux-gnu"}
1097 (@pxref{Configuration Names, GNU configuration triplets,, configure, GNU
1098 Configure and Build System}).
1099 @end deffn
1100
1101
1102 @node The Store
1103 @section The Store
1104
1105 @cindex store
1106 @cindex store paths
1107
1108 Conceptually, the @dfn{store} is where derivations that have been
1109 successfully built are stored---by default, under @file{/nix/store}.
1110 Sub-directories in the store are referred to as @dfn{store paths}. The
1111 store has an associated database that contains information such has the
1112 store paths referred to by each store path, and the list of @emph{valid}
1113 store paths---paths that result from a successful build.
1114
1115 The store is always accessed by the daemon on behalf of its clients
1116 (@pxref{Invoking guix-daemon}). To manipulate the store, clients
1117 connect to the daemon over a Unix-domain socket, send it requests, and
1118 read the result---these are remote procedure calls, or RPCs.
1119
1120 The @code{(guix store)} module provides procedures to connect to the
1121 daemon, and to perform RPCs. These are described below.
1122
1123 @deffn {Scheme Procedure} open-connection [@var{file}] [#:reserve-space? #t]
1124 Connect to the daemon over the Unix-domain socket at @var{file}. When
1125 @var{reserve-space?} is true, instruct it to reserve a little bit of
1126 extra space on the file system so that the garbage collector can still
1127 operate, should the disk become full. Return a server object.
1128
1129 @var{file} defaults to @var{%default-socket-path}, which is the normal
1130 location given the options that were passed to @command{configure}.
1131 @end deffn
1132
1133 @deffn {Scheme Procedure} close-connection @var{server}
1134 Close the connection to @var{server}.
1135 @end deffn
1136
1137 @defvr {Scheme Variable} current-build-output-port
1138 This variable is bound to a SRFI-39 parameter, which refers to the port
1139 where build and error logs sent by the daemon should be written.
1140 @end defvr
1141
1142 Procedures that make RPCs all take a server object as their first
1143 argument.
1144
1145 @deffn {Scheme Procedure} valid-path? @var{server} @var{path}
1146 Return @code{#t} when @var{path} is a valid store path.
1147 @end deffn
1148
1149 @deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}]
1150 Add @var{text} under file @var{name} in the store, and return its store
1151 path. @var{references} is the list of store paths referred to by the
1152 resulting store path.
1153 @end deffn
1154
1155 @deffn {Scheme Procedure} build-derivations @var{server} @var{derivations}
1156 Build @var{derivations} (a list of @code{<derivation>} objects or
1157 derivation paths), and return when the worker is done building them.
1158 Return @code{#t} on success.
1159 @end deffn
1160
1161 Note that the @code{(guix monads)} module provides a monad as well as
1162 monadic versions of the above procedures, with the goal of making it
1163 more convenient to work with code that accesses the store (@pxref{The
1164 Store Monad}).
1165
1166 @c FIXME
1167 @i{This section is currently incomplete.}
1168
1169 @node Derivations
1170 @section Derivations
1171
1172 @cindex derivations
1173 Low-level build actions and the environment in which they are performed
1174 are represented by @dfn{derivations}. A derivation contain the
1175 following pieces of information:
1176
1177 @itemize
1178 @item
1179 The outputs of the derivation---derivations produce at least one file or
1180 directory in the store, but may produce more.
1181
1182 @item
1183 The inputs of the derivations, which may be other derivations or plain
1184 files in the store (patches, build scripts, etc.)
1185
1186 @item
1187 The system type targeted by the derivation---e.g., @code{x86_64-linux}.
1188
1189 @item
1190 The file name of a build script in the store, along with the arguments
1191 to be passed.
1192
1193 @item
1194 A list of environment variables to be defined.
1195
1196 @end itemize
1197
1198 @cindex derivation path
1199 Derivations allow clients of the daemon to communicate build actions to
1200 the store. They exist in two forms: as an in-memory representation,
1201 both on the client- and daemon-side, and as files in the store whose
1202 name end in @code{.drv}---these files are referred to as @dfn{derivation
1203 paths}. Derivations paths can be passed to the @code{build-derivations}
1204 procedure to perform the build actions they prescribe (@pxref{The
1205 Store}).
1206
1207 The @code{(guix derivations)} module provides a representation of
1208 derivations as Scheme objects, along with procedures to create and
1209 otherwise manipulate derivations. The lowest-level primitive to create
1210 a derivation is the @code{derivation} procedure:
1211
1212 @deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] [#:hash-mode #f] [#:inputs '()] [#:env-vars '()] [#:system (%current-system)] [#:references-graphs #f]
1213 Build a derivation with the given arguments, and return the resulting
1214 @code{<derivation>} object.
1215
1216 When @var{hash}, @var{hash-algo}, and @var{hash-mode} are given, a
1217 @dfn{fixed-output derivation} is created---i.e., one whose result is
1218 known in advance, such as a file download.
1219
1220 When @var{references-graphs} is true, it must be a list of file
1221 name/store path pairs. In that case, the reference graph of each store
1222 path is exported in the build environment in the corresponding file, in
1223 a simple text format.
1224 @end deffn
1225
1226 @noindent
1227 Here's an example with a shell script as its builder, assuming
1228 @var{store} is an open connection to the daemon, and @var{bash} points
1229 to a Bash executable in the store:
1230
1231 @lisp
1232 (use-modules (guix utils)
1233 (guix store)
1234 (guix derivations))
1235
1236 (let ((builder ; add the Bash script to the store
1237 (add-text-to-store store "my-builder.sh"
1238 "echo hello world > $out\n" '())))
1239 (derivation store "foo"
1240 bash `("-e" ,builder)
1241 #:env-vars '(("HOME" . "/homeless"))))
1242 @result{} #<derivation /nix/store/@dots{}-foo.drv => /nix/store/@dots{}-foo>
1243 @end lisp
1244
1245 As can be guessed, this primitive is cumbersome to use directly. An
1246 improved variant is @code{build-expression->derivation}, which allows
1247 the caller to directly pass a Guile expression as the build script:
1248
1249 @deffn {Scheme Procedure} build-expression->derivation @var{store} @var{name} @var{system} @var{exp} @var{inputs} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] [#:env-vars '()] [#:modules '()] [#:references-graphs #f] [#:guile-for-build #f]
1250 Return a derivation that executes Scheme expression @var{exp} as a
1251 builder for derivation @var{name}. @var{inputs} must be a list of
1252 @code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
1253 @code{"out"} is assumed. @var{modules} is a list of names of Guile
1254 modules from the current search path to be copied in the store,
1255 compiled, and made available in the load path during the execution of
1256 @var{exp}---e.g., @code{((guix build utils) (guix build
1257 gnu-build-system))}.
1258
1259 @var{exp} is evaluated in an environment where @code{%outputs} is bound
1260 to a list of output/path pairs, and where @code{%build-inputs} is bound
1261 to a list of string/output-path pairs made from @var{inputs}.
1262 Optionally, @var{env-vars} is a list of string pairs specifying the name
1263 and value of environment variables visible to the builder. The builder
1264 terminates by passing the result of @var{exp} to @code{exit}; thus, when
1265 @var{exp} returns @code{#f}, the build is considered to have failed.
1266
1267 @var{exp} is built using @var{guile-for-build} (a derivation). When
1268 @var{guile-for-build} is omitted or is @code{#f}, the value of the
1269 @code{%guile-for-build} fluid is used instead.
1270
1271 See the @code{derivation} procedure for the meaning of @var{references-graphs}.
1272 @end deffn
1273
1274 @noindent
1275 Here's an example of a single-output derivation that creates a directory
1276 containing one file:
1277
1278 @lisp
1279 (let ((builder '(let ((out (assoc-ref %outputs "out")))
1280 (mkdir out) ; create /nix/store/@dots{}-goo
1281 (call-with-output-file (string-append out "/test")
1282 (lambda (p)
1283 (display '(hello guix) p))))))
1284 (build-expression->derivation store "goo" (%current-system)
1285 builder '()))
1286
1287 @result{} #<derivation /nix/store/@dots{}-goo.drv => @dots{}>
1288 @end lisp
1289
1290 @cindex strata of code
1291 Remember that the build expression passed to
1292 @code{build-expression->derivation} is run by a separate Guile process
1293 than the one that calls @code{build-expression->derivation}: it is run
1294 by a Guile process launched by the daemon, typically in a chroot. So,
1295 while there is a single language for both the @dfn{host} and the build
1296 side, there are really two @dfn{strata} of code: the host-side, and the
1297 build-side code@footnote{The term @dfn{stratum} in this context was
1298 coined by Manuel Serrano et al. in the context of their work on Hop.}.
1299 This distinction is important to keep in mind, notably when using
1300 higher-level constructs such as @var{gnu-build-system} (@pxref{Defining
1301 Packages}). For this reason, Guix modules that are meant to be used in
1302 the build stratum are kept in the @code{(guix build @dots{})} name
1303 space.
1304
1305 @node The Store Monad
1306 @section The Store Monad
1307
1308 @cindex monad
1309
1310 The procedures that operate on the store described in the previous
1311 sections all take an open connection to the build daemon as their first
1312 argument. Although the underlying model is functional, they either have
1313 side effects or depend on the current state of the store.
1314
1315 The former is inconvenient: the connection to the build daemon has to be
1316 carried around in all those functions, making it impossible to compose
1317 functions that do not take that parameter with functions that do. The
1318 latter can be problematic: since store operations have side effects
1319 and/or depend on external state, they have to be properly sequenced.
1320
1321 @cindex monadic values
1322 @cindex monadic functions
1323 This is where the @code{(guix monads)} module comes in. This module
1324 provides a framework for working with @dfn{monads}, and a particularly
1325 useful monad for our uses, the @dfn{store monad}. Monads are a
1326 construct that allows two things: associating ``context'' with values
1327 (in our case, the context is the store), and building sequences of
1328 computations (here computations includes accesses to the store.) Values
1329 in a monad---values that carry this additional context---are called
1330 @dfn{monadic values}; procedures that return such values are called
1331 @dfn{monadic procedures}.
1332
1333 Consider this ``normal'' procedure:
1334
1335 @example
1336 (define (profile.sh store)
1337 ;; Return the name of a shell script in the store that
1338 ;; initializes the 'PATH' environment variable.
1339 (let* ((drv (package-derivation store coreutils))
1340 (out (derivation->output-path drv)))
1341 (add-text-to-store store "profile.sh"
1342 (format #f "export PATH=~a/bin" out))))
1343 @end example
1344
1345 Using @code{(guix monads)}, it may be rewritten as a monadic function:
1346
1347 @example
1348 (define (profile.sh)
1349 ;; Same, but return a monadic value.
1350 (mlet %store-monad ((bin (package-file coreutils "bin")))
1351 (text-file "profile.sh"
1352 (string-append "export PATH=" bin))))
1353 @end example
1354
1355 There are two things to note in the second version: the @code{store}
1356 parameter is now implicit, and the monadic value returned by
1357 @code{package-file}---a wrapper around @code{package-derivation} and
1358 @code{derivation->output-path}---is @dfn{bound} using @code{mlet}
1359 instead of plain @code{let}.
1360
1361 Calling the monadic @code{profile.sh} has no effect. To get the desired
1362 effect, one must use @code{run-with-store}:
1363
1364 @example
1365 (run-with-store (open-connection) (profile.sh))
1366 @result{} /nix/store/...-profile.sh
1367 @end example
1368
1369 The main syntactic forms to deal with monads in general are described
1370 below.
1371
1372 @deffn {Scheme Syntax} with-monad @var{monad} @var{body} ...
1373 Evaluate any @code{>>=} or @code{return} forms in @var{body} as being
1374 in @var{monad}.
1375 @end deffn
1376
1377 @deffn {Scheme Syntax} return @var{val}
1378 Return a monadic value that encapsulates @var{val}.
1379 @end deffn
1380
1381 @deffn {Scheme Syntax} >>= @var{mval} @var{mproc}
1382 @dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic
1383 procedure @var{mproc}@footnote{This operation is commonly referred to as
1384 ``bind'', but that name denotes an unrelated procedure in Guile. Thus
1385 we use this somewhat cryptic symbol inherited from the Haskell
1386 language.}.
1387 @end deffn
1388
1389 @deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @
1390 @var{body} ...
1391 @deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @
1392 @var{body} ...
1393 Bind the variables @var{var} to the monadic values @var{mval} in
1394 @var{body}. The form (@var{var} -> @var{val}) binds @var{var} to the
1395 ``normal'' value @var{val}, as per @code{let}.
1396
1397 @code{mlet*} is to @code{mlet} what @code{let*} is to @code{let}
1398 (@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}).
1399 @end deffn
1400
1401 The interface to the store monad provided by @code{(guix monads)} is as
1402 follows.
1403
1404 @defvr {Scheme Variable} %store-monad
1405 The store monad. Values in the store monad encapsulate accesses to the
1406 store. When its effect is needed, a value of the store monad must be
1407 ``evaluated'' by passing it to the @code{run-with-store} procedure (see
1408 below.)
1409 @end defvr
1410
1411 @deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)]
1412 Run @var{mval}, a monadic value in the store monad, in @var{store}, an
1413 open store connection.
1414 @end deffn
1415
1416 @deffn {Monadic Procedure} text-file @var{name} @var{text}
1417 Return as a monadic value the absolute file name in the store of the file
1418 containing @var{text}.
1419 @end deffn
1420
1421 @deffn {Monadic Procedure} package-file @var{package} [@var{file}] @
1422 [#:system (%current-system)] [#:output "out"] Return as a monadic
1423 value in the absolute file name of @var{file} within the @var{output}
1424 directory of @var{package}. When @var{file} is omitted, return the name
1425 of the @var{output} directory of @var{package}.
1426 @end deffn
1427
1428 @deffn {Monadic Procedure} derivation-expression @var{name} @var{system} @
1429 @var{exp} @var{inputs} [#:outputs '("out")] [#:hash #f] @
1430 [#:hash-algo #f] [#:env-vars '()] [#:modules '()] @
1431 [#:references-graphs #f] [#:guile-for-build #f]
1432 Monadic version of @code{build-expression->derivation}
1433 (@pxref{Derivations}).
1434 @end deffn
1435
1436 @deffn {Monadic Procedure} package->derivation @var{package} [@var{system}]
1437 Monadic version of @code{package-derivation} (@pxref{Defining
1438 Packages}).
1439 @end deffn
1440
1441
1442 @c *********************************************************************
1443 @node Utilities
1444 @chapter Utilities
1445
1446 This section describes tools primarily targeted at developers and users
1447 who write new package definitions. They complement the Scheme
1448 programming interface of Guix in a convenient way.
1449
1450 @menu
1451 * Invoking guix build:: Building packages from the command line.
1452 * Invoking guix download:: Downloading a file and printing its hash.
1453 * Invoking guix hash:: Computing the cryptographic hash of a file.
1454 * Invoking guix refresh:: Updating package definitions.
1455 @end menu
1456
1457 @node Invoking guix build
1458 @section Invoking @command{guix build}
1459
1460 The @command{guix build} command builds packages or derivations and
1461 their dependencies, and prints the resulting store paths. Note that it
1462 does not modify the user's profile---this is the job of the
1463 @command{guix package} command (@pxref{Invoking guix package}). Thus,
1464 it is mainly useful for distribution developers.
1465
1466 The general syntax is:
1467
1468 @example
1469 guix build @var{options} @var{package-or-derivation}@dots{}
1470 @end example
1471
1472 @var{package-or-derivation} may be either the name of a package found in
1473 the software distribution such as @code{coreutils} or
1474 @code{coreutils-8.20}, or a derivation such as
1475 @file{/nix/store/@dots{}-coreutils-8.19.drv}. Alternatively, the
1476 @code{--expression} option may be used to specify a Scheme expression
1477 that evaluates to a package; this is useful when disambiguation among
1478 several same-named packages or package variants is needed.
1479
1480 The @var{options} may be zero or more of the following:
1481
1482 @table @code
1483
1484 @item --expression=@var{expr}
1485 @itemx -e @var{expr}
1486 Build the package @var{expr} evaluates to.
1487
1488 For example, @var{expr} may be @code{(@@ (gnu packages guile)
1489 guile-1.8)}, which unambiguously designates this specific variant of
1490 version 1.8 of Guile.
1491
1492 @item --source
1493 @itemx -S
1494 Build the packages' source derivations, rather than the packages
1495 themselves.
1496
1497 For instance, @code{guix build -S gcc} returns something like
1498 @file{/nix/store/@dots{}-gcc-4.7.2.tar.bz2}, which is GCC's source tarball.
1499
1500 The returned source tarball is the result of applying any patches and
1501 code snippets specified in the package's @code{origin} (@pxref{Defining
1502 Packages}).
1503
1504 @item --system=@var{system}
1505 @itemx -s @var{system}
1506 Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of
1507 the host's system type.
1508
1509 An example use of this is on Linux-based systems, which can emulate
1510 different personalities. For instance, passing
1511 @code{--system=i686-linux} on an @code{x86_64-linux} system allows users
1512 to build packages in a complete 32-bit environment.
1513
1514 @item --target=@var{triplet}
1515 @cindex cross-compilation
1516 Cross-build for @var{triplet}, which must be a valid GNU triplet, such
1517 as @code{"mips64el-linux-gnu"} (@pxref{Configuration Names, GNU
1518 configuration triplets,, configure, GNU Configure and Build System}).
1519
1520 @item --derivations
1521 @itemx -d
1522 Return the derivation paths, not the output paths, of the given
1523 packages.
1524
1525 @item --keep-failed
1526 @itemx -K
1527 Keep the build tree of failed builds. Thus, if a build fail, its build
1528 tree is kept under @file{/tmp}, in a directory whose name is shown at
1529 the end of the build log. This is useful when debugging build issues.
1530
1531 @item --dry-run
1532 @itemx -n
1533 Do not build the derivations.
1534
1535 @item --fallback
1536 When substituting a pre-built binary fails, fall back to building
1537 packages locally.
1538
1539 @item --no-substitutes
1540 Do not use substitutes for build products. That is, always build things
1541 locally instead of allowing downloads of pre-built binaries.
1542
1543 @item --max-silent-time=@var{seconds}
1544 When the build or substitution process remains silent for more than
1545 @var{seconds}, terminate it and report a build failure.
1546
1547 @item --cores=@var{n}
1548 @itemx -c @var{n}
1549 Allow the use of up to @var{n} CPU cores for the build. The special
1550 value @code{0} means to use as many CPU cores as available.
1551
1552 @item --root=@var{file}
1553 @itemx -r @var{file}
1554 Make @var{file} a symlink to the result, and register it as a garbage
1555 collector root.
1556
1557 @item --verbosity=@var{level}
1558 Use the given verbosity level. @var{level} must be an integer between 0
1559 and 5; higher means more verbose output. Setting a level of 4 or more
1560 may be helpful when debugging setup issues with the build daemon.
1561
1562 @item --log-file
1563 Return the build log file names for the given
1564 @var{package-or-derivation}s, or raise an error if build logs are
1565 missing.
1566
1567 This works regardless of how packages or derivations are specified. For
1568 instance, the following invocations are equivalent:
1569
1570 @example
1571 guix build --log-file `guix build -d guile`
1572 guix build --log-file `guix build guile`
1573 guix build --log-file guile
1574 guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)'
1575 @end example
1576
1577
1578 @end table
1579
1580 Behind the scenes, @command{guix build} is essentially an interface to
1581 the @code{package-derivation} procedure of the @code{(guix packages)}
1582 module, and to the @code{build-derivations} procedure of the @code{(guix
1583 store)} module.
1584
1585 @node Invoking guix download
1586 @section Invoking @command{guix download}
1587
1588 When writing a package definition, developers typically need to download
1589 the package's source tarball, compute its SHA256 hash, and write that
1590 hash in the package definition (@pxref{Defining Packages}). The
1591 @command{guix download} tool helps with this task: it downloads a file
1592 from the given URI, adds it to the store, and prints both its file name
1593 in the store and its SHA256 hash.
1594
1595 The fact that the downloaded file is added to the store saves bandwidth:
1596 when the developer eventually tries to build the newly defined package
1597 with @command{guix build}, the source tarball will not have to be
1598 downloaded again because it is already in the store. It is also a
1599 convenient way to temporarily stash files, which may be deleted
1600 eventually (@pxref{Invoking guix gc}).
1601
1602 The @command{guix download} command supports the same URIs as used in
1603 package definitions. In particular, it supports @code{mirror://} URIs.
1604 @code{https} URIs (HTTP over TLS) are supported @emph{provided} the
1605 Guile bindings for GnuTLS are available in the user's environment; when
1606 they are not available, an error is raised.
1607
1608 The following option is available:
1609
1610 @table @code
1611 @item --format=@var{fmt}
1612 @itemx -f @var{fmt}
1613 Write the hash in the format specified by @var{fmt}. For more
1614 information on the valid values for @var{fmt}, @ref{Invoking guix hash}.
1615 @end table
1616
1617 @node Invoking guix hash
1618 @section Invoking @command{guix hash}
1619
1620 The @command{guix hash} command computes the SHA256 hash of a file.
1621 It is primarily a convenience tool for anyone contributing to the
1622 distribution: it computes the cryptographic hash of a file, which can be
1623 used in the definition of a package (@pxref{Defining Packages}).
1624
1625 The general syntax is:
1626
1627 @example
1628 guix hash @var{option} @var{file}
1629 @end example
1630
1631 @command{guix hash} has the following option:
1632
1633 @table @code
1634
1635 @item --format=@var{fmt}
1636 @itemx -f @var{fmt}
1637 Write the hash in the format specified by @var{fmt}.
1638
1639 Supported formats: @code{nix-base32}, @code{base32}, @code{base16}
1640 (@code{hex} and @code{hexadecimal} can be used as well).
1641
1642 If the @option{--format} option is not specified, @command{guix hash}
1643 will output the hash in @code{nix-base32}. This representation is used
1644 in the definitions of packages.
1645
1646 @end table
1647
1648 @node Invoking guix refresh
1649 @section Invoking @command{guix refresh}
1650
1651 The primary audience of the @command{guix refresh} command is developers
1652 of the GNU software distribution. By default, it reports any packages
1653 provided by the distribution that are outdated compared to the latest
1654 upstream version, like this:
1655
1656 @example
1657 $ guix refresh
1658 gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1
1659 gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0
1660 @end example
1661
1662 It does so by browsing each package's FTP directory and determining the
1663 highest version number of the source tarballs
1664 therein@footnote{Currently, this only works for GNU packages.}.
1665
1666 When passed @code{--update}, it modifies distribution source files to
1667 update the version numbers and source tarball hashes of those packages'
1668 recipes (@pxref{Defining Packages}). This is achieved by downloading
1669 each package's latest source tarball and its associated OpenPGP
1670 signature, authenticating the downloaded tarball against its signature
1671 using @command{gpg}, and finally computing its hash. When the public
1672 key used to sign the tarball is missing from the user's keyring, an
1673 attempt is made to automatically retrieve it from a public key server;
1674 when it's successful, the key is added to the user's keyring; otherwise,
1675 @command{guix refresh} reports an error.
1676
1677 The following options are supported:
1678
1679 @table @code
1680
1681 @item --update
1682 @itemx -u
1683 Update distribution source files (package recipes) in place.
1684 @ref{Defining Packages}, for more information on package definitions.
1685
1686 @item --select=[@var{subset}]
1687 @itemx -s @var{subset}
1688 Select all the packages in @var{subset}, one of @code{core} or
1689 @code{non-core}.
1690
1691 The @code{core} subset refers to all the packages at the core of the
1692 distribution---i.e., packages that are used to build ``everything
1693 else''. This includes GCC, libc, Binutils, Bash, etc. Usually,
1694 changing one of these packages in the distribution entails a rebuild of
1695 all the others. Thus, such updates are an inconvenience to users in
1696 terms of build time or bandwidth used to achieve the upgrade.
1697
1698 The @code{non-core} subset refers to the remaining packages. It is
1699 typically useful in cases where an update of the core packages would be
1700 inconvenient.
1701
1702 @end table
1703
1704 In addition, @command{guix refresh} can be passed one or more package
1705 names, as in this example:
1706
1707 @example
1708 guix refresh -u emacs idutils
1709 @end example
1710
1711 @noindent
1712 The command above specifically updates the @code{emacs} and
1713 @code{idutils} packages. The @code{--select} option would have no
1714 effect in this case.
1715
1716 The following options can be used to customize GnuPG operation:
1717
1718 @table @code
1719
1720 @item --key-server=@var{host}
1721 Use @var{host} as the OpenPGP key server when importing a public key.
1722
1723 @item --gpg=@var{command}
1724 Use @var{command} as the GnuPG 2.x command. @var{command} is searched
1725 for in @code{$PATH}.
1726
1727 @end table
1728
1729
1730 @c *********************************************************************
1731 @node GNU Distribution
1732 @chapter GNU Distribution
1733
1734 Guix comes with a distribution of free software@footnote{The term
1735 ``free'' here refers to the
1736 @url{http://www.gnu.org/philosophy/free-sw.html,freedom provided to
1737 users of that software}.} that form the basis of the GNU system. This
1738 includes core GNU packages such as GNU libc, GCC, and Binutils, as well
1739 as many GNU and non-GNU applications. The complete list of available
1740 packages can be seen by running @command{guix package} (@pxref{Invoking
1741 guix package}):
1742
1743 @example
1744 guix package --list-available
1745 @end example
1746
1747 Our goal is to build a practical 100% free software distribution of
1748 Linux-based and other variants of GNU, with a focus on the promotion and
1749 tight integration of GNU components, and an emphasis on programs and
1750 tools that help users exert that freedom.
1751
1752 @menu
1753 * Installing Debugging Files:: Feeding the debugger.
1754 * Package Modules:: Packages from the programmer's viewpoint.
1755 * Packaging Guidelines:: Growing the distribution.
1756 * Bootstrapping:: GNU/Linux built from scratch.
1757 * Porting:: Targeting another platform or kernel.
1758 @end menu
1759
1760 Building this distribution is a cooperative effort, and you are invited
1761 to join! @ref{Contributing}, for information about how you can help.
1762
1763
1764 @node Installing Debugging Files
1765 @section Installing Debugging Files
1766
1767 Program binaries, as produced by the GCC compilers for instance, are
1768 typically written in the ELF format, with a section containing
1769 @dfn{debugging information}. Debugging information is what allows the
1770 debugger, GDB, to map binary code to source code; it is required to
1771 debug a compiled program in good conditions.
1772
1773 The problem with debugging information is that is takes up a fair amount
1774 of disk space. For example, debugging information for the GNU C Library
1775 weighs in at more than 60 MiB. Thus, as a user, keeping all the
1776 debugging info of all the installed programs is usually not an option.
1777 Yet, space savings should not come at the cost of an impediment to
1778 debugging---especially in the GNU system, which should make it easier
1779 for users to exert their computing freedom (@pxref{GNU Distribution}).
1780
1781 Thankfully, the GNU Binary Utilities (Binutils) and GDB provide a
1782 mechanism that allows users to get the best of both worlds: debugging
1783 information can be stripped from the binaries and stored in separate
1784 files. GDB is then able to load debugging information from those files,
1785 when they are available (@pxref{Separate Debug Files,,, gdb, Debugging
1786 with GDB}).
1787
1788 The GNU distribution takes advantage of this by storing debugging
1789 information in the @code{lib/debug} sub-directory of a separate package
1790 output unimaginatively called @code{debug} (@pxref{Packages with
1791 Multiple Outputs}). Users can choose to install the @code{debug} output
1792 of a package when they need it. For instance, the following command
1793 installs the debugging information for the GNU C Library and for GNU
1794 Guile:
1795
1796 @example
1797 guix package -i glibc:debug -i guile:debug
1798 @end example
1799
1800 GDB must then be told to look for debug files in the user's profile, by
1801 setting the @code{debug-file-directory} variable (consider setting it
1802 from the @file{~/.gdbinit} file, @pxref{Startup,,, gdb, Debugging with
1803 GDB}):
1804
1805 @example
1806 (gdb) set debug-file-directory ~/.guix-profile/lib/debug
1807 @end example
1808
1809 From there on, GDB will pick up debugging information from the
1810 @code{.debug} files under @file{~/.guix-profile/lib/debug}.
1811
1812 @c XXX: keep me up-to-date
1813 The @code{debug} output mechanism in Guix is implemented by the
1814 @code{gnu-build-system} (@pxref{Defining Packages}). Currently, it is
1815 opt-in---debugging information is available only for those packages
1816 whose definition explicitly declares a @code{debug} output. This may be
1817 changed to opt-out in the future, if our build farm servers can handle
1818 the load. To check whether a package has a @code{debug} output, use
1819 @command{guix package --list-available} (@pxref{Invoking guix package}).
1820
1821
1822 @node Package Modules
1823 @section Package Modules
1824
1825 From a programming viewpoint, the package definitions of the
1826 distribution are provided by Guile modules in the @code{(gnu packages
1827 ...)} name space (@pxref{Modules, Guile modules,, guile, GNU Guile
1828 Reference Manual}). For instance, the @code{(gnu packages emacs)}
1829 module exports a variable named @code{emacs}, which is bound to a
1830 @code{<package>} object (@pxref{Defining Packages}). The @code{(gnu
1831 packages)} module provides facilities for searching for packages.
1832
1833 The distribution is fully @dfn{bootstrapped} and @dfn{self-contained}:
1834 each package is built based solely on other packages in the
1835 distribution. The root of this dependency graph is a small set of
1836 @dfn{bootstrap binaries}, provided by the @code{(gnu packages
1837 bootstrap)} module. For more information on bootstrapping,
1838 @ref{Bootstrapping}.
1839
1840 @node Packaging Guidelines
1841 @section Packaging Guidelines
1842
1843 The GNU distribution is nascent and may well lack some of your favorite
1844 packages. This section describes how you can help make the distribution
1845 grow. @xref{Contributing}, for additional information on how you can
1846 help.
1847
1848 Free software packages are usually distributed in the form of
1849 @dfn{source code tarballs}---typically @file{tar.gz} files that contain
1850 all the source files. Adding a package to the distribution means
1851 essentially two things: adding a @dfn{recipe} that describes how to
1852 build the package, including a list of other packages required to build
1853 it, and adding @dfn{package meta-data} along with that recipe, such as a
1854 description and licensing information.
1855
1856 In Guix all this information is embodied in @dfn{package definitions}.
1857 Package definitions provide a high-level view of the package. They are
1858 written using the syntax of the Scheme programming language; in fact,
1859 for each package we define a variable bound to the package definition,
1860 and export that variable from a module (@pxref{Package Modules}).
1861 However, in-depth Scheme knowledge is @emph{not} a prerequisite for
1862 creating packages. For more information on package definitions,
1863 @ref{Defining Packages}.
1864
1865 Once a package definition is in place, stored in a file in the Guix
1866 source tree, it can be tested using the @command{guix build} command
1867 (@pxref{Invoking guix build}). For example, assuming the new package is
1868 called @code{gnew}, you may run this command from the Guix build tree:
1869
1870 @example
1871 ./pre-inst-env guix build gnew --keep-failed
1872 @end example
1873
1874 Using @code{--keep-failed} makes it easier to debug build failures since
1875 it provides access to the failed build tree.
1876
1877 Once your package builds correctly, please send us a patch
1878 (@pxref{Contributing}). Well, if you need help, we will be happy to
1879 help you too. Once the patch is committed in the Guix repository, the
1880 new package automatically gets built on the supported platforms by
1881 @url{http://hydra.gnu.org/gnu/master, our continuous integration
1882 system}.
1883
1884 @cindex substituter
1885 Users can obtain the new package definition simply by running
1886 @command{guix pull} (@pxref{Invoking guix pull}). When
1887 @code{hydra.gnu.org} is done building the package, installing the
1888 package automatically downloads binaries from there (except when using
1889 @code{--no-substitutes}). The only place where human intervention is
1890 needed is to review and apply the patch.
1891
1892
1893 @menu
1894 * Software Freedom:: What may go into the distribution.
1895 * Package Naming:: What's in a name?
1896 * Version Numbers:: When the name is not enough.
1897 * Python Modules:: Taming the snake.
1898 @end menu
1899
1900 @node Software Freedom
1901 @subsection Software Freedom
1902
1903 @c Adapted from http://www.gnu.org/philosophy/philosophy.html.
1904
1905 The GNU operating system has been developed so that users can have
1906 freedom in their computing. GNU is @dfn{free software}, meaning that
1907 users have the @url{http://www.gnu.org/philosophy/free-sw.html,four
1908 essential freedoms}: to run the program, to study and change the program
1909 in source code form, to redistribute exact copies, and to distribute
1910 modified versions. Packages found in the GNU distribution provide only
1911 software that conveys these four freedoms.
1912
1913 In addition, the GNU distribution follow the
1914 @url{http://www.gnu.org/distros/free-system-distribution-guidelines.html,free
1915 software distribution guidelines}. Among other things, these guidelines
1916 reject non-free firmware, recommendations of non-free software, and
1917 discuss ways to deal with trademarks and patents.
1918
1919 Some packages contain a small and optional subset that violates the
1920 above guidelines, for instance because this subset is itself non-free
1921 code. When that happens, the offending items are removed with
1922 appropriate patches or code snippets in the package definition's
1923 @code{origin} form (@pxref{Defining Packages}). That way, @code{guix
1924 build --source} returns the ``freed'' source rather than the unmodified
1925 upstream source.
1926
1927
1928 @node Package Naming
1929 @subsection Package Naming
1930
1931 A package has actually two names associated with it:
1932 First, there is the name of the @emph{Scheme variable}, the one following
1933 @code{define-public}. By this name, the package can be made known in the
1934 Scheme code, for instance as input to another package. Second, there is
1935 the string in the @code{name} field of a package definition. This name
1936 is used by package management commands such as
1937 @command{guix package} and @command{guix build}.
1938
1939 Both are usually the same and correspond to the lowercase conversion of the
1940 project name chosen upstream. For instance, the GNUnet project is packaged
1941 as @code{gnunet}. We do not add @code{lib} prefixes for library packages,
1942 unless these are already part of the official project name. But see
1943 @ref{Python Modules} for special rules concerning modules for
1944 the Python language.
1945
1946
1947 @node Version Numbers
1948 @subsection Version Numbers
1949
1950 We usually package only the latest version of a given free software
1951 project. But sometimes, for instance for incompatible library versions,
1952 two (or more) versions of the same package are needed. These require
1953 different Scheme variable names. We use the name as defined
1954 in @ref{Package Naming}
1955 for the most recent version; previous versions use the same name, suffixed
1956 by @code{-} and the smallest prefix of the version number that may
1957 distinguish the two versions.
1958
1959 The name inside the package definition is the same for all versions of a
1960 package and does not contain any version number.
1961
1962 For instance, the versions 2.24.20 and 3.9.12 of GTK+ may be packaged as follows:
1963
1964 @example
1965 (define-public gtk+
1966 (package
1967 (name "gtk+")
1968 (version "3.9.12")
1969 ...))
1970 (define-public gtk+-2
1971 (package
1972 (name "gtk+")
1973 (version "2.24.20")
1974 ...))
1975 @end example
1976 If we also wanted GTK+ 3.8.2, this would be packaged as
1977 @example
1978 (define-public gtk+-3.8
1979 (package
1980 (name "gtk+")
1981 (version "3.8.2")
1982 ...))
1983 @end example
1984
1985
1986 @node Python Modules
1987 @subsection Python Modules
1988
1989 We currently package Python 2 and Python 3, under the Scheme variable names
1990 @code{python-2} and @code{python} as explained in @ref{Version Numbers}.
1991 To avoid confusion and naming clashes with other programming languages, it
1992 seems desirable that the name of a package for a Python module contains
1993 the word @code{python}.
1994
1995 Some modules are compatible with only one version of Python, others with both.
1996 If the package Foo compiles only with Python 3, we name it
1997 @code{python-foo}; if it compiles only with Python 2, we name it
1998 @code{python2-foo}. If it is compatible with both versions, we create two
1999 packages with the corresponding names.
2000
2001 If a project already contains the word @code{python}, we drop this;
2002 for instance, the module python-dateutil is packaged under the names
2003 @code{python-dateutil} and @code{python2-dateutil}.
2004
2005
2006
2007
2008
2009 @node Bootstrapping
2010 @section Bootstrapping
2011
2012 @c Adapted from the ELS 2013 paper.
2013
2014 @cindex bootstrapping
2015
2016 Bootstrapping in our context refers to how the distribution gets built
2017 ``from nothing''. Remember that the build environment of a derivation
2018 contains nothing but its declared inputs (@pxref{Introduction}). So
2019 there's an obvious chicken-and-egg problem: how does the first package
2020 get built? How does the first compiler get compiled? Note that this is
2021 a question of interest only to the curious hacker, not to the regular
2022 user, so you can shamelessly skip this section if you consider yourself
2023 a ``regular user''.
2024
2025 @cindex bootstrap binaries
2026 The GNU system is primarily made of C code, with libc at its core. The
2027 GNU build system itself assumes the availability of a Bourne shell and
2028 command-line tools provided by GNU Coreutils, Awk, Findutils, `sed', and
2029 `grep'. Furthermore, build programs---programs that run
2030 @code{./configure}, @code{make}, etc.---are written in Guile Scheme
2031 (@pxref{Derivations}). Consequently, to be able to build anything at
2032 all, from scratch, Guix relies on pre-built binaries of Guile, GCC,
2033 Binutils, libc, and the other packages mentioned above---the
2034 @dfn{bootstrap binaries}.
2035
2036 These bootstrap binaries are ``taken for granted'', though we can also
2037 re-create them if needed (more on that later).
2038
2039 @unnumberedsubsec Preparing to Use the Bootstrap Binaries
2040
2041 @c As of Emacs 24.3, Info-mode displays the image, but since it's a
2042 @c large image, it's hard to scroll. Oh well.
2043 @image{images/bootstrap-graph,6in,,Dependency graph of the early bootstrap derivations}
2044
2045 The figure above shows the very beginning of the dependency graph of the
2046 distribution, corresponding to the package definitions of the @code{(gnu
2047 packages bootstrap)} module. At this level of detail, things are
2048 slightly complex. First, Guile itself consists of an ELF executable,
2049 along with many source and compiled Scheme files that are dynamically
2050 loaded when it runs. This gets stored in the @file{guile-2.0.7.tar.xz}
2051 tarball shown in this graph. This tarball is part of Guix's ``source''
2052 distribution, and gets inserted into the store with @code{add-to-store}
2053 (@pxref{The Store}).
2054
2055 But how do we write a derivation that unpacks this tarball and adds it
2056 to the store? To solve this problem, the @code{guile-bootstrap-2.0.drv}
2057 derivation---the first one that gets built---uses @code{bash} as its
2058 builder, which runs @code{build-bootstrap-guile.sh}, which in turn calls
2059 @code{tar} to unpack the tarball. Thus, @file{bash}, @file{tar},
2060 @file{xz}, and @file{mkdir} are statically-linked binaries, also part of
2061 the Guix source distribution, whose sole purpose is to allow the Guile
2062 tarball to be unpacked.
2063
2064 Once @code{guile-bootstrap-2.0.drv} is built, we have a functioning
2065 Guile that can be used to run subsequent build programs. Its first task
2066 is to download tarballs containing the other pre-built binaries---this
2067 is what the @code{.tar.xz.drv} derivations do. Guix modules such as
2068 @code{ftp-client.scm} are used for this purpose. The
2069 @code{module-import.drv} derivations import those modules in a directory
2070 in the store, using the original layout. The
2071 @code{module-import-compiled.drv} derivations compile those modules, and
2072 write them in an output directory with the right layout. This
2073 corresponds to the @code{#:modules} argument of
2074 @code{build-expression->derivation} (@pxref{Derivations}).
2075
2076 Finally, the various tarballs are unpacked by the
2077 derivations @code{gcc-bootstrap-0.drv}, @code{glibc-bootstrap-0.drv},
2078 etc., at which point we have a working C tool chain.
2079
2080
2081 @unnumberedsubsec Building the Build Tools
2082
2083 @c TODO: Add a package-level dependency graph generated from (gnu
2084 @c packages base).
2085
2086 Bootstrapping is complete when we have a full tool chain that does not
2087 depend on the pre-built bootstrap tools discussed above. This
2088 no-dependency requirement is verified by checking whether the files of
2089 the final tool chain contain references to the @file{/nix/store}
2090 directories of the bootstrap inputs. The process that leads to this
2091 ``final'' tool chain is described by the package definitions found in
2092 the @code{(gnu packages base)} module.
2093
2094 @c See <http://lists.gnu.org/archive/html/gnu-system-discuss/2012-10/msg00000.html>.
2095 The first tool that gets built with the bootstrap binaries is
2096 GNU Make, which is a prerequisite for all the following packages.
2097 From there Findutils and Diffutils get built.
2098
2099 Then come the first-stage Binutils and GCC, built as pseudo cross
2100 tools---i.e., with @code{--target} equal to @code{--host}. They are
2101 used to build libc. Thanks to this cross-build trick, this libc is
2102 guaranteed not to hold any reference to the initial tool chain.
2103
2104 From there the final Binutils and GCC are built. GCC uses @code{ld}
2105 from the final Binutils, and links programs against the just-built libc.
2106 This tool chain is used to build the other packages used by Guix and by
2107 the GNU Build System: Guile, Bash, Coreutils, etc.
2108
2109 And voilà! At this point we have the complete set of build tools that
2110 the GNU Build System expects. These are in the @code{%final-inputs}
2111 variables of the @code{(gnu packages base)} module, and are implicitly
2112 used by any package that uses @code{gnu-build-system} (@pxref{Defining
2113 Packages}).
2114
2115
2116 @unnumberedsubsec Building the Bootstrap Binaries
2117
2118 Because the final tool chain does not depend on the bootstrap binaries,
2119 those rarely need to be updated. Nevertheless, it is useful to have an
2120 automated way to produce them, should an update occur, and this is what
2121 the @code{(gnu packages make-bootstrap)} module provides.
2122
2123 The following command builds the tarballs containing the bootstrap
2124 binaries (Guile, Binutils, GCC, libc, and a tarball containing a mixture
2125 of Coreutils and other basic command-line tools):
2126
2127 @example
2128 guix build bootstrap-tarballs
2129 @end example
2130
2131 The generated tarballs are those that should be referred to in the
2132 @code{(gnu packages bootstrap)} module mentioned at the beginning of
2133 this section.
2134
2135 Still here? Then perhaps by now you've started to wonder: when do we
2136 reach a fixed point? That is an interesting question! The answer is
2137 unknown, but if you would like to investigate further (and have
2138 significant computational and storage resources to do so), then let us
2139 know.
2140
2141 @node Porting
2142 @section Porting to a New Platform
2143
2144 As discussed above, the GNU distribution is self-contained, and
2145 self-containment is achieved by relying on pre-built ``bootstrap
2146 binaries'' (@pxref{Bootstrapping}). These binaries are specific to an
2147 operating system kernel, CPU architecture, and application binary
2148 interface (ABI). Thus, to port the distribution to a platform that is
2149 not yet supported, one must build those bootstrap binaries, and update
2150 the @code{(gnu packages bootstrap)} module to use them on that platform.
2151
2152 Fortunately, Guix can @emph{cross compile} those bootstrap binaries.
2153 When everything goes well, and assuming the GNU tool chain supports the
2154 target platform, this can be as simple as running a command like this
2155 one:
2156
2157 @example
2158 guix build --target=armv5tel-linux-gnueabi bootstrap-tarballs
2159 @end example
2160
2161 Once these are built, the @code{(gnu packages bootstrap)} module needs
2162 to be updated to refer to these binaries on the target platform. In
2163 addition, the @code{glibc-dynamic-linker} procedure in that module must
2164 be augmented to return the right file name for libc's dynamic linker on
2165 that platform; likewise, @code{system->linux-architecture} in @code{(gnu
2166 packages linux)} must be taught about the new platform.
2167
2168 In practice, there may be some complications. First, it may be that the
2169 extended GNU triplet that specifies an ABI (like the @code{eabi} suffix
2170 above) is not recognized by all the GNU tools. Typically, glibc
2171 recognizes some of these, whereas GCC uses an extra @code{--with-abi}
2172 configure flag (see @code{gcc.scm} for examples of how to handle this).
2173 Second, some of the required packages could fail to build for that
2174 platform. Lastly, the generated binaries could be broken for some
2175 reason.
2176
2177
2178 @c *********************************************************************
2179 @node Contributing
2180 @chapter Contributing
2181
2182 This project is a cooperative effort, and we need your help to make it
2183 grow! Please get in touch with us on @email{guix-devel@@gnu.org}. We
2184 welcome ideas, bug reports, patches, and anything that may be helpful to
2185 the project. We particularly welcome help on packaging
2186 (@pxref{Packaging Guidelines}).
2187
2188 Please see the
2189 @url{http://git.savannah.gnu.org/cgit/guix.git/tree/HACKING,
2190 @file{HACKING} file} that comes with the Guix source code for practical
2191 details about contributions.
2192
2193
2194 @c *********************************************************************
2195 @node Acknowledgments
2196 @chapter Acknowledgments
2197
2198 Guix is based on the Nix package manager, which was designed and
2199 implemented by Eelco Dolstra. Nix pioneered functional package
2200 management, and promoted unprecedented features, such as transactional
2201 package upgrades and rollbacks, per-user profiles, and referentially
2202 transparent build processes. Without this work, Guix would not exist.
2203
2204 The Nix-based software distributions, Nixpkgs and NixOS, have also been
2205 an inspiration for Guix.
2206
2207 @c *********************************************************************
2208 @node GNU Free Documentation License
2209 @appendix GNU Free Documentation License
2210
2211 @include fdl-1.3.texi
2212
2213 @c *********************************************************************
2214 @node Concept Index
2215 @unnumbered Concept Index
2216 @printindex cp
2217
2218 @node Function Index
2219 @unnumbered Function Index
2220 @printindex fn
2221
2222 @bye
2223
2224 @c Local Variables:
2225 @c ispell-local-dictionary: "american";
2226 @c End: