guix build: Add '--log-file'.
[jackhill/guix/guix.git] / doc / guix.texi
1 \input texinfo
2 @c -*-texinfo-*-
3
4 @c %**start of header
5 @setfilename guix.info
6 @documentencoding UTF-8
7 @settitle GNU Guix Reference Manual
8 @c %**end of header
9
10 @include version.texi
11
12 @copying
13 Copyright @copyright{} 2012, 2013 Ludovic Courtès@*
14 Copyright @copyright{} 2013 Andreas Enge@*
15 Copyright @copyright{} 2013 Nikita Karetnikov
16
17 Permission is granted to copy, distribute and/or modify this document
18 under the terms of the GNU Free Documentation License, Version 1.3 or
19 any later version published by the Free Software Foundation; with no
20 Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
21 copy of the license is included in the section entitled ``GNU Free
22 Documentation License''.
23 @end copying
24
25 @dircategory Package management
26 @direntry
27 * guix: (guix). Guix, the functional package manager.
28 * guix package: (guix)Invoking guix package
29 Managing packages with Guix.
30 * guix build: (guix)Invoking guix build
31 Building packages with Guix.
32 @end direntry
33
34 @titlepage
35 @title GNU Guix Reference Manual
36 @subtitle Using the GNU Guix Functional Package Manager
37 @author Ludovic Courtès
38 @author Andreas Enge
39 @author Nikita Karetnikov
40
41 @page
42 @vskip 0pt plus 1filll
43 Edition @value{EDITION} @*
44 @value{UPDATED} @*
45
46 @insertcopying
47 @end titlepage
48
49 @contents
50
51 @c *********************************************************************
52 @node Top
53 @top GNU Guix
54
55 This document describes GNU Guix version @value{VERSION}, a functional
56 package management tool written for the GNU system.
57
58 @menu
59 * Introduction:: What is Guix about?
60 * Installation:: Installing Guix.
61 * Package Management:: Package installation, upgrade, etc.
62 * Programming Interface:: Using Guix in Scheme.
63 * Utilities:: Package management commands.
64 * GNU Distribution:: Software for your friendly GNU system.
65 * Contributing:: Your help needed!
66
67 * Acknowledgments:: Thanks!
68 * GNU Free Documentation License:: The license of this manual.
69 * Concept Index:: Concepts.
70 * Function Index:: Functions.
71 @end menu
72
73 @c *********************************************************************
74 @node Introduction
75 @chapter Introduction
76
77 GNU Guix@footnote{``Guix'' is pronounced like ``geeks'', or ``ɡiːks''
78 using the international phonetic alphabet (IPA).} is a functional
79 package management tool for the GNU system. Package management consists
80 of all activities that relate to building packages from sources,
81 honoring their build-time and run-time dependencies,
82 installing packages in user environments, upgrading installed packages
83 to new versions or rolling back to a previous set, removing unused
84 software packages, etc.
85
86 @cindex functional package management
87 The term @dfn{functional} refers to a specific package management
88 discipline. In Guix, the package build and installation process is seen
89 as a function, in the mathematical sense. That function takes inputs,
90 such as build scripts, a compiler, and libraries, and
91 returns an installed package. As a pure function, its result depends
92 solely on its inputs---for instance, it cannot refer to software or
93 scripts that were not explicitly passed as inputs. A build function
94 always produces the same result when passed a given set of inputs. It
95 cannot alter the system's environment in
96 any way; for instance, it cannot create, modify, or delete files outside
97 of its build and installation directories. This is achieved by running
98 build processes in isolated environments (or @dfn{chroots}), where only their
99 explicit inputs are visible.
100
101 @cindex store
102 The result of package build functions is @dfn{cached} in the file
103 system, in a special directory called @dfn{the store} (@pxref{The
104 Store}). Each package is installed in a directory of its own, in the
105 store---by default under @file{/nix/store}. The directory name contains
106 a hash of all the inputs used to build that package; thus, changing an
107 input yields a different directory name.
108
109 This approach is the foundation of Guix's salient features: support for
110 transactional package upgrade and rollback, per-user installation, and
111 garbage collection of packages (@pxref{Features}).
112
113 Guix has a command-line interface, which allows users to build, install,
114 upgrade, and remove packages, as well as a Scheme programming interface.
115
116 Last but not least, Guix is used to build a distribution of the GNU
117 system, with many GNU and non-GNU free software packages. @xref{GNU
118 Distribution}.
119
120 @c *********************************************************************
121 @node Installation
122 @chapter Installation
123
124 GNU Guix is available for download from its website at
125 @url{http://www.gnu.org/software/guix/}. This section describes the
126 software requirements of Guix, as well as how to install it and get
127 ready to use it.
128
129 The build procedure for Guix is the same as for other GNU software, and
130 is not covered here. Please see the files @file{README} and
131 @file{INSTALL} in the Guix source tree for additional details.
132
133 @menu
134 * Requirements:: Software needed to build and run Guix.
135 * Setting Up the Daemon:: Preparing the build daemon's environment.
136 * Invoking guix-daemon:: Running the build daemon.
137 @end menu
138
139 @node Requirements
140 @section Requirements
141
142 GNU Guix depends on the following packages:
143
144 @itemize
145 @item @url{http://gnu.org/software/guile/, GNU Guile}, version 2.0.5 or later;
146 @item @url{http://gnupg.org/, GNU libgcrypt}
147 @end itemize
148
149 Unless @code{--disable-daemon} was passed to @command{configure}, the
150 following packages are also needed:
151
152 @itemize
153 @item @url{http://sqlite.org, SQLite 3}
154 @item @url{http://www.bzip.org, libbz2}
155 @item @url{http://gcc.gnu.org, GCC's g++}
156 @end itemize
157
158 When a working installation of @url{http://nixos.org/nix/, the Nix package
159 manager} is available, you
160 can instead configure Guix with @code{--disable-daemon}. In that case,
161 Nix replaces the three dependencies above.
162
163 Guix is compatible with Nix, so it is possible to share the same store
164 between both. To do so, you must pass @command{configure} not only the
165 same @code{--with-store-dir} value, but also the same
166 @code{--localstatedir} value. The latter is essential because it
167 specifies where the database that stores metadata about the store is
168 located, among other things. The default values are
169 @code{--with-store-dir=/nix/store} and @code{--localstatedir=/nix/var}.
170 Note that @code{--disable-daemon} is not required if
171 your goal is to share the store with Nix.
172
173 @node Setting Up the Daemon
174 @section Setting Up the Daemon
175
176 @cindex daemon
177 Operations such as building a package or running the garbage collector
178 are all performed by a specialized process, the @dfn{Guix daemon}, on
179 behalf of clients. Only the daemon may access the store and its
180 associated database. Thus, any operation that manipulates the store
181 goes through the daemon. For instance, command-line tools such as
182 @command{guix package} and @command{guix build} communicate with the
183 daemon (@i{via} remote procedure calls) to instruct it what to do.
184
185 In a standard multi-user setup, Guix and its daemon---the
186 @command{guix-daemon} program---are installed by the system
187 administrator; @file{/nix/store} is owned by @code{root} and
188 @command{guix-daemon} runs as @code{root}. Unprivileged users may use
189 Guix tools to build packages or otherwise access the store, and the
190 daemon will do it on their behalf, ensuring that the store is kept in a
191 consistent state, and allowing built packages to be shared among users.
192
193 @cindex build users
194 When @command{guix-daemon} runs as @code{root}, you may not want package
195 build processes themselves to run as @code{root} too, for obvious
196 security reasons. To avoid that, a special pool of @dfn{build users}
197 should be created for use by build processes started by the daemon.
198 These build users need not have a shell and a home directory: they will
199 just be used when the daemon drops @code{root} privileges in build
200 processes. Having several such users allows the daemon to launch
201 distinct build processes under separate UIDs, which guarantees that they
202 do not interfere with each other---an essential feature since builds are
203 regarded as pure functions (@pxref{Introduction}).
204
205 On a GNU/Linux system, a build user pool may be created like this (using
206 Bash syntax and the @code{shadow} commands):
207
208 @c See http://lists.gnu.org/archive/html/bug-guix/2013-01/msg00239.html
209 @c for why `-G' is needed.
210 @example
211 # groupadd guix-builder
212 # for i in `seq 1 10`;
213 do
214 useradd -g guix-builder -G guix-builder \
215 -d /var/empty -s `which nologin` \
216 -c "Guix build user $i" guix-builder$i;
217 done
218 @end example
219
220 @noindent
221 The @code{guix-daemon} program may then be run as @code{root} with:
222
223 @example
224 # guix-daemon --build-users-group=guix-builder
225 @end example
226
227 @noindent
228 This way, the daemon starts build processes in a chroot, under one of
229 the @code{guix-builder} users. On GNU/Linux, by default, the chroot
230 environment contains nothing but the @code{/dev} and @code{/proc}
231 directories@footnote{On some systems @code{/dev/shm}, which supports
232 shared memory, is a symlink to another directory such as
233 @code{/run/shm}, that is @emph{not} is the chroot. When that is the
234 case, shared memory support is unavailable in the chroot environment.
235 The workaround is to make sure that @file{/dev/shm} is directly a
236 @code{tmpfs} mount point.}.
237
238 Guix may also be used in a single-user setup, with @command{guix-daemon}
239 running as an unprivileged user. However, to maximize non-interference
240 of build processes, the daemon still needs to perform certain operations
241 that are restricted to @code{root} on GNU/Linux: it should be able to
242 run build processes in a chroot, and to run them under different UIDs.
243 To that end, the @command{nix-setuid-helper} program is provided; it is
244 a small C program (less than 300 lines) that, if it is made setuid
245 @code{root}, can be executed by the daemon to perform these operations
246 on its behalf. The @code{root}-owned @file{/etc/nix-setuid.conf} file
247 is read by @command{nix-setuid-helper}; it should contain exactly two
248 words: the user name under which the authorized @command{guix-daemon}
249 runs, and the name of the build users group.
250
251 If you are installing Guix as an unprivileged user and do not have the
252 ability to make @file{nix-setuid-helper} setuid-@code{root}, it is still
253 possible to run @command{guix-daemon}. However, build processes will
254 not be isolated from one another, and not from the rest of the system.
255 Thus, build processes may interfere with each other, and may access
256 programs, libraries, and other files available on the system---making it
257 much harder to view them as @emph{pure} functions.
258
259 @node Invoking guix-daemon
260 @section Invoking @command{guix-daemon}
261
262 The @command{guix-daemon} program implements all the functionality to
263 access the store. This includes launching build processes, running the
264 garbage collector, querying the availability of a build result, etc. It
265 is normally run as @code{root} like this:
266
267 @example
268 # guix-daemon --build-users-group=guix-builder
269 @end example
270
271 @noindent
272 For details on how to set it up, @ref{Setting Up the Daemon}.
273
274 By default, @command{guix-daemon} launches build processes under
275 different UIDs, taken from the build group specified with
276 @code{--build-users-group}. In addition, each build process is run in a
277 chroot environment that only contains the subset of the store that the
278 build process depends on, as specified by its derivation
279 (@pxref{Programming Interface, derivation}), plus a set of specific
280 system directories. By default, the latter contains @file{/dev} and
281 @file{/dev/pts}.
282
283 The following command-line options are supported:
284
285 @table @code
286 @item --build-users-group=@var{group}
287 Take users from @var{group} to run build processes (@pxref{Setting Up
288 the Daemon, build users}).
289
290 @item --no-substitutes
291 @cindex substitutes
292 Do not use substitutes for build products. That is, always build things
293 locally instead of allowing downloads of pre-built binaries.
294
295 By default substitutes are used, unless the client---such as the
296 @command{guix package} command---is explicitly invoked with
297 @code{--no-substitutes}.
298
299 When the daemon runs with @code{--no-substitutes}, clients can still
300 explicitly enable substitution @i{via} the @code{set-build-options}
301 remote procedure call (@pxref{The Store}).
302
303 @item --cache-failures
304 Cache build failures. By default, only successful builds are cached.
305
306 @item --cores=@var{n}
307 @itemx -c @var{n}
308 Use @var{n} CPU cores to build each derivation; @code{0} means as many
309 as available.
310
311 The default value is @code{1}, but it may be overridden by clients, such
312 as the @code{--cores} option of @command{guix build} (@pxref{Invoking
313 guix build}).
314
315 The effect is to define the @code{NIX_BUILD_CORES} environment variable
316 in the build process, which can then use it to exploit internal
317 parallelism---for instance, by running @code{make -j$NIX_BUILD_CORES}.
318
319 @item --max-jobs=@var{n}
320 @itemx -M @var{n}
321 Allow at most @var{n} build jobs in parallel. The default value is
322 @code{1}.
323
324 @item --debug
325 Produce debugging output.
326
327 This is useful to debug daemon start-up issues, but then it may be
328 overridden by clients, for example the @code{--verbosity} option of
329 @command{guix build} (@pxref{Invoking guix build}).
330
331 @item --chroot-directory=@var{dir}
332 Add @var{dir} to the build chroot.
333
334 Doing this may change the result of build processes---for instance if
335 they use optional dependencies found in @var{dir} when it is available,
336 and not otherwise. For that reason, it is not recommended to do so.
337 Instead, make sure that each derivation declares all the inputs that it
338 needs.
339
340 @item --disable-chroot
341 Disable chroot builds.
342
343 Using this option is not recommended since, again, it would allow build
344 processes to gain access to undeclared dependencies.
345
346 @item --disable-log-compression
347 Disable compression of the build logs.
348
349 Unless @code{--lose-logs} is used, all the build logs are kept in the
350 @var{localstatedir}. To save space, the daemon automatically compresses
351 them with bzip2 by default. This option disables that.
352
353 @item --disable-store-optimization
354 Disable automatic file ``deduplication'' in the store.
355
356 By default, files added to the store are automatically ``deduplicated'':
357 if a newly added file is identical as another one found in the store,
358 the daemon makes the new file a hard link to the other file. This
359 slightly increases the input/output load at the end of a build process.
360 This option disables this.
361
362 @item --impersonate-linux-2.6
363 On Linux-based systems, impersonate Linux 2.6. This means that the
364 kernel's @code{uname} system call will report 2.6 as the release number.
365
366 This might be helpful to build programs that (usually wrongfully) depend
367 on the kernel version number.
368
369 @item --lose-logs
370 Do not keep build logs. By default they are kept under
371 @code{@var{localstatedir}/nix/log}.
372
373 @item --system=@var{system}
374 Assume @var{system} as the current system type. By default it is the
375 architecture/kernel pair found at configure time, such as
376 @code{x86_64-linux}.
377
378 @item --listen=@var{socket}
379 Listen for connections on @var{socket}, the file name of a Unix-domain
380 socket. The default socket is
381 @file{@var{localstatedir}/daemon-socket/socket}. This option is only
382 useful in exceptional circumstances, such as if you need to run several
383 daemons on the same machine.
384 @end table
385
386
387 @c *********************************************************************
388 @node Package Management
389 @chapter Package Management
390
391 The purpose of GNU Guix is to allow users to easily install, upgrade, and
392 remove software packages, without having to know about their build
393 procedure or dependencies. Guix also goes beyond this obvious set of
394 features.
395
396 This chapter describes the main features of Guix, as well as the package
397 management tools it provides.
398
399 @menu
400 * Features:: How Guix will make your life brighter.
401 * Invoking guix package:: Package installation, removal, etc.
402 * Packages with Multiple Outputs:: Single source package, multiple outputs.
403 * Invoking guix gc:: Running the garbage collector.
404 * Invoking guix pull:: Fetching the latest Guix and distribution.
405 @end menu
406
407 @node Features
408 @section Features
409
410 When using Guix, each package ends up in the @dfn{package store}, in its
411 own directory---something that resembles
412 @file{/nix/store/xxx-package-1.2}, where @code{xxx} is a base32 string.
413
414 Instead of referring to these directories, users have their own
415 @dfn{profile}, which points to the packages that they actually want to
416 use. These profiles are stored within each user's home directory, at
417 @code{$HOME/.guix-profile}.
418
419 For example, @code{alice} installs GCC 4.7.2. As a result,
420 @file{/home/alice/.guix-profile/bin/gcc} points to
421 @file{/nix/store/@dots{}-gcc-4.7.2/bin/gcc}. Now, on the same machine,
422 @code{bob} had already installed GCC 4.8.0. The profile of @code{bob}
423 simply continues to point to
424 @file{/nix/store/@dots{}-gcc-4.8.0/bin/gcc}---i.e., both versions of GCC
425 coexist on the same system without any interference.
426
427 The @command{guix package} command is the central tool to manage
428 packages (@pxref{Invoking guix package}). It operates on those per-user
429 profiles, and can be used @emph{with normal user privileges}.
430
431 The command provides the obvious install, remove, and upgrade
432 operations. Each invocation is actually a @emph{transaction}: either
433 the specified operation succeeds, or nothing happens. Thus, if the
434 @command{guix package} process is terminated during the transaction,
435 or if a power outage occurs during the transaction, then the user's
436 profile remains in its previous state, and remains usable.
437
438 In addition, any package transaction may be @emph{rolled back}. So, if,
439 for example, an upgrade installs a new version of a package that turns
440 out to have a serious bug, users may roll back to the previous instance
441 of their profile, which was known to work well.
442
443 All those packages in the package store may be @emph{garbage-collected}.
444 Guix can determine which packages are still referenced by the user
445 profiles, and remove those that are provably no longer referenced
446 (@pxref{Invoking guix gc}). Users may also explicitly remove old
447 generations of their profile so that the packages they refer to can be
448 collected.
449
450 Finally, Guix takes a @dfn{purely functional} approach to package
451 management, as described in the introduction (@pxref{Introduction}).
452 Each @file{/nix/store} package directory name contains a hash of all the
453 inputs that were used to build that package---compiler, libraries, build
454 scripts, etc. This direct correspondence allows users to make sure a
455 given package installation matches the current state of their
456 distribution, and helps maximize @dfn{reproducibility}.
457
458 @cindex substitute
459 This foundation allows Guix to support @dfn{transparent binary/source
460 deployment}. When a pre-built binary for a @file{/nix/store} path is
461 available from an external source---a @dfn{substitute}, Guix just
462 downloads it@footnote{@c XXX: Remove me when outdated.
463 As of version @value{VERSION}, substitutes are downloaded from
464 @url{http://hydra.gnu.org/} but are @emph{not} authenticated---i.e.,
465 Guix cannot tell whether binaries it downloaded have been tampered with,
466 nor whether they come from the genuine @code{gnu.org} build farm. This
467 will be fixed in future versions. In the meantime, concerned users can
468 opt for @code{--no-substitutes} (@pxref{Invoking guix-daemon}).};
469 otherwise, it builds the package from source, locally.
470
471 @node Invoking guix package
472 @section Invoking @command{guix package}
473
474 The @command{guix package} command is the tool that allows users to
475 install, upgrade, and remove packages, as well as rolling back to
476 previous configurations. It operates only on the user's own profile,
477 and works with normal user privileges (@pxref{Features}). Its syntax
478 is:
479
480 @example
481 guix package @var{options}
482 @end example
483
484 Primarily, @var{options} specifies the operations to be performed during
485 the transaction. Upon completion, a new profile is created, but
486 previous generations of the profile remain available, should the user
487 want to roll back.
488
489 For each user, a symlink to the user's default profile is automatically
490 created in @file{$HOME/.guix-profile}. This symlink always points to the
491 current generation of the user's default profile. Thus, users can add
492 @file{$HOME/.guix-profile/bin} to their @code{PATH} environment
493 variable, and so on.
494
495 In a multi-user setup, user profiles must be stored in a place
496 registered as a @dfn{garbage-collector root}, which
497 @file{$HOME/.guix-profile} points to (@pxref{Invoking guix gc}). That
498 directory is normally
499 @code{@var{localstatedir}/profiles/per-user/@var{user}}, where
500 @var{localstatedir} is the value passed to @code{configure} as
501 @code{--localstatedir}, and @var{user} is the user name. It must be
502 created by @code{root}, with @var{user} as the owner. When it does not
503 exist, or is not owned by @var{user}, @command{guix package} emits an
504 error about it.
505
506 The @var{options} can be among the following:
507
508 @table @code
509
510 @item --install=@var{package}
511 @itemx -i @var{package}
512 Install @var{package}.
513
514 @var{package} may specify either a simple package name, such as
515 @code{guile}, or a package name followed by a hyphen and version number,
516 such as @code{guile-1.8.8}. If no version number is specified, the
517 newest available version will be selected. In addition, @var{package}
518 may contain a colon, followed by the name of one of the outputs of the
519 package, as in @code{gcc:doc} or @code{binutils-2.22:lib}
520 (@pxref{Packages with Multiple Outputs}).
521
522 @cindex propagated inputs
523 Sometimes packages have @dfn{propagated inputs}: these are dependencies
524 that automatically get installed along with the required package.
525
526 An example is the GNU MPC library: its C header files refer to those of
527 the GNU MPFR library, which in turn refer to those of the GMP library.
528 Thus, when installing MPC, the MPFR and GMP libraries also get installed
529 in the profile; removing MPC also removes MPFR and GMP---unless they had
530 also been explicitly installed independently.
531
532 Besides, packages sometimes rely on the definition of environment
533 variables for their search paths (see explanation of
534 @code{--search-paths} below). Any missing or possibly incorrect
535 environment variable definitions are reported here.
536
537 @c XXX: keep me up-to-date
538 Finally, when installing a GNU package, the tool reports the
539 availability of a newer upstream version. In the future, it may provide
540 the option of installing directly from the upstream version, even if
541 that version is not yet in the distribution.
542
543 @item --install-from-expression=@var{exp}
544 @itemx -e @var{exp}
545 Install the package @var{exp} evaluates to.
546
547 @var{exp} must be a Scheme expression that evaluates to a
548 @code{<package>} object. This option is notably useful to disambiguate
549 between same-named variants of a package, with expressions such as
550 @code{(@@ (gnu packages base) guile-final)}.
551
552 Note that this option installs the first output of the specified
553 package, which may be insufficient when needing a specific output of a
554 multiple-output package.
555
556 @item --remove=@var{package}
557 @itemx -r @var{package}
558 Remove @var{package}.
559
560 As for @code{--install}, @var{package} may specify a version number
561 and/or output name in addition to the package name. For instance,
562 @code{-r glibc:debug} would remove the @code{debug} output of
563 @code{glibc}.
564
565 @item --upgrade[=@var{regexp}]
566 @itemx -u [@var{regexp}]
567 Upgrade all the installed packages. When @var{regexp} is specified, upgrade
568 only installed packages whose name matches @var{regexp}.
569
570 Note that this upgrades package to the latest version of packages found
571 in the distribution currently installed. To update your distribution,
572 you should regularly run @command{guix pull} (@pxref{Invoking guix
573 pull}).
574
575 @item --roll-back
576 Roll back to the previous @dfn{generation} of the profile---i.e., undo
577 the last transaction.
578
579 When combined with options such as @code{--install}, roll back occurs
580 before any other actions.
581
582 When rolling back from the first generation that actually contains
583 installed packages, the profile is made to point to the @dfn{zeroth
584 generation}, which contains no files apart from its own meta-data.
585
586 Installing, removing, or upgrading packages from a generation that has
587 been rolled back to overwrites previous future generations. Thus, the
588 history of a profile's generations is always linear.
589
590 @item --search-paths
591 @cindex search paths
592 Report environment variable definitions, in Bash syntax, that may be
593 needed in order to use the set of installed packages. These environment
594 variables are used to specify @dfn{search paths} for files used by some
595 of the installed packages.
596
597 For example, GCC needs the @code{CPATH} and @code{LIBRARY_PATH}
598 environment variables to be defined so it can look for headers and
599 libraries in the user's profile (@pxref{Environment Variables,,, gcc,
600 Using the GNU Compiler Collection (GCC)}). If GCC and, say, the C
601 library are installed in the profile, then @code{--search-paths} will
602 suggest setting these variables to @code{@var{profile}/include} and
603 @code{@var{profile}/lib}, respectively.
604
605 @item --profile=@var{profile}
606 @itemx -p @var{profile}
607 Use @var{profile} instead of the user's default profile.
608
609 @item --dry-run
610 @itemx -n
611 Show what would be done without actually doing it.
612
613 @item --fallback
614 When substituting a pre-built binary fails, fall back to building
615 packages locally.
616
617 @item --no-substitutes
618 Do not use substitutes for build products. That is, always build things
619 locally instead of allowing downloads of pre-built binaries.
620
621 @item --max-silent-time=@var{seconds}
622 Same as for @command{guix build} (@pxref{Invoking guix build}).
623
624 @item --verbose
625 Produce verbose output. In particular, emit the environment's build log
626 on the standard error port.
627
628 @item --bootstrap
629 Use the bootstrap Guile to build the profile. This option is only
630 useful to distribution developers.
631
632 @end table
633
634 In addition to these actions @command{guix package} supports the
635 following options to query the current state of a profile, or the
636 availability of packages:
637
638 @table @option
639
640 @item --search=@var{regexp}
641 @itemx -s @var{regexp}
642 List the available packages whose synopsis or description matches
643 @var{regexp}. Print all the meta-data of matching packages in
644 @code{recutils} format (@pxref{Top, GNU recutils databases,, recutils,
645 GNU recutils manual}).
646
647 This allows specific fields to be extracted using the @command{recsel}
648 command, for instance:
649
650 @example
651 $ guix package -s malloc | recsel -p name,version
652 name: glibc
653 version: 2.17
654
655 name: libgc
656 version: 7.2alpha6
657 @end example
658
659 @item --list-installed[=@var{regexp}]
660 @itemx -I [@var{regexp}]
661 List the currently installed packages in the specified profile, with the
662 most recently installed packages shown last. When @var{regexp} is
663 specified, list only installed packages whose name matches @var{regexp}.
664
665 For each installed package, print the following items, separated by
666 tabs: the package name, its version string, the part of the package that
667 is installed (for instance, @code{out} for the default output,
668 @code{include} for its headers, etc.), and the path of this package in
669 the store.
670
671 @item --list-available[=@var{regexp}]
672 @itemx -A [@var{regexp}]
673 List packages currently available in the software distribution
674 (@pxref{GNU Distribution}). When @var{regexp} is specified, list only
675 installed packages whose name matches @var{regexp}.
676
677 For each package, print the following items separated by tabs: its name,
678 its version string, the parts of the package (@pxref{Packages with
679 Multiple Outputs}), and the source location of its definition.
680
681 @item --list-generations[=@var{pattern}]
682 @itemx -l [@var{pattern}]
683 Return a list of generations along with their creation dates; for each
684 generation, show the installed packages, with the most recently
685 installed packages shown last. Note that the zeroth generation is never
686 shown.
687
688 For each installed package, print the following items, separated by
689 tabs: the name of a package, its version string, the part of the package
690 that is installed (@pxref{Packages with Multiple Outputs}), and the
691 location of this package in the store.
692
693 When @var{pattern} is used, the command returns only matching
694 generations. Valid patterns include:
695
696 @itemize
697 @item @emph{Integers and comma-separated integers}. Both patterns denote
698 generation numbers. For instance, @code{--list-generations=1} returns
699 the first one.
700
701 And @code{--list-generations=1,8,2} outputs three generations in the
702 specified order. Neither spaces nor trailing commas are allowed.
703
704 @item @emph{Ranges}. @code{--list-generations=2..9} prints the
705 specified generations and everything in between. Note that the start of
706 a range must be lesser than its end.
707
708 It is also possible to omit the endpoint. For example,
709 @code{--list-generations=2..}, returns all generations starting from the
710 second one.
711
712 @item @emph{Durations}. You can also get the last @emph{N}@tie{}days, weeks,
713 or months by passing an integer along with the first letter of the
714 duration. For example, @code{--list-generations=20d} lists generations
715 that are up to 20 days old.
716 @end itemize
717
718 @item --delete-generations[=@var{pattern}]
719 @itemx -d [@var{pattern}]
720 When @var{pattern} is omitted, delete all generations except the current
721 one.
722
723 This command accepts the same patterns as @option{--list-generations}.
724 When @var{pattern} is specified, delete the matching generations. When
725 @var{pattern} specifies a duration, generations @emph{older} than the
726 specified duration match. For instance, @code{--delete-generations=1m}
727 deletes generations that are more than one month old.
728
729 If the current generation matches, it is deleted atomically---i.e., by
730 switching to the previous available generation. Note that the zeroth
731 generation is never deleted.
732
733 Note that deleting generations prevents roll-back to them.
734 Consequently, this command must be used with care.
735
736 @end table
737
738 @node Packages with Multiple Outputs
739 @section Packages with Multiple Outputs
740
741 @cindex multiple-output packages
742 @cindex package outputs
743
744 Often, packages defined in Guix have a single @dfn{output}---i.e., the
745 source package leads exactly one directory in the store. When running
746 @command{guix package -i glibc}, one installs the default output of the
747 GNU libc package; the default output is called @code{out}, but its name
748 can be omitted as shown in this command. In this particular case, the
749 default output of @code{glibc} contains all the C header files, shared
750 libraries, static libraries, Info documentation, and other supporting
751 files.
752
753 Sometimes it is more appropriate to separate the various types of files
754 produced from a single source package into separate outputs. For
755 instance, the GLib C library (used by GTK+ and related packages)
756 installs more than 20 MiB of reference documentation as HTML pages.
757 To save space for users who do not need it, the documentation goes to a
758 separate output, called @code{doc}. To install the main GLib output,
759 which contains everything but the documentation, one would run:
760
761 @example
762 guix package -i glib
763 @end example
764
765 The command to install its documentation is:
766
767 @example
768 guix package -i glib:doc
769 @end example
770
771 Some packages install programs with different ``dependency footprints''.
772 For instance, the WordNet package install both command-line tools and
773 graphical user interfaces (GUIs). The former depend solely on the C
774 library, whereas the latter depend on Tcl/Tk and the underlying X
775 libraries. In this case, we leave the command-line tools in the default
776 output, whereas the GUIs are in a separate output. This allows users
777 who do not need the GUIs to save space.
778
779 There are several such multiple-output packages in the GNU distribution.
780 Other conventional output names include @code{lib} for libraries and
781 possibly header files, @code{bin} for stand-alone programs, and
782 @code{debug} for debugging information (@pxref{Installing Debugging
783 Files}). The outputs of a packages are listed in the third column of
784 the output of @command{guix package --list-available} (@pxref{Invoking
785 guix package}).
786
787
788 @node Invoking guix gc
789 @section Invoking @command{guix gc}
790
791 @cindex garbage collector
792 Packages that are installed but not used may be @dfn{garbage-collected}.
793 The @command{guix gc} command allows users to explicitly run the garbage
794 collector to reclaim space from the @file{/nix/store} directory.
795
796 The garbage collector has a set of known @dfn{roots}: any file under
797 @file{/nix/store} reachable from a root is considered @dfn{live} and
798 cannot be deleted; any other file is considered @dfn{dead} and may be
799 deleted. The set of garbage collector roots includes default user
800 profiles, and may be augmented with @command{guix build --root}, for
801 example (@pxref{Invoking guix build}).
802
803 Prior to running @code{guix gc --collect-garbage} to make space, it is
804 often useful to remove old generations from user profiles; that way, old
805 package builds referenced by those generations can be reclaimed. This
806 is achieved by running @code{guix package --delete-generations}
807 (@pxref{Invoking guix package}).
808
809 The @command{guix gc} command has three modes of operation: it can be
810 used to garbage-collect any dead files (the default), to delete specific
811 files (the @code{--delete} option), or to print garbage-collector
812 information. The available options are listed below:
813
814 @table @code
815 @item --collect-garbage[=@var{min}]
816 @itemx -C [@var{min}]
817 Collect garbage---i.e., unreachable @file{/nix/store} files and
818 sub-directories. This is the default operation when no option is
819 specified.
820
821 When @var{min} is given, stop once @var{min} bytes have been collected.
822 @var{min} may be a number of bytes, or it may include a unit as a
823 suffix, such as @code{MiB} for mebibytes and @code{GB} for gigabytes.
824
825 When @var{min} is omitted, collect all the garbage.
826
827 @item --delete
828 @itemx -d
829 Attempt to delete all the store files and directories specified as
830 arguments. This fails if some of the files are not in the store, or if
831 they are still live.
832
833 @item --list-dead
834 Show the list of dead files and directories still present in the
835 store---i.e., files and directories no longer reachable from any root.
836
837 @item --list-live
838 Show the list of live store files and directories.
839
840 @end table
841
842 In addition, the references among existing store files can be queried:
843
844 @table @code
845
846 @item --references
847 @itemx --referrers
848 List the references (respectively, the referrers) of store files given
849 as arguments.
850
851 @item --requisites
852 @itemx -R
853 List the requisites of the store files passed as arguments. Requisites
854 include the store files themselves, their references, and the references
855 of these, recursively. In other words, the returned list is the
856 @dfn{transitive closure} of the store files.
857
858 @end table
859
860
861 @node Invoking guix pull
862 @section Invoking @command{guix pull}
863
864 Packages are installed or upgraded to the latest version available in
865 the distribution currently available on your local machine. To update
866 that distribution, along with the Guix tools, you must run @command{guix
867 pull}: the command downloads the latest Guix source code and package
868 descriptions, and deploys it.
869
870 On completion, @command{guix package} will use packages and package
871 versions from this just-retrieved copy of Guix. Not only that, but all
872 the Guix commands and Scheme modules will also be taken from that latest
873 version. New @command{guix} sub-commands added by the update also
874 become available.
875
876 The @command{guix pull} command is usually invoked with no arguments,
877 but it supports the following options:
878
879 @table @code
880 @item --verbose
881 Produce verbose output, writing build logs to the standard error output.
882
883 @item --url=@var{url}
884 Download the source tarball of Guix from @var{url}.
885
886 By default, the tarball is taken from its canonical address at
887 @code{gnu.org}, for the stable branch of Guix.
888
889 @item --bootstrap
890 Use the bootstrap Guile to build the latest Guix. This option is only
891 useful to Guix developers.
892 @end table
893
894 @c *********************************************************************
895 @node Programming Interface
896 @chapter Programming Interface
897
898 GNU Guix provides several Scheme programming interfaces (APIs) to
899 define, build, and query packages. The first interface allows users to
900 write high-level package definitions. These definitions refer to
901 familiar packaging concepts, such as the name and version of a package,
902 its build system, and its dependencies. These definitions can then be
903 turned into concrete build actions.
904
905 Build actions are performed by the Guix daemon, on behalf of users. In a
906 standard setup, the daemon has write access to the store---the
907 @file{/nix/store} directory---whereas users do not. The recommended
908 setup also has the daemon perform builds in chroots, under a specific
909 build users, to minimize interference with the rest of the system.
910
911 @cindex derivation
912 Lower-level APIs are available to interact with the daemon and the
913 store. To instruct the daemon to perform a build action, users actually
914 provide it with a @dfn{derivation}. A derivation is a low-level
915 representation of the build actions to be taken, and the environment in
916 which they should occur---derivations are to package definitions what
917 assembly is to C programs.
918
919 This chapter describes all these APIs in turn, starting from high-level
920 package definitions.
921
922 @menu
923 * Defining Packages:: Defining new packages.
924 * The Store:: Manipulating the package store.
925 * Derivations:: Low-level interface to package derivations.
926 * The Store Monad:: Purely functional interface to the store.
927 @end menu
928
929 @node Defining Packages
930 @section Defining Packages
931
932 The high-level interface to package definitions is implemented in the
933 @code{(guix packages)} and @code{(guix build-system)} modules. As an
934 example, the package definition, or @dfn{recipe}, for the GNU Hello
935 package looks like this:
936
937 @example
938 (use-modules (guix packages)
939 (guix download)
940 (guix build-system gnu)
941 (guix licenses))
942
943 (define hello
944 (package
945 (name "hello")
946 (version "2.8")
947 (source (origin
948 (method url-fetch)
949 (uri (string-append "mirror://gnu/hello/hello-" version
950 ".tar.gz"))
951 (sha256
952 (base32 "0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6"))))
953 (build-system gnu-build-system)
954 (inputs `(("gawk" ,gawk)))
955 (synopsis "GNU Hello")
956 (description "Yeah...")
957 (home-page "http://www.gnu.org/software/hello/")
958 (license gpl3+)))
959 @end example
960
961 @noindent
962 Without being a Scheme expert, the reader may have guessed the meaning
963 of the various fields here. This expression binds variable @var{hello}
964 to a @code{<package>} object, which is essentially a record
965 (@pxref{SRFI-9, Scheme records,, guile, GNU Guile Reference Manual}).
966 This package object can be inspected using procedures found in the
967 @code{(guix packages)} module; for instance, @code{(package-name hello)}
968 returns---surprise!---@code{"hello"}.
969
970 There are a few points worth noting in the above package definition:
971
972 @itemize
973 @item
974 The @code{source} field of the package is an @code{<origin>} object.
975 Here, the @code{url-fetch} method from @code{(guix download)} is used,
976 meaning that the source is a file to be downloaded over FTP or HTTP.
977
978 The @code{mirror://gnu} prefix instructs @code{url-fetch} to use one of
979 the GNU mirrors defined in @code{(guix download)}.
980
981 The @code{sha256} field specifies the expected SHA256 hash of the file
982 being downloaded. It is mandatory, and allows Guix to check the
983 integrity of the file. The @code{(base32 @dots{})} form introduces the
984 base32 representation of the hash. You can obtain this information with
985 @code{guix download} (@pxref{Invoking guix download}) and @code{guix
986 hash} (@pxref{Invoking guix hash}).
987
988 @cindex patches
989 When needed, the @code{origin} form can also have a @code{patches} field
990 listing patches to be applied, and a @code{snippet} field giving a
991 Scheme expression to modify the source code.
992
993 @item
994 @cindex GNU Build System
995 The @code{build-system} field is set to @var{gnu-build-system}. The
996 @var{gnu-build-system} variable is defined in the @code{(guix
997 build-system gnu)} module, and is bound to a @code{<build-system>}
998 object.
999
1000 Naturally, @var{gnu-build-system} represents the familiar GNU Build
1001 System, and variants thereof (@pxref{Configuration, configuration and
1002 makefile conventions,, standards, GNU Coding Standards}). In a
1003 nutshell, packages using the GNU Build System may be configured, built,
1004 and installed with the usual @code{./configure && make && make check &&
1005 make install} command sequence. This is what @var{gnu-build-system}
1006 does.
1007
1008 In addition, @var{gnu-build-system} ensures that the ``standard''
1009 environment for GNU packages is available. This includes tools such as
1010 GCC, Coreutils, Bash, Make, Diffutils, and Patch.
1011
1012 @item
1013 The @code{inputs} field specifies inputs to the build process---i.e.,
1014 build-time or run-time dependencies of the package. Here, we define an
1015 input called @code{"gawk"} whose value is that of the @var{gawk}
1016 variable; @var{gawk} is itself bound to a @code{<package>} object.
1017
1018 Note that GCC, Coreutils, Bash, and other essential tools do not need to
1019 be specified as inputs here. Instead, @var{gnu-build-system} takes care
1020 of ensuring that they are present.
1021
1022 However, any other dependencies need to be specified in the
1023 @code{inputs} field. Any dependency not specified here will simply be
1024 unavailable to the build process, possibly leading to a build failure.
1025 @end itemize
1026
1027 There are other fields that package definitions may provide. Of
1028 particular interest is the @code{arguments} field. When specified, it
1029 must be bound to a list of additional arguments to be passed to the
1030 build system. For instance, the above definition could be augmented
1031 with the following field initializer:
1032
1033 @example
1034 (arguments `(#:tests? #f
1035 #:configure-flags '("--enable-silent-rules")))
1036 @end example
1037
1038 @noindent
1039 These are keyword arguments (@pxref{Optional Arguments, keyword
1040 arguments in Guile,, guile, GNU Guile Reference Manual}). They are
1041 passed to @var{gnu-build-system}, which interprets them as meaning ``do
1042 not run @code{make check}'', and ``run @file{configure} with the
1043 @code{--enable-silent-rules} flag''. The value of these keyword
1044 parameters is actually evaluated in the @dfn{build stratum}---i.e., by a
1045 Guile process launched by the daemon (@pxref{Derivations}).
1046
1047 Once a package definition is in place@footnote{Simple package
1048 definitions like the one above may be automatically converted from the
1049 Nixpkgs distribution using the @command{guix import} command.}, the
1050 package may actually be built using the @code{guix build} command-line
1051 tool (@pxref{Invoking guix build}). Eventually, updating the package
1052 definition to a new upstream version can be partly automated by the
1053 @command{guix refresh} command (@pxref{Invoking guix refresh}).
1054
1055 Behind the scenes, a derivation corresponding to the @code{<package>}
1056 object is first computed by the @code{package-derivation} procedure.
1057 That derivation is stored in a @code{.drv} file under @file{/nix/store}.
1058 The build actions it prescribes may then be realized by using the
1059 @code{build-derivations} procedure (@pxref{The Store}).
1060
1061 @deffn {Scheme Procedure} package-derivation @var{store} @var{package} [@var{system}]
1062 Return the @code{<derivation>} object of @var{package} for @var{system}
1063 (@pxref{Derivations}).
1064
1065 @var{package} must be a valid @code{<package>} object, and @var{system}
1066 must be a string denoting the target system type---e.g.,
1067 @code{"x86_64-linux"} for an x86_64 Linux-based GNU system. @var{store}
1068 must be a connection to the daemon, which operates on the store
1069 (@pxref{The Store}).
1070 @end deffn
1071
1072 @noindent
1073 @cindex cross-compilation
1074 Similarly, it is possible to compute a derivation that cross-builds a
1075 package for some other system:
1076
1077 @deffn {Scheme Procedure} package-cross-derivation @var{store} @
1078 @var{package} @var{target} [@var{system}]
1079 Return the @code{<derivation>} object of @var{package} cross-built from
1080 @var{system} to @var{target}.
1081
1082 @var{target} must be a valid GNU triplet denoting the target hardware
1083 and operating system, such as @code{"mips64el-linux-gnu"}
1084 (@pxref{Configuration Names, GNU configuration triplets,, configure, GNU
1085 Configure and Build System}).
1086 @end deffn
1087
1088
1089 @node The Store
1090 @section The Store
1091
1092 @cindex store
1093 @cindex store paths
1094
1095 Conceptually, the @dfn{store} is where derivations that have been
1096 successfully built are stored---by default, under @file{/nix/store}.
1097 Sub-directories in the store are referred to as @dfn{store paths}. The
1098 store has an associated database that contains information such has the
1099 store paths referred to by each store path, and the list of @emph{valid}
1100 store paths---paths that result from a successful build.
1101
1102 The store is always accessed by the daemon on behalf of its clients
1103 (@pxref{Invoking guix-daemon}). To manipulate the store, clients
1104 connect to the daemon over a Unix-domain socket, send it requests, and
1105 read the result---these are remote procedure calls, or RPCs.
1106
1107 The @code{(guix store)} module provides procedures to connect to the
1108 daemon, and to perform RPCs. These are described below.
1109
1110 @deffn {Scheme Procedure} open-connection [@var{file}] [#:reserve-space? #t]
1111 Connect to the daemon over the Unix-domain socket at @var{file}. When
1112 @var{reserve-space?} is true, instruct it to reserve a little bit of
1113 extra space on the file system so that the garbage collector can still
1114 operate, should the disk become full. Return a server object.
1115
1116 @var{file} defaults to @var{%default-socket-path}, which is the normal
1117 location given the options that were passed to @command{configure}.
1118 @end deffn
1119
1120 @deffn {Scheme Procedure} close-connection @var{server}
1121 Close the connection to @var{server}.
1122 @end deffn
1123
1124 @defvr {Scheme Variable} current-build-output-port
1125 This variable is bound to a SRFI-39 parameter, which refers to the port
1126 where build and error logs sent by the daemon should be written.
1127 @end defvr
1128
1129 Procedures that make RPCs all take a server object as their first
1130 argument.
1131
1132 @deffn {Scheme Procedure} valid-path? @var{server} @var{path}
1133 Return @code{#t} when @var{path} is a valid store path.
1134 @end deffn
1135
1136 @deffn {Scheme Procedure} add-text-to-store @var{server} @var{name} @var{text} [@var{references}]
1137 Add @var{text} under file @var{name} in the store, and return its store
1138 path. @var{references} is the list of store paths referred to by the
1139 resulting store path.
1140 @end deffn
1141
1142 @deffn {Scheme Procedure} build-derivations @var{server} @var{derivations}
1143 Build @var{derivations} (a list of @code{<derivation>} objects or
1144 derivation paths), and return when the worker is done building them.
1145 Return @code{#t} on success.
1146 @end deffn
1147
1148 Note that the @code{(guix monads)} module provides a monad as well as
1149 monadic versions of the above procedures, with the goal of making it
1150 more convenient to work with code that accesses the store (@pxref{The
1151 Store Monad}).
1152
1153 @c FIXME
1154 @i{This section is currently incomplete.}
1155
1156 @node Derivations
1157 @section Derivations
1158
1159 @cindex derivations
1160 Low-level build actions and the environment in which they are performed
1161 are represented by @dfn{derivations}. A derivation contain the
1162 following pieces of information:
1163
1164 @itemize
1165 @item
1166 The outputs of the derivation---derivations produce at least one file or
1167 directory in the store, but may produce more.
1168
1169 @item
1170 The inputs of the derivations, which may be other derivations or plain
1171 files in the store (patches, build scripts, etc.)
1172
1173 @item
1174 The system type targeted by the derivation---e.g., @code{x86_64-linux}.
1175
1176 @item
1177 The file name of a build script in the store, along with the arguments
1178 to be passed.
1179
1180 @item
1181 A list of environment variables to be defined.
1182
1183 @end itemize
1184
1185 @cindex derivation path
1186 Derivations allow clients of the daemon to communicate build actions to
1187 the store. They exist in two forms: as an in-memory representation,
1188 both on the client- and daemon-side, and as files in the store whose
1189 name end in @code{.drv}---these files are referred to as @dfn{derivation
1190 paths}. Derivations paths can be passed to the @code{build-derivations}
1191 procedure to perform the build actions they prescribe (@pxref{The
1192 Store}).
1193
1194 The @code{(guix derivations)} module provides a representation of
1195 derivations as Scheme objects, along with procedures to create and
1196 otherwise manipulate derivations. The lowest-level primitive to create
1197 a derivation is the @code{derivation} procedure:
1198
1199 @deffn {Scheme Procedure} derivation @var{store} @var{name} @var{builder} @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] [#:hash-mode #f] [#:inputs '()] [#:env-vars '()] [#:system (%current-system)] [#:references-graphs #f]
1200 Build a derivation with the given arguments, and return the resulting
1201 @code{<derivation>} object.
1202
1203 When @var{hash}, @var{hash-algo}, and @var{hash-mode} are given, a
1204 @dfn{fixed-output derivation} is created---i.e., one whose result is
1205 known in advance, such as a file download.
1206
1207 When @var{references-graphs} is true, it must be a list of file
1208 name/store path pairs. In that case, the reference graph of each store
1209 path is exported in the build environment in the corresponding file, in
1210 a simple text format.
1211 @end deffn
1212
1213 @noindent
1214 Here's an example with a shell script as its builder, assuming
1215 @var{store} is an open connection to the daemon, and @var{bash} points
1216 to a Bash executable in the store:
1217
1218 @lisp
1219 (use-modules (guix utils)
1220 (guix store)
1221 (guix derivations))
1222
1223 (let ((builder ; add the Bash script to the store
1224 (add-text-to-store store "my-builder.sh"
1225 "echo hello world > $out\n" '())))
1226 (derivation store "foo"
1227 bash `("-e" ,builder)
1228 #:env-vars '(("HOME" . "/homeless"))))
1229 @result{} #<derivation /nix/store/@dots{}-foo.drv => /nix/store/@dots{}-foo>
1230 @end lisp
1231
1232 As can be guessed, this primitive is cumbersome to use directly. An
1233 improved variant is @code{build-expression->derivation}, which allows
1234 the caller to directly pass a Guile expression as the build script:
1235
1236 @deffn {Scheme Procedure} build-expression->derivation @var{store} @var{name} @var{system} @var{exp} @var{inputs} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] [#:env-vars '()] [#:modules '()] [#:references-graphs #f] [#:guile-for-build #f]
1237 Return a derivation that executes Scheme expression @var{exp} as a
1238 builder for derivation @var{name}. @var{inputs} must be a list of
1239 @code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
1240 @code{"out"} is assumed. @var{modules} is a list of names of Guile
1241 modules from the current search path to be copied in the store,
1242 compiled, and made available in the load path during the execution of
1243 @var{exp}---e.g., @code{((guix build utils) (guix build
1244 gnu-build-system))}.
1245
1246 @var{exp} is evaluated in an environment where @code{%outputs} is bound
1247 to a list of output/path pairs, and where @code{%build-inputs} is bound
1248 to a list of string/output-path pairs made from @var{inputs}.
1249 Optionally, @var{env-vars} is a list of string pairs specifying the name
1250 and value of environment variables visible to the builder. The builder
1251 terminates by passing the result of @var{exp} to @code{exit}; thus, when
1252 @var{exp} returns @code{#f}, the build is considered to have failed.
1253
1254 @var{exp} is built using @var{guile-for-build} (a derivation). When
1255 @var{guile-for-build} is omitted or is @code{#f}, the value of the
1256 @code{%guile-for-build} fluid is used instead.
1257
1258 See the @code{derivation} procedure for the meaning of @var{references-graphs}.
1259 @end deffn
1260
1261 @noindent
1262 Here's an example of a single-output derivation that creates a directory
1263 containing one file:
1264
1265 @lisp
1266 (let ((builder '(let ((out (assoc-ref %outputs "out")))
1267 (mkdir out) ; create /nix/store/@dots{}-goo
1268 (call-with-output-file (string-append out "/test")
1269 (lambda (p)
1270 (display '(hello guix) p))))))
1271 (build-expression->derivation store "goo" (%current-system)
1272 builder '()))
1273
1274 @result{} #<derivation /nix/store/@dots{}-goo.drv => @dots{}>
1275 @end lisp
1276
1277 @cindex strata of code
1278 Remember that the build expression passed to
1279 @code{build-expression->derivation} is run by a separate Guile process
1280 than the one that calls @code{build-expression->derivation}: it is run
1281 by a Guile process launched by the daemon, typically in a chroot. So,
1282 while there is a single language for both the @dfn{host} and the build
1283 side, there are really two @dfn{strata} of code: the host-side, and the
1284 build-side code@footnote{The term @dfn{stratum} in this context was
1285 coined by Manuel Serrano et al. in the context of their work on Hop.}.
1286 This distinction is important to keep in mind, notably when using
1287 higher-level constructs such as @var{gnu-build-system} (@pxref{Defining
1288 Packages}). For this reason, Guix modules that are meant to be used in
1289 the build stratum are kept in the @code{(guix build @dots{})} name
1290 space.
1291
1292 @node The Store Monad
1293 @section The Store Monad
1294
1295 @cindex monad
1296
1297 The procedures that operate on the store described in the previous
1298 sections all take an open connection to the build daemon as their first
1299 argument. Although the underlying model is functional, they either have
1300 side effects or depend on the current state of the store.
1301
1302 The former is inconvenient: the connection to the build daemon has to be
1303 carried around in all those functions, making it impossible to compose
1304 functions that do not take that parameter with functions that do. The
1305 latter can be problematic: since store operations have side effects
1306 and/or depend on external state, they have to be properly sequenced.
1307
1308 @cindex monadic values
1309 @cindex monadic functions
1310 This is where the @code{(guix monads)} module comes in. This module
1311 provides a framework for working with @dfn{monads}, and a particularly
1312 useful monad for our uses, the @dfn{store monad}. Monads are a
1313 construct that allows two things: associating ``context'' with values
1314 (in our case, the context is the store), and building sequences of
1315 computations (here computations includes accesses to the store.) Values
1316 in a monad---values that carry this additional context---are called
1317 @dfn{monadic values}; procedures that return such values are called
1318 @dfn{monadic procedures}.
1319
1320 Consider this ``normal'' procedure:
1321
1322 @example
1323 (define (profile.sh store)
1324 ;; Return the name of a shell script in the store that
1325 ;; initializes the 'PATH' environment variable.
1326 (let* ((drv (package-derivation store coreutils))
1327 (out (derivation->output-path drv)))
1328 (add-text-to-store store "profile.sh"
1329 (format #f "export PATH=~a/bin" out))))
1330 @end example
1331
1332 Using @code{(guix monads)}, it may be rewritten as a monadic function:
1333
1334 @example
1335 (define (profile.sh)
1336 ;; Same, but return a monadic value.
1337 (mlet %store-monad ((bin (package-file coreutils "bin")))
1338 (text-file "profile.sh"
1339 (string-append "export PATH=" bin))))
1340 @end example
1341
1342 There are two things to note in the second version: the @code{store}
1343 parameter is now implicit, and the monadic value returned by
1344 @code{package-file}---a wrapper around @code{package-derivation} and
1345 @code{derivation->output-path}---is @dfn{bound} using @code{mlet}
1346 instead of plain @code{let}.
1347
1348 Calling the monadic @code{profile.sh} has no effect. To get the desired
1349 effect, one must use @code{run-with-store}:
1350
1351 @example
1352 (run-with-store (open-connection) (profile.sh))
1353 @result{} /nix/store/...-profile.sh
1354 @end example
1355
1356 The main syntactic forms to deal with monads in general are described
1357 below.
1358
1359 @deffn {Scheme Syntax} with-monad @var{monad} @var{body} ...
1360 Evaluate any @code{>>=} or @code{return} forms in @var{body} as being
1361 in @var{monad}.
1362 @end deffn
1363
1364 @deffn {Scheme Syntax} return @var{val}
1365 Return a monadic value that encapsulates @var{val}.
1366 @end deffn
1367
1368 @deffn {Scheme Syntax} >>= @var{mval} @var{mproc}
1369 @dfn{Bind} monadic value @var{mval}, passing its ``contents'' to monadic
1370 procedure @var{mproc}@footnote{This operation is commonly referred to as
1371 ``bind'', but that name denotes an unrelated procedure in Guile. Thus
1372 we use this somewhat cryptic symbol inherited from the Haskell
1373 language.}.
1374 @end deffn
1375
1376 @deffn {Scheme Syntax} mlet @var{monad} ((@var{var} @var{mval}) ...) @
1377 @var{body} ...
1378 @deffnx {Scheme Syntax} mlet* @var{monad} ((@var{var} @var{mval}) ...) @
1379 @var{body} ...
1380 Bind the variables @var{var} to the monadic values @var{mval} in
1381 @var{body}. The form (@var{var} -> @var{val}) binds @var{var} to the
1382 ``normal'' value @var{val}, as per @code{let}.
1383
1384 @code{mlet*} is to @code{mlet} what @code{let*} is to @code{let}
1385 (@pxref{Local Bindings,,, guile, GNU Guile Reference Manual}).
1386 @end deffn
1387
1388 The interface to the store monad provided by @code{(guix monads)} is as
1389 follows.
1390
1391 @defvr {Scheme Variable} %store-monad
1392 The store monad. Values in the store monad encapsulate accesses to the
1393 store. When its effect is needed, a value of the store monad must be
1394 ``evaluated'' by passing it to the @code{run-with-store} procedure (see
1395 below.)
1396 @end defvr
1397
1398 @deffn {Scheme Procedure} run-with-store @var{store} @var{mval} [#:guile-for-build] [#:system (%current-system)]
1399 Run @var{mval}, a monadic value in the store monad, in @var{store}, an
1400 open store connection.
1401 @end deffn
1402
1403 @deffn {Monadic Procedure} text-file @var{name} @var{text}
1404 Return as a monadic value the absolute file name in the store of the file
1405 containing @var{text}.
1406 @end deffn
1407
1408 @deffn {Monadic Procedure} package-file @var{package} [@var{file}] @
1409 [#:system (%current-system)] [#:output "out"] Return as a monadic
1410 value in the absolute file name of @var{file} within the @var{output}
1411 directory of @var{package}. When @var{file} is omitted, return the name
1412 of the @var{output} directory of @var{package}.
1413 @end deffn
1414
1415 @deffn {Monadic Procedure} derivation-expression @var{name} @var{system} @
1416 @var{exp} @var{inputs} [#:outputs '("out")] [#:hash #f] @
1417 [#:hash-algo #f] [#:env-vars '()] [#:modules '()] @
1418 [#:references-graphs #f] [#:guile-for-build #f]
1419 Monadic version of @code{build-expression->derivation}
1420 (@pxref{Derivations}).
1421 @end deffn
1422
1423 @deffn {Monadic Procedure} package->derivation @var{package} [@var{system}]
1424 Monadic version of @code{package-derivation} (@pxref{Defining
1425 Packages}).
1426 @end deffn
1427
1428
1429 @c *********************************************************************
1430 @node Utilities
1431 @chapter Utilities
1432
1433 This section describes tools primarily targeted at developers and users
1434 who write new package definitions. They complement the Scheme
1435 programming interface of Guix in a convenient way.
1436
1437 @menu
1438 * Invoking guix build:: Building packages from the command line.
1439 * Invoking guix download:: Downloading a file and printing its hash.
1440 * Invoking guix hash:: Computing the cryptographic hash of a file.
1441 * Invoking guix refresh:: Updating package definitions.
1442 @end menu
1443
1444 @node Invoking guix build
1445 @section Invoking @command{guix build}
1446
1447 The @command{guix build} command builds packages or derivations and
1448 their dependencies, and prints the resulting store paths. Note that it
1449 does not modify the user's profile---this is the job of the
1450 @command{guix package} command (@pxref{Invoking guix package}). Thus,
1451 it is mainly useful for distribution developers.
1452
1453 The general syntax is:
1454
1455 @example
1456 guix build @var{options} @var{package-or-derivation}@dots{}
1457 @end example
1458
1459 @var{package-or-derivation} may be either the name of a package found in
1460 the software distribution such as @code{coreutils} or
1461 @code{coreutils-8.20}, or a derivation such as
1462 @file{/nix/store/@dots{}-coreutils-8.19.drv}. Alternatively, the
1463 @code{--expression} option may be used to specify a Scheme expression
1464 that evaluates to a package; this is useful when disambiguation among
1465 several same-named packages or package variants is needed.
1466
1467 The @var{options} may be zero or more of the following:
1468
1469 @table @code
1470
1471 @item --expression=@var{expr}
1472 @itemx -e @var{expr}
1473 Build the package @var{expr} evaluates to.
1474
1475 For example, @var{expr} may be @code{(@@ (gnu packages guile)
1476 guile-1.8)}, which unambiguously designates this specific variant of
1477 version 1.8 of Guile.
1478
1479 @item --source
1480 @itemx -S
1481 Build the packages' source derivations, rather than the packages
1482 themselves.
1483
1484 For instance, @code{guix build -S gcc} returns something like
1485 @file{/nix/store/@dots{}-gcc-4.7.2.tar.bz2}, which is GCC's source tarball.
1486
1487 The returned source tarball is the result of applying any patches and
1488 code snippets specified in the package's @code{origin} (@pxref{Defining
1489 Packages}).
1490
1491 @item --system=@var{system}
1492 @itemx -s @var{system}
1493 Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of
1494 the host's system type.
1495
1496 An example use of this is on Linux-based systems, which can emulate
1497 different personalities. For instance, passing
1498 @code{--system=i686-linux} on an @code{x86_64-linux} system allows users
1499 to build packages in a complete 32-bit environment.
1500
1501 @item --target=@var{triplet}
1502 @cindex cross-compilation
1503 Cross-build for @var{triplet}, which must be a valid GNU triplet, such
1504 as @code{"mips64el-linux-gnu"} (@pxref{Configuration Names, GNU
1505 configuration triplets,, configure, GNU Configure and Build System}).
1506
1507 @item --derivations
1508 @itemx -d
1509 Return the derivation paths, not the output paths, of the given
1510 packages.
1511
1512 @item --keep-failed
1513 @itemx -K
1514 Keep the build tree of failed builds. Thus, if a build fail, its build
1515 tree is kept under @file{/tmp}, in a directory whose name is shown at
1516 the end of the build log. This is useful when debugging build issues.
1517
1518 @item --dry-run
1519 @itemx -n
1520 Do not build the derivations.
1521
1522 @item --fallback
1523 When substituting a pre-built binary fails, fall back to building
1524 packages locally.
1525
1526 @item --no-substitutes
1527 Do not use substitutes for build products. That is, always build things
1528 locally instead of allowing downloads of pre-built binaries.
1529
1530 @item --max-silent-time=@var{seconds}
1531 When the build or substitution process remains silent for more than
1532 @var{seconds}, terminate it and report a build failure.
1533
1534 @item --cores=@var{n}
1535 @itemx -c @var{n}
1536 Allow the use of up to @var{n} CPU cores for the build. The special
1537 value @code{0} means to use as many CPU cores as available.
1538
1539 @item --root=@var{file}
1540 @itemx -r @var{file}
1541 Make @var{file} a symlink to the result, and register it as a garbage
1542 collector root.
1543
1544 @item --verbosity=@var{level}
1545 Use the given verbosity level. @var{level} must be an integer between 0
1546 and 5; higher means more verbose output. Setting a level of 4 or more
1547 may be helpful when debugging setup issues with the build daemon.
1548
1549 @item --log-file
1550 Return the build log file names for the given
1551 @var{package-or-derivation}s, or raise an error if build logs are
1552 missing.
1553
1554 This works regardless of how packages or derivations are specified. For
1555 instance, the following invocations are equivalent:
1556
1557 @example
1558 guix build --log-file `guix build -d guile`
1559 guix build --log-file `guix build guile`
1560 guix build --log-file guile
1561 guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)'
1562 @end example
1563
1564
1565 @end table
1566
1567 Behind the scenes, @command{guix build} is essentially an interface to
1568 the @code{package-derivation} procedure of the @code{(guix packages)}
1569 module, and to the @code{build-derivations} procedure of the @code{(guix
1570 store)} module.
1571
1572 @node Invoking guix download
1573 @section Invoking @command{guix download}
1574
1575 When writing a package definition, developers typically need to download
1576 the package's source tarball, compute its SHA256 hash, and write that
1577 hash in the package definition (@pxref{Defining Packages}). The
1578 @command{guix download} tool helps with this task: it downloads a file
1579 from the given URI, adds it to the store, and prints both its file name
1580 in the store and its SHA256 hash.
1581
1582 The fact that the downloaded file is added to the store saves bandwidth:
1583 when the developer eventually tries to build the newly defined package
1584 with @command{guix build}, the source tarball will not have to be
1585 downloaded again because it is already in the store. It is also a
1586 convenient way to temporarily stash files, which may be deleted
1587 eventually (@pxref{Invoking guix gc}).
1588
1589 The @command{guix download} command supports the same URIs as used in
1590 package definitions. In particular, it supports @code{mirror://} URIs.
1591 @code{https} URIs (HTTP over TLS) are supported @emph{provided} the
1592 Guile bindings for GnuTLS are available in the user's environment; when
1593 they are not available, an error is raised.
1594
1595 The following option is available:
1596
1597 @table @code
1598 @item --format=@var{fmt}
1599 @itemx -f @var{fmt}
1600 Write the hash in the format specified by @var{fmt}. For more
1601 information on the valid values for @var{fmt}, @ref{Invoking guix hash}.
1602 @end table
1603
1604 @node Invoking guix hash
1605 @section Invoking @command{guix hash}
1606
1607 The @command{guix hash} command computes the SHA256 hash of a file.
1608 It is primarily a convenience tool for anyone contributing to the
1609 distribution: it computes the cryptographic hash of a file, which can be
1610 used in the definition of a package (@pxref{Defining Packages}).
1611
1612 The general syntax is:
1613
1614 @example
1615 guix hash @var{option} @var{file}
1616 @end example
1617
1618 @command{guix hash} has the following option:
1619
1620 @table @code
1621
1622 @item --format=@var{fmt}
1623 @itemx -f @var{fmt}
1624 Write the hash in the format specified by @var{fmt}.
1625
1626 Supported formats: @code{nix-base32}, @code{base32}, @code{base16}
1627 (@code{hex} and @code{hexadecimal} can be used as well).
1628
1629 If the @option{--format} option is not specified, @command{guix hash}
1630 will output the hash in @code{nix-base32}. This representation is used
1631 in the definitions of packages.
1632
1633 @end table
1634
1635 @node Invoking guix refresh
1636 @section Invoking @command{guix refresh}
1637
1638 The primary audience of the @command{guix refresh} command is developers
1639 of the GNU software distribution. By default, it reports any packages
1640 provided by the distribution that are outdated compared to the latest
1641 upstream version, like this:
1642
1643 @example
1644 $ guix refresh
1645 gnu/packages/gettext.scm:29:13: gettext would be upgraded from 0.18.1.1 to 0.18.2.1
1646 gnu/packages/glib.scm:77:12: glib would be upgraded from 2.34.3 to 2.37.0
1647 @end example
1648
1649 It does so by browsing each package's FTP directory and determining the
1650 highest version number of the source tarballs
1651 therein@footnote{Currently, this only works for GNU packages.}.
1652
1653 When passed @code{--update}, it modifies distribution source files to
1654 update the version numbers and source tarball hashes of those packages'
1655 recipes (@pxref{Defining Packages}). This is achieved by downloading
1656 each package's latest source tarball and its associated OpenPGP
1657 signature, authenticating the downloaded tarball against its signature
1658 using @command{gpg}, and finally computing its hash. When the public
1659 key used to sign the tarball is missing from the user's keyring, an
1660 attempt is made to automatically retrieve it from a public key server;
1661 when it's successful, the key is added to the user's keyring; otherwise,
1662 @command{guix refresh} reports an error.
1663
1664 The following options are supported:
1665
1666 @table @code
1667
1668 @item --update
1669 @itemx -u
1670 Update distribution source files (package recipes) in place.
1671 @ref{Defining Packages}, for more information on package definitions.
1672
1673 @item --select=[@var{subset}]
1674 @itemx -s @var{subset}
1675 Select all the packages in @var{subset}, one of @code{core} or
1676 @code{non-core}.
1677
1678 The @code{core} subset refers to all the packages at the core of the
1679 distribution---i.e., packages that are used to build ``everything
1680 else''. This includes GCC, libc, Binutils, Bash, etc. Usually,
1681 changing one of these packages in the distribution entails a rebuild of
1682 all the others. Thus, such updates are an inconvenience to users in
1683 terms of build time or bandwidth used to achieve the upgrade.
1684
1685 The @code{non-core} subset refers to the remaining packages. It is
1686 typically useful in cases where an update of the core packages would be
1687 inconvenient.
1688
1689 @end table
1690
1691 In addition, @command{guix refresh} can be passed one or more package
1692 names, as in this example:
1693
1694 @example
1695 guix refresh -u emacs idutils
1696 @end example
1697
1698 @noindent
1699 The command above specifically updates the @code{emacs} and
1700 @code{idutils} packages. The @code{--select} option would have no
1701 effect in this case.
1702
1703 The following options can be used to customize GnuPG operation:
1704
1705 @table @code
1706
1707 @item --key-server=@var{host}
1708 Use @var{host} as the OpenPGP key server when importing a public key.
1709
1710 @item --gpg=@var{command}
1711 Use @var{command} as the GnuPG 2.x command. @var{command} is searched
1712 for in @code{$PATH}.
1713
1714 @end table
1715
1716
1717 @c *********************************************************************
1718 @node GNU Distribution
1719 @chapter GNU Distribution
1720
1721 Guix comes with a distribution of free software@footnote{The term
1722 ``free'' here refers to the
1723 @url{http://www.gnu.org/philosophy/free-sw.html,freedom provided to
1724 users of that software}.} that form the basis of the GNU system. This
1725 includes core GNU packages such as GNU libc, GCC, and Binutils, as well
1726 as many GNU and non-GNU applications. The complete list of available
1727 packages can be seen by running @command{guix package} (@pxref{Invoking
1728 guix package}):
1729
1730 @example
1731 guix package --list-available
1732 @end example
1733
1734 Our goal is to build a practical 100% free software distribution of
1735 Linux-based and other variants of GNU, with a focus on the promotion and
1736 tight integration of GNU components, and an emphasis on programs and
1737 tools that help users exert that freedom.
1738
1739 @menu
1740 * Installing Debugging Files:: Feeding the debugger.
1741 * Package Modules:: Packages from the programmer's viewpoint.
1742 * Packaging Guidelines:: Growing the distribution.
1743 * Bootstrapping:: GNU/Linux built from scratch.
1744 * Porting:: Targeting another platform or kernel.
1745 @end menu
1746
1747 Building this distribution is a cooperative effort, and you are invited
1748 to join! @ref{Contributing}, for information about how you can help.
1749
1750
1751 @node Installing Debugging Files
1752 @section Installing Debugging Files
1753
1754 Program binaries, as produced by the GCC compilers for instance, are
1755 typically written in the ELF format, with a section containing
1756 @dfn{debugging information}. Debugging information is what allows the
1757 debugger, GDB, to map binary code to source code; it is required to
1758 debug a compiled program in good conditions.
1759
1760 The problem with debugging information is that is takes up a fair amount
1761 of disk space. For example, debugging information for the GNU C Library
1762 weighs in at more than 60 MiB. Thus, as a user, keeping all the
1763 debugging info of all the installed programs is usually not an option.
1764 Yet, space savings should not come at the cost of an impediment to
1765 debugging---especially in the GNU system, which should make it easier
1766 for users to exert their computing freedom (@pxref{GNU Distribution}).
1767
1768 Thankfully, the GNU Binary Utilities (Binutils) and GDB provide a
1769 mechanism that allows users to get the best of both worlds: debugging
1770 information can be stripped from the binaries and stored in separate
1771 files. GDB is then able to load debugging information from those files,
1772 when they are available (@pxref{Separate Debug Files,,, gdb, Debugging
1773 with GDB}).
1774
1775 The GNU distribution takes advantage of this by storing debugging
1776 information in the @code{lib/debug} sub-directory of a separate package
1777 output unimaginatively called @code{debug} (@pxref{Packages with
1778 Multiple Outputs}). Users can choose to install the @code{debug} output
1779 of a package when they need it. For instance, the following command
1780 installs the debugging information for the GNU C Library and for GNU
1781 Guile:
1782
1783 @example
1784 guix package -i glibc:debug -i guile:debug
1785 @end example
1786
1787 GDB must then be told to look for debug files in the user's profile, by
1788 setting the @code{debug-file-directory} variable (consider setting it
1789 from the @file{~/.gdbinit} file, @pxref{Startup,,, gdb, Debugging with
1790 GDB}):
1791
1792 @example
1793 (gdb) set debug-file-directory ~/.guix-profile/lib/debug
1794 @end example
1795
1796 From there on, GDB will pick up debugging information from the
1797 @code{.debug} files under @file{~/.guix-profile/lib/debug}.
1798
1799 @c XXX: keep me up-to-date
1800 The @code{debug} output mechanism in Guix is implemented by the
1801 @code{gnu-build-system} (@pxref{Defining Packages}). Currently, it is
1802 opt-in---debugging information is available only for those packages
1803 whose definition explicitly declares a @code{debug} output. This may be
1804 changed to opt-out in the future, if our build farm servers can handle
1805 the load. To check whether a package has a @code{debug} output, use
1806 @command{guix package --list-available} (@pxref{Invoking guix package}).
1807
1808
1809 @node Package Modules
1810 @section Package Modules
1811
1812 From a programming viewpoint, the package definitions of the
1813 distribution are provided by Guile modules in the @code{(gnu packages
1814 ...)} name space (@pxref{Modules, Guile modules,, guile, GNU Guile
1815 Reference Manual}). For instance, the @code{(gnu packages emacs)}
1816 module exports a variable named @code{emacs}, which is bound to a
1817 @code{<package>} object (@pxref{Defining Packages}). The @code{(gnu
1818 packages)} module provides facilities for searching for packages.
1819
1820 The distribution is fully @dfn{bootstrapped} and @dfn{self-contained}:
1821 each package is built based solely on other packages in the
1822 distribution. The root of this dependency graph is a small set of
1823 @dfn{bootstrap binaries}, provided by the @code{(gnu packages
1824 bootstrap)} module. For more information on bootstrapping,
1825 @ref{Bootstrapping}.
1826
1827 @node Packaging Guidelines
1828 @section Packaging Guidelines
1829
1830 The GNU distribution is nascent and may well lack some of your favorite
1831 packages. This section describes how you can help make the distribution
1832 grow. @xref{Contributing}, for additional information on how you can
1833 help.
1834
1835 Free software packages are usually distributed in the form of
1836 @dfn{source code tarballs}---typically @file{tar.gz} files that contain
1837 all the source files. Adding a package to the distribution means
1838 essentially two things: adding a @dfn{recipe} that describes how to
1839 build the package, including a list of other packages required to build
1840 it, and adding @dfn{package meta-data} along with that recipe, such as a
1841 description and licensing information.
1842
1843 In Guix all this information is embodied in @dfn{package definitions}.
1844 Package definitions provide a high-level view of the package. They are
1845 written using the syntax of the Scheme programming language; in fact,
1846 for each package we define a variable bound to the package definition,
1847 and export that variable from a module (@pxref{Package Modules}).
1848 However, in-depth Scheme knowledge is @emph{not} a prerequisite for
1849 creating packages. For more information on package definitions,
1850 @ref{Defining Packages}.
1851
1852 Once a package definition is in place, stored in a file in the Guix
1853 source tree, it can be tested using the @command{guix build} command
1854 (@pxref{Invoking guix build}). For example, assuming the new package is
1855 called @code{gnew}, you may run this command from the Guix build tree:
1856
1857 @example
1858 ./pre-inst-env guix build gnew --keep-failed
1859 @end example
1860
1861 Using @code{--keep-failed} makes it easier to debug build failures since
1862 it provides access to the failed build tree.
1863
1864 Once your package builds correctly, please send us a patch
1865 (@pxref{Contributing}). Well, if you need help, we will be happy to
1866 help you too. Once the patch is committed in the Guix repository, the
1867 new package automatically gets built on the supported platforms by
1868 @url{http://hydra.gnu.org/gnu/master, our continuous integration
1869 system}.
1870
1871 @cindex substituter
1872 Users can obtain the new package definition simply by running
1873 @command{guix pull} (@pxref{Invoking guix pull}). When
1874 @code{hydra.gnu.org} is done building the package, installing the
1875 package automatically downloads binaries from there (except when using
1876 @code{--no-substitutes}). The only place where human intervention is
1877 needed is to review and apply the patch.
1878
1879
1880 @menu
1881 * Software Freedom:: What may go into the distribution.
1882 * Package Naming:: What's in a name?
1883 * Version Numbers:: When the name is not enough.
1884 * Python Modules:: Taming the snake.
1885 @end menu
1886
1887 @node Software Freedom
1888 @subsection Software Freedom
1889
1890 @c Adapted from http://www.gnu.org/philosophy/philosophy.html.
1891
1892 The GNU operating system has been developed so that users can have
1893 freedom in their computing. GNU is @dfn{free software}, meaning that
1894 users have the @url{http://www.gnu.org/philosophy/free-sw.html,four
1895 essential freedoms}: to run the program, to study and change the program
1896 in source code form, to redistribute exact copies, and to distribute
1897 modified versions. Packages found in the GNU distribution provide only
1898 software that conveys these four freedoms.
1899
1900 In addition, the GNU distribution follow the
1901 @url{http://www.gnu.org/distros/free-system-distribution-guidelines.html,free
1902 software distribution guidelines}. Among other things, these guidelines
1903 reject non-free firmware, recommendations of non-free software, and
1904 discuss ways to deal with trademarks and patents.
1905
1906 Some packages contain a small and optional subset that violates the
1907 above guidelines, for instance because this subset is itself non-free
1908 code. When that happens, the offending items are removed with
1909 appropriate patches or code snippets in the package definition's
1910 @code{origin} form (@pxref{Defining Packages}). That way, @code{guix
1911 build --source} returns the ``freed'' source rather than the unmodified
1912 upstream source.
1913
1914
1915 @node Package Naming
1916 @subsection Package Naming
1917
1918 A package has actually two names associated with it:
1919 First, there is the name of the @emph{Scheme variable}, the one following
1920 @code{define-public}. By this name, the package can be made known in the
1921 Scheme code, for instance as input to another package. Second, there is
1922 the string in the @code{name} field of a package definition. This name
1923 is used by package management commands such as
1924 @command{guix package} and @command{guix build}.
1925
1926 Both are usually the same and correspond to the lowercase conversion of the
1927 project name chosen upstream. For instance, the GNUnet project is packaged
1928 as @code{gnunet}. We do not add @code{lib} prefixes for library packages,
1929 unless these are already part of the official project name. But see
1930 @ref{Python Modules} for special rules concerning modules for
1931 the Python language.
1932
1933
1934 @node Version Numbers
1935 @subsection Version Numbers
1936
1937 We usually package only the latest version of a given free software
1938 project. But sometimes, for instance for incompatible library versions,
1939 two (or more) versions of the same package are needed. These require
1940 different Scheme variable names. We use the name as defined
1941 in @ref{Package Naming}
1942 for the most recent version; previous versions use the same name, suffixed
1943 by @code{-} and the smallest prefix of the version number that may
1944 distinguish the two versions.
1945
1946 The name inside the package definition is the same for all versions of a
1947 package and does not contain any version number.
1948
1949 For instance, the versions 2.24.20 and 3.9.12 of GTK+ may be packaged as follows:
1950
1951 @example
1952 (define-public gtk+
1953 (package
1954 (name "gtk+")
1955 (version "3.9.12")
1956 ...))
1957 (define-public gtk+-2
1958 (package
1959 (name "gtk+")
1960 (version "2.24.20")
1961 ...))
1962 @end example
1963 If we also wanted GTK+ 3.8.2, this would be packaged as
1964 @example
1965 (define-public gtk+-3.8
1966 (package
1967 (name "gtk+")
1968 (version "3.8.2")
1969 ...))
1970 @end example
1971
1972
1973 @node Python Modules
1974 @subsection Python Modules
1975
1976 We currently package Python 2 and Python 3, under the Scheme variable names
1977 @code{python-2} and @code{python} as explained in @ref{Version Numbers}.
1978 To avoid confusion and naming clashes with other programming languages, it
1979 seems desirable that the name of a package for a Python module contains
1980 the word @code{python}.
1981
1982 Some modules are compatible with only one version of Python, others with both.
1983 If the package Foo compiles only with Python 3, we name it
1984 @code{python-foo}; if it compiles only with Python 2, we name it
1985 @code{python2-foo}. If it is compatible with both versions, we create two
1986 packages with the corresponding names.
1987
1988 If a project already contains the word @code{python}, we drop this;
1989 for instance, the module python-dateutil is packaged under the names
1990 @code{python-dateutil} and @code{python2-dateutil}.
1991
1992
1993
1994
1995
1996 @node Bootstrapping
1997 @section Bootstrapping
1998
1999 @c Adapted from the ELS 2013 paper.
2000
2001 @cindex bootstrapping
2002
2003 Bootstrapping in our context refers to how the distribution gets built
2004 ``from nothing''. Remember that the build environment of a derivation
2005 contains nothing but its declared inputs (@pxref{Introduction}). So
2006 there's an obvious chicken-and-egg problem: how does the first package
2007 get built? How does the first compiler get compiled? Note that this is
2008 a question of interest only to the curious hacker, not to the regular
2009 user, so you can shamelessly skip this section if you consider yourself
2010 a ``regular user''.
2011
2012 @cindex bootstrap binaries
2013 The GNU system is primarily made of C code, with libc at its core. The
2014 GNU build system itself assumes the availability of a Bourne shell and
2015 command-line tools provided by GNU Coreutils, Awk, Findutils, `sed', and
2016 `grep'. Furthermore, build programs---programs that run
2017 @code{./configure}, @code{make}, etc.---are written in Guile Scheme
2018 (@pxref{Derivations}). Consequently, to be able to build anything at
2019 all, from scratch, Guix relies on pre-built binaries of Guile, GCC,
2020 Binutils, libc, and the other packages mentioned above---the
2021 @dfn{bootstrap binaries}.
2022
2023 These bootstrap binaries are ``taken for granted'', though we can also
2024 re-create them if needed (more on that later).
2025
2026 @unnumberedsubsec Preparing to Use the Bootstrap Binaries
2027
2028 @c As of Emacs 24.3, Info-mode displays the image, but since it's a
2029 @c large image, it's hard to scroll. Oh well.
2030 @image{images/bootstrap-graph,6in,,Dependency graph of the early bootstrap derivations}
2031
2032 The figure above shows the very beginning of the dependency graph of the
2033 distribution, corresponding to the package definitions of the @code{(gnu
2034 packages bootstrap)} module. At this level of detail, things are
2035 slightly complex. First, Guile itself consists of an ELF executable,
2036 along with many source and compiled Scheme files that are dynamically
2037 loaded when it runs. This gets stored in the @file{guile-2.0.7.tar.xz}
2038 tarball shown in this graph. This tarball is part of Guix's ``source''
2039 distribution, and gets inserted into the store with @code{add-to-store}
2040 (@pxref{The Store}).
2041
2042 But how do we write a derivation that unpacks this tarball and adds it
2043 to the store? To solve this problem, the @code{guile-bootstrap-2.0.drv}
2044 derivation---the first one that gets built---uses @code{bash} as its
2045 builder, which runs @code{build-bootstrap-guile.sh}, which in turn calls
2046 @code{tar} to unpack the tarball. Thus, @file{bash}, @file{tar},
2047 @file{xz}, and @file{mkdir} are statically-linked binaries, also part of
2048 the Guix source distribution, whose sole purpose is to allow the Guile
2049 tarball to be unpacked.
2050
2051 Once @code{guile-bootstrap-2.0.drv} is built, we have a functioning
2052 Guile that can be used to run subsequent build programs. Its first task
2053 is to download tarballs containing the other pre-built binaries---this
2054 is what the @code{.tar.xz.drv} derivations do. Guix modules such as
2055 @code{ftp-client.scm} are used for this purpose. The
2056 @code{module-import.drv} derivations import those modules in a directory
2057 in the store, using the original layout. The
2058 @code{module-import-compiled.drv} derivations compile those modules, and
2059 write them in an output directory with the right layout. This
2060 corresponds to the @code{#:modules} argument of
2061 @code{build-expression->derivation} (@pxref{Derivations}).
2062
2063 Finally, the various tarballs are unpacked by the
2064 derivations @code{gcc-bootstrap-0.drv}, @code{glibc-bootstrap-0.drv},
2065 etc., at which point we have a working C tool chain.
2066
2067
2068 @unnumberedsubsec Building the Build Tools
2069
2070 @c TODO: Add a package-level dependency graph generated from (gnu
2071 @c packages base).
2072
2073 Bootstrapping is complete when we have a full tool chain that does not
2074 depend on the pre-built bootstrap tools discussed above. This
2075 no-dependency requirement is verified by checking whether the files of
2076 the final tool chain contain references to the @file{/nix/store}
2077 directories of the bootstrap inputs. The process that leads to this
2078 ``final'' tool chain is described by the package definitions found in
2079 the @code{(gnu packages base)} module.
2080
2081 @c See <http://lists.gnu.org/archive/html/gnu-system-discuss/2012-10/msg00000.html>.
2082 The first tool that gets built with the bootstrap binaries is
2083 GNU Make, which is a prerequisite for all the following packages.
2084 From there Findutils and Diffutils get built.
2085
2086 Then come the first-stage Binutils and GCC, built as pseudo cross
2087 tools---i.e., with @code{--target} equal to @code{--host}. They are
2088 used to build libc. Thanks to this cross-build trick, this libc is
2089 guaranteed not to hold any reference to the initial tool chain.
2090
2091 From there the final Binutils and GCC are built. GCC uses @code{ld}
2092 from the final Binutils, and links programs against the just-built libc.
2093 This tool chain is used to build the other packages used by Guix and by
2094 the GNU Build System: Guile, Bash, Coreutils, etc.
2095
2096 And voilà! At this point we have the complete set of build tools that
2097 the GNU Build System expects. These are in the @code{%final-inputs}
2098 variables of the @code{(gnu packages base)} module, and are implicitly
2099 used by any package that uses @code{gnu-build-system} (@pxref{Defining
2100 Packages}).
2101
2102
2103 @unnumberedsubsec Building the Bootstrap Binaries
2104
2105 Because the final tool chain does not depend on the bootstrap binaries,
2106 those rarely need to be updated. Nevertheless, it is useful to have an
2107 automated way to produce them, should an update occur, and this is what
2108 the @code{(gnu packages make-bootstrap)} module provides.
2109
2110 The following command builds the tarballs containing the bootstrap
2111 binaries (Guile, Binutils, GCC, libc, and a tarball containing a mixture
2112 of Coreutils and other basic command-line tools):
2113
2114 @example
2115 guix build bootstrap-tarballs
2116 @end example
2117
2118 The generated tarballs are those that should be referred to in the
2119 @code{(gnu packages bootstrap)} module mentioned at the beginning of
2120 this section.
2121
2122 Still here? Then perhaps by now you've started to wonder: when do we
2123 reach a fixed point? That is an interesting question! The answer is
2124 unknown, but if you would like to investigate further (and have
2125 significant computational and storage resources to do so), then let us
2126 know.
2127
2128 @node Porting
2129 @section Porting to a New Platform
2130
2131 As discussed above, the GNU distribution is self-contained, and
2132 self-containment is achieved by relying on pre-built ``bootstrap
2133 binaries'' (@pxref{Bootstrapping}). These binaries are specific to an
2134 operating system kernel, CPU architecture, and application binary
2135 interface (ABI). Thus, to port the distribution to a platform that is
2136 not yet supported, one must build those bootstrap binaries, and update
2137 the @code{(gnu packages bootstrap)} module to use them on that platform.
2138
2139 Fortunately, Guix can @emph{cross compile} those bootstrap binaries.
2140 When everything goes well, and assuming the GNU tool chain supports the
2141 target platform, this can be as simple as running a command like this
2142 one:
2143
2144 @example
2145 guix build --target=armv5tel-linux-gnueabi bootstrap-tarballs
2146 @end example
2147
2148 Once these are built, the @code{(gnu packages bootstrap)} module needs
2149 to be updated to refer to these binaries on the target platform. In
2150 addition, the @code{glibc-dynamic-linker} procedure in that module must
2151 be augmented to return the right file name for libc's dynamic linker on
2152 that platform; likewise, @code{system->linux-architecture} in @code{(gnu
2153 packages linux)} must be taught about the new platform.
2154
2155 In practice, there may be some complications. First, it may be that the
2156 extended GNU triplet that specifies an ABI (like the @code{eabi} suffix
2157 above) is not recognized by all the GNU tools. Typically, glibc
2158 recognizes some of these, whereas GCC uses an extra @code{--with-abi}
2159 configure flag (see @code{gcc.scm} for examples of how to handle this).
2160 Second, some of the required packages could fail to build for that
2161 platform. Lastly, the generated binaries could be broken for some
2162 reason.
2163
2164
2165 @c *********************************************************************
2166 @node Contributing
2167 @chapter Contributing
2168
2169 This project is a cooperative effort, and we need your help to make it
2170 grow! Please get in touch with us on @email{guix-devel@@gnu.org}. We
2171 welcome ideas, bug reports, patches, and anything that may be helpful to
2172 the project. We particularly welcome help on packaging
2173 (@pxref{Packaging Guidelines}).
2174
2175 Please see the
2176 @url{http://git.savannah.gnu.org/cgit/guix.git/tree/HACKING,
2177 @file{HACKING} file} that comes with the Guix source code for practical
2178 details about contributions.
2179
2180
2181 @c *********************************************************************
2182 @node Acknowledgments
2183 @chapter Acknowledgments
2184
2185 Guix is based on the Nix package manager, which was designed and
2186 implemented by Eelco Dolstra. Nix pioneered functional package
2187 management, and promoted unprecedented features, such as transactional
2188 package upgrades and rollbacks, per-user profiles, and referentially
2189 transparent build processes. Without this work, Guix would not exist.
2190
2191 The Nix-based software distributions, Nixpkgs and NixOS, have also been
2192 an inspiration for Guix.
2193
2194 @c *********************************************************************
2195 @node GNU Free Documentation License
2196 @appendix GNU Free Documentation License
2197
2198 @include fdl-1.3.texi
2199
2200 @c *********************************************************************
2201 @node Concept Index
2202 @unnumbered Concept Index
2203 @printindex cp
2204
2205 @node Function Index
2206 @unnumbered Function Index
2207 @printindex fn
2208
2209 @bye
2210
2211 @c Local Variables:
2212 @c ispell-local-dictionary: "american";
2213 @c End: