@copying
Copyright @copyright{} 2012, 2013, 2014 Ludovic Courtès@*
-Copyright @copyright{} 2013 Andreas Enge@*
+Copyright @copyright{} 2013, 2014 Andreas Enge@*
Copyright @copyright{} 2013 Nikita Karetnikov
Permission is granted to copy, distribute and/or modify this document
@c Keep this list in sync with libstore/build.cc! -----------------------
@itemize
@item
-the @code{/dev} and @code{/proc} directories@footnote{On some systems
-@code{/dev/shm}, which supports shared memory, is a symlink to another
-directory such as @code{/run/shm}, that is @emph{not} is the chroot.
-When that is the case, shared memory support is unavailable in the
-chroot environment. The workaround is to make sure that @file{/dev/shm}
-is directly a @code{tmpfs} mount point.};
+a minimal @code{/dev} directory, created mostly independently from the
+host @code{/dev}@footnote{``Mostly'', because while the set of files
+that appear in the chroot's @code{/dev} is fixed, most of these files
+can only be created if the host has them.};
+
+@item
+the @code{/proc} directory; it only shows the container's processes
+since a separate PID name space is used;
@item
@file{/etc/passwd} with an entry for the current user and an entry for
@table @code
+@item port
+Port number of the machine's SSH server (default: 22).
+
@item private-key
The SSH private key file to use when connecting to the machine.
@item --no-substitutes
@cindex substitutes
Do not use substitutes for build products. That is, always build things
-locally instead of allowing downloads of pre-built binaries.
+locally instead of allowing downloads of pre-built binaries
+(@pxref{Substitutes}).
By default substitutes are used, unless the client---such as the
@command{guix package} command---is explicitly invoked with
@menu
* Features:: How Guix will make your life brighter.
* Invoking guix package:: Package installation, removal, etc.
+* Substitutes:: Downloading pre-built binaries.
* Packages with Multiple Outputs:: Single source package, multiple outputs.
* Invoking guix gc:: Running the garbage collector.
* Invoking guix pull:: Fetching the latest Guix and distribution.
is likely to yield bit-identical files when performed on different
machines (@pxref{Invoking guix-daemon, container}).
-@cindex substitute
+@cindex substitutes
This foundation allows Guix to support @dfn{transparent binary/source
-deployment}. When a pre-built binary for a @file{/gnu/store} path is
+deployment}. When a pre-built binary for a @file{/gnu/store} item is
available from an external source---a @dfn{substitute}, Guix just
-downloads it@footnote{@c XXX: Remove me when outdated.
-As of version @value{VERSION}, substitutes are downloaded from
-@url{http://hydra.gnu.org/} but are @emph{not} authenticated---i.e.,
-Guix cannot tell whether binaries it downloaded have been tampered with,
-nor whether they come from the genuine @code{gnu.org} build farm. This
-will be fixed in future versions. In the meantime, concerned users can
-opt for @code{--no-substitutes} (@pxref{Invoking guix-daemon}).};
-otherwise, it builds the package from source, locally.
+downloads it and unpacks it;
+otherwise, it builds the package from source, locally
+(@pxref{Substitutes}).
@node Invoking guix package
@section Invoking @command{guix package}
version: 7.2alpha6
@end example
+Similarly, to show the name of all the packages available under the
+terms of the GNU@tie{}LGPL version 3:
+
+@example
+$ guix package -s "" | recsel -p name -e 'license ~ "LGPL 3"'
+name: elfutils
+
+name: gmp
+@dots{}
+@end example
+
@item --list-installed[=@var{regexp}]
@itemx -I [@var{regexp}]
List the currently installed packages in the specified profile, with the
processes, it supports all the common build options that @command{guix
build} supports (@pxref{Invoking guix build, common build options}).
+@node Substitutes
+@section Substitutes
+
+@cindex substitutes
+@cindex pre-built binaries
+Guix supports transparent source/binary deployment, which means that it
+can either build things locally, or download pre-built items from a
+server. We call these pre-built items @dfn{substitutes}---they are
+substitutes for local build results. In many cases, downloading a
+substitute is much faster than building things locally.
+
+Substitutes can be anything resulting from a derivation build
+(@pxref{Derivations}). Of course, in the common case, they are
+pre-built package binaries, but source tarballs, for instance, which
+also result from derivation builds, can be available as substitutes.
+
+The @code{hydra.gnu.org} server is a front-end to a build farm that
+builds packages from the GNU distribution continuously for some
+architectures, and makes them available as substitutes.
+
+@cindex security
+@cindex digital signatures
+To allow Guix to download substitutes from @code{hydra.gnu.org}, you
+must add its public key to the access control list (ACL) of archive
+imports, using the @command{guix archive} command (@pxref{Invoking guix
+archive}). Doing so implies that you trust @code{hydra.gnu.org} to not
+be compromised and to serve genuine substitutes.
+
+This public key is installed along with Guix, in
+@code{@var{prefix}/share/guix/hydra.gnu.org.pub}, where @var{prefix} is
+the installation prefix of Guix. If you installed Guix from source,
+make sure you checked the GPG signature of
+@file{guix-@value{VERSION}.tar.gz}, which contains this public key file.
+Then, you can run something like this:
+
+@example
+# guix archive --authorize < hydra.gnu.org.pub
+@end example
+
+Once this is in place, the output of a command like @code{guix build}
+should change from something like:
+
+@example
+$ guix build emacs --dry-run
+The following derivations would be built:
+ /gnu/store/yr7bnx8xwcayd6j95r2clmkdl1qh688w-emacs-24.3.drv
+ /gnu/store/x8qsh1hlhgjx6cwsjyvybnfv2i37z23w-dbus-1.6.4.tar.gz.drv
+ /gnu/store/1ixwp12fl950d15h2cj11c73733jay0z-alsa-lib-1.0.27.1.tar.bz2.drv
+ /gnu/store/nlma1pw0p603fpfiqy7kn4zm105r5dmw-util-linux-2.21.drv
+@dots{}
+@end example
+
+@noindent
+to something like:
+
+@example
+$ guix build emacs --dry-run
+The following files would be downloaded:
+ /gnu/store/pk3n22lbq6ydamyymqkkz7i69wiwjiwi-emacs-24.3
+ /gnu/store/2ygn4ncnhrpr61rssa6z0d9x22si0va3-libjpeg-8d
+ /gnu/store/71yz6lgx4dazma9dwn2mcjxaah9w77jq-cairo-1.12.16
+ /gnu/store/7zdhgp0n1518lvfn8mb96sxqfmvqrl7v-libxrender-0.9.7
+@dots{}
+@end example
+
+@noindent
+This indicates that substitutes from @code{hydra.gnu.org} are usable and
+will be downloaded, when possible, for future builds.
+
+Guix ignores substitutes that are not signed, or that are not signed by
+one of the keys listed in the ACL. It also detects and raises an error
+when attempting to use a substitute that has been tampered with.
+
+The substitute mechanism can be disabled globally by running
+@code{guix-daemon} with @code{--no-substitutes} (@pxref{Invoking
+guix-daemon}). It can also be disabled temporarily by passing the
+@code{--no-substitutes} option to @command{guix package}, @command{guix
+build}, and other command-line tools.
+
+
+Today, each individual's control over their own computing is at the
+mercy of institutions, corporations, and groups with enough power and
+determination to subvert the computing infrastructure and exploit its
+weaknesses. While using @code{hydra.gnu.org} substitutes can be
+convenient, we encourage users to also build on their own, or even run
+their own build farm, such that @code{hydra.gnu.org} is less of an
+interesting target.
+
+Guix has the foundations to maximize build reproducibility
+(@pxref{Features}). In most cases, independent builds of a given
+package or derivation should yield bit-identical results. Thus, through
+a diverse set of independent package builds, we can strengthen the
+integrity of our systems.
+
+In the future, we want Guix to have support to publish and retrieve
+binaries to/from other users, in a peer-to-peer fashion. If you would
+like to discuss this project, join us on @email{guix-devel@@gnu.org}.
+
+
@node Packages with Multiple Outputs
@section Packages with Multiple Outputs
provide it with a @dfn{derivation}. A derivation is a low-level
representation of the build actions to be taken, and the environment in
which they should occur---derivations are to package definitions what
-assembly is to C programs.
+assembly is to C programs. The term ``derivation'' comes from the fact
+that build results @emph{derive} from them.
This chapter describes all these APIs in turn, starting from high-level
package definitions.
@menu
* Defining Packages:: Defining new packages.
+* Build Systems:: Specifying how packages are built.
* The Store:: Manipulating the package store.
* Derivations:: Low-level interface to package derivations.
* The Store Monad:: Purely functional interface to the store.
+* G-Expressions:: Manipulating build expressions.
@end menu
@node Defining Packages
(sha256
(base32 "0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6"))))
(build-system gnu-build-system)
+ (arguments `(#:configure-flags '("--enable-silent-rules")))
(inputs `(("gawk" ,gawk)))
- (synopsis "GNU Hello")
- (description "Yeah...")
+ (synopsis "Hello, GNU world: An example GNU package")
+ (description "Guess what GNU Hello prints!")
(home-page "http://www.gnu.org/software/hello/")
(license gpl3+)))
@end example
@item
@cindex GNU Build System
-The @code{build-system} field is set to @var{gnu-build-system}. The
-@var{gnu-build-system} variable is defined in the @code{(guix
-build-system gnu)} module, and is bound to a @code{<build-system>}
-object.
-
-Naturally, @var{gnu-build-system} represents the familiar GNU Build
-System, and variants thereof (@pxref{Configuration, configuration and
-makefile conventions,, standards, GNU Coding Standards}). In a
-nutshell, packages using the GNU Build System may be configured, built,
-and installed with the usual @code{./configure && make && make check &&
-make install} command sequence. This is what @var{gnu-build-system}
-does.
-
-In addition, @var{gnu-build-system} ensures that the ``standard''
-environment for GNU packages is available. This includes tools such as
-GCC, Coreutils, Bash, Make, Diffutils, and Patch.
+The @code{build-system} field specifies the procedure to build the
+package (@pxref{Build Systems}). Here, @var{gnu-build-system}
+represents the familiar GNU Build System, where packages may be
+configured, built, and installed with the usual @code{./configure &&
+make && make check && make install} command sequence.
+
+@item
+The @code{arguments} field specifies options for the build system
+(@pxref{Build Systems}). Here it is interpreted by
+@var{gnu-build-system} as a request run @file{configure} with the
+@code{--enable-silent-rules} flag.
@item
The @code{inputs} field specifies inputs to the build process---i.e.,
Note that GCC, Coreutils, Bash, and other essential tools do not need to
be specified as inputs here. Instead, @var{gnu-build-system} takes care
-of ensuring that they are present.
+of ensuring that they are present (@pxref{Build Systems}).
However, any other dependencies need to be specified in the
@code{inputs} field. Any dependency not specified here will simply be
unavailable to the build process, possibly leading to a build failure.
@end itemize
-There are other fields that package definitions may provide. Of
-particular interest is the @code{arguments} field. When specified, it
-must be bound to a list of additional arguments to be passed to the
-build system. For instance, the above definition could be augmented
-with the following field initializer:
-
-@example
- (arguments `(#:tests? #f
- #:configure-flags '("--enable-silent-rules")))
-@end example
-
-@noindent
-These are keyword arguments (@pxref{Optional Arguments, keyword
-arguments in Guile,, guile, GNU Guile Reference Manual}). They are
-passed to @var{gnu-build-system}, which interprets them as meaning ``do
-not run @code{make check}'', and ``run @file{configure} with the
-@code{--enable-silent-rules} flag''. The value of these keyword
-parameters is actually evaluated in the @dfn{build stratum}---i.e., by a
-Guile process launched by the daemon (@pxref{Derivations}).
-
Once a package definition is in place@footnote{Simple package
definitions like the one above may be automatically converted from the
Nixpkgs distribution using the @command{guix import} command.}, the
package may actually be built using the @code{guix build} command-line
-tool (@pxref{Invoking guix build}). Eventually, updating the package
-definition to a new upstream version can be partly automated by the
-@command{guix refresh} command (@pxref{Invoking guix refresh}).
+tool (@pxref{Invoking guix build}). @xref{Packaging Guidelines}, for
+more information on how to test package definitions.
+
+Eventually, updating the package definition to a new upstream version
+can be partly automated by the @command{guix refresh} command
+(@pxref{Invoking guix refresh}).
Behind the scenes, a derivation corresponding to the @code{<package>}
object is first computed by the @code{package-derivation} procedure.
@end deffn
+@node Build Systems
+@section Build Systems
+
+@cindex build system
+Each package definition specifies a @dfn{build system} and arguments for
+that build system (@pxref{Defining Packages}). This @code{build-system}
+field represents the build procedure of the package, as well implicit
+dependencies of that build procedure.
+
+Build systems are @code{<build-system>} objects. The interface to
+create and manipulate them is provided by the @code{(guix build-system)}
+module, and actual build systems are exported by specific modules.
+
+Build systems accept an optional list of @dfn{arguments}. In package
+definitions, these are passed @i{via} the @code{arguments} field
+(@pxref{Defining Packages}). They are typically keyword arguments
+(@pxref{Optional Arguments, keyword arguments in Guile,, guile, GNU
+Guile Reference Manual}). The value of these arguments is usually
+evaluated in the @dfn{build stratum}---i.e., by a Guile process launched
+by the daemon (@pxref{Derivations}).
+
+The main build system is @var{gnu-build-system}, which implements the
+standard build procedure for GNU packages and many other packages. It
+is provided by the @code{(guix build-system gnu)} module.
+
+@defvr {Scheme Variable} gnu-build-system
+@var{gnu-build-system} represents the GNU Build System, and variants
+thereof (@pxref{Configuration, configuration and makefile conventions,,
+standards, GNU Coding Standards}).
+
+@cindex build phases
+In a nutshell, packages using it configured, built, and installed with
+the usual @code{./configure && make && make check && make install}
+command sequence. In practice, a few additional steps are often needed.
+All these steps are split up in separate @dfn{phases},
+notably@footnote{Please see the @code{(guix build gnu-build-system)}
+modules for more details about the build phases.}:
+
+@table @code
+@item unpack
+Unpack the source tarball, and change the current directory to the
+extracted source tree. If the source is actually a directory, copy it
+to the build tree, and enter that directory.
+
+@item patch-source-shebangs
+Patch shebangs encountered in source files so they refer to the right
+store file names. For instance, this changes @code{#!/bin/sh} to
+@code{#!/gnu/store/@dots{}-bash-4.3/bin/sh}.
+
+@item configure
+Run the @file{configure} script with a number of default options, such
+as @code{--prefix=/gnu/store/@dots{}}, as well as the options specified
+by the @code{#:configure-flags} argument.
+
+@item build
+Run @code{make} with the list of flags specified with
+@code{#:make-flags}. If the @code{#:parallel-builds?} argument is true
+(the default), build with @code{make -j}.
+
+@item check
+Run @code{make check}, or some other target specified with
+@code{#:test-target}, unless @code{#:tests? #f} is passed. If the
+@code{#:parallel-tests?} argument is true (the default), run @code{make
+check -j}.
+
+@item install
+Run @code{make install} with the flags listed in @code{#:make-flags}.
+
+@item patch-shebangs
+Patch shebangs on the installed executable files.
+
+@item strip
+Strip debugging symbols from ELF files (unless @code{#:strip-binaries?}
+is false), copying them to the @code{debug} output when available
+(@pxref{Installing Debugging Files}).
+@end table
+
+@vindex %standard-phases
+The build-side module @code{(guix build gnu-build-system)} defines
+@var{%standard-phases} as the default list of build phases.
+@var{%standard-phases} is a list of symbol/procedure pairs, where the
+procedure implements the actual phase.
+
+The list of phases used for a particular package can be changed with the
+@code{#:phases} parameter. For instance, passing:
+
+@example
+#:phases (alist-delete 'configure %standard-phases)
+@end example
+
+means that all the phases described above will be used, except the
+@code{configure} phase.
+
+In addition, this build system ensures that the ``standard'' environment
+for GNU packages is available. This includes tools such as GCC, libc,
+Coreutils, Bash, Make, Diffutils, grep, and sed (see the @code{(guix
+build-system gnu)} module for a complete list.) We call these the
+@dfn{implicit inputs} of a package, because package definitions don't
+have to mention them.
+@end defvr
+
+Other @code{<build-system>} objects are defined to support other
+conventions and tools used by free software packages. They inherit most
+of @var{gnu-build-system}, and differ mainly in the set of inputs
+implicitly added to the build process, and in the list of phases
+executed. Some of these build systems are listed below.
+
+@defvr {Scheme Variable} cmake-build-system
+This variable is exported by @code{(guix build-system cmake)}. It
+implements the build procedure for packages using the
+@url{http://www.cmake.org, CMake build tool}.
+
+It automatically adds the @code{cmake} package to the set of inputs.
+Which package is used can be specified with the @code{#:cmake}
+parameter.
+@end defvr
+
+@defvr {Scheme Variable} python-build-system
+This variable is exported by @code{(guix build-system python)}. It
+implements the more or less standard build procedure used by Python
+packages, which consists in running @code{python setup.py build} and
+then @code{python setup.py install --prefix=/gnu/store/@dots{}}.
+
+For packages that install stand-alone Python programs under @code{bin/},
+it takes care of wrapping these programs so their @code{PYTHONPATH}
+environment variable points to all the Python libraries they depend on.
+
+Which Python package is used can be specified with the @code{#:python}
+parameter.
+@end defvr
+
+@defvr {Scheme Variable} perl-build-system
+This variable is exported by @code{(guix build-system perl)}. It
+implements the standard build procedure for Perl packages, which
+consists in running @code{perl Makefile.PL PREFIX=/gnu/store/@dots{}},
+followed by @code{make} and @code{make install}.
+
+The initial @code{perl Makefile.PL} invocation passes flags specified by
+the @code{#:make-maker-flags} parameter.
+
+Which Perl package is used can be specified with @code{#:perl}.
+@end defvr
+
+
+Lastly, for packages that do not need anything as sophisticated, a
+``trivial'' build system is provided. It is trivial in the sense that
+it provides basically no support: it does not pull any implicit inputs,
+and does not have a notion of build phases.
+
+@defvr {Scheme Variable} trivial-build-system
+This variable is exported by @code{(guix build-system trivial)}.
+
+This build system requires a @code{#:builder} argument. This argument
+must be a Scheme expression that builds the package's output(s)---as
+with @code{build-expression->derivation} (@pxref{Derivations,
+@code{build-expression->derivation}}).
+@end defvr
+
@node The Store
@section The Store
@var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:inputs '()] [#:env-vars '()] @
[#:system (%current-system)] [#:references-graphs #f] @
- [#:local-build? #f]
+ [#:allowed-references #f] [#:local-build? #f]
Build a derivation with the given arguments, and return the resulting
@code{<derivation>} object.
path is exported in the build environment in the corresponding file, in
a simple text format.
+When @var{allowed-references} is true, it must be a list of store items
+or outputs that the derivation's output may refer to.
+
When @var{local-build?} is true, declare that the derivation is not a
good candidate for offloading and should rather be built locally
(@pxref{Daemon Offload Setup}). This is the case for small derivations
"echo hello world > $out\n" '())))
(derivation store "foo"
bash `("-e" ,builder)
+ #:inputs `((,bash) (,builder))
#:env-vars '(("HOME" . "/homeless"))))
@result{} #<derivation /gnu/store/@dots{}-foo.drv => /gnu/store/@dots{}-foo>
@end lisp
-As can be guessed, this primitive is cumbersome to use directly. An
-improved variant is @code{build-expression->derivation}, which allows
-the caller to directly pass a Guile expression as the build script:
+As can be guessed, this primitive is cumbersome to use directly. A
+better approach is to write build scripts in Scheme, of course! The
+best course of action for that is to write the build code as a
+``G-expression'', and to pass it to @code{gexp->derivation}. For more
+information, @ref{G-Expressions}.
+
+Once upon a time, @code{gexp->derivation} did not exist and constructing
+derivations with build code written in Scheme was achieved with
+@code{build-expression->derivation}, documented below. This procedure
+is now deprecated in favor of the much nicer @code{gexp->derivation}.
@deffn {Scheme Procedure} build-expression->derivation @var{store} @
@var{name} @var{exp} @
[#:system (%current-system)] [#:inputs '()] @
[#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:env-vars '()] [#:modules '()] @
- [#:references-graphs #f] [#:local-build? #f] [#:guile-for-build #f]
+ [#:references-graphs #f] [#:allowed-references #f] @
+ [#:local-build? #f] [#:guile-for-build #f]
Return a derivation that executes Scheme expression @var{exp} as a
builder for derivation @var{name}. @var{inputs} must be a list of
@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
@var{guile-for-build} is omitted or is @code{#f}, the value of the
@code{%guile-for-build} fluid is used instead.
-See the @code{derivation} procedure for the meaning of @var{references-graphs}
-and @var{local-build?}.
+See the @code{derivation} procedure for the meaning of
+@var{references-graphs}, @var{allowed-references}, and @var{local-build?}.
@end deffn
@noindent
@result{} #<derivation /gnu/store/@dots{}-goo.drv => @dots{}>
@end lisp
-@cindex strata of code
-Remember that the build expression passed to
-@code{build-expression->derivation} is run by a separate Guile process
-than the one that calls @code{build-expression->derivation}: it is run
-by a Guile process launched by the daemon, typically in a chroot. So,
-while there is a single language for both the @dfn{host} and the build
-side, there are really two @dfn{strata} of code: the host-side, and the
-build-side code@footnote{The term @dfn{stratum} in this context was
-coined by Manuel Serrano et al. in the context of their work on Hop.}.
-This distinction is important to keep in mind, notably when using
-higher-level constructs such as @var{gnu-build-system} (@pxref{Defining
-Packages}). For this reason, Guix modules that are meant to be used in
-the build stratum are kept in the @code{(guix build @dots{})} name
-space.
@node The Store Monad
@section The Store Monad
Using @code{(guix monads)}, it may be rewritten as a monadic function:
+@c FIXME: Find a better example, one that uses 'mlet'.
@example
(define (sh-symlink)
;; Same, but return a monadic value.
- (mlet %store-monad ((sh (package-file bash "bin")))
- (derivation-expression "sh" `(symlink ,sh %output))))
+ (gexp->derivation "sh"
+ #~(symlink (string-append #$bash "/bin/bash") #$output)))
@end example
There are two things to note in the second version: the @code{store}
of the @var{output} directory of @var{package}.
@end deffn
-@deffn {Monadic Procedure} derivation-expression @var{name} @var{exp} @
- [#:system (%current-system)] [#:inputs '()] @
- [#:outputs '("out")] [#:hash #f] @
- [#:hash-algo #f] [#:env-vars '()] [#:modules '()] @
- [#:references-graphs #f] [#:guile-for-build #f]
-Monadic version of @code{build-expression->derivation}
-(@pxref{Derivations}).
-@end deffn
-
@deffn {Monadic Procedure} package->derivation @var{package} [@var{system}]
Monadic version of @code{package-derivation} (@pxref{Defining
Packages}).
@end deffn
+@node G-Expressions
+@section G-Expressions
+
+@cindex G-expression
+@cindex build code quoting
+So we have ``derivations'', which represent a sequence of build actions
+to be performed to produce an item in the store (@pxref{Derivations}).
+Those build actions are performed when asking the daemon to actually
+build the derivations; they are run by the daemon in a container
+(@pxref{Invoking guix-daemon}).
+
+@cindex strata of code
+It should come as no surprise that we like to write those build actions
+in Scheme. When we do that, we end up with two @dfn{strata} of Scheme
+code@footnote{The term @dfn{stratum} in this context was coined by
+Manuel Serrano et al.@: in the context of their work on Hop. Oleg
+Kiselyov, who has written insightful
+@url{http://okmij.org/ftp/meta-programming/#meta-scheme, essays and code
+on this topic}, refers to this kind of code generation as
+@dfn{staging}.}: the ``host code''---code that defines packages, talks
+to the daemon, etc.---and the ``build code''---code that actually
+performs build actions, such as making directories, invoking
+@command{make}, etc.
+
+To describe a derivation and its build actions, one typically needs to
+embed build code inside host code. It boils down to manipulating build
+code as data, and Scheme's homoiconicity---code has a direct
+representation as data---comes in handy for that. But we need more than
+Scheme's normal @code{quasiquote} mechanism to construct build
+expressions.
+
+The @code{(guix gexp)} module implements @dfn{G-expressions}, a form of
+S-expressions adapted to build expressions. G-expressions, or
+@dfn{gexps}, consist essentially in three syntactic forms: @code{gexp},
+@code{ungexp}, and @code{ungexp-splicing} (or simply: @code{#~},
+@code{#$}, and @code{#$@@}), which are comparable respectively to
+@code{quasiquote}, @code{unquote}, and @code{unquote-splicing}
+(@pxref{Expression Syntax, @code{quasiquote},, guile, GNU Guile
+Reference Manual}). However, there are major differences:
+
+@itemize
+@item
+Gexps are meant to be written to a file and run or manipulated by other
+processes.
+
+@item
+When a package or derivation is unquoted inside a gexp, the result is as
+if its output file name had been introduced.
+
+@item
+Gexps carry information about the packages or derivations they refer to,
+and these dependencies are automatically added as inputs to the build
+processes that use them.
+@end itemize
+
+To illustrate the idea, here is an example of a gexp:
+
+@example
+(define build-exp
+ #~(begin
+ (mkdir #$output)
+ (chdir #$output)
+ (symlink (string-append #$coreutils "/bin/ls")
+ "list-files")))
+@end example
+
+This gexp can be passed to @code{gexp->derivation}; we obtain a
+derivation that builds a directory containing exactly one symlink to
+@file{/gnu/store/@dots{}-coreutils-8.22/bin/ls}:
+
+@example
+(gexp->derivation "the-thing" build-exp)
+@end example
+
+As one would expect, the @code{"/gnu/store/@dots{}-coreutils-8.22"} string is
+substituted to the reference to the @var{coreutils} package in the
+actual build code, and @var{coreutils} is automatically made an input to
+the derivation. Likewise, @code{#$output} (equivalent to @code{(ungexp
+output)}) is replaced by a string containing the derivation's output
+directory name. The syntactic form to construct gexps is summarized
+below.
+
+@deffn {Scheme Syntax} #~@var{exp}
+@deffnx {Scheme Syntax} (gexp @var{exp})
+Return a G-expression containing @var{exp}. @var{exp} may contain one
+or more of the following forms:
+
+@table @code
+@item #$@var{obj}
+@itemx (ungexp @var{obj})
+Introduce a reference to @var{obj}. @var{obj} may be a package or a
+derivation, in which case the @code{ungexp} form is replaced by its
+output file name---e.g., @code{"/gnu/store/@dots{}-coreutils-8.22}.
+
+If @var{obj} is a list, it is traversed and any package or derivation
+references are substituted similarly.
+
+If @var{obj} is another gexp, its contents are inserted and its
+dependencies are added to those of the containing gexp.
+
+If @var{obj} is another kind of object, it is inserted as is.
+
+@item #$@var{package-or-derivation}:@var{output}
+@itemx (ungexp @var{package-or-derivation} @var{output})
+This is like the form above, but referring explicitly to the
+@var{output} of @var{package-or-derivation}---this is useful when
+@var{package-or-derivation} produces multiple outputs (@pxref{Packages
+with Multiple Outputs}).
+
+@item #$output[:@var{output}]
+@itemx (ungexp output [@var{output}])
+Insert a reference to derivation output @var{output}, or to the main
+output when @var{output} is omitted.
+
+This only makes sense for gexps passed to @code{gexp->derivation}.
+
+@item #$@@@var{lst}
+@itemx (ungexp-splicing @var{lst})
+Like the above, but splices the contents of @var{lst} inside the
+containing list.
+
+@end table
+
+G-expressions created by @code{gexp} or @code{#~} are run-time objects
+of the @code{gexp?} type (see below.)
+@end deffn
+
+@deffn {Scheme Procedure} gexp? @var{obj}
+Return @code{#t} if @var{obj} is a G-expression.
+@end deffn
+
+G-expressions are meant to be written to disk, either as code building
+some derivation, or as plain files in the store. The monadic procedures
+below allow you to do that (@pxref{The Store Monad}, for more
+information about monads.)
+
+@deffn {Monadic Procedure} gexp->derivation @var{name} @var{exp} @
+ [#:system (%current-system)] [#:inputs '()] @
+ [#:hash #f] [#:hash-algo #f] @
+ [#:recursive? #f] [#:env-vars '()] [#:modules '()] @
+ [#:references-graphs #f] [#:local-build? #f] @
+ [#:guile-for-build #f]
+Return a derivation @var{name} that runs @var{exp} (a gexp) with
+@var{guile-for-build} (a derivation) on @var{system}.
+
+Make @var{modules} available in the evaluation context of @var{EXP};
+@var{MODULES} is a list of names of Guile modules from the current
+search path to be copied in the store, compiled, and made available in
+the load path during the execution of @var{exp}---e.g., @code{((guix
+build utils) (guix build gnu-build-system))}.
+
+The other arguments are as for @code{derivation} (@pxref{Derivations}).
+@end deffn
+
+@deffn {Monadic Procedure} gexp->script @var{name} @var{exp}
+Return an executable script @var{name} that runs @var{exp} using
+@var{guile} with @var{modules} in its search path.
+
+The example below builds a script that simply invokes the @command{ls}
+command:
+
+@example
+(use-modules (guix gexp) (gnu packages base))
+
+(gexp->script "list-files"
+ #~(execl (string-append #$coreutils "/bin/ls")
+ "ls"))
+@end example
+
+When ``running'' it through the store (@pxref{The Store Monad,
+@code{run-with-store}}), we obtain a derivation that produces an
+executable file @file{/gnu/store/@dots{}-list-files} along these lines:
+
+@example
+#!/gnu/store/@dots{}-guile-2.0.11/bin/guile -ds
+!#
+(execl (string-append "/gnu/store/@dots{}-coreutils-8.22"/bin/ls")
+ "ls")
+@end example
+@end deffn
+
+@deffn {Monadic Procedure} gexp->file @var{name} @var{exp}
+Return a derivation that builds a file @var{name} containing @var{exp}.
+
+The resulting file holds references to all the dependencies of @var{exp}
+or a subset thereof.
+@end deffn
+
+Of course, in addition to gexps embedded in ``host'' code, there are
+also modules containing build tools. To make it clear that they are
+meant to be used in the build stratum, these modules are kept in the
+@code{(guix build @dots{})} name space.
+
+
@c *********************************************************************
@node Utilities
@chapter Utilities
guile-1.8)}, which unambiguously designates this specific variant of
version 1.8 of Guile.
-Alternately, @var{expr} may refer to a zero-argument monadic procedure
+Alternately, @var{expr} may be a G-expression, in which case it is used
+as a build program passed to @code{gexp->derivation}
+(@pxref{G-Expressions}).
+
+Lastly, @var{expr} may refer to a zero-argument monadic procedure
(@pxref{The Store Monad}). The procedure must return a derivation as a
monadic value, which is then passed through @code{run-with-store}.
as @code{"mips64el-linux-gnu"} (@pxref{Configuration Names, GNU
configuration triplets,, configure, GNU Configure and Build System}).
+@item --with-source=@var{source}
+Use @var{source} as the source of the corresponding package.
+@var{source} must be a file name or a URL, as for @command{guix
+download} (@pxref{Invoking guix download}).
+
+The ``corresponding package'' is taken to be one specified on the
+command line whose name matches the base of @var{source}---e.g., if
+@var{source} is @code{/src/guile-2.0.10.tar.gz}, the corresponding
+package is @code{guile}. Likewise, the version string is inferred from
+@var{source}; in the previous example, it's @code{2.0.10}.
+
+This option allows users to try out versions of packages other than the
+one provided by the distribution. The example below downloads
+@file{ed-1.7.tar.gz} from a GNU mirror and uses that as the source for
+the @code{ed} package:
+
+@example
+guix build ed --with-source=mirror://gnu/ed/ed-1.7.tar.gz
+@end example
+
+As a developer, @code{--with-source} makes it easy to test release
+candidates:
+
+@example
+guix build guile --with-source=../guile-2.0.9.219-e1bb7.tar.xz
+@end example
+
+
@item --derivations
@itemx -d
Return the derivation paths, not the output paths, of the given
@item --no-substitutes
Do not use substitutes for build products. That is, always build things
-locally instead of allowing downloads of pre-built binaries.
+locally instead of allowing downloads of pre-built binaries
+(@pxref{Substitutes}).
@item --no-build-hook
Do not attempt to offload builds @i{via} the daemon's ``build hook''
@node Installing Debugging Files
@section Installing Debugging Files
+@cindex debugging files
Program binaries, as produced by the GCC compilers for instance, are
typically written in the ELF format, with a section containing
@dfn{debugging information}. Debugging information is what allows the
Guile:
@example
-guix package -i glibc:debug -i guile:debug
+guix package -i glibc:debug guile:debug
@end example
GDB must then be told to look for debug files in the user's profile, by
From there on, GDB will pick up debugging information from the
@code{.debug} files under @file{~/.guix-profile/lib/debug}.
+In addition, you will most likely want GDB to be able to show the source
+code being debugged. To do that, you will have to unpack the source
+code of the package of interest (obtained with @code{guix build
+--source}, @pxref{Invoking guix build}), and to point GDB to that source
+directory using the @code{directory} command (@pxref{Source Path,
+@code{directory},, gdb, Debugging with GDB}).
+
@c XXX: keep me up-to-date
The @code{debug} output mechanism in Guix is implemented by the
-@code{gnu-build-system} (@pxref{Defining Packages}). Currently, it is
+@code{gnu-build-system} (@pxref{Build Systems}). Currently, it is
opt-in---debugging information is available only for those packages
whose definition explicitly declares a @code{debug} output. This may be
changed to opt-out in the future, if our build farm servers can handle
@end example
Using @code{--keep-failed} makes it easier to debug build failures since
-it provides access to the failed build tree.
+it provides access to the failed build tree. Another useful
+command-line option when debugging is @code{--log-file}, to access the
+build log.
If the package is unknown to the @command{guix} command, it may be that
the source file contains a syntax error, or lacks a @code{define-public}
Users can obtain the new package definition simply by running
@command{guix pull} (@pxref{Invoking guix pull}). When
@code{hydra.gnu.org} is done building the package, installing the
-package automatically downloads binaries from there (except when using
-@code{--no-substitutes}). The only place where human intervention is
+package automatically downloads binaries from there
+(@pxref{Substitutes}). The only place where human intervention is
needed is to review and apply the patch.
* Package Naming:: What's in a name?
* Version Numbers:: When the name is not enough.
* Python Modules:: Taming the snake.
+* Perl Modules:: Little pearls.
@end menu
@node Software Freedom
is used by package management commands such as
@command{guix package} and @command{guix build}.
-Both are usually the same and correspond to the lowercase conversion of the
-project name chosen upstream. For instance, the GNUnet project is packaged
-as @code{gnunet}. We do not add @code{lib} prefixes for library packages,
-unless these are already part of the official project name. But see
-@ref{Python Modules} for special rules concerning modules for
-the Python language.
+Both are usually the same and correspond to the lowercase conversion of
+the project name chosen upstream, with underscores replaced with
+hyphens. For instance, GNUnet is available as @code{gnunet}, and
+SDL_net as @code{sdl-net}.
+
+We do not add @code{lib} prefixes for library packages, unless these are
+already part of the official project name. But see @pxref{Python
+Modules} and @ref{Perl Modules} for special rules concerning modules for
+the Python and Perl languages.
@node Version Numbers
@code{python-dateutil} and @code{python2-dateutil}.
+@node Perl Modules
+@subsection Perl Modules
+
+Perl programs standing for themselves are named as any other package,
+using the lowercase upstream name.
+For Perl packages containing a single class, we use the lowercase class name,
+replace all occurrences of @code{::} by dashes and prepend the prefix
+@code{perl-}.
+So the class @code{XML::Parser} becomes @code{perl-xml-parser}.
+Modules containing several classes keep their lowercase upstream name and
+are also prepended by @code{perl-}. Such modules tend to have the word
+@code{perl} somewhere in their name, which gets dropped in favor of the
+prefix. For instance, @code{libwww-perl} becomes @code{perl-libwww}.
@menu
* Using the Configuration System:: Customizing your GNU system.
+* File Systems:: Configuring file system mounts.
+* User Accounts:: Specifying user accounts.
+* Services:: Specifying system services.
* Invoking guix system:: Instantiating a system configuration.
* Defining Services:: Adding new service definitions.
@end menu
@node Using the Configuration System
@subsection Using the Configuration System
-The operating system is configured by filling in an
-@code{operating-system} structure, as defined by the @code{(gnu system)}
-module. A simple setup, with the default system services, the default
-Linux-Libre kernel, initial RAM disk, and boot loader looks like this:
+The operating system is configured by providing an
+@code{operating-system} declaration in a file that can then be passed to
+the @command{guix system} command (@pxref{Invoking guix system}). A
+simple setup, with the default system services, the default Linux-Libre
+kernel, initial RAM disk, and boot loader looks like this:
@findex operating-system
@lisp
-(use-modules (gnu services base) ; for '%base-services'
- (gnu services ssh) ; for 'lsh-service'
- (gnu system shadow) ; for 'user-account'
- (gnu packages base) ; Coreutils, grep, etc.
- (gnu packages bash) ; Bash
- (gnu packages admin) ; dmd, Inetutils
- (gnu packages zile) ; Zile
- (gnu packages less) ; less
- (gnu packages guile) ; Guile
- (gnu packages linux)) ; procps, psmisc
-
-(define komputilo
- (operating-system
- (host-name "komputilo")
- (timezone "Europe/Paris")
- (locale "fr_FR.UTF-8")
- (users (list (user-account
- (name "alice")
- (password "")
- (uid 1000) (gid 100)
- (comment "Bob's sister")
- (home-directory "/home/alice"))))
- (packages (list coreutils bash guile-2.0
- guix dmd
- inetutils
- findutils grep sed
- procps psmisc
- zile less))
- (services (cons (lsh-service #:port 2222 #:allow-root-login? #t)
- %base-services))))
+(use-modules (gnu) ; for 'user-account', '%base-services', etc.
+ (gnu packages emacs) ; for 'emacs'
+ (gnu services ssh)) ; for 'lsh-service'
+
+(operating-system
+ (host-name "komputilo")
+ (timezone "Europe/Paris")
+ (locale "fr_FR.UTF-8")
+ (bootloader (grub-configuration
+ (device "/dev/sda")))
+ (file-systems (list (file-system
+ (device "/dev/sda1") ; or partition label
+ (mount-point "/")
+ (type "ext3"))))
+ (users (list (user-account
+ (name "alice")
+ (password "")
+ (uid 1000) (gid 100)
+ (comment "Bob's sister")
+ (home-directory "/home/alice"))))
+ (packages (cons emacs %base-packages))
+ (services (cons (lsh-service #:port 2222 #:allow-root-login? #t)
+ %base-services)))
@end lisp
-This example should be self-describing. The @code{packages} field lists
-packages provided by the various @code{(gnu packages ...)} modules above
-(@pxref{Package Modules}). These are the packages that will be globally
-visible on the system, for all user accounts---i.e., in every user's
-@code{PATH} environment variable---in addition to the per-user profiles
-(@pxref{Invoking guix package}).
+This example should be self-describing. Some of the fields defined
+above, such as @code{host-name} and @code{bootloader}, are mandatory.
+Others, such as @code{packages} and @code{services}, can be omitted, in
+which case they get a default value.
+
+@vindex %base-packages
+The @code{packages} field lists
+packages that will be globally visible on the system, for all user
+accounts---i.e., in every user's @code{PATH} environment variable---in
+addition to the per-user profiles (@pxref{Invoking guix package}). The
+@var{%base-packages} variable provides all the tools one would expect
+for basic user and administrator tasks---including the GNU Core
+Utilities, the GNU Networking Utilities, the GNU Zile lightweight text
+editor, @command{find}, @command{grep}, etc. The example above adds
+Emacs to those, taken from the @code{(gnu packages emacs)} module
+(@pxref{Package Modules}).
@vindex %base-services
The @code{services} field lists @dfn{system services} to be made
generated as needed (@pxref{Defining Services}).
Assuming the above snippet is stored in the @file{my-system-config.scm}
-file, the @command{guix system boot my-system-config.scm} command
+file, the @command{guix system reconfigure my-system-config.scm} command
instantiates that configuration, and makes it the default GRUB boot
entry (@pxref{Invoking guix system}). The normal way to change the
system's configuration is by updating this file and re-running the
instantiate @var{os}.
@end deffn
+@node File Systems
+@subsection File Systems
+
+The list of file systems to be mounted is specified in the
+@code{file-systems} field of the operating system's declaration
+(@pxref{Using the Configuration System}). Each file system is declared
+using the @code{file-system} form, like this:
+
+@example
+(file-system
+ (mount-point "/home")
+ (device "/dev/sda3")
+ (type "ext4"))
+@end example
+
+As usual, some of the fields are mandatory---those shown in the example
+above---while others can be omitted. These are described below.
+
+@deftp {Data Type} file-system
+Objects of these types represent file systems to be mounted. They
+contain the following members:
+
+@table @asis
+@item @code{type}
+This is a string specifying the type of the file system---e.g.,
+@code{"ext4"}.
+
+@item @code{mount-point}
+This designates the place where the file system is to be mounted.
+
+@item @code{device}
+This names the ``source'' of the file system. By default it is the name
+of a node under @file{/dev}, but its meaning depends on the @code{title}
+field described below.
+
+@item @code{title} (default: @code{'device})
+This is a symbol that specifies how the @code{device} field is to be
+interpreted.
+
+When it is the symbol @code{device}, then the @code{device} field is
+interpreted as a file name; when it is @code{label}, then @code{device}
+is interpreted as a partition label name; when it is @code{uuid},
+@code{device} is interpreted as a partition unique identifier (UUID).
+
+The @code{label} and @code{uuid} options offer a way to refer to disk
+partitions without having to hard-code their actual device name.
+
+@item @code{flags} (default: @code{'()})
+This is a list of symbols denoting mount flags. Recognized flags
+include @code{read-only} and @code{bind-mount}.
+
+@item @code{options} (default: @code{#f})
+This is either @code{#f}, or a string denoting mount options.
+
+@item @code{needed-for-boot?} (default: @code{#f})
+This Boolean value indicates whether the file system is needed when
+booting. If that is true, then the file system is mounted when the
+initial RAM disk (initrd) is loaded. This is always the case, for
+instance, for the root file system.
+
+@item @code{check?} (default: @code{#t})
+This Boolean indicates whether the file system needs to be checked for
+errors before being mounted.
+
+@end table
+@end deftp
+
+@node User Accounts
+@subsection User Accounts
+
+TODO
+
+@node Services
+@subsection Services
+
+TODO
+
@node Invoking guix system
@subsection Invoking @code{guix system}
@var{file} must be the name of a file containing an
@code{operating-system} declaration. @var{action} specifies how the
-operating system is instantiate. Currently only one value is supported:
+operating system is instantiate. Currently the following values are
+supported:
@table @code
+@item reconfigure
+Build the operating system described in @var{file}, activate it, and
+switch to it@footnote{This action is usable only on systems already
+running GNU.}.
+
+This effects all the configuration specified in @var{file}: user
+accounts, system services, global package list, setuid programs, etc.
+
+It also adds a GRUB menu entry for the new OS configuration, and moves
+entries for older configurations to a submenu---unless
+@option{--no-grub} is passed.
+
+@item build
+Build the operating system's derivation, which includes all the
+configuration files and programs needed to boot and run the system.
+This action does not actually install anything.
+
+@item init
+Populate the given directory with all the files necessary to run the
+operating system specified in @var{file}. This is useful for first-time
+installations of the GNU system. For instance:
+
+@example
+guix system init my-os-config.scm /mnt
+@end example
+
+copies to @file{/mnt} all the store items required by the configuration
+specified in @file{my-os-config.scm}. This includes configuration
+files, packages, and so on. It also creates other essential files
+needed for the system to operate correctly---e.g., the @file{/etc},
+@file{/var}, and @file{/run} directories, and the @file{/bin/sh} file.
+
+This command also installs GRUB on the device specified in
+@file{my-os-config}, unless the @option{--no-grub} option was passed.
+
@item vm
@cindex virtual machine
Build a virtual machine that contain the operating system declared in
@var{file}, and return a script to run that virtual machine (VM).
The VM shares its store with the host system.
+
+@item vm-image
+@itemx disk-image
+Return a virtual machine or disk image of the operating system declared
+in @var{file} that stands alone. Use the @option{--image-size} option
+to specify the size of the image.
+
+When using @code{vm-image}, the returned image is in qcow2 format, which
+the QEMU emulator can efficiently use.
+
+When using @code{disk-image}, a raw disk image is produced; it can be
+copied as is to a USB stick, for instance. Assuming @code{/dev/sdc} is
+the device corresponding to a USB stick, one can copy the image on it
+using the following command:
+
+@example
+# dd if=$(guix system disk-image my-os.scm) of=/dev/sdc
+@end example
+
@end table
@var{options} can contain any of the common build options provided by
@command{guix build} (@pxref{Invoking guix build}).
+Note that all the actions above, except @code{build} and @code{init},
+rely on KVM support in the Linux-Libre kernel. Specifically, the
+machine should have hardware virtualization support, the corresponding
+KVM kernel module should be loaded, and the @file{/dev/kvm} device node
+must exist and be readable and writable by the user and by the daemon's
+build users.
@node Defining Services
@subsection Defining Services
@lisp
(define (nscd-service)
- (mlet %store-monad ((nscd (package-file glibc "sbin/nscd")))
+ (with-monad %store-monad
(return (service
(documentation "Run libc's name service cache daemon.")
(provision '(nscd))
- (start `(make-forkexec-constructor ,nscd "-f" "/dev/null"
- "--foreground"))
- (stop `(make-kill-destructor))
-
- (respawn? #f)
- (inputs `(("glibc" ,glibc)))))))
+ (activate #~(begin
+ (use-modules (guix build utils))
+ (mkdir-p "/var/run/nscd")))
+ (start #~(make-forkexec-constructor
+ (string-append #$glibc "/sbin/nscd")
+ "-f" "/dev/null" "--foreground"))
+ (stop #~(make-kill-destructor))
+ (respawn? #f)))))
@end lisp
@noindent
-The @code{inputs} field specifies that this service depends on the
-@var{glibc} package---the package that contains the @command{nscd}
-program. The @code{start} and @code{stop} fields are expressions that
-make use of dmd's facilities to start and stop processes (@pxref{Service
-De- and Constructors,,, dmd, GNU dmd Manual}). The @code{provision}
-field specifies the name under which this service is known to dmd, and
-@code{documentation} specifies on-line documentation. Thus, the
-commands @command{deco start ncsd}, @command{deco stop nscd}, and
-@command{deco doc nscd} will do what you would expect (@pxref{Invoking
-deco,,, dmd, GNU dmd Manual}).
+The @code{activate}, @code{start}, and @code{stop} fields are G-expressions
+(@pxref{G-Expressions}). The @code{activate} field contains a script to
+run at ``activation'' time; it makes sure that the @file{/var/run/nscd}
+directory exists before @command{nscd} is started.
+
+The @code{start} and @code{stop} fields refer to dmd's facilities to
+start and stop processes (@pxref{Service De- and Constructors,,, dmd,
+GNU dmd Manual}). The @code{provision} field specifies the name under
+which this service is known to dmd, and @code{documentation} specifies
+on-line documentation. Thus, the commands @command{deco start ncsd},
+@command{deco stop nscd}, and @command{deco doc nscd} will do what you
+would expect (@pxref{Invoking deco,,, dmd, GNU dmd Manual}).
@c *********************************************************************
HCoop / jackhill / guix / guix.git / blobdiff