gnu: vowpal-wabbit: Install more headers.
[jackhill/guix/guix.git] / doc / guix-cookbook.texi
CommitLineData
7bc46ecc
RW
1\input texinfo
2@c -*-texinfo-*-
3
4@c %**start of header
5@setfilename guix-cookbook.info
6@documentencoding UTF-8
7@settitle GNU Guix Cookbook
8@c %**end of header
9
10@copying
11Copyright @copyright{} 2019 Ricardo Wurmus@*
12Copyright @copyright{} 2019 Efraim Flashner@*
13Copyright @copyright{} 2019 Pierre Neidhardt@*
65051fdc 14Copyright @copyright{} 2020 Oleg Pykhalov@*
60651dd9 15Copyright @copyright{} 2020 Matthew Brooks@*
145df67b 16Copyright @copyright{} 2020 Marcin Karpezo@*
c987b723 17Copyright @copyright{} 2020 Brice Waegeneire@*
e3d2e618 18Copyright @copyright{} 2020 André Batista@*
7bc46ecc
RW
19
20Permission is granted to copy, distribute and/or modify this document
21under the terms of the GNU Free Documentation License, Version 1.3 or
22any later version published by the Free Software Foundation; with no
23Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
24copy of the license is included in the section entitled ``GNU Free
25Documentation License''.
26@end copying
27
28@dircategory System administration
29@direntry
30* Guix cookbook: (guix-cookbook). Tutorials and examples for GNU Guix.
31@end direntry
32
33@titlepage
34@title GNU Guix Cookbook
35@subtitle Tutorials and examples for using the GNU Guix Functional Package Manager
36@author The GNU Guix Developers
37
38@page
39@vskip 0pt plus 1filll
40
41@insertcopying
42@end titlepage
43
44@contents
45
46@c *********************************************************************
47@node Top
48@top GNU Guix Cookbook
49
50This document presents tutorials and detailed examples for GNU@tie{}Guix, a
51functional package management tool written for the GNU system. Please
52@pxref{Top,,, guix, GNU Guix reference manual} for details about the system,
53its API, and related concepts.
54
55@c TRANSLATORS: You can replace the following paragraph with information on
56@c how to join your own translation team and how to report issues with the
57@c translation.
58If you would like to translate this document in your native language, consider
59joining the @uref{https://translationproject.org/domain/guix-cookbook.html,
60Translation Project}.
61
62@menu
63* Scheme tutorials:: Meet your new favorite language!
64* Packaging:: Packaging tutorials
65* System Configuration:: Customizing the GNU System
4c463569 66* Advanced package management:: Power to the users!
d0fc1e83 67* Environment management:: Control environment
7bc46ecc
RW
68
69* Acknowledgments:: Thanks!
70* GNU Free Documentation License:: The license of this document.
71* Concept Index:: Concepts.
72
73@detailmenu
74 --- The Detailed Node Listing ---
75
76Scheme tutorials
77
78* A Scheme Crash Course:: Learn the basics of Scheme
79
80Packaging
81
82* Packaging Tutorial:: Let's add a package to Guix!
83
84System Configuration
85
86* Customizing the Kernel:: Creating and using a custom Linux kernel
87
88
89@end detailmenu
90@end menu
91
92@c *********************************************************************
93@node Scheme tutorials
94@chapter Scheme tutorials
95
96GNU@tie{}Guix is written in the general purpose programming language Scheme,
97and many of its features can be accessed and manipulated programmatically.
98You can use Scheme to generate package definitions, to modify them, to build
99them, to deploy whole operating systems, etc.
100
101Knowing the basics of how to program in Scheme will unlock many of the
102advanced features Guix provides --- and you don't even need to be an
103experienced programmer to use them!
104
105Let's get started!
106
107@node A Scheme Crash Course
108@section A Scheme Crash Course
109
110@cindex Scheme, crash course
111
112Guix uses the Guile implementation of Scheme. To start playing with the
113language, install it with @code{guix install guile} and start a
96a95aa9
LC
114@dfn{REPL}---short for @uref{https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop,
115@dfn{read-eval-print loop}}---by running @code{guile} from the command line.
7bc46ecc
RW
116
117Alternatively you can also run @code{guix environment --ad-hoc guile -- guile}
118if you'd rather not have Guile installed in your user profile.
119
cf1e6f5f
LC
120In the following examples, lines show what you would type at the REPL;
121lines starting with ``@result{}'' show evaluation results, while lines
122starting with ``@print{}'' show things that get printed. @xref{Using Guile
96a95aa9 123Interactively,,, guile, GNU Guile Reference Manual}, for more details on the
7bc46ecc
RW
124REPL.
125
126@itemize
127@item
128Scheme syntax boils down to a tree of expressions (or @emph{s-expression} in
129Lisp lingo). An expression can be a literal such as numbers and strings, or a
130compound which is a parenthesized list of compounds and literals. @code{#t}
0cbef07b 131and @code{#f} stand for the Booleans ``true'' and ``false'', respectively.
7bc46ecc
RW
132
133Examples of valid expressions:
134
b1eecb5c 135@lisp
7bc46ecc 136"Hello World!"
cf1e6f5f
LC
137@result{} "Hello World!"
138
7bc46ecc 13917
cf1e6f5f
LC
140@result{} 17
141
142(display (string-append "Hello " "Guix" "\n"))
143@print{} Hello Guix!
144@result{} #<unspecified>
b1eecb5c 145@end lisp
7bc46ecc
RW
146
147@item
148This last example is a function call nested in another function call. When a
149parenthesized expression is evaluated, the first term is the function and the
150rest are the arguments passed to the function. Every function returns the
151last evaluated expression as its return value.
152
153@item
154Anonymous functions are declared with the @code{lambda} term:
155
b1eecb5c 156@lisp
cf1e6f5f
LC
157(lambda (x) (* x x))
158@result{} #<procedure 120e348 at <unknown port>:24:0 (x)>
b1eecb5c 159@end lisp
7bc46ecc
RW
160
161The above procedure returns the square of its argument. Since everything is
162an expression, the @code{lambda} expression returns an anonymous procedure,
163which can in turn be applied to an argument:
164
b1eecb5c 165@lisp
cf1e6f5f
LC
166((lambda (x) (* x x)) 3)
167@result{} 9
b1eecb5c 168@end lisp
7bc46ecc
RW
169
170@item
171Anything can be assigned a global name with @code{define}:
172
b1eecb5c 173@lisp
cf1e6f5f
LC
174(define a 3)
175(define square (lambda (x) (* x x)))
176(square a)
177@result{} 9
b1eecb5c 178@end lisp
7bc46ecc
RW
179
180@item
181Procedures can be defined more concisely with the following syntax:
182
b1eecb5c 183@lisp
7bc46ecc 184(define (square x) (* x x))
b1eecb5c 185@end lisp
7bc46ecc
RW
186
187@item
188A list structure can be created with the @code{list} procedure:
189
b1eecb5c 190@lisp
cf1e6f5f
LC
191(list 2 a 5 7)
192@result{} (2 3 5 7)
b1eecb5c 193@end lisp
7bc46ecc
RW
194
195@item
6a893fa8
LC
196The @dfn{quote} disables evaluation of a parenthesized expression: the
197first term is not called over the other terms (@pxref{Expression Syntax,
198quote,, guile, GNU Guile Reference Manual}). Thus it effectively
199returns a list of terms.
7bc46ecc 200
b1eecb5c 201@lisp
cf1e6f5f
LC
202'(display (string-append "Hello " "Guix" "\n"))
203@result{} (display (string-append "Hello " "Guix" "\n"))
204
205'(2 a 5 7)
206@result{} (2 a 5 7)
b1eecb5c 207@end lisp
7bc46ecc
RW
208
209@item
6a893fa8
LC
210The @dfn{quasiquote} disables evaluation of a parenthesized expression
211until @dfn{unquote} (a comma) re-enables it. Thus it provides us with
212fine-grained control over what is evaluated and what is not.
7bc46ecc 213
b1eecb5c 214@lisp
cf1e6f5f
LC
215`(2 a 5 7 (2 ,a 5 ,(+ a 4)))
216@result{} (2 a 5 7 (2 3 5 7))
b1eecb5c 217@end lisp
7bc46ecc
RW
218
219Note that the above result is a list of mixed elements: numbers, symbols (here
220@code{a}) and the last element is a list itself.
221
222@item
6a893fa8
LC
223Multiple variables can be named locally with @code{let} (@pxref{Local
224Bindings,,, guile, GNU Guile Reference Manual}):
7bc46ecc 225
b1eecb5c 226@lisp
cf1e6f5f
LC
227(define x 10)
228(let ((x 2)
229 (y 3))
230 (list x y))
231@result{} (2 3)
232
233x
234@result{} 10
235
236y
237@error{} In procedure module-lookup: Unbound variable: y
b1eecb5c 238@end lisp
7bc46ecc
RW
239
240Use @code{let*} to allow later variable declarations to refer to earlier
241definitions.
242
b1eecb5c 243@lisp
cf1e6f5f
LC
244(let* ((x 2)
245 (y (* x 3)))
246 (list x y))
247@result{} (2 6)
b1eecb5c 248@end lisp
7bc46ecc
RW
249
250@item
251The keyword syntax is @code{#:}; it is used to create unique identifiers.
252@pxref{Keywords,,, guile, GNU Guile Reference Manual}.
253
254@item
255The percentage @code{%} is typically used for read-only global variables in
256the build stage. Note that it is merely a convention, like @code{_} in C.
257Scheme treats @code{%} exactly the same as any other letter.
258
259@item
6a893fa8
LC
260Modules are created with @code{define-module} (@pxref{Creating Guile
261Modules,,, guile, GNU Guile Reference Manual}). For instance
7bc46ecc 262
b1eecb5c 263@lisp
7bc46ecc
RW
264(define-module (guix build-system ruby)
265 #:use-module (guix store)
266 #:export (ruby-build
267 ruby-build-system))
b1eecb5c 268@end lisp
7bc46ecc
RW
269
270defines the module @code{guix build-system ruby} which must be located in
271@file{guix/build-system/ruby.scm} somewhere in the Guile load path. It
272depends on the @code{(guix store)} module and it exports two variables,
273@code{ruby-build} and @code{ruby-build-system}.
274@end itemize
275
276For a more detailed introduction, check out
277@uref{http://www.troubleshooters.com/codecorn/scheme_guile/hello.htm, Scheme
278at a Glance}, by Steve Litt.
279
280One of the reference Scheme books is the seminal ``Structure and
281Interpretation of Computer Programs'', by Harold Abelson and Gerald Jay
282Sussman, with Julie Sussman. You'll find a
283@uref{https://mitpress.mit.edu/sites/default/files/sicp/index.html, free copy
284online}, together with
285@uref{https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/,
286videos of the lectures by the authors}. The book is available in Texinfo
287format as the @code{sicp} Guix package. Go ahead, run @code{guix install
ed79636c 288sicp} and start reading with @code{info sicp} (@pxref{,,, sicp, Structure and Interpretation of Computer Programs}).
7bc46ecc
RW
289An @uref{https://sarabander.github.io/sicp/, unofficial ebook is also
290available}.
291
292You'll find more books, tutorials and other resources at
293@url{https://schemers.org/}.
294
295
296@c *********************************************************************
297@node Packaging
298@chapter Packaging
299
300@cindex packaging
301
302This chapter is dedicated to teaching you how to add packages to the
303collection of packages that come with GNU Guix. This involves writing package
304definitions in Guile Scheme, organizing them in package modules, and building
305them.
306
307@menu
308* Packaging Tutorial:: A tutorial on how to add packages to Guix.
309@end menu
310
311@node Packaging Tutorial
312@section Packaging Tutorial
313
314GNU Guix stands out as the @emph{hackable} package manager, mostly because it
315uses @uref{https://www.gnu.org/software/guile/, GNU Guile}, a powerful
316high-level programming language, one of the
317@uref{https://en.wikipedia.org/wiki/Scheme_%28programming_language%29, Scheme}
318dialects from the
319@uref{https://en.wikipedia.org/wiki/Lisp_%28programming_language%29, Lisp family}.
320
321Package definitions are also written in Scheme, which empowers Guix in some
322very unique ways, unlike most other package managers that use shell scripts or
323simple languages.
324
325@itemize
326@item
327Use functions, structures, macros and all of Scheme expressiveness for your
328package definitions.
329
330@item
331Inheritance makes it easy to customize a package by inheriting from it and
332modifying only what is needed.
5c3e94cd 333
7bc46ecc
RW
334@item
335Batch processing: the whole package collection can be parsed, filtered and
336processed. Building a headless server with all graphical interfaces stripped
337out? It's possible. Want to rebuild everything from source using specific
338compiler optimization flags? Pass the @code{#:make-flags "..."} argument to
339the list of packages. It wouldn't be a stretch to think
340@uref{https://wiki.gentoo.org/wiki/USE_flag, Gentoo USE flags} here, but this
341goes even further: the changes don't have to be thought out beforehand by the
342packager, they can be @emph{programmed} by the user!
343@end itemize
344
345The following tutorial covers all the basics around package creation with Guix.
346It does not assume much knowledge of the Guix system nor of the Lisp language.
347The reader is only expected to be familiar with the command line and to have some
348basic programming knowledge.
349
0cbef07b
LC
350@node A ``Hello World'' package
351@subsection A ``Hello World'' package
7bc46ecc 352
0cbef07b 353The ``Defining Packages'' section of the manual introduces the basics of Guix
7bc46ecc
RW
354packaging (@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}). In
355the following section, we will partly go over those basics again.
356
0cbef07b 357GNU@tie{}Hello is a dummy project that serves as an idiomatic example for
7bc46ecc
RW
358packaging. It uses the GNU build system (@code{./configure && make && make
359install}). Guix already provides a package definition which is a perfect
360example to start with. You can look up its declaration with @code{guix edit
361hello} from the command line. Let's see how it looks:
362
b1eecb5c 363@lisp
7bc46ecc
RW
364(define-public hello
365 (package
366 (name "hello")
367 (version "2.10")
368 (source (origin
369 (method url-fetch)
370 (uri (string-append "mirror://gnu/hello/hello-" version
371 ".tar.gz"))
372 (sha256
373 (base32
374 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
375 (build-system gnu-build-system)
376 (synopsis "Hello, GNU world: An example GNU package")
377 (description
378 "GNU Hello prints the message \"Hello, world!\" and then exits. It
379serves as an example of standard GNU coding practices. As such, it supports
380command-line arguments, multiple languages, and so on.")
381 (home-page "https://www.gnu.org/software/hello/")
382 (license gpl3+)))
b1eecb5c 383@end lisp
7bc46ecc
RW
384
385As you can see, most of it is rather straightforward. But let's review the
386fields together:
387
388@table @samp
389@item name
390The project name. Using Scheme conventions, we prefer to keep it
391lower case, without underscore and using dash-separated words.
392
393@item source
394This field contains a description of the source code origin. The
395@code{origin} record contains these fields:
396
397@enumerate
398@item The method, here @code{url-fetch} to download via HTTP/FTP, but other methods
399 exist, such as @code{git-fetch} for Git repositories.
400@item The URI, which is typically some @code{https://} location for @code{url-fetch}. Here
401 the special `mirror://gnu` refers to a set of well known locations, all of
402 which can be used by Guix to fetch the source, should some of them fail.
403@item The @code{sha256} checksum of the requested file. This is essential to ensure
404 the source is not corrupted. Note that Guix works with base32 strings,
405 hence the call to the @code{base32} function.
406@end enumerate
407
408@item build-system
409
410This is where the power of abstraction provided by the Scheme language really
411shines: in this case, the @code{gnu-build-system} abstracts away the famous
412@code{./configure && make && make install} shell invocations. Other build
413systems include the @code{trivial-build-system} which does not do anything and
414requires from the packager to program all the build steps, the
415@code{python-build-system}, the @code{emacs-build-system}, and many more
416(@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
417
418@item synopsis
419It should be a concise summary of what the package does. For many packages a
420tagline from the project's home page can be used as the synopsis.
421
422@item description
423Same as for the synopsis, it's fine to re-use the project description from the
424homepage. Note that Guix uses Texinfo syntax.
425
426@item home-page
427Use HTTPS if available.
428
429@item license
430See @code{guix/licenses.scm} in the project source for a full list of
431available licenses.
432@end table
433
434Time to build our first package! Nothing fancy here for now: we will stick to a
0cbef07b 435dummy @code{my-hello}, a copy of the above declaration.
7bc46ecc 436
0cbef07b
LC
437As with the ritualistic ``Hello World'' taught with most programming languages,
438this will possibly be the most ``manual'' approach. We will work out an ideal
7bc46ecc
RW
439setup later; for now we will go the simplest route.
440
441Save the following to a file @file{my-hello.scm}.
442
b1eecb5c 443@lisp
7bc46ecc
RW
444(use-modules (guix packages)
445 (guix download)
446 (guix build-system gnu)
447 (guix licenses))
448
449(package
450 (name "my-hello")
451 (version "2.10")
452 (source (origin
453 (method url-fetch)
454 (uri (string-append "mirror://gnu/hello/hello-" version
455 ".tar.gz"))
456 (sha256
457 (base32
458 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
459 (build-system gnu-build-system)
460 (synopsis "Hello, Guix world: An example custom Guix package")
461 (description
462 "GNU Hello prints the message \"Hello, world!\" and then exits. It
463serves as an example of standard GNU coding practices. As such, it supports
464command-line arguments, multiple languages, and so on.")
465 (home-page "https://www.gnu.org/software/hello/")
466 (license gpl3+))
b1eecb5c 467@end lisp
7bc46ecc
RW
468
469We will explain the extra code in a moment.
470
471Feel free to play with the different values of the various fields. If you
472change the source, you'll need to update the checksum. Indeed, Guix refuses to
473build anything if the given checksum does not match the computed checksum of the
474source code. To obtain the correct checksum of the package declaration, we
475need to download the source, compute the sha256 checksum and convert it to
476base32.
477
478Thankfully, Guix can automate this task for us; all we need is to provide the
479URI:
480
481@c TRANSLATORS: This is example shell output.
482@example sh
483$ guix download mirror://gnu/hello/hello-2.10.tar.gz
484
485Starting download of /tmp/guix-file.JLYgL7
486From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
487following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
488 …10.tar.gz 709KiB 2.5MiB/s 00:00 [##################] 100.0%
489/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
4900ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
491@end example
492
493In this specific case the output tells us which mirror was chosen.
494If the result of the above command is not the same as in the above snippet,
495update your @code{my-hello} declaration accordingly.
496
497Note that GNU package tarballs come with an OpenPGP signature, so you
498should definitely check the signature of this tarball with `gpg` to
499authenticate it before going further:
500
501@c TRANSLATORS: This is example shell output.
502@example sh
503$ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig
504
505Starting download of /tmp/guix-file.03tFfb
506From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
507following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
508 ….tar.gz.sig 819B 1.2MiB/s 00:00 [##################] 100.0%
509/gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
5100q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
511$ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
512gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
513gpg: using RSA key A9553245FDE9B739
514gpg: Good signature from "Sami Kerola <kerolasa@@iki.fi>" [unknown]
515gpg: aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@@iki.fi>" [unknown]
516gpg: WARNING: This key is not certified with a trusted signature!
517gpg: There is no indication that the signature belongs to the owner.
518Primary key fingerprint: 8ED3 96E3 7E38 D471 A005 30D3 A955 3245 FDE9 B739
519@end example
520
521You can then happily run
522
523@c TRANSLATORS: Do not translate this command
524@example sh
525$ guix package --install-from-file=my-hello.scm
526@end example
527
528You should now have @code{my-hello} in your profile!
529
530@c TRANSLATORS: Do not translate this command
531@example sh
532$ guix package --list-installed=my-hello
533my-hello 2.10 out
534/gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10
535@end example
536
537We've gone as far as we could without any knowledge of Scheme. Before moving
538on to more complex packages, now is the right time to brush up on your Scheme
539knowledge. @pxref{A Scheme Crash Course} to get up to speed.
540
ffe059af
PN
541@node Setup
542@subsection Setup
7bc46ecc 543
ffe059af
PN
544In the rest of this chapter we will rely on some basic Scheme
545programming knowledge. Now let's detail the different possible setups
546for working on Guix packages.
547
548There are several ways to set up a Guix packaging environment.
549
550We recommend you work directly on the Guix source checkout since it makes it
551easier for everyone to contribute to the project.
552
553But first, let's look at other possibilities.
554
555@node Local file
556@subsubsection Local file
557
558This is what we previously did with @samp{my-hello}. With the Scheme basics we've
559covered, we are now able to explain the leading chunks. As stated in @code{guix
560package --help}:
561
562@example
563 -f, --install-from-file=FILE
564 install the package that the code within FILE
565 evaluates to
566@end example
567
568Thus the last expression @emph{must} return a package, which is the case in our
569earlier example.
570
571The @code{use-modules} expression tells which of the modules we need in the file.
572Modules are a collection of values and procedures. They are commonly called
0cbef07b 573``libraries'' or ``packages'' in other programming languages.
ffe059af
PN
574
575@node @samp{GUIX_PACKAGE_PATH}
576@subsubsection @samp{GUIX_PACKAGE_PATH}
577
0cbef07b 578@emph{Note: Starting from Guix 0.16, the more flexible Guix @dfn{channels} are the
ffe059af
PN
579preferred way and supersede @samp{GUIX_PACKAGE_PATH}. See next section.}
580
581It can be tedious to specify the file from the command line instead of simply
582calling @code{guix package --install my-hello} as you would do with the official
583packages.
584
0cbef07b
LC
585Guix makes it possible to streamline the process by adding as many ``package
586declaration directories'' as you want.
ffe059af 587
2c8305d1 588Create a directory, say @file{~./guix-packages} and add it to the @samp{GUIX_PACKAGE_PATH}
ffe059af
PN
589environment variable:
590
591@example
592$ mkdir ~/guix-packages
593$ export GUIX_PACKAGE_PATH=~/guix-packages
594@end example
595
596To add several directories, separate them with a colon (@code{:}).
597
598Our previous @samp{my-hello} needs some adjustments though:
599
d482e13f 600@lisp
ffe059af
PN
601(define-module (my-hello)
602 #:use-module (guix licenses)
603 #:use-module (guix packages)
604 #:use-module (guix build-system gnu)
605 #:use-module (guix download))
606
607(define-public my-hello
608 (package
609 (name "my-hello")
610 (version "2.10")
611 (source (origin
612 (method url-fetch)
613 (uri (string-append "mirror://gnu/hello/hello-" version
614 ".tar.gz"))
615 (sha256
616 (base32
617 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
618 (build-system gnu-build-system)
619 (synopsis "Hello, Guix world: An example custom Guix package")
620 (description
621 "GNU Hello prints the message \"Hello, world!\" and then exits. It
622serves as an example of standard GNU coding practices. As such, it supports
623command-line arguments, multiple languages, and so on.")
624 (home-page "https://www.gnu.org/software/hello/")
625 (license gpl3+)))
d482e13f 626@end lisp
ffe059af
PN
627
628Note that we have assigned the package value to an exported variable name with
629@code{define-public}. This is effectively assigning the package to the @code{my-hello}
630variable so that it can be referenced, among other as dependency of other
631packages.
632
633If you use @code{guix package --install-from-file=my-hello.scm} on the above file, it
634will fail because the last expression, @code{define-public}, does not return a
635package. If you want to use @code{define-public} in this use-case nonetheless, make
636sure the file ends with an evaluation of @code{my-hello}:
637
d482e13f 638@lisp
ffe059af
PN
639; ...
640(define-public my-hello
641 ; ...
642 )
643
644my-hello
d482e13f 645@end lisp
ffe059af
PN
646
647This last example is not very typical.
648
649Now @samp{my-hello} should be part of the package collection like all other official
650packages. You can verify this with:
651
652@example
653$ guix package --show=my-hello
654@end example
655
656@node Guix channels
657@subsubsection Guix channels
658
659Guix 0.16 features channels, which is very similar to @samp{GUIX_PACKAGE_PATH} but
660provides better integration and provenance tracking. Channels are not
661necessarily local, they can be maintained as a public Git repository for
662instance. Of course, several channels can be used at the same time.
663
664@xref{Channels,,, guix, GNU Guix Reference Manual} for setup details.
665
666@node Direct checkout hacking
667@subsubsection Direct checkout hacking
668
669Working directly on the Guix project is recommended: it reduces the friction
670when the time comes to submit your changes upstream to let the community benefit
671from your hard work!
672
673Unlike most software distributions, the Guix repository holds in one place both
674the tooling (including the package manager) and the package definitions. This
675choice was made so that it would give developers the flexibility to modify the
676API without breakage by updating all packages at the same time. This reduces
677development inertia.
678
679Check out the official @uref{https://git-scm.com/, Git} repository:
680
681@example
682$ git clone https://git.savannah.gnu.org/git/guix.git
683@end example
684
685In the rest of this article, we use @samp{$GUIX_CHECKOUT} to refer to the location of
686the checkout.
687
688
e97a4a29 689Follow the instructions in the manual (@pxref{Contributing,,, guix, GNU Guix
ffe059af
PN
690Reference Manual}) to set up the repository environment.
691
692Once ready, you should be able to use the package definitions from the
693repository environment.
694
695Feel free to edit package definitions found in @samp{$GUIX_CHECKOUT/gnu/packages}.
696
697The @samp{$GUIX_CHECKOUT/pre-inst-env} script lets you use @samp{guix} over the package
e97a4a29
LC
698collection of the repository (@pxref{Running Guix Before It Is
699Installed,,, guix, GNU Guix Reference Manual}).
ffe059af
PN
700
701@itemize
702@item
703Search packages, such as Ruby:
704
705@example
706 $ cd $GUIX_CHECKOUT
707 $ ./pre-inst-env guix package --list-available=ruby
708 ruby 1.8.7-p374 out gnu/packages/ruby.scm:119:2
709 ruby 2.1.6 out gnu/packages/ruby.scm:91:2
710 ruby 2.2.2 out gnu/packages/ruby.scm:39:2
711@end example
712
713@item
714Build a package, here Ruby version 2.1:
715
716@example
717 $ ./pre-inst-env guix build --keep-failed ruby@@2.1
718 /gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
719@end example
720
721@item
722Install it to your user profile:
723
724@example
725 $ ./pre-inst-env guix package --install ruby@@2.1
726@end example
727
728@item
729Check for common mistakes:
730
731@example
732 $ ./pre-inst-env guix lint ruby@@2.1
733@end example
734@end itemize
735
736Guix strives at maintaining a high packaging standard; when contributing to the
737Guix project, remember to
738
739@itemize
740@item
741follow the coding style (@pxref{Coding Style,,, guix, GNU Guix Reference Manual}),
742@item
743and review the check list from the manual (@pxref{Submitting Patches,,, guix, GNU Guix Reference Manual}).
744@end itemize
745
746Once you are happy with the result, you are welcome to send your contribution to
747make it part of Guix. This process is also detailed in the manual. (@pxref{Contributing,,, guix, GNU Guix Reference Manual})
748
749
750It's a community effort so the more join in, the better Guix becomes!
751
752@node Extended example
753@subsection Extended example
754
0cbef07b 755The above ``Hello World'' example is as simple as it goes. Packages can be more
ffe059af
PN
756complex than that and Guix can handle more advanced scenarios. Let's look at
757another, more sophisticated package (slightly modified from the source):
758
d482e13f 759@lisp
ffe059af
PN
760(define-module (gnu packages version-control)
761 #:use-module ((guix licenses) #:prefix license:)
762 #:use-module (guix utils)
763 #:use-module (guix packages)
764 #:use-module (guix git-download)
765 #:use-module (guix build-system cmake)
766 #:use-module (gnu packages ssh)
767 #:use-module (gnu packages web)
768 #:use-module (gnu packages pkg-config)
769 #:use-module (gnu packages python)
770 #:use-module (gnu packages compression)
771 #:use-module (gnu packages tls))
772
773(define-public my-libgit2
774 (let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
775 (revision "1"))
776 (package
777 (name "my-libgit2")
778 (version (git-version "0.26.6" revision commit))
779 (source (origin
780 (method git-fetch)
781 (uri (git-reference
782 (url "https://github.com/libgit2/libgit2/")
783 (commit commit)))
784 (file-name (git-file-name name version))
785 (sha256
786 (base32
787 "17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
788 (patches (search-patches "libgit2-mtime-0.patch"))
789 (modules '((guix build utils)))
790 (snippet '(begin
791 ;; Remove bundled software.
792 (delete-file-recursively "deps")
793 #t))))
794 (build-system cmake-build-system)
795 (outputs '("out" "debug"))
796 (arguments
797 `(#:tests? #t ; Run the test suite (this is the default)
798 #:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
799 #:phases
800 (modify-phases %standard-phases
801 (add-after 'unpack 'fix-hardcoded-paths
802 (lambda _
803 (substitute* "tests/repo/init.c"
804 (("#!/bin/sh") (string-append "#!" (which "sh"))))
805 (substitute* "tests/clar/fs.h"
806 (("/bin/cp") (which "cp"))
807 (("/bin/rm") (which "rm")))
808 #t))
809 ;; Run checks more verbosely.
810 (replace 'check
811 (lambda _ (invoke "./libgit2_clar" "-v" "-Q")))
812 (add-after 'unpack 'make-files-writable-for-tests
813 (lambda _ (for-each make-file-writable (find-files "." ".*")))))))
814 (inputs
815 `(("libssh2" ,libssh2)
816 ("http-parser" ,http-parser)
817 ("python" ,python-wrapper)))
818 (native-inputs
819 `(("pkg-config" ,pkg-config)))
820 (propagated-inputs
821 ;; These two libraries are in 'Requires.private' in libgit2.pc.
822 `(("openssl" ,openssl)
823 ("zlib" ,zlib)))
824 (home-page "https://libgit2.github.com/")
825 (synopsis "Library providing Git core methods")
826 (description
827 "Libgit2 is a portable, pure C implementation of the Git core methods
828provided as a re-entrant linkable library with a solid API, allowing you to
829write native speed custom Git applications in any language with bindings.")
830 ;; GPLv2 with linking exception
831 (license license:gpl2))))
d482e13f 832@end lisp
ffe059af
PN
833
834(In those cases were you only want to tweak a few fields from a package
835definition, you should rely on inheritance instead of copy-pasting everything.
836See below.)
837
838Let's discuss those fields in depth.
839
840@subsubsection @code{git-fetch} method
841
842Unlike the @code{url-fetch} method, @code{git-fetch} expects a @code{git-reference} which takes
843a Git repository and a commit. The commit can be any Git reference such as
844tags, so if the @code{version} is tagged, then it can be used directly. Sometimes
845the tag is prefixed with a @code{v}, in which case you'd use @code{(commit (string-append
846"v" version))}.
847
ab83e1f0
LC
848To ensure that the source code from the Git repository is stored in a
849directory with a descriptive name, we use @code{(file-name (git-file-name name
ffe059af
PN
850version))}.
851
4ebbd92c 852The @code{git-version} procedure can be used to derive the
ab83e1f0
LC
853version when packaging programs for a specific commit, following the
854Guix contributor guidelines (@pxref{Version Numbers,,, guix, GNU Guix
855Reference Manual}).
ffe059af 856
2f562699
LC
857How does one obtain the @code{sha256} hash that's in there, you ask? By
858invoking @command{guix hash} on a checkout of the desired commit, along
4ebbd92c 859these lines:
2f562699
LC
860
861@example
862git clone https://github.com/libgit2/libgit2/
863cd libgit2
864git checkout v0.26.6
865guix hash -rx .
866@end example
867
868@command{guix hash -rx} computes a SHA256 hash over the whole directory,
869excluding the @file{.git} sub-directory (@pxref{Invoking guix hash,,,
870guix, GNU Guix Reference Manual}).
871
872In the future, @command{guix download} will hopefully be able to do
873these steps for you, just like it does for regular downloads.
874
ffe059af
PN
875@subsubsection Snippets
876
877Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching
2c8305d1 878the source. They are a Guix-y alternative to the traditional @file{.patch} files.
ffe059af 879Because of the quote, the code in only evaluated when passed to the Guix daemon
0cbef07b 880for building. There can be as many snippets as needed.
ffe059af
PN
881
882Snippets might need additional Guile modules which can be imported from the
883@code{modules} field.
884
885@subsubsection Inputs
886
887First, a syntactic comment: See the quasi-quote / comma syntax?
888
d482e13f 889@lisp
ffe059af
PN
890 (native-inputs
891 `(("pkg-config" ,pkg-config)))
d482e13f 892@end lisp
ffe059af
PN
893
894is equivalent to
895
d482e13f 896@lisp
ffe059af
PN
897 (native-inputs
898 (list (list "pkg-config" pkg-config)))
d482e13f 899@end lisp
ffe059af
PN
900
901You'll mostly see the former because it's shorter.
902
903There are 3 different input types. In short:
904
905@table @asis
906@item native-inputs
907Required for building but not runtime -- installing a package
908through a substitute won't install these inputs.
909@item inputs
910Installed in the store but not in the profile, as well as being
911present at build time.
912@item propagated-inputs
913Installed in the store and in the profile, as well as
914being present at build time.
915@end table
916
917@xref{Package Reference,,, guix, GNU Guix Reference Manual} for more details.
918
919The distinction between the various inputs is important: if a dependency can be
920handled as an @emph{input} instead of a @emph{propagated input}, it should be done so, or
0cbef07b 921else it ``pollutes'' the user profile for no good reason.
ffe059af
PN
922
923For instance, a user installing a graphical program that depends on a
924command line tool might only be interested in the graphical part, so there is no
925need to force the command line tool into the user profile. The dependency is a
926concern to the package, not to the user. @emph{Inputs} make it possible to handle
927dependencies without bugging the user by adding undesired executable files (or
928libraries) to their profile.
929
930Same goes for @emph{native-inputs}: once the program is installed, build-time
931dependencies can be safely garbage-collected.
932It also matters when a substitute is available, in which case only the @emph{inputs}
933and @emph{propagated inputs} will be fetched: the @emph{native inputs} are not required to
934install a package from a substitute.
935
936@subsubsection Outputs
937
938Just like how a package can have multiple inputs, it can also produce multiple
939outputs.
940
941Each output corresponds to a separate directory in the store.
942
943The user can choose which output to install; this is useful to save space or
944to avoid polluting the user profile with unwanted executables or libraries.
945
946Output separation is optional. When the @code{outputs} field is left out, the
947default and only output (the complete package) is referred to as @code{"out"}.
948
949Typical separate output names include @code{debug} and @code{doc}.
950
951It's advised to separate outputs only when you've shown it's worth it: if the
952output size is significant (compare with @code{guix size}) or in case the package is
953modular.
954
955@subsubsection Build system arguments
956
957The @code{arguments} is a keyword-value list used to configure the build process.
958
959The simplest argument @code{#:tests?} can be used to disable the test suite when
960building the package. This is mostly useful when the package does not feature
961any test suite. It's strongly recommended to keep the test suite on if there is
962one.
963
964Another common argument is @code{:make-flags}, which specifies a list of flags to
965append when running make, as you would from the command line. For instance, the
966following flags
967
d482e13f 968@lisp
ffe059af
PN
969#:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
970 "CC=gcc")
d482e13f 971@end lisp
ffe059af
PN
972
973translate into
974
975@example
976$ make CC=gcc prefix=/gnu/store/...-<out>
977@end example
978
979This sets the C compiler to @code{gcc} and the @code{prefix} variable (the installation
980directory in Make parlance) to @code{(assoc-ref %outputs "out")}, which is a build-stage
981global variable pointing to the destination directory in the store (something like
2c8305d1 982@file{/gnu/store/...-my-libgit2-20180408}).
ffe059af 983
0cbef07b 984Similarly, it's possible to set the configure flags:
ffe059af 985
d482e13f 986@lisp
ffe059af 987#:configure-flags '("-DUSE_SHA1DC=ON")
d482e13f 988@end lisp
ffe059af
PN
989
990The @code{%build-inputs} variable is also generated in scope. It's an association
991table that maps the input names to their store directories.
992
993The @code{phases} keyword lists the sequential steps of the build system. Typically
994phases include @code{unpack}, @code{configure}, @code{build}, @code{install} and @code{check}. To know
995more about those phases, you need to work out the appropriate build system
996definition in @samp{$GUIX_CHECKOUT/guix/build/gnu-build-system.scm}:
997
d482e13f 998@lisp
ffe059af
PN
999(define %standard-phases
1000 ;; Standard build phases, as a list of symbol/procedure pairs.
1001 (let-syntax ((phases (syntax-rules ()
1002 ((_ p ...) `((p . ,p) ...)))))
1003 (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
1004 bootstrap
1005 patch-usr-bin-file
1006 patch-source-shebangs configure patch-generated-file-shebangs
1007 build check install
1008 patch-shebangs strip
1009 validate-runpath
1010 validate-documentation-location
1011 delete-info-dir-file
1012 patch-dot-desktop-files
1013 install-license-files
1014 reset-gzip-timestamps
1015 compress-documentation)))
d482e13f 1016@end lisp
ffe059af
PN
1017
1018Or from the REPL:
1019
d482e13f 1020@lisp
cf1e6f5f
LC
1021(add-to-load-path "/path/to/guix/checkout")
1022,use (guix build gnu-build-system)
1023(map first %standard-phases)
1024@result{} (set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap patch-usr-bin-file patch-source-shebangs configure patch-generated-file-shebangs build check install patch-shebangs strip validate-runpath validate-documentation-location delete-info-dir-file patch-dot-desktop-files install-license-files reset-gzip-timestamps compress-documentation)
d482e13f 1025@end lisp
ffe059af
PN
1026
1027If you want to know more about what happens during those phases, consult the
1028associated procedures.
1029
1030For instance, as of this writing the definition of @code{unpack} for the GNU build
1031system is
1032
d482e13f 1033@lisp
ffe059af
PN
1034(define* (unpack #:key source #:allow-other-keys)
1035 "Unpack SOURCE in the working directory, and change directory within the
1036source. When SOURCE is a directory, copy it in a sub-directory of the current
1037working directory."
1038 (if (file-is-directory? source)
1039 (begin
1040 (mkdir "source")
1041 (chdir "source")
1042
1043 ;; Preserve timestamps (set to the Epoch) on the copied tree so that
1044 ;; things work deterministically.
1045 (copy-recursively source "."
1046 #:keep-mtime? #t))
1047 (begin
1048 (if (string-suffix? ".zip" source)
1049 (invoke "unzip" source)
1050 (invoke "tar" "xvf" source))
1051 (chdir (first-subdirectory "."))))
1052 #t)
d482e13f 1053@end lisp
ffe059af
PN
1054
1055Note the @code{chdir} call: it changes the working directory to where the source was
1056unpacked.
1057Thus every phase following the @code{unpack} will use the source as a working
1058directory, which is why we can directly work on the source files.
1059That is to say, unless a later phase changes the working directory to something
1060else.
1061
1062We modify the list of @code{%standard-phases} of the build system with the
1063@code{modify-phases} macro as per the list of specified modifications, which may have
1064the following forms:
1065
1066@itemize
1067@item
1068@code{(add-before PHASE NEW-PHASE PROCEDURE)}: Run @code{PROCEDURE} named @code{NEW-PHASE} before @code{PHASE}.
1069@item
1070@code{(add-after PHASE NEW-PHASE PROCEDURE)}: Same, but afterwards.
1071@item
1072@code{(replace PHASE PROCEDURE)}.
1073@item
1074@code{(delete PHASE)}.
1075@end itemize
1076
1077The @code{PROCEDURE} supports the keyword arguments @code{inputs} and @code{outputs}. Each
1078input (whether @emph{native}, @emph{propagated} or not) and output directory is referenced
1079by their name in those variables. Thus @code{(assoc-ref outputs "out")} is the store
1080directory of the main output of the package. A phase procedure may look like
1081this:
1082
d482e13f 1083@lisp
ffe059af
PN
1084(lambda* (#:key inputs outputs #:allow-other-keys)
1085 (let (((bash-directory (assoc-ref inputs "bash"))
1086 (output-directory (assoc-ref outputs "out"))
1087 (doc-directory (assoc-ref outputs "doc"))
1088 ; ...
1089 #t)
d482e13f 1090@end lisp
ffe059af
PN
1091
1092The procedure must return @code{#t} on success. It's brittle to rely on the return
1093value of the last expression used to tweak the phase because there is no
1094guarantee it would be a @code{#t}. Hence the trailing @code{#t} to ensure the right value
1095is returned on success.
1096
1097@subsubsection Code staging
1098
1099The astute reader may have noticed the quasi-quote and comma syntax in the
1100argument field. Indeed, the build code in the package declaration should not be
1101evaluated on the client side, but only when passed to the Guix daemon. This
1102mechanism of passing code around two running processes is called @uref{https://arxiv.org/abs/1709.00833, code staging}.
1103
0cbef07b 1104@subsubsection Utility functions
ffe059af
PN
1105
1106When customizing @code{phases}, we often need to write code that mimics the
cc765e47 1107equivalent system invocations (@code{make}, @code{mkdir}, @code{cp}, etc.)@: commonly used during
0cbef07b 1108regular ``Unix-style'' installations.
ffe059af
PN
1109
1110Some like @code{chmod} are native to Guile.
1111@xref{,,, guile, Guile reference manual} for a complete list.
1112
1113Guix provides additional helper functions which prove especially handy in the
1114context of package management.
1115
1116Some of those functions can be found in
1117@samp{$GUIX_CHECKOUT/guix/guix/build/utils.scm}. Most of them mirror the behaviour
1118of the traditional Unix system commands:
1119
1120@table @asis
1121@item which
1122Like the @samp{which} system command.
1123@item find-files
1124Akin to the @samp{find} system command.
1125@item mkdir-p
1126Like @samp{mkdir -p}, which creates all parents as needed.
1127@item install-file
1128Similar to @samp{install} when installing a file to a (possibly
1129non-existing) directory. Guile has @code{copy-file} which works
1130like @samp{cp}.
1131@item copy-recursively
1132Like @samp{cp -r}.
1133@item delete-file-recursively
1134Like @samp{rm -rf}.
1135@item invoke
1136Run an executable. This should be used instead of @code{system*}.
1137@item with-directory-excursion
1138Run the body in a different working directory,
1139then restore the previous working directory.
1140@item substitute*
0cbef07b 1141A ``@command{sed}-like'' function.
ffe059af
PN
1142@end table
1143
1144@subsubsection Module prefix
1145
1146The license in our last example needs a prefix: this is because of how the
1147@code{license} module was imported in the package, as @code{#:use-module ((guix licenses)
1148#:prefix license:)}. The Guile module import mechanism
1149(@pxref{Using Guile Modules,,, guile, Guile reference manual})
1150gives the user full control over namespacing: this is needed to avoid
1151clashes between, say, the
1152@samp{zlib} variable from @samp{licenses.scm} (a @emph{license} value) and the @samp{zlib} variable
1153from @samp{compression.scm} (a @emph{package} value).
1154
1155@node Other build systems
1156@subsection Other build systems
1157
1158What we've seen so far covers the majority of packages using a build system
1159other than the @code{trivial-build-system}. The latter does not automate anything
1160and leaves you to build everything manually. This can be more demanding and we
1161won't cover it here for now, but thankfully it is rarely necessary to fall back
1162on this system.
1163
1164For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the
1165process is very similar to the GNU build system except for a few specialized
1166arguments.
1167
a9e25512
LC
1168@xref{Build Systems,,, guix, GNU Guix Reference Manual}, for more
1169information on build systems, or check the source code in the
1170@samp{$GUIX_CHECKOUT/guix/build} and
ffe059af 1171@samp{$GUIX_CHECKOUT/guix/build-system} directories.
ffe059af
PN
1172
1173@node Programmable and automated package definition
1174@subsection Programmable and automated package definition
1175
1176We can't repeat it enough: having a full-fledged programming language at hand
1177empowers us in ways that reach far beyond traditional package management.
1178
1179Let's illustrate this with some awesome features of Guix!
1180
1181@node Recursive importers
1182@subsubsection Recursive importers
1183
1184You might find some build systems good enough that there is little to do at all
1185to write a package, to the point that it becomes repetitive and tedious after a
1186while. A @emph{raison d'être} of computers is to replace human beings at those
1187boring tasks. So let's tell Guix to do this for us and create the package
1188definition of an R package from CRAN (the output is trimmed for conciseness):
1189
1190@example
1191$ guix import cran --recursive walrus
1192
1193(define-public r-mc2d
1194 ; ...
1195 (license gpl2+)))
1196
1197(define-public r-jmvcore
1198 ; ...
1199 (license gpl2+)))
1200
1201(define-public r-wrs2
1202 ; ...
1203 (license gpl3)))
1204
1205(define-public r-walrus
1206 (package
1207 (name "r-walrus")
1208 (version "1.0.3")
1209 (source
1210 (origin
1211 (method url-fetch)
1212 (uri (cran-uri "walrus" version))
1213 (sha256
1214 (base32
1215 "1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
1216 (build-system r-build-system)
1217 (propagated-inputs
1218 `(("r-ggplot2" ,r-ggplot2)
1219 ("r-jmvcore" ,r-jmvcore)
1220 ("r-r6" ,r-r6)
1221 ("r-wrs2" ,r-wrs2)))
1222 (home-page "https://github.com/jamovi/walrus")
1223 (synopsis "Robust Statistical Methods")
1224 (description
1225 "This package provides a toolbox of common robust statistical
1226tests, including robust descriptives, robust t-tests, and robust ANOVA.
1227It is also available as a module for 'jamovi' (see
1228<https://www.jamovi.org> for more information). Walrus is based on the
1229WRS2 package by Patrick Mair, which is in turn based on the scripts and
1230work of Rand Wilcox. These analyses are described in depth in the book
1231'Introduction to Robust Estimation & Hypothesis Testing'.")
1232 (license gpl3)))
1233@end example
1234
1235The recursive importer won't import packages for which Guix already has package
1236definitions, except for the very first.
1237
1238Not all applications can be packaged this way, only those relying on a select
1239number of supported systems. Read about the full list of importers in
1240the guix import section of the manual
1241(@pxref{Invoking guix import,,, guix, GNU Guix Reference Manual}).
1242
1243@node Automatic update
1244@subsubsection Automatic update
1245
1246Guix can be smart enough to check for updates on systems it knows. It can
1247report outdated package definitions with
1248
1249@example
1250$ guix refresh hello
1251@end example
1252
1253In most cases, updating a package to a newer version requires little more than
1254changing the version number and the checksum. Guix can do that automatically as
1255well:
1256
1257@example
1258$ guix refresh hello --update
1259@end example
1260
1261@node Inheritance
1262@subsubsection Inheritance
1263
1264If you've started browsing the existing package definitions, you might have
1265noticed that a significant number of them have a @code{inherit} field:
1266
d482e13f 1267@lisp
ffe059af
PN
1268(define-public adwaita-icon-theme
1269 (package (inherit gnome-icon-theme)
1270 (name "adwaita-icon-theme")
1271 (version "3.26.1")
1272 (source (origin
1273 (method url-fetch)
1274 (uri (string-append "mirror://gnome/sources/" name "/"
1275 (version-major+minor version) "/"
1276 name "-" version ".tar.xz"))
1277 (sha256
1278 (base32
1279 "17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
1280 (native-inputs
1281 `(("gtk-encode-symbolic-svg" ,gtk+ "bin")))))
d482e13f 1282@end lisp
ffe059af
PN
1283
1284All unspecified fields are inherited from the parent package. This is very
1285convenient to create alternative packages, for instance with different source,
1286version or compilation options.
1287
1288@node Getting help
1289@subsection Getting help
1290
1291Sadly, some applications can be tough to package. Sometimes they need a patch to
e892b9c3 1292work with the non-standard file system hierarchy enforced by the store.
ffe059af
PN
1293Sometimes the tests won't run properly. (They can be skipped but this is not
1294recommended.) Other times the resulting package won't be reproducible.
1295
1296Should you be stuck, unable to figure out how to fix any sort of packaging
1297issue, don't hesitate to ask the community for help.
1298
1299See the @uref{https://www.gnu.org/software/guix/contact/, Guix homepage} for information on the mailing lists, IRC, etc.
1300
1301@node Conclusion
1302@subsection Conclusion
1303
1304This tutorial was a showcase of the sophisticated package management that Guix
1305boasts. At this point we have mostly restricted this introduction to the
1306@code{gnu-build-system} which is a core abstraction layer on which more advanced
1307abstractions are based.
1308
1309Where do we go from here? Next we ought to dissect the innards of the build
1310system by removing all abstractions, using the @code{trivial-build-system}: this
1311should give us a thorough understanding of the process before investigating some
1312more advanced packaging techniques and edge cases.
1313
1314Other features worth exploring are the interactive editing and debugging
1315capabilities of Guix provided by the Guile REPL@.
1316
1317Those fancy features are completely optional and can wait; now is a good time
1318to take a well-deserved break. With what we've introduced here you should be
1319well armed to package lots of programs. You can get started right away and
1320hopefully we will see your contributions soon!
1321
1322@node References
1323@subsection References
1324
1325@itemize
1326@item
1327The @uref{https://www.gnu.org/software/guix/manual/en/html_node/Defining-Packages.html, package reference in the manual}
1328
1329@item
1330@uref{https://gitlab.com/pjotrp/guix-notes/blob/master/HACKING.org, Pjotr’s hacking guide to GNU Guix}
1331
1332@item
0cbef07b 1333@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
ffe059af 1334@end itemize
7bc46ecc
RW
1335
1336@c *********************************************************************
1337@node System Configuration
1338@chapter System Configuration
1339
1340Guix offers a flexible language for declaratively configuring your Guix
1341System. This flexibility can at times be overwhelming. The purpose of this
1342chapter is to demonstrate some advanced configuration concepts.
1343
1344@pxref{System Configuration,,, guix, GNU Guix Reference Manual} for a complete
1345reference.
1346
1347@menu
65051fdc 1348* Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
5c3e94cd 1349* Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
65051fdc 1350* Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
60651dd9 1351* Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
c987b723 1352* Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
7bc46ecc
RW
1353@end menu
1354
1355@node Customizing the Kernel
1356@section Customizing the Kernel
1357
1358Guix is, at its core, a source based distribution with substitutes
1359(@pxref{Substitutes,,, guix, GNU Guix Reference Manual}), and as such building
1360packages from their source code is an expected part of regular package
1361installations and upgrades. Given this starting point, it makes sense that
1362efforts are made to reduce the amount of time spent compiling packages, and
1363recent changes and upgrades to the building and distribution of substitutes
1364continues to be a topic of discussion within Guix.
1365
1366The kernel, while not requiring an overabundance of RAM to build, does take a
1367rather long time on an average machine. The official kernel configuration, as
1368is the case with many GNU/Linux distributions, errs on the side of
1369inclusiveness, and this is really what causes the build to take such a long
1370time when the kernel is built from source.
1371
1372The Linux kernel, however, can also just be described as a regular old
1373package, and as such can be customized just like any other package. The
1374procedure is a little bit different, although this is primarily due to the
1375nature of how the package definition is written.
1376
1377The @code{linux-libre} kernel package definition is actually a procedure which
1378creates a package.
1379
b1eecb5c 1380@lisp
7bc46ecc
RW
1381(define* (make-linux-libre version hash supported-systems
1382 #:key
1383 ;; A function that takes an arch and a variant.
1384 ;; See kernel-config for an example.
1385 (extra-version #f)
1386 (configuration-file #f)
1387 (defconfig "defconfig")
1388 (extra-options %default-extra-linux-options)
1389 (patches (list %boot-logo-patch)))
1390 ...)
b1eecb5c 1391@end lisp
7bc46ecc
RW
1392
1393The current @code{linux-libre} package is for the 5.1.x series, and is
1394declared like this:
1395
b1eecb5c 1396@lisp
7bc46ecc
RW
1397(define-public linux-libre
1398 (make-linux-libre %linux-libre-version
1399 %linux-libre-hash
1400 '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux")
1401 #:patches %linux-libre-5.1-patches
1402 #:configuration-file kernel-config))
b1eecb5c 1403@end lisp
7bc46ecc
RW
1404
1405Any keys which are not assigned values inherit their default value from the
1406@code{make-linux-libre} definition. When comparing the two snippets above,
1407you may notice that the code comment in the first doesn't actually refer to
1408the @code{#:extra-version} keyword; it is actually for
1409@code{#:configuration-file}. Because of this, it is not actually easy to
1410include a custom kernel configuration from the definition, but don't worry,
1411there are other ways to work with what we do have.
1412
1413There are two ways to create a kernel with a custom kernel configuration. The
1414first is to provide a standard @file{.config} file during the build process by
1415including an actual @file{.config} file as a native input to our custom
1416kernel. The following is a snippet from the custom @code{'configure} phase of
1417the @code{make-linux-libre} package definition:
1418
b1eecb5c 1419@lisp
7bc46ecc
RW
1420(let ((build (assoc-ref %standard-phases 'build))
1421 (config (assoc-ref (or native-inputs inputs) "kconfig")))
1422
1423 ;; Use a custom kernel configuration file or a default
1424 ;; configuration file.
1425 (if config
1426 (begin
1427 (copy-file config ".config")
1428 (chmod ".config" #o666))
1429 (invoke "make" ,defconfig))
b1eecb5c 1430@end lisp
7bc46ecc
RW
1431
1432Below is a sample kernel package. The @code{linux-libre} package is nothing
1433special and can be inherited from and have its fields overridden like any
1434other package:
1435
b1eecb5c 1436@lisp
7bc46ecc
RW
1437(define-public linux-libre/E2140
1438 (package
1439 (inherit linux-libre)
1440 (native-inputs
1441 `(("kconfig" ,(local-file "E2140.config"))
1442 ,@@(alist-delete "kconfig"
1443 (package-native-inputs linux-libre))))))
b1eecb5c 1444@end lisp
7bc46ecc
RW
1445
1446In the same directory as the file defining @code{linux-libre-E2140} is a file
1447named @file{E2140.config}, which is an actual kernel configuration file. The
1448@code{defconfig} keyword of @code{make-linux-libre} is left blank here, so the
1449only kernel configuration in the package is the one which was included in the
1450@code{native-inputs} field.
1451
1452The second way to create a custom kernel is to pass a new value to the
1453@code{extra-options} keyword of the @code{make-linux-libre} procedure. The
1454@code{extra-options} keyword works with another function defined right below
1455it:
1456
b1eecb5c 1457@lisp
7bc46ecc
RW
1458(define %default-extra-linux-options
1459 `(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
1460 ("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #t)
1461 ;; Modules required for initrd:
1462 ("CONFIG_NET_9P" . m)
1463 ("CONFIG_NET_9P_VIRTIO" . m)
1464 ("CONFIG_VIRTIO_BLK" . m)
1465 ("CONFIG_VIRTIO_NET" . m)
1466 ("CONFIG_VIRTIO_PCI" . m)
1467 ("CONFIG_VIRTIO_BALLOON" . m)
1468 ("CONFIG_VIRTIO_MMIO" . m)
1469 ("CONFIG_FUSE_FS" . m)
1470 ("CONFIG_CIFS" . m)
1471 ("CONFIG_9P_FS" . m)))
1472
1473(define (config->string options)
1474 (string-join (map (match-lambda
1475 ((option . 'm)
1476 (string-append option "=m"))
1477 ((option . #t)
1478 (string-append option "=y"))
1479 ((option . #f)
1480 (string-append option "=n")))
1481 options)
1482 "\n"))
b1eecb5c 1483@end lisp
7bc46ecc
RW
1484
1485And in the custom configure script from the `make-linux-libre` package:
1486
b1eecb5c 1487@lisp
7bc46ecc
RW
1488;; Appending works even when the option wasn't in the
1489;; file. The last one prevails if duplicated.
1490(let ((port (open-file ".config" "a"))
1491 (extra-configuration ,(config->string extra-options)))
1492 (display extra-configuration port)
1493 (close-port port))
1494
1495(invoke "make" "oldconfig"))))
b1eecb5c 1496@end lisp
7bc46ecc
RW
1497
1498So by not providing a configuration-file the @file{.config} starts blank, and
1499then we write into it the collection of flags that we want. Here's another
1500custom kernel:
1501
b1eecb5c 1502@lisp
7bc46ecc
RW
1503(define %macbook41-full-config
1504 (append %macbook41-config-options
e892b9c3 1505 %file-systems
7bc46ecc
RW
1506 %efi-support
1507 %emulation
1508 (@@@@ (gnu packages linux) %default-extra-linux-options)))
1509
1510(define-public linux-libre-macbook41
1511 ;; XXX: Access the internal 'make-linux-libre' procedure, which is
1512 ;; private and unexported, and is liable to change in the future.
1513 ((@@@@ (gnu packages linux) make-linux-libre) (@@@@ (gnu packages linux) %linux-libre-version)
1514 (@@@@ (gnu packages linux) %linux-libre-hash)
1515 '("x86_64-linux")
1516 #:extra-version "macbook41"
1517 #:patches (@@@@ (gnu packages linux) %linux-libre-5.1-patches)
1518 #:extra-options %macbook41-config-options))
b1eecb5c 1519@end lisp
7bc46ecc 1520
e892b9c3
TGR
1521In the above example @code{%file-systems} is a collection of flags enabling
1522different file system support, @code{%efi-support} enables EFI support and
7bc46ecc
RW
1523@code{%emulation} enables a x86_64-linux machine to act in 32-bit mode also.
1524@code{%default-extra-linux-options} are the ones quoted above, which had to be
1525added in since they were replaced in the @code{extra-options} keyword.
1526
1527This all sounds like it should be doable, but how does one even know which
1528modules are required for a particular system? Two places that can be helpful
1529in trying to answer this question is the
1530@uref{https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel, Gentoo
1531Handbook} and the
1532@uref{https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig,
1533documentation from the kernel itself}. From the kernel documentation, it
1534seems that @code{make localmodconfig} is the command we want.
1535
1536In order to actually run @code{make localmodconfig} we first need to get and
1537unpack the kernel source code:
1538
1539@example shell
1540tar xf $(guix build linux-libre --source)
1541@end example
1542
1543Once inside the directory containing the source code run @code{touch .config}
1544to create an initial, empty @file{.config} to start with. @code{make
1545localmodconfig} works by seeing what you already have in @file{.config} and
1546letting you know what you're missing. If the file is blank then you're
1547missing everything. The next step is to run:
1548
1549@example shell
1550guix environment linux-libre -- make localmodconfig
1551@end example
1552
1553and note the output. Do note that the @file{.config} file is still empty.
1554The output generally contains two types of warnings. The first start with
1555"WARNING" and can actually be ignored in our case. The second read:
1556
1557@example shell
1558module pcspkr did not have configs CONFIG_INPUT_PCSPKR
1559@end example
1560
1561For each of these lines, copy the @code{CONFIG_XXXX_XXXX} portion into the
1562@file{.config} in the directory, and append @code{=m}, so in the end it looks
1563like this:
1564
1565@example shell
1566CONFIG_INPUT_PCSPKR=m
1567CONFIG_VIRTIO=m
1568@end example
1569
1570After copying all the configuration options, run @code{make localmodconfig}
0cbef07b 1571again to make sure that you don't have any output starting with ``module''.
7bc46ecc
RW
1572After all of these machine specific modules there are a couple more left that
1573are also needed. @code{CONFIG_MODULES} is necessary so that you can build and
1574load modules separately and not have everything built into the kernel.
1575@code{CONFIG_BLK_DEV_SD} is required for reading from hard drives. It is
1576possible that there are other modules which you will need.
1577
1578This post does not aim to be a guide to configuring your own kernel however,
1579so if you do decide to build a custom kernel you'll have to seek out other
1580guides to create a kernel which is just right for your needs.
1581
1582The second way to setup the kernel configuration makes more use of Guix's
1583features and allows you to share configuration segments between different
1584kernels. For example, all machines using EFI to boot have a number of EFI
1585configuration flags that they need. It is likely that all the kernels will
e892b9c3 1586share a list of file systems to support. By using variables it is easier to
7bc46ecc
RW
1587see at a glance what features are enabled and to make sure you don't have
1588features in one kernel but missing in another.
1589
1590Left undiscussed however, is Guix's initrd and its customization. It is
1591likely that you'll need to modify the initrd on a machine using a custom
1592kernel, since certain modules which are expected to be built may not be
1593available for inclusion into the initrd.
1594
5c3e94cd
MK
1595@node Connecting to Wireguard VPN
1596@section Connecting to Wireguard VPN
1597
1598To connect to a Wireguard VPN server you need the kernel module to be
1599loaded in memory and a package providing networking tools that support
1600it (e.g. @code{wireguard-tools} or @code{network-manager}).
1601
1602Here is a configuration example for Linux-Libre < 5.6, where the module
1603is out of tree and need to be loaded manually---following revisions of
1604the kernel have it built-in and so don't need such configuration:
1605
1606@lisp
1607(use-modules (gnu))
1608(use-service-modules desktop)
1609(use-package-modules vpn)
1610
1611(operating-system
1612 ;; …
1613 (services (cons (simple-service 'wireguard-module
1614 kernel-module-loader-service-type
1615 '("wireguard"))
1616 %desktop-services))
1617 (packages (cons wireguard-tools %base-packages))
1618 (kernel-loadable-modules (list wireguard-linux-compat)))
1619@end lisp
1620
1621After reconfiguring and restarting your system you can either use
1622Wireguard tools or NetworkManager to connect to a VPN server.
1623
1624@subsection Using Wireguard tools
1625
1626To test your Wireguard setup it is convenient to use @command{wg-quick}.
1627Just give it a configuration file @command{wg-quick up ./wg0.conf}; or
1628put that file in @file{/etc/wireguard} and run @command{wg-quick up wg0}
1629instead.
1630
1631@quotation Note
1632Be warned that the author described this command as a: “[…] very quick
1633and dirty bash script […]”.
1634@end quotation
1635
1636@subsection Using NetworkManager
1637
1638Thanks to NetworkManager support for Wireguard we can connect to our VPN
1639using @command{nmcli} command. Up to this point this guide assumes that
1640you're using Network Manager service provided by
1641@code{%desktop-services}. Ortherwise you need to adjust your services
1642list to load @code{network-manager-service-type} and reconfigure your
1643Guix system.
1644
1645To import your VPN configuration execute nmcli import command:
1646
1647@example shell
1648# nmcli connection import type wireguard file wg0.conf
1649Connection 'wg0' (edbee261-aa5a-42db-b032-6c7757c60fde) successfully added
1650@end example
1651
1652This will create a configuration file in
1653@file{/etc/NetworkManager/wg0.nmconnection}. Next connect to the
1654Wireguard server:
1655
1656@example shell
1657$ nmcli connection up wg0
1658Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
1659@end example
1660
1661By default NetworkManager will connect automatically on system boot. To
1662change that behaviour you need to edit your config:
1663
1664@example shell
1665# nmcli connection modify wg0 connection.autoconnect no
1666@end example
1667
1668For more specific information about NetworkManager and wireguard
1669@uref{https://blogs.gnome.org/thaller/2019/03/15/wireguard-in-networkmanager/,see
1670this post by thaller}.
1671
65051fdc
OP
1672@node Customizing a Window Manager
1673@section Customizing a Window Manager
1674@cindex wm
1675
1676@node StumpWM
1677@subsection StumpWM
1678@cindex stumpwm
1679
1680You could install StumpWM with a Guix system by adding
145df67b 1681@code{stumpwm} and optionally @code{`(,stumpwm "lib")}
cc765e47 1682packages to a system configuration file, e.g.@: @file{/etc/config.scm}.
65051fdc
OP
1683
1684An example configuration can look like this:
1685
1686@lisp
1687(use-modules (gnu))
1688(use-package-modules wm)
1689
1690(operating-system
1691 ;; …
145df67b 1692 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
65051fdc
OP
1693 %base-packages)))
1694@end lisp
1695
1696@cindex stumpwm fonts
1697By default StumpWM uses X11 fonts, which could be small or pixelated on
1698your system. You could fix this by installing StumpWM contrib Lisp
751d1f01 1699module @code{sbcl-ttf-fonts}, adding it to Guix system packages:
65051fdc
OP
1700
1701@lisp
1702(use-modules (gnu))
1703(use-package-modules fonts wm)
1704
1705(operating-system
1706 ;; …
145df67b 1707 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
751d1f01 1708 sbcl-ttf-fonts font-dejavu %base-packages)))
65051fdc
OP
1709@end lisp
1710
1711Then you need to add the following code to a StumpWM configuration file
1712@file{~/.stumpwm.d/init.lisp}:
1713
1714@lisp
1715(require :ttf-fonts)
1716(setf xft:*font-dirs* '("/run/current-system/profile/share/fonts/"))
1717(setf clx-truetype:+font-cache-filename+ (concat (getenv "HOME") "/.fonts/font-cache.sexp"))
1718(xft:cache-fonts)
1719(set-font (make-instance 'xft:font :family "DejaVu Sans Mono" :subfamily "Book" :size 11))
1720@end lisp
1721
3142191d
CB
1722@node Session lock
1723@subsection Session lock
1724@cindex sessionlock
1725
1726Depending on your environment, locking the screen of your session might come built in
1727or it might be something you have to set up yourself. If you use a desktop environment
1728like GNOME or KDE, it's usually built in. If you use a plain window manager like
1729StumpWM or EXWM, you might have to set it up yourself.
1730
1731@node Xorg
1732@subsubsection Xorg
1733
1734If you use Xorg, you can use the utility
1735@uref{https://www.mankier.com/1/xss-lock, xss-lock} to lock the screen of your session.
1736xss-lock is triggered by DPMS which since Xorg 1.8 is auto-detected and enabled if
1737ACPI is also enabled at kernel runtime.
1738
1739To use xss-lock, you can simple execute it and put it into the background before
1740you start your window manager from e.g. your @file{~/.xsession}:
1741
1742@example
1743xss-lock -- slock &
1744exec stumpwm
1745@end example
1746
1747In this example, xss-lock uses @code{slock} to do the actual locking of the screen when
1748it determines it's appropriate, like when you suspend your device.
1749
1750For slock to be allowed to be a screen locker for the graphical session, it needs to
1751be made setuid-root so it can authenticate users, and it needs a PAM service. This
1752can be achieved by adding the following service to your @file{config.scm}:
1753
1754@lisp
1755(screen-locker-service slock)
1756@end lisp
1757
1758If you manually lock your screen, e.g. by directly calling slock when you want to lock
1759your screen but not suspend it, it's a good idea to notify xss-lock about this so no
1760confusion occurs. This can be done by executing @code{xset s activate} immediately
1761before you execute slock.
1762
60651dd9
MB
1763@node Setting up a bind mount
1764@section Setting up a bind mount
1765
1766To bind mount a file system, one must first set up some definitions
cc765e47 1767before the @code{operating-system} section of the system definition. In
60651dd9 1768this example we will bind mount a folder from a spinning disk drive to
2c8305d1
NG
1769@file{/tmp}, to save wear and tear on the primary SSD, without
1770dedicating an entire partition to be mounted as @file{/tmp}.
60651dd9
MB
1771
1772First, the source drive that hosts the folder we wish to bind mount
1773should be defined, so that the bind mount can depend on it.
1774
1775@lisp
1776(define source-drive ;; "source-drive" can be named anything you want.
1777 (file-system
1778 (device (uuid "UUID goes here"))
1779 (mount-point "/path-to-spinning-disk-goes-here")
1780 (type "ext4"))) ;; Make sure to set this to the appropriate type for your drive.
1781@end lisp
1782
1783The source folder must also be defined, so that guix will know it's not
1784a regular block device, but a folder.
1785@lisp
1786(define (%source-directory) "/path-to-spinning-disk-goes-here/tmp") ;; "source-directory" can be named any valid variable name.
1787@end lisp
1788
1789Finally, inside the @code{file-systems} definition, we must add the
1790mount itself.
1791
1792@lisp
1793(file-systems (cons*
1794
1795 ...<other drives omitted for clarity>...
1796
1797 source-drive ;; Must match the name you gave the source drive in the earlier definition.
1798
1799 (file-system
1800 (device (%source-directory)) ;; Make sure "source-directory" matches your earlier definition.
1801 (mount-point "/tmp")
1802 (type "none") ;; We are mounting a folder, not a partition, so this type needs to be "none"
1803 (flags '(bind-mount))
1804 (dependencies (list source-drive)) ;; Ensure "source-drive" matches what you've named the variable for the drive.
1805 )
1806
1807 ...<other drives omitted for clarity>...
1808
1809 ))
1810@end lisp
1811
c987b723
BW
1812@node Getting substitutes from Tor
1813@section Getting substitutes from Tor
1814
1815Guix daemon can use a HTTP proxy to get substitutes, here we are
1816configuring it to get them via Tor.
1817
1818@quotation Warning
1819@emph{Not all} Guix daemon's traffic will go through Tor! Only
1820HTTP/HTTPS will get proxied; FTP, Git protocol, SSH, etc connections
1821will still go through the clearnet. Again, this configuration isn't
1822foolproof some of your traffic won't get routed by Tor at all. Use it
1823at your own risk.
f8945734
AB
1824
1825Also note that the procedure described here applies only to package
1826substitution. When you update your guix distribution with
1827@command{guix pull}, you still need to use @command{torsocks} if
1828you want to route the connection to guix's git repository servers
1829through Tor.
c987b723
BW
1830@end quotation
1831
1832Guix's substitute server is available as a Onion service, if you want
f8945734 1833to use it to get your substitutes through Tor configure your system as
c987b723
BW
1834follow:
1835
1836@lisp
1837(use-modules (gnu))
1838(use-service-module base networking)
1839
1840(operating-system
1841
1842 (services
1843 (cons
1844 (service tor-service-type
1845 (tor-configuration
1846 (config-file (plain-file "tor-config"
1847 "HTTPTunnelPort 127.0.0.1:9250"))))
1848 (modify-services %base-services
1849 (guix-service-type
1850 config => (guix-configuration
1851 (inherit config)
1852 ;; ci.guix.gnu.org's Onion service
1853 (substitute-urls "https://bp7o7ckwlewr4slm.onion")
1854 (http-proxy "http://localhost:9250")))))))
1855@end lisp
1856
1857This will keep a tor process running that provides a HTTP CONNECT tunnel
1858which will be used by @command{guix-daemon}. The daemon can use other
1859protocols than HTTP(S) to get remote resources, request using those
1860protocols won't go through Tor since we are only setting a HTTP tunnel
1861here. Note that @code{substitutes-urls} is using HTTPS and not HTTP or
1862it won't work, that's a limitation of Tor's tunnel; you may want to use
1863@command{privoxy} instead to avoid such limitations.
1864
1865If you don't want to always get substitutes through Tor but using it just
1866some of the times, then skip the @code{guix-configuration}. When you
1867want to get a substitute from the Tor tunnel run:
1868
1869@example
1870sudo herd set-http-proxy guix-daemon http://localhost:9250
1871guix build --substitute-urls=https://bp7o7ckwlewr4slm.onion …
1872@end example
1873
4c463569
PN
1874@c *********************************************************************
1875@node Advanced package management
1876@chapter Advanced package management
1877
1878Guix is a functional package manager that offers many features beyond
1879what more traditional package managers can do. To the uninitiated,
1880those features might not have obvious use cases at first. The purpose
1881of this chapter is to demonstrate some advanced package management
1882concepts.
1883
1884@pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
1885reference.
1886
1887@menu
1888* Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
1889@end menu
1890
1891@node Guix Profiles in Practice
1892@section Guix Profiles in Practice
1893
1894Guix provides a very useful feature that may be quite foreign to newcomers:
1895@emph{profiles}. They are a way to group package installations together and all users
f6c27c55 1896on the same system are free to use as many profiles as they want.
4c463569
PN
1897
1898Whether you're a developer or not, you may find that multiple profiles bring you
1899great power and flexibility. While they shift the paradigm somewhat compared to
1900@emph{traditional package managers}, they are very convenient to use once you've
1901understood how to set them up.
1902
1903If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
1904kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
1905just Python software. Furthermore, profiles are self-sufficient: they capture
1906all the runtime dependencies which guarantees that all programs within a profile
1907will always work at any point in time.
1908
1909Multiple profiles have many benefits:
1910
1911@itemize
1912@item
1913Clean semantic separation of the various packages a user needs for different contexts.
1914
1915@item
1916Multiple profiles can be made available into the environment either on login
1917or within a dedicated shell.
1918
1919@item
1920Profiles can be loaded on demand. For instance, the user can use multiple
1921shells, each of them running different profiles.
1922
1923@item
1924Isolation: Programs from one profile will not use programs from the other, and
f6c27c55 1925the user can even install different versions of the same programs to the two
4c463569
PN
1926profiles without conflict.
1927
1928@item
1929Deduplication: Profiles share dependencies that happens to be the exact same.
1930This makes multiple profiles storage-efficient.
1931
1932@item
1933Reproducible: when used with declarative manifests, a profile can be fully
1934specified by the Guix commit that was active when it was set up. This means
f6c27c55
PN
1935that the exact same profile can be
1936@uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
1937set up anywhere and anytime}, with just the commit information. See the
1938section on @ref{Reproducible profiles}.
4c463569
PN
1939
1940@item
1941Easier upgrades and maintenance: Multiple profiles make it easy to keep
1942package listings at hand and make upgrades completely friction-less.
1943@end itemize
1944
1945Concretely, here follows some typical profiles:
1946
1947@itemize
1948@item
1949The dependencies of a project you are working on.
1950
1951@item
1952Your favourite programming language libraries.
1953
1954@item
1955Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
1956
1957@item
1958@TeX{}live (this one can be really useful when you need to install just one
1959package for this one document you've just received over email).
1960
1961@item
1962Games.
1963@end itemize
1964
1965Let's dive in the set up!
1966
1967@node Basic setup with manifests
1968@subsection Basic setup with manifests
1969
1970A Guix profile can be set up @emph{via} a so-called @emph{manifest specification} that looks like
1971this:
1972
b1eecb5c 1973@lisp
4c463569
PN
1974(specifications->manifest
1975 '("package-1"
1976 ;; Version 1.3 of package-2.
1977 "package-2@@1.3"
1978 ;; The "lib" output of package-3.
1979 "package-3:lib"
1980 ; ...
1981 "package-N"))
b1eecb5c 1982@end lisp
4c463569 1983
b1eecb5c 1984@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual}, for
4c463569
PN
1985the syntax details.
1986
1987We can create a manifest specification per profile and install them this way:
1988
1989@example
1990GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
1991mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
1992guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
1993@end example
1994
1995Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
1996where we will store our profiles in the rest of this article.
1997
1998Placing all your profiles in a single directory, with each profile getting its
1999own sub-directory, is somewhat cleaner. This way, each sub-directory will
128b136e
NG
2000contain all the symlinks for precisely one profile. Besides, ``looping over
2001profiles'' becomes obvious from any programming language (e.g.@: a shell script) by
4c463569
PN
2002simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
2003
2004Note that it's also possible to loop over the output of
2005
2006@example
2007guix package --list-profiles
2008@end example
2009
2c8305d1 2010although you'll probably have to filter out @file{~/.config/guix/current}.
4c463569 2011
2c8305d1 2012To enable all profiles on login, add this to your @file{~/.bash_profile} (or similar):
4c463569
PN
2013
2014@example
2015for i in $GUIX_EXTRA_PROFILES/*; do
2016 profile=$i/$(basename "$i")
2017 if [ -f "$profile"/etc/profile ]; then
2018 GUIX_PROFILE="$profile"
2019 . "$GUIX_PROFILE"/etc/profile
2020 fi
2021 unset profile
2022done
2023@end example
2024
2025Note to Guix System users: the above reflects how your default profile
2c8305d1
NG
2026@file{~/.guix-profile} is activated from @file{/etc/profile}, that latter being loaded by
2027@file{~/.bashrc} by default.
4c463569
PN
2028
2029You can obviously choose to only enable a subset of them:
2030
2031@example
2032for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
2033 profile=$i/$(basename "$i")
2034 if [ -f "$profile"/etc/profile ]; then
2035 GUIX_PROFILE="$profile"
2036 . "$GUIX_PROFILE"/etc/profile
2037 fi
2038 unset profile
2039done
2040@end example
2041
2042When a profile is off, it's straightforward to enable it for an individual shell
2043without "polluting" the rest of the user session:
2044
2045@example
2046GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
2047@end example
2048
2049The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. This file
2050contains shell code that exports the right environment variables necessary to
2051activate the software contained in the profile. It is built automatically by
2052Guix and meant to be sourced.
2053It contains the same variables you would get if you ran:
2054
2055@example
2056guix package --search-paths=prefix --profile=$my_profile"
2057@end example
2058
2059Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual})
2060for the command line options.
2061
2062To upgrade a profile, simply install the manifest again:
2063
2064@example
2065guix package -m /path/to/guix-my-project-manifest.scm -p "$GUIX_EXTRA_PROFILES"/my-project/my-project
2066@end example
2067
2068To upgrade all profiles, it's easy enough to loop over them. For instance,
2069assuming your manifest specifications are stored in
2c8305d1 2070@file{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
cc765e47 2071of the profile (e.g.@: "project1"), you could do the following in Bourne shell:
4c463569
PN
2072
2073@example
2074for profile in "$GUIX_EXTRA_PROFILES"/*; do
2075 guix package --profile="$profile" --manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
2076done
2077@end example
2078
2079Each profile has its own generations:
2080
2081@example
2082guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
2083@end example
2084
2085You can roll-back to any generation of a given profile:
2086
2087@example
2088guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
2089@end example
cb7b501d
PN
2090
2091Finally, if you want to switch to a profile without inheriting from the
2092current environment, you can activate it from an empty shell:
2093
2094@example
2095env -i $(which bash) --login --noprofile --norc
2096. my-project/etc/profile
2097@end example
4c463569
PN
2098
2099@node Required packages
2100@subsection Required packages
2101
2102Activating a profile essentially boils down to exporting a bunch of
2103environmental variables. This is the role of the @samp{etc/profile} within the
2104profile.
2105
2106@emph{Note: Only the environmental variables of the packages that consume them will
2107be set.}
2108
2109For instance, @samp{MANPATH} won't be set if there is no consumer application for man
2110pages within the profile. So if you need to transparently access man pages once
2111the profile is loaded, you've got two options:
2112
2113@itemize
2114@item
2115Either export the variable manually, e.g.
2116@example
f6c27c55 2117export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH
4c463569
PN
2118@end example
2119
2120@item
2121Or include @samp{man-db} to the profile manifest.
2122@end itemize
2123
2124The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
2125@samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
2126
2127@node Default profile
2128@subsection Default profile
2129
2c8305d1 2130What about the default profile that Guix keeps in @file{~/.guix-profile}?
4c463569
PN
2131
2132You can assign it the role you want. Typically you would install the manifest
2133of the packages you want to use all the time.
2134
128b136e 2135Alternatively, you could keep it ``manifest-less'' for throw-away packages
4c463569
PN
2136that you would just use for a couple of days.
2137This way makes it convenient to run
2138
2139@example
2140guix install package-foo
2141guix upgrade package-bar
2142@end example
2143
2144without having to specify the path to a profile.
2145
2146@node The benefits of manifests
2147@subsection The benefits of manifests
2148
2149Manifests are a convenient way to keep your package lists around and, say,
2150to synchronize them across multiple machines using a version control system.
2151
2152A common complaint about manifests is that they can be slow to install when they
2153contain large number of packages. This is especially cumbersome when you just
2154want get an upgrade for one package within a big manifest.
2155
2156This is one more reason to use multiple profiles, which happen to be just
2157perfect to break down manifests into multiple sets of semantically connected
2158packages. Using multiple, small profiles provides more flexibility and
2159usability.
2160
2161Manifests come with multiple benefits. In particular, they ease maintenance:
2162
2163@itemize
2164@item
2165When a profile is set up from a manifest, the manifest itself is
128b136e 2166self-sufficient to keep a ``package listing'' around and reinstall the profile
4c463569
PN
2167later or on a different system. For ad-hoc profiles, we would need to
2168generate a manifest specification manually and maintain the package versions
2169for the packages that don't use the default version.
2170
2171@item
2172@code{guix package --upgrade} always tries to update the packages that have
2173propagated inputs, even if there is nothing to do. Guix manifests remove this
2174problem.
2175
2176@item
2177When partially upgrading a profile, conflicts may arise (due to diverging
2178dependencies between the updated and the non-updated packages) and they can be
2179annoying to resolve manually. Manifests remove this problem altogether since
2180all packages are always upgraded at once.
2181
2182@item
2183As mentioned above, manifests allow for reproducible profiles, while the
2184imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
2185different profiles every time even when they hold the same packages. See
2186@uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
2187
2188@item
2189Manifest specifications are usable by other @samp{guix} commands. For example, you
2190can run @code{guix weather -m manifest.scm} to see how many substitutes are
2191available, which can help you decide whether you want to try upgrading today
2192or wait a while. Another example: you can run @code{guix pack -m manifest.scm} to
2193create a pack containing all the packages in the manifest (and their
2194transitive references).
2195
2196@item
2197Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
2198They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
2199@end itemize
2200
2201It's important to understand that while manifests can be used to declare
2202profiles, they are not strictly equivalent: profiles have the side effect that
128b136e 2203they ``pin'' packages in the store, which prevents them from being
4c463569
PN
2204garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
2205and ensures that they will still be available at any point in
2206the future.
2207
2208Let's take an example:
2209
2210@enumerate
2211@item
2212We have an environment for hacking on a project for which there isn't a Guix
2213package yet. We build the environment using a manifest, and then run @code{guix
2214 environment -m manifest.scm}. So far so good.
2215
2216@item
2217Many weeks pass and we have run a couple of @code{guix pull} in the mean time.
2218Maybe a dependency from our manifest has been updated; or we may have run
2219@code{guix gc} and some packages needed by our manifest have been
2220garbage-collected.
2221
2222@item
2223Eventually, we set to work on that project again, so we run @code{guix environment
2224 -m manifest.scm}. But now we have to wait for Guix to build and install
2225stuff!
2226@end enumerate
2227
2228Ideally, we could spare the rebuild time. And indeed we can, all we need is to
2229install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
2230. "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
2231hacking environment will be available at all times.
2232
2233@emph{Security warning:} While keeping old profiles around can be convenient, keep in
2234mind that outdated packages may not have received the latest security fixes.
2235
2236@node Reproducible profiles
2237@subsection Reproducible profiles
2238
2239To reproduce a profile bit-for-bit, we need two pieces of information:
2240
2241@itemize
2242@item
2243a manifest,
2244@item
2245a Guix channel specification.
2246@end itemize
2247
2248Indeed, manifests alone might not be enough: different Guix versions (or
2249different channels) can produce different outputs for a given manifest.
2250
2251You can output the Guix channel specification with @samp{guix describe
2252--format=channels}.
2253Save this to a file, say @samp{channel-specs.scm}.
2254
2255On another computer, you can use the channel specification file and the manifest
2256to reproduce the exact same profile:
2257
2258@example
2259GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
2260GUIX_EXTRA=$HOME/.guix-extra
2261
2262mkdir "$GUIX_EXTRA"/my-project
2263guix pull --channels=channel-specs.scm --profile "$GUIX_EXTRA/my-project/guix"
2264
2265mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
2266"$GUIX_EXTRA"/my-project/guix/bin/guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
2267@end example
2268
2269It's safe to delete the Guix channel profile you've just installed with the
2270channel specification, the project profile does not depend on it.
2271
d0fc1e83
OP
2272@c *********************************************************************
2273@node Environment management
2274@chapter Environment management
2275
2276Guix provides multiple tools to manage environment. This chapter
2277demonstrate such utilities.
2278
2279@menu
2280* Guix environment via direnv:: Setup Guix environment with direnv
2281@end menu
2282
2283@node Guix environment via direnv
2284@section Guix environment via direnv
2285
2286Guix provides a @samp{direnv} package, which could extend shell after
2287directory change. This tool could be used to prepare a pure Guix
2288environment.
2289
2290The following example provides a shell function for @file{~/.direnvrc}
2291file, which could be used from Guix Git repository in
2292@file{~/src/guix/.envrc} file to setup a build environment similar to
2293described in @pxref{Building from Git,,, guix, GNU Guix Reference
2294Manual}.
2295
2296Create a @file{~/.direnvrc} with a Bash code:
2297
2298@example
2299# Thanks <https://github.com/direnv/direnv/issues/73#issuecomment-152284914>
2300export_function()
2301@{
2302 local name=$1
2303 local alias_dir=$PWD/.direnv/aliases
2304 mkdir -p "$alias_dir"
2305 PATH_add "$alias_dir"
2306 local target="$alias_dir/$name"
2307 if declare -f "$name" >/dev/null; then
2308 echo "#!$SHELL" > "$target"
2309 declare -f "$name" >> "$target" 2>/dev/null
2310 # Notice that we add shell variables to the function trigger.
2311 echo "$name \$*" >> "$target"
2312 chmod +x "$target"
2313 fi
2314@}
2315
2316use_guix()
2317@{
2318 # Set GitHub token.
2319 export GUIX_GITHUB_TOKEN="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
2320
2321 # Unset 'GUIX_PACKAGE_PATH'.
2322 export GUIX_PACKAGE_PATH=""
2323
2324 # Recreate a garbage collector root.
2325 gcroots="$HOME/.config/guix/gcroots"
2326 mkdir -p "$gcroots"
2327 gcroot="$gcroots/guix"
2328 if [ -L "$gcroot" ]
2329 then
2330 rm -v "$gcroot"
2331 fi
2332
2333 # Miscellaneous packages.
2334 PACKAGES_MAINTENANCE=(
2335 direnv
2336 git
2337 git:send-email
2338 git-cal
2339 gnupg
2340 guile-colorized
2341 guile-readline
2342 less
2343 ncurses
2344 openssh
2345 xdot
2346 )
2347
2348 # Environment packages.
2349 PACKAGES=(help2man guile-sqlite3 guile-gcrypt)
2350
2351 # Thanks <https://lists.gnu.org/archive/html/guix-devel/2016-09/msg00859.html>
2352 eval "$(guix environment --search-paths --root="$gcroot" --pure guix --ad-hoc $@{PACKAGES[@@]@} $@{PACKAGES_MAINTENANCE[@@]@} "$@@")"
2353
2354 # Predefine configure flags.
2355 configure()
2356 @{
2357 ./configure --localstatedir=/var --prefix=
2358 @}
2359 export_function configure
2360
2361 # Run make and optionally build something.
2362 build()
2363 @{
2364 make -j 2
2365 if [ $# -gt 0 ]
2366 then
2367 ./pre-inst-env guix build "$@@"
2368 fi
2369 @}
2370 export_function build
2371
2372 # Predefine push Git command.
2373 push()
2374 @{
2375 git push --set-upstream origin
2376 @}
2377 export_function push
2378
2379 clear # Clean up the screen.
2380 git-cal --author='Your Name' # Show contributions calendar.
2381
2382 # Show commands help.
2383 echo "
2384build build a package or just a project if no argument provided
2385configure run ./configure with predefined parameters
2386push push to upstream Git repository
2387"
2388@}
2389@end example
2390
2391Every project containing @file{.envrc} with a string @code{use guix}
2392will have predefined environment variables and procedures.
2393
2394Run @command{direnv allow} to setup the environment for the first time.
2395
7bc46ecc
RW
2396@c *********************************************************************
2397@node Acknowledgments
2398@chapter Acknowledgments
2399
2400Guix is based on the @uref{https://nixos.org/nix/, Nix package manager},
2401which was designed and
2402implemented by Eelco Dolstra, with contributions from other people (see
2403the @file{nix/AUTHORS} file in Guix.) Nix pioneered functional package
2404management, and promoted unprecedented features, such as transactional
2405package upgrades and rollbacks, per-user profiles, and referentially
2406transparent build processes. Without this work, Guix would not exist.
2407
2408The Nix-based software distributions, Nixpkgs and NixOS, have also been
2409an inspiration for Guix.
2410
2411GNU@tie{}Guix itself is a collective work with contributions from a
2412number of people. See the @file{AUTHORS} file in Guix for more
2413information on these fine people. The @file{THANKS} file lists people
2414who have helped by reporting bugs, taking care of the infrastructure,
2415providing artwork and themes, making suggestions, and more---thank you!
2416
2417This document includes adapted sections from articles that have previously
2418been published on the Guix blog at @uref{https://guix.gnu.org/blog}.
2419
2420
2421@c *********************************************************************
2422@node GNU Free Documentation License
2423@appendix GNU Free Documentation License
2424@cindex license, GNU Free Documentation License
2425@include fdl-1.3.texi
2426
2427@c *********************************************************************
2428@node Concept Index
2429@unnumbered Concept Index
2430@printindex cp
2431
2432@bye
2433
2434@c Local Variables:
2435@c ispell-local-dictionary: "american";
2436@c End: