doc: Document 'mount-may-fail?' field.
[jackhill/guix/guix.git] / doc / guix-cookbook.texi
1 \input texinfo
2 @c -*-texinfo-*-
3
4 @c %**start of header
5 @setfilename guix-cookbook.info
6 @documentencoding UTF-8
7 @settitle GNU Guix Cookbook
8 @c %**end of header
9
10 @copying
11 Copyright @copyright{} 2019 Ricardo Wurmus@*
12 Copyright @copyright{} 2019 Efraim Flashner@*
13 Copyright @copyright{} 2019 Pierre Neidhardt@*
14 Copyright @copyright{} 2020 Oleg Pykhalov@*
15 Copyright @copyright{} 2020 Matthew Brooks@*
16 Copyright @copyright{} 2020 Marcin Karpezo@*
17 Copyright @copyright{} 2020 Brice Waegeneire@*
18 Copyright @copyright{} 2020 André Batista@*
19
20 Permission is granted to copy, distribute and/or modify this document
21 under the terms of the GNU Free Documentation License, Version 1.3 or
22 any later version published by the Free Software Foundation; with no
23 Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
24 copy of the license is included in the section entitled ``GNU Free
25 Documentation License''.
26 @end copying
27
28 @dircategory System administration
29 @direntry
30 * Guix cookbook: (guix-cookbook). Tutorials and examples for GNU Guix.
31 @end direntry
32
33 @titlepage
34 @title GNU Guix Cookbook
35 @subtitle Tutorials and examples for using the GNU Guix Functional Package Manager
36 @author The GNU Guix Developers
37
38 @page
39 @vskip 0pt plus 1filll
40
41 @insertcopying
42 @end titlepage
43
44 @contents
45
46 @c *********************************************************************
47 @node Top
48 @top GNU Guix Cookbook
49
50 This document presents tutorials and detailed examples for GNU@tie{}Guix, a
51 functional package management tool written for the GNU system. Please
52 @pxref{Top,,, guix, GNU Guix reference manual} for details about the system,
53 its API, and related concepts.
54
55 @c TRANSLATORS: You can replace the following paragraph with information on
56 @c how to join your own translation team and how to report issues with the
57 @c translation.
58 If you would like to translate this document in your native language, consider
59 joining the @uref{https://translationproject.org/domain/guix-cookbook.html,
60 Translation Project}.
61
62 @menu
63 * Scheme tutorials:: Meet your new favorite language!
64 * Packaging:: Packaging tutorials
65 * System Configuration:: Customizing the GNU System
66 * Advanced package management:: Power to the users!
67
68 * Acknowledgments:: Thanks!
69 * GNU Free Documentation License:: The license of this document.
70 * Concept Index:: Concepts.
71
72 @detailmenu
73 --- The Detailed Node Listing ---
74
75 Scheme tutorials
76
77 * A Scheme Crash Course:: Learn the basics of Scheme
78
79 Packaging
80
81 * Packaging Tutorial:: Let's add a package to Guix!
82
83 System Configuration
84
85 * Customizing the Kernel:: Creating and using a custom Linux kernel
86
87
88 @end detailmenu
89 @end menu
90
91 @c *********************************************************************
92 @node Scheme tutorials
93 @chapter Scheme tutorials
94
95 GNU@tie{}Guix is written in the general purpose programming language Scheme,
96 and many of its features can be accessed and manipulated programmatically.
97 You can use Scheme to generate package definitions, to modify them, to build
98 them, to deploy whole operating systems, etc.
99
100 Knowing the basics of how to program in Scheme will unlock many of the
101 advanced features Guix provides --- and you don't even need to be an
102 experienced programmer to use them!
103
104 Let's get started!
105
106 @node A Scheme Crash Course
107 @section A Scheme Crash Course
108
109 @cindex Scheme, crash course
110
111 Guix uses the Guile implementation of Scheme. To start playing with the
112 language, install it with @code{guix install guile} and start a
113 @dfn{REPL}---short for @uref{https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop,
114 @dfn{read-eval-print loop}}---by running @code{guile} from the command line.
115
116 Alternatively you can also run @code{guix environment --ad-hoc guile -- guile}
117 if you'd rather not have Guile installed in your user profile.
118
119 In the following examples, lines show what you would type at the REPL;
120 lines starting with ``@result{}'' show evaluation results, while lines
121 starting with ``@print{}'' show things that get printed. @xref{Using Guile
122 Interactively,,, guile, GNU Guile Reference Manual}, for more details on the
123 REPL.
124
125 @itemize
126 @item
127 Scheme syntax boils down to a tree of expressions (or @emph{s-expression} in
128 Lisp lingo). An expression can be a literal such as numbers and strings, or a
129 compound which is a parenthesized list of compounds and literals. @code{#t}
130 and @code{#f} stand for the Booleans ``true'' and ``false'', respectively.
131
132 Examples of valid expressions:
133
134 @lisp
135 "Hello World!"
136 @result{} "Hello World!"
137
138 17
139 @result{} 17
140
141 (display (string-append "Hello " "Guix" "\n"))
142 @print{} Hello Guix!
143 @result{} #<unspecified>
144 @end lisp
145
146 @item
147 This last example is a function call nested in another function call. When a
148 parenthesized expression is evaluated, the first term is the function and the
149 rest are the arguments passed to the function. Every function returns the
150 last evaluated expression as its return value.
151
152 @item
153 Anonymous functions are declared with the @code{lambda} term:
154
155 @lisp
156 (lambda (x) (* x x))
157 @result{} #<procedure 120e348 at <unknown port>:24:0 (x)>
158 @end lisp
159
160 The above procedure returns the square of its argument. Since everything is
161 an expression, the @code{lambda} expression returns an anonymous procedure,
162 which can in turn be applied to an argument:
163
164 @lisp
165 ((lambda (x) (* x x)) 3)
166 @result{} 9
167 @end lisp
168
169 @item
170 Anything can be assigned a global name with @code{define}:
171
172 @lisp
173 (define a 3)
174 (define square (lambda (x) (* x x)))
175 (square a)
176 @result{} 9
177 @end lisp
178
179 @item
180 Procedures can be defined more concisely with the following syntax:
181
182 @lisp
183 (define (square x) (* x x))
184 @end lisp
185
186 @item
187 A list structure can be created with the @code{list} procedure:
188
189 @lisp
190 (list 2 a 5 7)
191 @result{} (2 3 5 7)
192 @end lisp
193
194 @item
195 The @dfn{quote} disables evaluation of a parenthesized expression: the
196 first term is not called over the other terms (@pxref{Expression Syntax,
197 quote,, guile, GNU Guile Reference Manual}). Thus it effectively
198 returns a list of terms.
199
200 @lisp
201 '(display (string-append "Hello " "Guix" "\n"))
202 @result{} (display (string-append "Hello " "Guix" "\n"))
203
204 '(2 a 5 7)
205 @result{} (2 a 5 7)
206 @end lisp
207
208 @item
209 The @dfn{quasiquote} disables evaluation of a parenthesized expression
210 until @dfn{unquote} (a comma) re-enables it. Thus it provides us with
211 fine-grained control over what is evaluated and what is not.
212
213 @lisp
214 `(2 a 5 7 (2 ,a 5 ,(+ a 4)))
215 @result{} (2 a 5 7 (2 3 5 7))
216 @end lisp
217
218 Note that the above result is a list of mixed elements: numbers, symbols (here
219 @code{a}) and the last element is a list itself.
220
221 @item
222 Multiple variables can be named locally with @code{let} (@pxref{Local
223 Bindings,,, guile, GNU Guile Reference Manual}):
224
225 @lisp
226 (define x 10)
227 (let ((x 2)
228 (y 3))
229 (list x y))
230 @result{} (2 3)
231
232 x
233 @result{} 10
234
235 y
236 @error{} In procedure module-lookup: Unbound variable: y
237 @end lisp
238
239 Use @code{let*} to allow later variable declarations to refer to earlier
240 definitions.
241
242 @lisp
243 (let* ((x 2)
244 (y (* x 3)))
245 (list x y))
246 @result{} (2 6)
247 @end lisp
248
249 @item
250 The keyword syntax is @code{#:}; it is used to create unique identifiers.
251 @pxref{Keywords,,, guile, GNU Guile Reference Manual}.
252
253 @item
254 The percentage @code{%} is typically used for read-only global variables in
255 the build stage. Note that it is merely a convention, like @code{_} in C.
256 Scheme treats @code{%} exactly the same as any other letter.
257
258 @item
259 Modules are created with @code{define-module} (@pxref{Creating Guile
260 Modules,,, guile, GNU Guile Reference Manual}). For instance
261
262 @lisp
263 (define-module (guix build-system ruby)
264 #:use-module (guix store)
265 #:export (ruby-build
266 ruby-build-system))
267 @end lisp
268
269 defines the module @code{guix build-system ruby} which must be located in
270 @file{guix/build-system/ruby.scm} somewhere in the Guile load path. It
271 depends on the @code{(guix store)} module and it exports two variables,
272 @code{ruby-build} and @code{ruby-build-system}.
273 @end itemize
274
275 For a more detailed introduction, check out
276 @uref{http://www.troubleshooters.com/codecorn/scheme_guile/hello.htm, Scheme
277 at a Glance}, by Steve Litt.
278
279 One of the reference Scheme books is the seminal ``Structure and
280 Interpretation of Computer Programs'', by Harold Abelson and Gerald Jay
281 Sussman, with Julie Sussman. You'll find a
282 @uref{https://mitpress.mit.edu/sites/default/files/sicp/index.html, free copy
283 online}, together with
284 @uref{https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/,
285 videos of the lectures by the authors}. The book is available in Texinfo
286 format as the @code{sicp} Guix package. Go ahead, run @code{guix install
287 sicp} and start reading with @code{info sicp} (@pxref{,,, sicp, Structure and Interpretation of Computer Programs}).
288 An @uref{https://sarabander.github.io/sicp/, unofficial ebook is also
289 available}.
290
291 You'll find more books, tutorials and other resources at
292 @url{https://schemers.org/}.
293
294
295 @c *********************************************************************
296 @node Packaging
297 @chapter Packaging
298
299 @cindex packaging
300
301 This chapter is dedicated to teaching you how to add packages to the
302 collection of packages that come with GNU Guix. This involves writing package
303 definitions in Guile Scheme, organizing them in package modules, and building
304 them.
305
306 @menu
307 * Packaging Tutorial:: A tutorial on how to add packages to Guix.
308 @end menu
309
310 @node Packaging Tutorial
311 @section Packaging Tutorial
312
313 GNU Guix stands out as the @emph{hackable} package manager, mostly because it
314 uses @uref{https://www.gnu.org/software/guile/, GNU Guile}, a powerful
315 high-level programming language, one of the
316 @uref{https://en.wikipedia.org/wiki/Scheme_%28programming_language%29, Scheme}
317 dialects from the
318 @uref{https://en.wikipedia.org/wiki/Lisp_%28programming_language%29, Lisp family}.
319
320 Package definitions are also written in Scheme, which empowers Guix in some
321 very unique ways, unlike most other package managers that use shell scripts or
322 simple languages.
323
324 @itemize
325 @item
326 Use functions, structures, macros and all of Scheme expressiveness for your
327 package definitions.
328
329 @item
330 Inheritance makes it easy to customize a package by inheriting from it and
331 modifying only what is needed.
332
333 @item
334 Batch processing: the whole package collection can be parsed, filtered and
335 processed. Building a headless server with all graphical interfaces stripped
336 out? It's possible. Want to rebuild everything from source using specific
337 compiler optimization flags? Pass the @code{#:make-flags "..."} argument to
338 the list of packages. It wouldn't be a stretch to think
339 @uref{https://wiki.gentoo.org/wiki/USE_flag, Gentoo USE flags} here, but this
340 goes even further: the changes don't have to be thought out beforehand by the
341 packager, they can be @emph{programmed} by the user!
342 @end itemize
343
344 The following tutorial covers all the basics around package creation with Guix.
345 It does not assume much knowledge of the Guix system nor of the Lisp language.
346 The reader is only expected to be familiar with the command line and to have some
347 basic programming knowledge.
348
349 @node A ``Hello World'' package
350 @subsection A ``Hello World'' package
351
352 The ``Defining Packages'' section of the manual introduces the basics of Guix
353 packaging (@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}). In
354 the following section, we will partly go over those basics again.
355
356 GNU@tie{}Hello is a dummy project that serves as an idiomatic example for
357 packaging. It uses the GNU build system (@code{./configure && make && make
358 install}). Guix already provides a package definition which is a perfect
359 example to start with. You can look up its declaration with @code{guix edit
360 hello} from the command line. Let's see how it looks:
361
362 @lisp
363 (define-public hello
364 (package
365 (name "hello")
366 (version "2.10")
367 (source (origin
368 (method url-fetch)
369 (uri (string-append "mirror://gnu/hello/hello-" version
370 ".tar.gz"))
371 (sha256
372 (base32
373 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
374 (build-system gnu-build-system)
375 (synopsis "Hello, GNU world: An example GNU package")
376 (description
377 "GNU Hello prints the message \"Hello, world!\" and then exits. It
378 serves as an example of standard GNU coding practices. As such, it supports
379 command-line arguments, multiple languages, and so on.")
380 (home-page "https://www.gnu.org/software/hello/")
381 (license gpl3+)))
382 @end lisp
383
384 As you can see, most of it is rather straightforward. But let's review the
385 fields together:
386
387 @table @samp
388 @item name
389 The project name. Using Scheme conventions, we prefer to keep it
390 lower case, without underscore and using dash-separated words.
391
392 @item source
393 This field contains a description of the source code origin. The
394 @code{origin} record contains these fields:
395
396 @enumerate
397 @item The method, here @code{url-fetch} to download via HTTP/FTP, but other methods
398 exist, such as @code{git-fetch} for Git repositories.
399 @item The URI, which is typically some @code{https://} location for @code{url-fetch}. Here
400 the special `mirror://gnu` refers to a set of well known locations, all of
401 which can be used by Guix to fetch the source, should some of them fail.
402 @item The @code{sha256} checksum of the requested file. This is essential to ensure
403 the source is not corrupted. Note that Guix works with base32 strings,
404 hence the call to the @code{base32} function.
405 @end enumerate
406
407 @item build-system
408
409 This is where the power of abstraction provided by the Scheme language really
410 shines: in this case, the @code{gnu-build-system} abstracts away the famous
411 @code{./configure && make && make install} shell invocations. Other build
412 systems include the @code{trivial-build-system} which does not do anything and
413 requires from the packager to program all the build steps, the
414 @code{python-build-system}, the @code{emacs-build-system}, and many more
415 (@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
416
417 @item synopsis
418 It should be a concise summary of what the package does. For many packages a
419 tagline from the project's home page can be used as the synopsis.
420
421 @item description
422 Same as for the synopsis, it's fine to re-use the project description from the
423 homepage. Note that Guix uses Texinfo syntax.
424
425 @item home-page
426 Use HTTPS if available.
427
428 @item license
429 See @code{guix/licenses.scm} in the project source for a full list of
430 available licenses.
431 @end table
432
433 Time to build our first package! Nothing fancy here for now: we will stick to a
434 dummy @code{my-hello}, a copy of the above declaration.
435
436 As with the ritualistic ``Hello World'' taught with most programming languages,
437 this will possibly be the most ``manual'' approach. We will work out an ideal
438 setup later; for now we will go the simplest route.
439
440 Save the following to a file @file{my-hello.scm}.
441
442 @lisp
443 (use-modules (guix packages)
444 (guix download)
445 (guix build-system gnu)
446 (guix licenses))
447
448 (package
449 (name "my-hello")
450 (version "2.10")
451 (source (origin
452 (method url-fetch)
453 (uri (string-append "mirror://gnu/hello/hello-" version
454 ".tar.gz"))
455 (sha256
456 (base32
457 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
458 (build-system gnu-build-system)
459 (synopsis "Hello, Guix world: An example custom Guix package")
460 (description
461 "GNU Hello prints the message \"Hello, world!\" and then exits. It
462 serves as an example of standard GNU coding practices. As such, it supports
463 command-line arguments, multiple languages, and so on.")
464 (home-page "https://www.gnu.org/software/hello/")
465 (license gpl3+))
466 @end lisp
467
468 We will explain the extra code in a moment.
469
470 Feel free to play with the different values of the various fields. If you
471 change the source, you'll need to update the checksum. Indeed, Guix refuses to
472 build anything if the given checksum does not match the computed checksum of the
473 source code. To obtain the correct checksum of the package declaration, we
474 need to download the source, compute the sha256 checksum and convert it to
475 base32.
476
477 Thankfully, Guix can automate this task for us; all we need is to provide the
478 URI:
479
480 @c TRANSLATORS: This is example shell output.
481 @example sh
482 $ guix download mirror://gnu/hello/hello-2.10.tar.gz
483
484 Starting download of /tmp/guix-file.JLYgL7
485 From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
486 following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
487 …10.tar.gz 709KiB 2.5MiB/s 00:00 [##################] 100.0%
488 /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
489 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
490 @end example
491
492 In this specific case the output tells us which mirror was chosen.
493 If the result of the above command is not the same as in the above snippet,
494 update your @code{my-hello} declaration accordingly.
495
496 Note that GNU package tarballs come with an OpenPGP signature, so you
497 should definitely check the signature of this tarball with `gpg` to
498 authenticate it before going further:
499
500 @c TRANSLATORS: This is example shell output.
501 @example sh
502 $ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig
503
504 Starting download of /tmp/guix-file.03tFfb
505 From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
506 following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
507 ….tar.gz.sig 819B 1.2MiB/s 00:00 [##################] 100.0%
508 /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
509 0q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
510 $ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
511 gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
512 gpg: using RSA key A9553245FDE9B739
513 gpg: Good signature from "Sami Kerola <kerolasa@@iki.fi>" [unknown]
514 gpg: aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@@iki.fi>" [unknown]
515 gpg: WARNING: This key is not certified with a trusted signature!
516 gpg: There is no indication that the signature belongs to the owner.
517 Primary key fingerprint: 8ED3 96E3 7E38 D471 A005 30D3 A955 3245 FDE9 B739
518 @end example
519
520 You can then happily run
521
522 @c TRANSLATORS: Do not translate this command
523 @example sh
524 $ guix package --install-from-file=my-hello.scm
525 @end example
526
527 You should now have @code{my-hello} in your profile!
528
529 @c TRANSLATORS: Do not translate this command
530 @example sh
531 $ guix package --list-installed=my-hello
532 my-hello 2.10 out
533 /gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10
534 @end example
535
536 We've gone as far as we could without any knowledge of Scheme. Before moving
537 on to more complex packages, now is the right time to brush up on your Scheme
538 knowledge. @pxref{A Scheme Crash Course} to get up to speed.
539
540 @node Setup
541 @subsection Setup
542
543 In the rest of this chapter we will rely on some basic Scheme
544 programming knowledge. Now let's detail the different possible setups
545 for working on Guix packages.
546
547 There are several ways to set up a Guix packaging environment.
548
549 We recommend you work directly on the Guix source checkout since it makes it
550 easier for everyone to contribute to the project.
551
552 But first, let's look at other possibilities.
553
554 @node Local file
555 @subsubsection Local file
556
557 This is what we previously did with @samp{my-hello}. With the Scheme basics we've
558 covered, we are now able to explain the leading chunks. As stated in @code{guix
559 package --help}:
560
561 @example
562 -f, --install-from-file=FILE
563 install the package that the code within FILE
564 evaluates to
565 @end example
566
567 Thus the last expression @emph{must} return a package, which is the case in our
568 earlier example.
569
570 The @code{use-modules} expression tells which of the modules we need in the file.
571 Modules are a collection of values and procedures. They are commonly called
572 ``libraries'' or ``packages'' in other programming languages.
573
574 @node @samp{GUIX_PACKAGE_PATH}
575 @subsubsection @samp{GUIX_PACKAGE_PATH}
576
577 @emph{Note: Starting from Guix 0.16, the more flexible Guix @dfn{channels} are the
578 preferred way and supersede @samp{GUIX_PACKAGE_PATH}. See next section.}
579
580 It can be tedious to specify the file from the command line instead of simply
581 calling @code{guix package --install my-hello} as you would do with the official
582 packages.
583
584 Guix makes it possible to streamline the process by adding as many ``package
585 declaration directories'' as you want.
586
587 Create a directory, say @file{~./guix-packages} and add it to the @samp{GUIX_PACKAGE_PATH}
588 environment variable:
589
590 @example
591 $ mkdir ~/guix-packages
592 $ export GUIX_PACKAGE_PATH=~/guix-packages
593 @end example
594
595 To add several directories, separate them with a colon (@code{:}).
596
597 Our previous @samp{my-hello} needs some adjustments though:
598
599 @lisp
600 (define-module (my-hello)
601 #:use-module (guix licenses)
602 #:use-module (guix packages)
603 #:use-module (guix build-system gnu)
604 #:use-module (guix download))
605
606 (define-public my-hello
607 (package
608 (name "my-hello")
609 (version "2.10")
610 (source (origin
611 (method url-fetch)
612 (uri (string-append "mirror://gnu/hello/hello-" version
613 ".tar.gz"))
614 (sha256
615 (base32
616 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
617 (build-system gnu-build-system)
618 (synopsis "Hello, Guix world: An example custom Guix package")
619 (description
620 "GNU Hello prints the message \"Hello, world!\" and then exits. It
621 serves as an example of standard GNU coding practices. As such, it supports
622 command-line arguments, multiple languages, and so on.")
623 (home-page "https://www.gnu.org/software/hello/")
624 (license gpl3+)))
625 @end lisp
626
627 Note that we have assigned the package value to an exported variable name with
628 @code{define-public}. This is effectively assigning the package to the @code{my-hello}
629 variable so that it can be referenced, among other as dependency of other
630 packages.
631
632 If you use @code{guix package --install-from-file=my-hello.scm} on the above file, it
633 will fail because the last expression, @code{define-public}, does not return a
634 package. If you want to use @code{define-public} in this use-case nonetheless, make
635 sure the file ends with an evaluation of @code{my-hello}:
636
637 @lisp
638 ; ...
639 (define-public my-hello
640 ; ...
641 )
642
643 my-hello
644 @end lisp
645
646 This last example is not very typical.
647
648 Now @samp{my-hello} should be part of the package collection like all other official
649 packages. You can verify this with:
650
651 @example
652 $ guix package --show=my-hello
653 @end example
654
655 @node Guix channels
656 @subsubsection Guix channels
657
658 Guix 0.16 features channels, which is very similar to @samp{GUIX_PACKAGE_PATH} but
659 provides better integration and provenance tracking. Channels are not
660 necessarily local, they can be maintained as a public Git repository for
661 instance. Of course, several channels can be used at the same time.
662
663 @xref{Channels,,, guix, GNU Guix Reference Manual} for setup details.
664
665 @node Direct checkout hacking
666 @subsubsection Direct checkout hacking
667
668 Working directly on the Guix project is recommended: it reduces the friction
669 when the time comes to submit your changes upstream to let the community benefit
670 from your hard work!
671
672 Unlike most software distributions, the Guix repository holds in one place both
673 the tooling (including the package manager) and the package definitions. This
674 choice was made so that it would give developers the flexibility to modify the
675 API without breakage by updating all packages at the same time. This reduces
676 development inertia.
677
678 Check out the official @uref{https://git-scm.com/, Git} repository:
679
680 @example
681 $ git clone https://git.savannah.gnu.org/git/guix.git
682 @end example
683
684 In the rest of this article, we use @samp{$GUIX_CHECKOUT} to refer to the location of
685 the checkout.
686
687
688 Follow the instructions in the manual (@pxref{Contributing,,, guix, GNU Guix
689 Reference Manual}) to set up the repository environment.
690
691 Once ready, you should be able to use the package definitions from the
692 repository environment.
693
694 Feel free to edit package definitions found in @samp{$GUIX_CHECKOUT/gnu/packages}.
695
696 The @samp{$GUIX_CHECKOUT/pre-inst-env} script lets you use @samp{guix} over the package
697 collection of the repository (@pxref{Running Guix Before It Is
698 Installed,,, guix, GNU Guix Reference Manual}).
699
700 @itemize
701 @item
702 Search packages, such as Ruby:
703
704 @example
705 $ cd $GUIX_CHECKOUT
706 $ ./pre-inst-env guix package --list-available=ruby
707 ruby 1.8.7-p374 out gnu/packages/ruby.scm:119:2
708 ruby 2.1.6 out gnu/packages/ruby.scm:91:2
709 ruby 2.2.2 out gnu/packages/ruby.scm:39:2
710 @end example
711
712 @item
713 Build a package, here Ruby version 2.1:
714
715 @example
716 $ ./pre-inst-env guix build --keep-failed ruby@@2.1
717 /gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
718 @end example
719
720 @item
721 Install it to your user profile:
722
723 @example
724 $ ./pre-inst-env guix package --install ruby@@2.1
725 @end example
726
727 @item
728 Check for common mistakes:
729
730 @example
731 $ ./pre-inst-env guix lint ruby@@2.1
732 @end example
733 @end itemize
734
735 Guix strives at maintaining a high packaging standard; when contributing to the
736 Guix project, remember to
737
738 @itemize
739 @item
740 follow the coding style (@pxref{Coding Style,,, guix, GNU Guix Reference Manual}),
741 @item
742 and review the check list from the manual (@pxref{Submitting Patches,,, guix, GNU Guix Reference Manual}).
743 @end itemize
744
745 Once you are happy with the result, you are welcome to send your contribution to
746 make it part of Guix. This process is also detailed in the manual. (@pxref{Contributing,,, guix, GNU Guix Reference Manual})
747
748
749 It's a community effort so the more join in, the better Guix becomes!
750
751 @node Extended example
752 @subsection Extended example
753
754 The above ``Hello World'' example is as simple as it goes. Packages can be more
755 complex than that and Guix can handle more advanced scenarios. Let's look at
756 another, more sophisticated package (slightly modified from the source):
757
758 @lisp
759 (define-module (gnu packages version-control)
760 #:use-module ((guix licenses) #:prefix license:)
761 #:use-module (guix utils)
762 #:use-module (guix packages)
763 #:use-module (guix git-download)
764 #:use-module (guix build-system cmake)
765 #:use-module (gnu packages ssh)
766 #:use-module (gnu packages web)
767 #:use-module (gnu packages pkg-config)
768 #:use-module (gnu packages python)
769 #:use-module (gnu packages compression)
770 #:use-module (gnu packages tls))
771
772 (define-public my-libgit2
773 (let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
774 (revision "1"))
775 (package
776 (name "my-libgit2")
777 (version (git-version "0.26.6" revision commit))
778 (source (origin
779 (method git-fetch)
780 (uri (git-reference
781 (url "https://github.com/libgit2/libgit2/")
782 (commit commit)))
783 (file-name (git-file-name name version))
784 (sha256
785 (base32
786 "17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
787 (patches (search-patches "libgit2-mtime-0.patch"))
788 (modules '((guix build utils)))
789 (snippet '(begin
790 ;; Remove bundled software.
791 (delete-file-recursively "deps")
792 #t))))
793 (build-system cmake-build-system)
794 (outputs '("out" "debug"))
795 (arguments
796 `(#:tests? #t ; Run the test suite (this is the default)
797 #:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
798 #:phases
799 (modify-phases %standard-phases
800 (add-after 'unpack 'fix-hardcoded-paths
801 (lambda _
802 (substitute* "tests/repo/init.c"
803 (("#!/bin/sh") (string-append "#!" (which "sh"))))
804 (substitute* "tests/clar/fs.h"
805 (("/bin/cp") (which "cp"))
806 (("/bin/rm") (which "rm")))
807 #t))
808 ;; Run checks more verbosely.
809 (replace 'check
810 (lambda _ (invoke "./libgit2_clar" "-v" "-Q")))
811 (add-after 'unpack 'make-files-writable-for-tests
812 (lambda _ (for-each make-file-writable (find-files "." ".*")))))))
813 (inputs
814 `(("libssh2" ,libssh2)
815 ("http-parser" ,http-parser)
816 ("python" ,python-wrapper)))
817 (native-inputs
818 `(("pkg-config" ,pkg-config)))
819 (propagated-inputs
820 ;; These two libraries are in 'Requires.private' in libgit2.pc.
821 `(("openssl" ,openssl)
822 ("zlib" ,zlib)))
823 (home-page "https://libgit2.github.com/")
824 (synopsis "Library providing Git core methods")
825 (description
826 "Libgit2 is a portable, pure C implementation of the Git core methods
827 provided as a re-entrant linkable library with a solid API, allowing you to
828 write native speed custom Git applications in any language with bindings.")
829 ;; GPLv2 with linking exception
830 (license license:gpl2))))
831 @end lisp
832
833 (In those cases were you only want to tweak a few fields from a package
834 definition, you should rely on inheritance instead of copy-pasting everything.
835 See below.)
836
837 Let's discuss those fields in depth.
838
839 @subsubsection @code{git-fetch} method
840
841 Unlike the @code{url-fetch} method, @code{git-fetch} expects a @code{git-reference} which takes
842 a Git repository and a commit. The commit can be any Git reference such as
843 tags, so if the @code{version} is tagged, then it can be used directly. Sometimes
844 the tag is prefixed with a @code{v}, in which case you'd use @code{(commit (string-append
845 "v" version))}.
846
847 To ensure that the source code from the Git repository is stored in a
848 directory with a descriptive name, we use @code{(file-name (git-file-name name
849 version))}.
850
851 The @code{git-version} procedure can be used to derive the
852 version when packaging programs for a specific commit, following the
853 Guix contributor guidelines (@pxref{Version Numbers,,, guix, GNU Guix
854 Reference Manual}).
855
856 How does one obtain the @code{sha256} hash that's in there, you ask? By
857 invoking @command{guix hash} on a checkout of the desired commit, along
858 these lines:
859
860 @example
861 git clone https://github.com/libgit2/libgit2/
862 cd libgit2
863 git checkout v0.26.6
864 guix hash -rx .
865 @end example
866
867 @command{guix hash -rx} computes a SHA256 hash over the whole directory,
868 excluding the @file{.git} sub-directory (@pxref{Invoking guix hash,,,
869 guix, GNU Guix Reference Manual}).
870
871 In the future, @command{guix download} will hopefully be able to do
872 these steps for you, just like it does for regular downloads.
873
874 @subsubsection Snippets
875
876 Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching
877 the source. They are a Guix-y alternative to the traditional @file{.patch} files.
878 Because of the quote, the code in only evaluated when passed to the Guix daemon
879 for building. There can be as many snippets as needed.
880
881 Snippets might need additional Guile modules which can be imported from the
882 @code{modules} field.
883
884 @subsubsection Inputs
885
886 First, a syntactic comment: See the quasi-quote / comma syntax?
887
888 @lisp
889 (native-inputs
890 `(("pkg-config" ,pkg-config)))
891 @end lisp
892
893 is equivalent to
894
895 @lisp
896 (native-inputs
897 (list (list "pkg-config" pkg-config)))
898 @end lisp
899
900 You'll mostly see the former because it's shorter.
901
902 There are 3 different input types. In short:
903
904 @table @asis
905 @item native-inputs
906 Required for building but not runtime -- installing a package
907 through a substitute won't install these inputs.
908 @item inputs
909 Installed in the store but not in the profile, as well as being
910 present at build time.
911 @item propagated-inputs
912 Installed in the store and in the profile, as well as
913 being present at build time.
914 @end table
915
916 @xref{Package Reference,,, guix, GNU Guix Reference Manual} for more details.
917
918 The distinction between the various inputs is important: if a dependency can be
919 handled as an @emph{input} instead of a @emph{propagated input}, it should be done so, or
920 else it ``pollutes'' the user profile for no good reason.
921
922 For instance, a user installing a graphical program that depends on a
923 command line tool might only be interested in the graphical part, so there is no
924 need to force the command line tool into the user profile. The dependency is a
925 concern to the package, not to the user. @emph{Inputs} make it possible to handle
926 dependencies without bugging the user by adding undesired executable files (or
927 libraries) to their profile.
928
929 Same goes for @emph{native-inputs}: once the program is installed, build-time
930 dependencies can be safely garbage-collected.
931 It also matters when a substitute is available, in which case only the @emph{inputs}
932 and @emph{propagated inputs} will be fetched: the @emph{native inputs} are not required to
933 install a package from a substitute.
934
935 @subsubsection Outputs
936
937 Just like how a package can have multiple inputs, it can also produce multiple
938 outputs.
939
940 Each output corresponds to a separate directory in the store.
941
942 The user can choose which output to install; this is useful to save space or
943 to avoid polluting the user profile with unwanted executables or libraries.
944
945 Output separation is optional. When the @code{outputs} field is left out, the
946 default and only output (the complete package) is referred to as @code{"out"}.
947
948 Typical separate output names include @code{debug} and @code{doc}.
949
950 It's advised to separate outputs only when you've shown it's worth it: if the
951 output size is significant (compare with @code{guix size}) or in case the package is
952 modular.
953
954 @subsubsection Build system arguments
955
956 The @code{arguments} is a keyword-value list used to configure the build process.
957
958 The simplest argument @code{#:tests?} can be used to disable the test suite when
959 building the package. This is mostly useful when the package does not feature
960 any test suite. It's strongly recommended to keep the test suite on if there is
961 one.
962
963 Another common argument is @code{:make-flags}, which specifies a list of flags to
964 append when running make, as you would from the command line. For instance, the
965 following flags
966
967 @lisp
968 #:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
969 "CC=gcc")
970 @end lisp
971
972 translate into
973
974 @example
975 $ make CC=gcc prefix=/gnu/store/...-<out>
976 @end example
977
978 This sets the C compiler to @code{gcc} and the @code{prefix} variable (the installation
979 directory in Make parlance) to @code{(assoc-ref %outputs "out")}, which is a build-stage
980 global variable pointing to the destination directory in the store (something like
981 @file{/gnu/store/...-my-libgit2-20180408}).
982
983 Similarly, it's possible to set the configure flags:
984
985 @lisp
986 #:configure-flags '("-DUSE_SHA1DC=ON")
987 @end lisp
988
989 The @code{%build-inputs} variable is also generated in scope. It's an association
990 table that maps the input names to their store directories.
991
992 The @code{phases} keyword lists the sequential steps of the build system. Typically
993 phases include @code{unpack}, @code{configure}, @code{build}, @code{install} and @code{check}. To know
994 more about those phases, you need to work out the appropriate build system
995 definition in @samp{$GUIX_CHECKOUT/guix/build/gnu-build-system.scm}:
996
997 @lisp
998 (define %standard-phases
999 ;; Standard build phases, as a list of symbol/procedure pairs.
1000 (let-syntax ((phases (syntax-rules ()
1001 ((_ p ...) `((p . ,p) ...)))))
1002 (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
1003 bootstrap
1004 patch-usr-bin-file
1005 patch-source-shebangs configure patch-generated-file-shebangs
1006 build check install
1007 patch-shebangs strip
1008 validate-runpath
1009 validate-documentation-location
1010 delete-info-dir-file
1011 patch-dot-desktop-files
1012 install-license-files
1013 reset-gzip-timestamps
1014 compress-documentation)))
1015 @end lisp
1016
1017 Or from the REPL:
1018
1019 @lisp
1020 (add-to-load-path "/path/to/guix/checkout")
1021 ,use (guix build gnu-build-system)
1022 (map first %standard-phases)
1023 @result{} (set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap patch-usr-bin-file patch-source-shebangs configure patch-generated-file-shebangs build check install patch-shebangs strip validate-runpath validate-documentation-location delete-info-dir-file patch-dot-desktop-files install-license-files reset-gzip-timestamps compress-documentation)
1024 @end lisp
1025
1026 If you want to know more about what happens during those phases, consult the
1027 associated procedures.
1028
1029 For instance, as of this writing the definition of @code{unpack} for the GNU build
1030 system is
1031
1032 @lisp
1033 (define* (unpack #:key source #:allow-other-keys)
1034 "Unpack SOURCE in the working directory, and change directory within the
1035 source. When SOURCE is a directory, copy it in a sub-directory of the current
1036 working directory."
1037 (if (file-is-directory? source)
1038 (begin
1039 (mkdir "source")
1040 (chdir "source")
1041
1042 ;; Preserve timestamps (set to the Epoch) on the copied tree so that
1043 ;; things work deterministically.
1044 (copy-recursively source "."
1045 #:keep-mtime? #t))
1046 (begin
1047 (if (string-suffix? ".zip" source)
1048 (invoke "unzip" source)
1049 (invoke "tar" "xvf" source))
1050 (chdir (first-subdirectory "."))))
1051 #t)
1052 @end lisp
1053
1054 Note the @code{chdir} call: it changes the working directory to where the source was
1055 unpacked.
1056 Thus every phase following the @code{unpack} will use the source as a working
1057 directory, which is why we can directly work on the source files.
1058 That is to say, unless a later phase changes the working directory to something
1059 else.
1060
1061 We modify the list of @code{%standard-phases} of the build system with the
1062 @code{modify-phases} macro as per the list of specified modifications, which may have
1063 the following forms:
1064
1065 @itemize
1066 @item
1067 @code{(add-before PHASE NEW-PHASE PROCEDURE)}: Run @code{PROCEDURE} named @code{NEW-PHASE} before @code{PHASE}.
1068 @item
1069 @code{(add-after PHASE NEW-PHASE PROCEDURE)}: Same, but afterwards.
1070 @item
1071 @code{(replace PHASE PROCEDURE)}.
1072 @item
1073 @code{(delete PHASE)}.
1074 @end itemize
1075
1076 The @code{PROCEDURE} supports the keyword arguments @code{inputs} and @code{outputs}. Each
1077 input (whether @emph{native}, @emph{propagated} or not) and output directory is referenced
1078 by their name in those variables. Thus @code{(assoc-ref outputs "out")} is the store
1079 directory of the main output of the package. A phase procedure may look like
1080 this:
1081
1082 @lisp
1083 (lambda* (#:key inputs outputs #:allow-other-keys)
1084 (let (((bash-directory (assoc-ref inputs "bash"))
1085 (output-directory (assoc-ref outputs "out"))
1086 (doc-directory (assoc-ref outputs "doc"))
1087 ; ...
1088 #t)
1089 @end lisp
1090
1091 The procedure must return @code{#t} on success. It's brittle to rely on the return
1092 value of the last expression used to tweak the phase because there is no
1093 guarantee it would be a @code{#t}. Hence the trailing @code{#t} to ensure the right value
1094 is returned on success.
1095
1096 @subsubsection Code staging
1097
1098 The astute reader may have noticed the quasi-quote and comma syntax in the
1099 argument field. Indeed, the build code in the package declaration should not be
1100 evaluated on the client side, but only when passed to the Guix daemon. This
1101 mechanism of passing code around two running processes is called @uref{https://arxiv.org/abs/1709.00833, code staging}.
1102
1103 @subsubsection Utility functions
1104
1105 When customizing @code{phases}, we often need to write code that mimics the
1106 equivalent system invocations (@code{make}, @code{mkdir}, @code{cp}, etc.)@: commonly used during
1107 regular ``Unix-style'' installations.
1108
1109 Some like @code{chmod} are native to Guile.
1110 @xref{,,, guile, Guile reference manual} for a complete list.
1111
1112 Guix provides additional helper functions which prove especially handy in the
1113 context of package management.
1114
1115 Some of those functions can be found in
1116 @samp{$GUIX_CHECKOUT/guix/guix/build/utils.scm}. Most of them mirror the behaviour
1117 of the traditional Unix system commands:
1118
1119 @table @asis
1120 @item which
1121 Like the @samp{which} system command.
1122 @item find-files
1123 Akin to the @samp{find} system command.
1124 @item mkdir-p
1125 Like @samp{mkdir -p}, which creates all parents as needed.
1126 @item install-file
1127 Similar to @samp{install} when installing a file to a (possibly
1128 non-existing) directory. Guile has @code{copy-file} which works
1129 like @samp{cp}.
1130 @item copy-recursively
1131 Like @samp{cp -r}.
1132 @item delete-file-recursively
1133 Like @samp{rm -rf}.
1134 @item invoke
1135 Run an executable. This should be used instead of @code{system*}.
1136 @item with-directory-excursion
1137 Run the body in a different working directory,
1138 then restore the previous working directory.
1139 @item substitute*
1140 A ``@command{sed}-like'' function.
1141 @end table
1142
1143 @subsubsection Module prefix
1144
1145 The license in our last example needs a prefix: this is because of how the
1146 @code{license} module was imported in the package, as @code{#:use-module ((guix licenses)
1147 #:prefix license:)}. The Guile module import mechanism
1148 (@pxref{Using Guile Modules,,, guile, Guile reference manual})
1149 gives the user full control over namespacing: this is needed to avoid
1150 clashes between, say, the
1151 @samp{zlib} variable from @samp{licenses.scm} (a @emph{license} value) and the @samp{zlib} variable
1152 from @samp{compression.scm} (a @emph{package} value).
1153
1154 @node Other build systems
1155 @subsection Other build systems
1156
1157 What we've seen so far covers the majority of packages using a build system
1158 other than the @code{trivial-build-system}. The latter does not automate anything
1159 and leaves you to build everything manually. This can be more demanding and we
1160 won't cover it here for now, but thankfully it is rarely necessary to fall back
1161 on this system.
1162
1163 For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the
1164 process is very similar to the GNU build system except for a few specialized
1165 arguments.
1166
1167 @xref{Build Systems,,, guix, GNU Guix Reference Manual}, for more
1168 information on build systems, or check the source code in the
1169 @samp{$GUIX_CHECKOUT/guix/build} and
1170 @samp{$GUIX_CHECKOUT/guix/build-system} directories.
1171
1172 @node Programmable and automated package definition
1173 @subsection Programmable and automated package definition
1174
1175 We can't repeat it enough: having a full-fledged programming language at hand
1176 empowers us in ways that reach far beyond traditional package management.
1177
1178 Let's illustrate this with some awesome features of Guix!
1179
1180 @node Recursive importers
1181 @subsubsection Recursive importers
1182
1183 You might find some build systems good enough that there is little to do at all
1184 to write a package, to the point that it becomes repetitive and tedious after a
1185 while. A @emph{raison d'être} of computers is to replace human beings at those
1186 boring tasks. So let's tell Guix to do this for us and create the package
1187 definition of an R package from CRAN (the output is trimmed for conciseness):
1188
1189 @example
1190 $ guix import cran --recursive walrus
1191
1192 (define-public r-mc2d
1193 ; ...
1194 (license gpl2+)))
1195
1196 (define-public r-jmvcore
1197 ; ...
1198 (license gpl2+)))
1199
1200 (define-public r-wrs2
1201 ; ...
1202 (license gpl3)))
1203
1204 (define-public r-walrus
1205 (package
1206 (name "r-walrus")
1207 (version "1.0.3")
1208 (source
1209 (origin
1210 (method url-fetch)
1211 (uri (cran-uri "walrus" version))
1212 (sha256
1213 (base32
1214 "1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
1215 (build-system r-build-system)
1216 (propagated-inputs
1217 `(("r-ggplot2" ,r-ggplot2)
1218 ("r-jmvcore" ,r-jmvcore)
1219 ("r-r6" ,r-r6)
1220 ("r-wrs2" ,r-wrs2)))
1221 (home-page "https://github.com/jamovi/walrus")
1222 (synopsis "Robust Statistical Methods")
1223 (description
1224 "This package provides a toolbox of common robust statistical
1225 tests, including robust descriptives, robust t-tests, and robust ANOVA.
1226 It is also available as a module for 'jamovi' (see
1227 <https://www.jamovi.org> for more information). Walrus is based on the
1228 WRS2 package by Patrick Mair, which is in turn based on the scripts and
1229 work of Rand Wilcox. These analyses are described in depth in the book
1230 'Introduction to Robust Estimation & Hypothesis Testing'.")
1231 (license gpl3)))
1232 @end example
1233
1234 The recursive importer won't import packages for which Guix already has package
1235 definitions, except for the very first.
1236
1237 Not all applications can be packaged this way, only those relying on a select
1238 number of supported systems. Read about the full list of importers in
1239 the guix import section of the manual
1240 (@pxref{Invoking guix import,,, guix, GNU Guix Reference Manual}).
1241
1242 @node Automatic update
1243 @subsubsection Automatic update
1244
1245 Guix can be smart enough to check for updates on systems it knows. It can
1246 report outdated package definitions with
1247
1248 @example
1249 $ guix refresh hello
1250 @end example
1251
1252 In most cases, updating a package to a newer version requires little more than
1253 changing the version number and the checksum. Guix can do that automatically as
1254 well:
1255
1256 @example
1257 $ guix refresh hello --update
1258 @end example
1259
1260 @node Inheritance
1261 @subsubsection Inheritance
1262
1263 If you've started browsing the existing package definitions, you might have
1264 noticed that a significant number of them have a @code{inherit} field:
1265
1266 @lisp
1267 (define-public adwaita-icon-theme
1268 (package (inherit gnome-icon-theme)
1269 (name "adwaita-icon-theme")
1270 (version "3.26.1")
1271 (source (origin
1272 (method url-fetch)
1273 (uri (string-append "mirror://gnome/sources/" name "/"
1274 (version-major+minor version) "/"
1275 name "-" version ".tar.xz"))
1276 (sha256
1277 (base32
1278 "17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
1279 (native-inputs
1280 `(("gtk-encode-symbolic-svg" ,gtk+ "bin")))))
1281 @end lisp
1282
1283 All unspecified fields are inherited from the parent package. This is very
1284 convenient to create alternative packages, for instance with different source,
1285 version or compilation options.
1286
1287 @node Getting help
1288 @subsection Getting help
1289
1290 Sadly, some applications can be tough to package. Sometimes they need a patch to
1291 work with the non-standard file system hierarchy enforced by the store.
1292 Sometimes the tests won't run properly. (They can be skipped but this is not
1293 recommended.) Other times the resulting package won't be reproducible.
1294
1295 Should you be stuck, unable to figure out how to fix any sort of packaging
1296 issue, don't hesitate to ask the community for help.
1297
1298 See the @uref{https://www.gnu.org/software/guix/contact/, Guix homepage} for information on the mailing lists, IRC, etc.
1299
1300 @node Conclusion
1301 @subsection Conclusion
1302
1303 This tutorial was a showcase of the sophisticated package management that Guix
1304 boasts. At this point we have mostly restricted this introduction to the
1305 @code{gnu-build-system} which is a core abstraction layer on which more advanced
1306 abstractions are based.
1307
1308 Where do we go from here? Next we ought to dissect the innards of the build
1309 system by removing all abstractions, using the @code{trivial-build-system}: this
1310 should give us a thorough understanding of the process before investigating some
1311 more advanced packaging techniques and edge cases.
1312
1313 Other features worth exploring are the interactive editing and debugging
1314 capabilities of Guix provided by the Guile REPL@.
1315
1316 Those fancy features are completely optional and can wait; now is a good time
1317 to take a well-deserved break. With what we've introduced here you should be
1318 well armed to package lots of programs. You can get started right away and
1319 hopefully we will see your contributions soon!
1320
1321 @node References
1322 @subsection References
1323
1324 @itemize
1325 @item
1326 The @uref{https://www.gnu.org/software/guix/manual/en/html_node/Defining-Packages.html, package reference in the manual}
1327
1328 @item
1329 @uref{https://gitlab.com/pjotrp/guix-notes/blob/master/HACKING.org, Pjotr’s hacking guide to GNU Guix}
1330
1331 @item
1332 @uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
1333 @end itemize
1334
1335 @c *********************************************************************
1336 @node System Configuration
1337 @chapter System Configuration
1338
1339 Guix offers a flexible language for declaratively configuring your Guix
1340 System. This flexibility can at times be overwhelming. The purpose of this
1341 chapter is to demonstrate some advanced configuration concepts.
1342
1343 @pxref{System Configuration,,, guix, GNU Guix Reference Manual} for a complete
1344 reference.
1345
1346 @menu
1347 * Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
1348 * Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
1349 * Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
1350 * Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
1351 * Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
1352 @end menu
1353
1354 @node Customizing the Kernel
1355 @section Customizing the Kernel
1356
1357 Guix is, at its core, a source based distribution with substitutes
1358 (@pxref{Substitutes,,, guix, GNU Guix Reference Manual}), and as such building
1359 packages from their source code is an expected part of regular package
1360 installations and upgrades. Given this starting point, it makes sense that
1361 efforts are made to reduce the amount of time spent compiling packages, and
1362 recent changes and upgrades to the building and distribution of substitutes
1363 continues to be a topic of discussion within Guix.
1364
1365 The kernel, while not requiring an overabundance of RAM to build, does take a
1366 rather long time on an average machine. The official kernel configuration, as
1367 is the case with many GNU/Linux distributions, errs on the side of
1368 inclusiveness, and this is really what causes the build to take such a long
1369 time when the kernel is built from source.
1370
1371 The Linux kernel, however, can also just be described as a regular old
1372 package, and as such can be customized just like any other package. The
1373 procedure is a little bit different, although this is primarily due to the
1374 nature of how the package definition is written.
1375
1376 The @code{linux-libre} kernel package definition is actually a procedure which
1377 creates a package.
1378
1379 @lisp
1380 (define* (make-linux-libre version hash supported-systems
1381 #:key
1382 ;; A function that takes an arch and a variant.
1383 ;; See kernel-config for an example.
1384 (extra-version #f)
1385 (configuration-file #f)
1386 (defconfig "defconfig")
1387 (extra-options %default-extra-linux-options)
1388 (patches (list %boot-logo-patch)))
1389 ...)
1390 @end lisp
1391
1392 The current @code{linux-libre} package is for the 5.1.x series, and is
1393 declared like this:
1394
1395 @lisp
1396 (define-public linux-libre
1397 (make-linux-libre %linux-libre-version
1398 %linux-libre-hash
1399 '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux")
1400 #:patches %linux-libre-5.1-patches
1401 #:configuration-file kernel-config))
1402 @end lisp
1403
1404 Any keys which are not assigned values inherit their default value from the
1405 @code{make-linux-libre} definition. When comparing the two snippets above,
1406 you may notice that the code comment in the first doesn't actually refer to
1407 the @code{#:extra-version} keyword; it is actually for
1408 @code{#:configuration-file}. Because of this, it is not actually easy to
1409 include a custom kernel configuration from the definition, but don't worry,
1410 there are other ways to work with what we do have.
1411
1412 There are two ways to create a kernel with a custom kernel configuration. The
1413 first is to provide a standard @file{.config} file during the build process by
1414 including an actual @file{.config} file as a native input to our custom
1415 kernel. The following is a snippet from the custom @code{'configure} phase of
1416 the @code{make-linux-libre} package definition:
1417
1418 @lisp
1419 (let ((build (assoc-ref %standard-phases 'build))
1420 (config (assoc-ref (or native-inputs inputs) "kconfig")))
1421
1422 ;; Use a custom kernel configuration file or a default
1423 ;; configuration file.
1424 (if config
1425 (begin
1426 (copy-file config ".config")
1427 (chmod ".config" #o666))
1428 (invoke "make" ,defconfig))
1429 @end lisp
1430
1431 Below is a sample kernel package. The @code{linux-libre} package is nothing
1432 special and can be inherited from and have its fields overridden like any
1433 other package:
1434
1435 @lisp
1436 (define-public linux-libre/E2140
1437 (package
1438 (inherit linux-libre)
1439 (native-inputs
1440 `(("kconfig" ,(local-file "E2140.config"))
1441 ,@@(alist-delete "kconfig"
1442 (package-native-inputs linux-libre))))))
1443 @end lisp
1444
1445 In the same directory as the file defining @code{linux-libre-E2140} is a file
1446 named @file{E2140.config}, which is an actual kernel configuration file. The
1447 @code{defconfig} keyword of @code{make-linux-libre} is left blank here, so the
1448 only kernel configuration in the package is the one which was included in the
1449 @code{native-inputs} field.
1450
1451 The second way to create a custom kernel is to pass a new value to the
1452 @code{extra-options} keyword of the @code{make-linux-libre} procedure. The
1453 @code{extra-options} keyword works with another function defined right below
1454 it:
1455
1456 @lisp
1457 (define %default-extra-linux-options
1458 `(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
1459 ("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #t)
1460 ;; Modules required for initrd:
1461 ("CONFIG_NET_9P" . m)
1462 ("CONFIG_NET_9P_VIRTIO" . m)
1463 ("CONFIG_VIRTIO_BLK" . m)
1464 ("CONFIG_VIRTIO_NET" . m)
1465 ("CONFIG_VIRTIO_PCI" . m)
1466 ("CONFIG_VIRTIO_BALLOON" . m)
1467 ("CONFIG_VIRTIO_MMIO" . m)
1468 ("CONFIG_FUSE_FS" . m)
1469 ("CONFIG_CIFS" . m)
1470 ("CONFIG_9P_FS" . m)))
1471
1472 (define (config->string options)
1473 (string-join (map (match-lambda
1474 ((option . 'm)
1475 (string-append option "=m"))
1476 ((option . #t)
1477 (string-append option "=y"))
1478 ((option . #f)
1479 (string-append option "=n")))
1480 options)
1481 "\n"))
1482 @end lisp
1483
1484 And in the custom configure script from the `make-linux-libre` package:
1485
1486 @lisp
1487 ;; Appending works even when the option wasn't in the
1488 ;; file. The last one prevails if duplicated.
1489 (let ((port (open-file ".config" "a"))
1490 (extra-configuration ,(config->string extra-options)))
1491 (display extra-configuration port)
1492 (close-port port))
1493
1494 (invoke "make" "oldconfig"))))
1495 @end lisp
1496
1497 So by not providing a configuration-file the @file{.config} starts blank, and
1498 then we write into it the collection of flags that we want. Here's another
1499 custom kernel:
1500
1501 @lisp
1502 (define %macbook41-full-config
1503 (append %macbook41-config-options
1504 %file-systems
1505 %efi-support
1506 %emulation
1507 (@@@@ (gnu packages linux) %default-extra-linux-options)))
1508
1509 (define-public linux-libre-macbook41
1510 ;; XXX: Access the internal 'make-linux-libre' procedure, which is
1511 ;; private and unexported, and is liable to change in the future.
1512 ((@@@@ (gnu packages linux) make-linux-libre) (@@@@ (gnu packages linux) %linux-libre-version)
1513 (@@@@ (gnu packages linux) %linux-libre-hash)
1514 '("x86_64-linux")
1515 #:extra-version "macbook41"
1516 #:patches (@@@@ (gnu packages linux) %linux-libre-5.1-patches)
1517 #:extra-options %macbook41-config-options))
1518 @end lisp
1519
1520 In the above example @code{%file-systems} is a collection of flags enabling
1521 different file system support, @code{%efi-support} enables EFI support and
1522 @code{%emulation} enables a x86_64-linux machine to act in 32-bit mode also.
1523 @code{%default-extra-linux-options} are the ones quoted above, which had to be
1524 added in since they were replaced in the @code{extra-options} keyword.
1525
1526 This all sounds like it should be doable, but how does one even know which
1527 modules are required for a particular system? Two places that can be helpful
1528 in trying to answer this question is the
1529 @uref{https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel, Gentoo
1530 Handbook} and the
1531 @uref{https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig,
1532 documentation from the kernel itself}. From the kernel documentation, it
1533 seems that @code{make localmodconfig} is the command we want.
1534
1535 In order to actually run @code{make localmodconfig} we first need to get and
1536 unpack the kernel source code:
1537
1538 @example shell
1539 tar xf $(guix build linux-libre --source)
1540 @end example
1541
1542 Once inside the directory containing the source code run @code{touch .config}
1543 to create an initial, empty @file{.config} to start with. @code{make
1544 localmodconfig} works by seeing what you already have in @file{.config} and
1545 letting you know what you're missing. If the file is blank then you're
1546 missing everything. The next step is to run:
1547
1548 @example shell
1549 guix environment linux-libre -- make localmodconfig
1550 @end example
1551
1552 and note the output. Do note that the @file{.config} file is still empty.
1553 The output generally contains two types of warnings. The first start with
1554 "WARNING" and can actually be ignored in our case. The second read:
1555
1556 @example shell
1557 module pcspkr did not have configs CONFIG_INPUT_PCSPKR
1558 @end example
1559
1560 For each of these lines, copy the @code{CONFIG_XXXX_XXXX} portion into the
1561 @file{.config} in the directory, and append @code{=m}, so in the end it looks
1562 like this:
1563
1564 @example shell
1565 CONFIG_INPUT_PCSPKR=m
1566 CONFIG_VIRTIO=m
1567 @end example
1568
1569 After copying all the configuration options, run @code{make localmodconfig}
1570 again to make sure that you don't have any output starting with ``module''.
1571 After all of these machine specific modules there are a couple more left that
1572 are also needed. @code{CONFIG_MODULES} is necessary so that you can build and
1573 load modules separately and not have everything built into the kernel.
1574 @code{CONFIG_BLK_DEV_SD} is required for reading from hard drives. It is
1575 possible that there are other modules which you will need.
1576
1577 This post does not aim to be a guide to configuring your own kernel however,
1578 so if you do decide to build a custom kernel you'll have to seek out other
1579 guides to create a kernel which is just right for your needs.
1580
1581 The second way to setup the kernel configuration makes more use of Guix's
1582 features and allows you to share configuration segments between different
1583 kernels. For example, all machines using EFI to boot have a number of EFI
1584 configuration flags that they need. It is likely that all the kernels will
1585 share a list of file systems to support. By using variables it is easier to
1586 see at a glance what features are enabled and to make sure you don't have
1587 features in one kernel but missing in another.
1588
1589 Left undiscussed however, is Guix's initrd and its customization. It is
1590 likely that you'll need to modify the initrd on a machine using a custom
1591 kernel, since certain modules which are expected to be built may not be
1592 available for inclusion into the initrd.
1593
1594 @node Connecting to Wireguard VPN
1595 @section Connecting to Wireguard VPN
1596
1597 To connect to a Wireguard VPN server you need the kernel module to be
1598 loaded in memory and a package providing networking tools that support
1599 it (e.g. @code{wireguard-tools} or @code{network-manager}).
1600
1601 Here is a configuration example for Linux-Libre < 5.6, where the module
1602 is out of tree and need to be loaded manually---following revisions of
1603 the kernel have it built-in and so don't need such configuration:
1604
1605 @lisp
1606 (use-modules (gnu))
1607 (use-service-modules desktop)
1608 (use-package-modules vpn)
1609
1610 (operating-system
1611 ;; …
1612 (services (cons (simple-service 'wireguard-module
1613 kernel-module-loader-service-type
1614 '("wireguard"))
1615 %desktop-services))
1616 (packages (cons wireguard-tools %base-packages))
1617 (kernel-loadable-modules (list wireguard-linux-compat)))
1618 @end lisp
1619
1620 After reconfiguring and restarting your system you can either use
1621 Wireguard tools or NetworkManager to connect to a VPN server.
1622
1623 @subsection Using Wireguard tools
1624
1625 To test your Wireguard setup it is convenient to use @command{wg-quick}.
1626 Just give it a configuration file @command{wg-quick up ./wg0.conf}; or
1627 put that file in @file{/etc/wireguard} and run @command{wg-quick up wg0}
1628 instead.
1629
1630 @quotation Note
1631 Be warned that the author described this command as a: “[…] very quick
1632 and dirty bash script […]”.
1633 @end quotation
1634
1635 @subsection Using NetworkManager
1636
1637 Thanks to NetworkManager support for Wireguard we can connect to our VPN
1638 using @command{nmcli} command. Up to this point this guide assumes that
1639 you're using Network Manager service provided by
1640 @code{%desktop-services}. Ortherwise you need to adjust your services
1641 list to load @code{network-manager-service-type} and reconfigure your
1642 Guix system.
1643
1644 To import your VPN configuration execute nmcli import command:
1645
1646 @example shell
1647 # nmcli connection import type wireguard file wg0.conf
1648 Connection 'wg0' (edbee261-aa5a-42db-b032-6c7757c60fde) successfully added
1649 @end example
1650
1651 This will create a configuration file in
1652 @file{/etc/NetworkManager/wg0.nmconnection}. Next connect to the
1653 Wireguard server:
1654
1655 @example shell
1656 $ nmcli connection up wg0
1657 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
1658 @end example
1659
1660 By default NetworkManager will connect automatically on system boot. To
1661 change that behaviour you need to edit your config:
1662
1663 @example shell
1664 # nmcli connection modify wg0 connection.autoconnect no
1665 @end example
1666
1667 For more specific information about NetworkManager and wireguard
1668 @uref{https://blogs.gnome.org/thaller/2019/03/15/wireguard-in-networkmanager/,see
1669 this post by thaller}.
1670
1671 @node Customizing a Window Manager
1672 @section Customizing a Window Manager
1673 @cindex wm
1674
1675 @node StumpWM
1676 @subsection StumpWM
1677 @cindex stumpwm
1678
1679 You could install StumpWM with a Guix system by adding
1680 @code{stumpwm} and optionally @code{`(,stumpwm "lib")}
1681 packages to a system configuration file, e.g.@: @file{/etc/config.scm}.
1682
1683 An example configuration can look like this:
1684
1685 @lisp
1686 (use-modules (gnu))
1687 (use-package-modules wm)
1688
1689 (operating-system
1690 ;; …
1691 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
1692 %base-packages)))
1693 @end lisp
1694
1695 @cindex stumpwm fonts
1696 By default StumpWM uses X11 fonts, which could be small or pixelated on
1697 your system. You could fix this by installing StumpWM contrib Lisp
1698 module @code{sbcl-ttf-fonts}, adding it to Guix system packages:
1699
1700 @lisp
1701 (use-modules (gnu))
1702 (use-package-modules fonts wm)
1703
1704 (operating-system
1705 ;; …
1706 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
1707 sbcl-ttf-fonts font-dejavu %base-packages)))
1708 @end lisp
1709
1710 Then you need to add the following code to a StumpWM configuration file
1711 @file{~/.stumpwm.d/init.lisp}:
1712
1713 @lisp
1714 (require :ttf-fonts)
1715 (setf xft:*font-dirs* '("/run/current-system/profile/share/fonts/"))
1716 (setf clx-truetype:+font-cache-filename+ (concat (getenv "HOME") "/.fonts/font-cache.sexp"))
1717 (xft:cache-fonts)
1718 (set-font (make-instance 'xft:font :family "DejaVu Sans Mono" :subfamily "Book" :size 11))
1719 @end lisp
1720
1721 @node Session lock
1722 @subsection Session lock
1723 @cindex sessionlock
1724
1725 Depending on your environment, locking the screen of your session might come built in
1726 or it might be something you have to set up yourself. If you use a desktop environment
1727 like GNOME or KDE, it's usually built in. If you use a plain window manager like
1728 StumpWM or EXWM, you might have to set it up yourself.
1729
1730 @node Xorg
1731 @subsubsection Xorg
1732
1733 If you use Xorg, you can use the utility
1734 @uref{https://www.mankier.com/1/xss-lock, xss-lock} to lock the screen of your session.
1735 xss-lock is triggered by DPMS which since Xorg 1.8 is auto-detected and enabled if
1736 ACPI is also enabled at kernel runtime.
1737
1738 To use xss-lock, you can simple execute it and put it into the background before
1739 you start your window manager from e.g. your @file{~/.xsession}:
1740
1741 @example
1742 xss-lock -- slock &
1743 exec stumpwm
1744 @end example
1745
1746 In this example, xss-lock uses @code{slock} to do the actual locking of the screen when
1747 it determines it's appropriate, like when you suspend your device.
1748
1749 For slock to be allowed to be a screen locker for the graphical session, it needs to
1750 be made setuid-root so it can authenticate users, and it needs a PAM service. This
1751 can be achieved by adding the following service to your @file{config.scm}:
1752
1753 @lisp
1754 (screen-locker-service slock)
1755 @end lisp
1756
1757 If you manually lock your screen, e.g. by directly calling slock when you want to lock
1758 your screen but not suspend it, it's a good idea to notify xss-lock about this so no
1759 confusion occurs. This can be done by executing @code{xset s activate} immediately
1760 before you execute slock.
1761
1762 @node Setting up a bind mount
1763 @section Setting up a bind mount
1764
1765 To bind mount a file system, one must first set up some definitions
1766 before the @code{operating-system} section of the system definition. In
1767 this example we will bind mount a folder from a spinning disk drive to
1768 @file{/tmp}, to save wear and tear on the primary SSD, without
1769 dedicating an entire partition to be mounted as @file{/tmp}.
1770
1771 First, the source drive that hosts the folder we wish to bind mount
1772 should be defined, so that the bind mount can depend on it.
1773
1774 @lisp
1775 (define source-drive ;; "source-drive" can be named anything you want.
1776 (file-system
1777 (device (uuid "UUID goes here"))
1778 (mount-point "/path-to-spinning-disk-goes-here")
1779 (type "ext4"))) ;; Make sure to set this to the appropriate type for your drive.
1780 @end lisp
1781
1782 The source folder must also be defined, so that guix will know it's not
1783 a regular block device, but a folder.
1784 @lisp
1785 (define (%source-directory) "/path-to-spinning-disk-goes-here/tmp") ;; "source-directory" can be named any valid variable name.
1786 @end lisp
1787
1788 Finally, inside the @code{file-systems} definition, we must add the
1789 mount itself.
1790
1791 @lisp
1792 (file-systems (cons*
1793
1794 ...<other drives omitted for clarity>...
1795
1796 source-drive ;; Must match the name you gave the source drive in the earlier definition.
1797
1798 (file-system
1799 (device (%source-directory)) ;; Make sure "source-directory" matches your earlier definition.
1800 (mount-point "/tmp")
1801 (type "none") ;; We are mounting a folder, not a partition, so this type needs to be "none"
1802 (flags '(bind-mount))
1803 (dependencies (list source-drive)) ;; Ensure "source-drive" matches what you've named the variable for the drive.
1804 )
1805
1806 ...<other drives omitted for clarity>...
1807
1808 ))
1809 @end lisp
1810
1811 @node Getting substitutes from Tor
1812 @section Getting substitutes from Tor
1813
1814 Guix daemon can use a HTTP proxy to get substitutes, here we are
1815 configuring it to get them via Tor.
1816
1817 @quotation Warning
1818 @emph{Not all} Guix daemon's traffic will go through Tor! Only
1819 HTTP/HTTPS will get proxied; FTP, Git protocol, SSH, etc connections
1820 will still go through the clearnet. Again, this configuration isn't
1821 foolproof some of your traffic won't get routed by Tor at all. Use it
1822 at your own risk.
1823
1824 Also note that the procedure described here applies only to package
1825 substitution. When you update your guix distribution with
1826 @command{guix pull}, you still need to use @command{torsocks} if
1827 you want to route the connection to guix's git repository servers
1828 through Tor.
1829 @end quotation
1830
1831 Guix's substitute server is available as a Onion service, if you want
1832 to use it to get your substitutes through Tor configure your system as
1833 follow:
1834
1835 @lisp
1836 (use-modules (gnu))
1837 (use-service-module base networking)
1838
1839 (operating-system
1840
1841 (services
1842 (cons
1843 (service tor-service-type
1844 (tor-configuration
1845 (config-file (plain-file "tor-config"
1846 "HTTPTunnelPort 127.0.0.1:9250"))))
1847 (modify-services %base-services
1848 (guix-service-type
1849 config => (guix-configuration
1850 (inherit config)
1851 ;; ci.guix.gnu.org's Onion service
1852 (substitute-urls "https://bp7o7ckwlewr4slm.onion")
1853 (http-proxy "http://localhost:9250")))))))
1854 @end lisp
1855
1856 This will keep a tor process running that provides a HTTP CONNECT tunnel
1857 which will be used by @command{guix-daemon}. The daemon can use other
1858 protocols than HTTP(S) to get remote resources, request using those
1859 protocols won't go through Tor since we are only setting a HTTP tunnel
1860 here. Note that @code{substitutes-urls} is using HTTPS and not HTTP or
1861 it won't work, that's a limitation of Tor's tunnel; you may want to use
1862 @command{privoxy} instead to avoid such limitations.
1863
1864 If you don't want to always get substitutes through Tor but using it just
1865 some of the times, then skip the @code{guix-configuration}. When you
1866 want to get a substitute from the Tor tunnel run:
1867
1868 @example
1869 sudo herd set-http-proxy guix-daemon http://localhost:9250
1870 guix build --substitute-urls=https://bp7o7ckwlewr4slm.onion …
1871 @end example
1872
1873 @c *********************************************************************
1874 @node Advanced package management
1875 @chapter Advanced package management
1876
1877 Guix is a functional package manager that offers many features beyond
1878 what more traditional package managers can do. To the uninitiated,
1879 those features might not have obvious use cases at first. The purpose
1880 of this chapter is to demonstrate some advanced package management
1881 concepts.
1882
1883 @pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
1884 reference.
1885
1886 @menu
1887 * Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
1888 @end menu
1889
1890 @node Guix Profiles in Practice
1891 @section Guix Profiles in Practice
1892
1893 Guix provides a very useful feature that may be quite foreign to newcomers:
1894 @emph{profiles}. They are a way to group package installations together and all users
1895 on the same system are free to use as many profiles as they want.
1896
1897 Whether you're a developer or not, you may find that multiple profiles bring you
1898 great power and flexibility. While they shift the paradigm somewhat compared to
1899 @emph{traditional package managers}, they are very convenient to use once you've
1900 understood how to set them up.
1901
1902 If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
1903 kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
1904 just Python software. Furthermore, profiles are self-sufficient: they capture
1905 all the runtime dependencies which guarantees that all programs within a profile
1906 will always work at any point in time.
1907
1908 Multiple profiles have many benefits:
1909
1910 @itemize
1911 @item
1912 Clean semantic separation of the various packages a user needs for different contexts.
1913
1914 @item
1915 Multiple profiles can be made available into the environment either on login
1916 or within a dedicated shell.
1917
1918 @item
1919 Profiles can be loaded on demand. For instance, the user can use multiple
1920 shells, each of them running different profiles.
1921
1922 @item
1923 Isolation: Programs from one profile will not use programs from the other, and
1924 the user can even install different versions of the same programs to the two
1925 profiles without conflict.
1926
1927 @item
1928 Deduplication: Profiles share dependencies that happens to be the exact same.
1929 This makes multiple profiles storage-efficient.
1930
1931 @item
1932 Reproducible: when used with declarative manifests, a profile can be fully
1933 specified by the Guix commit that was active when it was set up. This means
1934 that the exact same profile can be
1935 @uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
1936 set up anywhere and anytime}, with just the commit information. See the
1937 section on @ref{Reproducible profiles}.
1938
1939 @item
1940 Easier upgrades and maintenance: Multiple profiles make it easy to keep
1941 package listings at hand and make upgrades completely friction-less.
1942 @end itemize
1943
1944 Concretely, here follows some typical profiles:
1945
1946 @itemize
1947 @item
1948 The dependencies of a project you are working on.
1949
1950 @item
1951 Your favourite programming language libraries.
1952
1953 @item
1954 Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
1955
1956 @item
1957 @TeX{}live (this one can be really useful when you need to install just one
1958 package for this one document you've just received over email).
1959
1960 @item
1961 Games.
1962 @end itemize
1963
1964 Let's dive in the set up!
1965
1966 @node Basic setup with manifests
1967 @subsection Basic setup with manifests
1968
1969 A Guix profile can be set up @emph{via} a so-called @emph{manifest specification} that looks like
1970 this:
1971
1972 @lisp
1973 (specifications->manifest
1974 '("package-1"
1975 ;; Version 1.3 of package-2.
1976 "package-2@@1.3"
1977 ;; The "lib" output of package-3.
1978 "package-3:lib"
1979 ; ...
1980 "package-N"))
1981 @end lisp
1982
1983 @pxref{Invoking guix package,,, guix, GNU Guix Reference Manual}, for
1984 the syntax details.
1985
1986 We can create a manifest specification per profile and install them this way:
1987
1988 @example
1989 GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
1990 mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
1991 guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
1992 @end example
1993
1994 Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
1995 where we will store our profiles in the rest of this article.
1996
1997 Placing all your profiles in a single directory, with each profile getting its
1998 own sub-directory, is somewhat cleaner. This way, each sub-directory will
1999 contain all the symlinks for precisely one profile. Besides, ``looping over
2000 profiles'' becomes obvious from any programming language (e.g.@: a shell script) by
2001 simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
2002
2003 Note that it's also possible to loop over the output of
2004
2005 @example
2006 guix package --list-profiles
2007 @end example
2008
2009 although you'll probably have to filter out @file{~/.config/guix/current}.
2010
2011 To enable all profiles on login, add this to your @file{~/.bash_profile} (or similar):
2012
2013 @example
2014 for i in $GUIX_EXTRA_PROFILES/*; do
2015 profile=$i/$(basename "$i")
2016 if [ -f "$profile"/etc/profile ]; then
2017 GUIX_PROFILE="$profile"
2018 . "$GUIX_PROFILE"/etc/profile
2019 fi
2020 unset profile
2021 done
2022 @end example
2023
2024 Note to Guix System users: the above reflects how your default profile
2025 @file{~/.guix-profile} is activated from @file{/etc/profile}, that latter being loaded by
2026 @file{~/.bashrc} by default.
2027
2028 You can obviously choose to only enable a subset of them:
2029
2030 @example
2031 for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
2032 profile=$i/$(basename "$i")
2033 if [ -f "$profile"/etc/profile ]; then
2034 GUIX_PROFILE="$profile"
2035 . "$GUIX_PROFILE"/etc/profile
2036 fi
2037 unset profile
2038 done
2039 @end example
2040
2041 When a profile is off, it's straightforward to enable it for an individual shell
2042 without "polluting" the rest of the user session:
2043
2044 @example
2045 GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
2046 @end example
2047
2048 The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. This file
2049 contains shell code that exports the right environment variables necessary to
2050 activate the software contained in the profile. It is built automatically by
2051 Guix and meant to be sourced.
2052 It contains the same variables you would get if you ran:
2053
2054 @example
2055 guix package --search-paths=prefix --profile=$my_profile"
2056 @end example
2057
2058 Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual})
2059 for the command line options.
2060
2061 To upgrade a profile, simply install the manifest again:
2062
2063 @example
2064 guix package -m /path/to/guix-my-project-manifest.scm -p "$GUIX_EXTRA_PROFILES"/my-project/my-project
2065 @end example
2066
2067 To upgrade all profiles, it's easy enough to loop over them. For instance,
2068 assuming your manifest specifications are stored in
2069 @file{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
2070 of the profile (e.g.@: "project1"), you could do the following in Bourne shell:
2071
2072 @example
2073 for profile in "$GUIX_EXTRA_PROFILES"/*; do
2074 guix package --profile="$profile" --manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
2075 done
2076 @end example
2077
2078 Each profile has its own generations:
2079
2080 @example
2081 guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
2082 @end example
2083
2084 You can roll-back to any generation of a given profile:
2085
2086 @example
2087 guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
2088 @end example
2089
2090 Finally, if you want to switch to a profile without inheriting from the
2091 current environment, you can activate it from an empty shell:
2092
2093 @example
2094 env -i $(which bash) --login --noprofile --norc
2095 . my-project/etc/profile
2096 @end example
2097
2098 @node Required packages
2099 @subsection Required packages
2100
2101 Activating a profile essentially boils down to exporting a bunch of
2102 environmental variables. This is the role of the @samp{etc/profile} within the
2103 profile.
2104
2105 @emph{Note: Only the environmental variables of the packages that consume them will
2106 be set.}
2107
2108 For instance, @samp{MANPATH} won't be set if there is no consumer application for man
2109 pages within the profile. So if you need to transparently access man pages once
2110 the profile is loaded, you've got two options:
2111
2112 @itemize
2113 @item
2114 Either export the variable manually, e.g.
2115 @example
2116 export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH
2117 @end example
2118
2119 @item
2120 Or include @samp{man-db} to the profile manifest.
2121 @end itemize
2122
2123 The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
2124 @samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
2125
2126 @node Default profile
2127 @subsection Default profile
2128
2129 What about the default profile that Guix keeps in @file{~/.guix-profile}?
2130
2131 You can assign it the role you want. Typically you would install the manifest
2132 of the packages you want to use all the time.
2133
2134 Alternatively, you could keep it ``manifest-less'' for throw-away packages
2135 that you would just use for a couple of days.
2136 This way makes it convenient to run
2137
2138 @example
2139 guix install package-foo
2140 guix upgrade package-bar
2141 @end example
2142
2143 without having to specify the path to a profile.
2144
2145 @node The benefits of manifests
2146 @subsection The benefits of manifests
2147
2148 Manifests are a convenient way to keep your package lists around and, say,
2149 to synchronize them across multiple machines using a version control system.
2150
2151 A common complaint about manifests is that they can be slow to install when they
2152 contain large number of packages. This is especially cumbersome when you just
2153 want get an upgrade for one package within a big manifest.
2154
2155 This is one more reason to use multiple profiles, which happen to be just
2156 perfect to break down manifests into multiple sets of semantically connected
2157 packages. Using multiple, small profiles provides more flexibility and
2158 usability.
2159
2160 Manifests come with multiple benefits. In particular, they ease maintenance:
2161
2162 @itemize
2163 @item
2164 When a profile is set up from a manifest, the manifest itself is
2165 self-sufficient to keep a ``package listing'' around and reinstall the profile
2166 later or on a different system. For ad-hoc profiles, we would need to
2167 generate a manifest specification manually and maintain the package versions
2168 for the packages that don't use the default version.
2169
2170 @item
2171 @code{guix package --upgrade} always tries to update the packages that have
2172 propagated inputs, even if there is nothing to do. Guix manifests remove this
2173 problem.
2174
2175 @item
2176 When partially upgrading a profile, conflicts may arise (due to diverging
2177 dependencies between the updated and the non-updated packages) and they can be
2178 annoying to resolve manually. Manifests remove this problem altogether since
2179 all packages are always upgraded at once.
2180
2181 @item
2182 As mentioned above, manifests allow for reproducible profiles, while the
2183 imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
2184 different profiles every time even when they hold the same packages. See
2185 @uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
2186
2187 @item
2188 Manifest specifications are usable by other @samp{guix} commands. For example, you
2189 can run @code{guix weather -m manifest.scm} to see how many substitutes are
2190 available, which can help you decide whether you want to try upgrading today
2191 or wait a while. Another example: you can run @code{guix pack -m manifest.scm} to
2192 create a pack containing all the packages in the manifest (and their
2193 transitive references).
2194
2195 @item
2196 Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
2197 They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
2198 @end itemize
2199
2200 It's important to understand that while manifests can be used to declare
2201 profiles, they are not strictly equivalent: profiles have the side effect that
2202 they ``pin'' packages in the store, which prevents them from being
2203 garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
2204 and ensures that they will still be available at any point in
2205 the future.
2206
2207 Let's take an example:
2208
2209 @enumerate
2210 @item
2211 We have an environment for hacking on a project for which there isn't a Guix
2212 package yet. We build the environment using a manifest, and then run @code{guix
2213 environment -m manifest.scm}. So far so good.
2214
2215 @item
2216 Many weeks pass and we have run a couple of @code{guix pull} in the mean time.
2217 Maybe a dependency from our manifest has been updated; or we may have run
2218 @code{guix gc} and some packages needed by our manifest have been
2219 garbage-collected.
2220
2221 @item
2222 Eventually, we set to work on that project again, so we run @code{guix environment
2223 -m manifest.scm}. But now we have to wait for Guix to build and install
2224 stuff!
2225 @end enumerate
2226
2227 Ideally, we could spare the rebuild time. And indeed we can, all we need is to
2228 install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
2229 . "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
2230 hacking environment will be available at all times.
2231
2232 @emph{Security warning:} While keeping old profiles around can be convenient, keep in
2233 mind that outdated packages may not have received the latest security fixes.
2234
2235 @node Reproducible profiles
2236 @subsection Reproducible profiles
2237
2238 To reproduce a profile bit-for-bit, we need two pieces of information:
2239
2240 @itemize
2241 @item
2242 a manifest,
2243 @item
2244 a Guix channel specification.
2245 @end itemize
2246
2247 Indeed, manifests alone might not be enough: different Guix versions (or
2248 different channels) can produce different outputs for a given manifest.
2249
2250 You can output the Guix channel specification with @samp{guix describe
2251 --format=channels}.
2252 Save this to a file, say @samp{channel-specs.scm}.
2253
2254 On another computer, you can use the channel specification file and the manifest
2255 to reproduce the exact same profile:
2256
2257 @example
2258 GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
2259 GUIX_EXTRA=$HOME/.guix-extra
2260
2261 mkdir "$GUIX_EXTRA"/my-project
2262 guix pull --channels=channel-specs.scm --profile "$GUIX_EXTRA/my-project/guix"
2263
2264 mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
2265 "$GUIX_EXTRA"/my-project/guix/bin/guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
2266 @end example
2267
2268 It's safe to delete the Guix channel profile you've just installed with the
2269 channel specification, the project profile does not depend on it.
2270
2271 @c *********************************************************************
2272 @node Acknowledgments
2273 @chapter Acknowledgments
2274
2275 Guix is based on the @uref{https://nixos.org/nix/, Nix package manager},
2276 which was designed and
2277 implemented by Eelco Dolstra, with contributions from other people (see
2278 the @file{nix/AUTHORS} file in Guix.) Nix pioneered functional package
2279 management, and promoted unprecedented features, such as transactional
2280 package upgrades and rollbacks, per-user profiles, and referentially
2281 transparent build processes. Without this work, Guix would not exist.
2282
2283 The Nix-based software distributions, Nixpkgs and NixOS, have also been
2284 an inspiration for Guix.
2285
2286 GNU@tie{}Guix itself is a collective work with contributions from a
2287 number of people. See the @file{AUTHORS} file in Guix for more
2288 information on these fine people. The @file{THANKS} file lists people
2289 who have helped by reporting bugs, taking care of the infrastructure,
2290 providing artwork and themes, making suggestions, and more---thank you!
2291
2292 This document includes adapted sections from articles that have previously
2293 been published on the Guix blog at @uref{https://guix.gnu.org/blog}.
2294
2295
2296 @c *********************************************************************
2297 @node GNU Free Documentation License
2298 @appendix GNU Free Documentation License
2299 @cindex license, GNU Free Documentation License
2300 @include fdl-1.3.texi
2301
2302 @c *********************************************************************
2303 @node Concept Index
2304 @unnumbered Concept Index
2305 @printindex cp
2306
2307 @bye
2308
2309 @c Local Variables:
2310 @c ispell-local-dictionary: "american";
2311 @c End: