doc: cookbook: Add entry about getting substitutes through Tor.
[jackhill/guix/guix.git] / doc / guix-cookbook.texi
1 \input texinfo
2 @c -*-texinfo-*-
3
4 @c %**start of header
5 @setfilename guix-cookbook.info
6 @documentencoding UTF-8
7 @settitle GNU Guix Cookbook
8 @c %**end of header
9
10 @copying
11 Copyright @copyright{} 2019 Ricardo Wurmus@*
12 Copyright @copyright{} 2019 Efraim Flashner@*
13 Copyright @copyright{} 2019 Pierre Neidhardt@*
14 Copyright @copyright{} 2020 Oleg Pykhalov@*
15 Copyright @copyright{} 2020 Matthew Brooks@*
16 Copyright @copyright{} 2020 Marcin Karpezo@*
17 Copyright @copyright{} 2020 Brice Waegeneire@*
18
19 Permission is granted to copy, distribute and/or modify this document
20 under the terms of the GNU Free Documentation License, Version 1.3 or
21 any later version published by the Free Software Foundation; with no
22 Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
23 copy of the license is included in the section entitled ``GNU Free
24 Documentation License''.
25 @end copying
26
27 @dircategory System administration
28 @direntry
29 * Guix cookbook: (guix-cookbook). Tutorials and examples for GNU Guix.
30 @end direntry
31
32 @titlepage
33 @title GNU Guix Cookbook
34 @subtitle Tutorials and examples for using the GNU Guix Functional Package Manager
35 @author The GNU Guix Developers
36
37 @page
38 @vskip 0pt plus 1filll
39
40 @insertcopying
41 @end titlepage
42
43 @contents
44
45 @c *********************************************************************
46 @node Top
47 @top GNU Guix Cookbook
48
49 This document presents tutorials and detailed examples for GNU@tie{}Guix, a
50 functional package management tool written for the GNU system. Please
51 @pxref{Top,,, guix, GNU Guix reference manual} for details about the system,
52 its API, and related concepts.
53
54 @c TRANSLATORS: You can replace the following paragraph with information on
55 @c how to join your own translation team and how to report issues with the
56 @c translation.
57 If you would like to translate this document in your native language, consider
58 joining the @uref{https://translationproject.org/domain/guix-cookbook.html,
59 Translation Project}.
60
61 @menu
62 * Scheme tutorials:: Meet your new favorite language!
63 * Packaging:: Packaging tutorials
64 * System Configuration:: Customizing the GNU System
65 * Advanced package management:: Power to the users!
66
67 * Acknowledgments:: Thanks!
68 * GNU Free Documentation License:: The license of this document.
69 * Concept Index:: Concepts.
70
71 @detailmenu
72 --- The Detailed Node Listing ---
73
74 Scheme tutorials
75
76 * A Scheme Crash Course:: Learn the basics of Scheme
77
78 Packaging
79
80 * Packaging Tutorial:: Let's add a package to Guix!
81
82 System Configuration
83
84 * Customizing the Kernel:: Creating and using a custom Linux kernel
85
86
87 @end detailmenu
88 @end menu
89
90 @c *********************************************************************
91 @node Scheme tutorials
92 @chapter Scheme tutorials
93
94 GNU@tie{}Guix is written in the general purpose programming language Scheme,
95 and many of its features can be accessed and manipulated programmatically.
96 You can use Scheme to generate package definitions, to modify them, to build
97 them, to deploy whole operating systems, etc.
98
99 Knowing the basics of how to program in Scheme will unlock many of the
100 advanced features Guix provides --- and you don't even need to be an
101 experienced programmer to use them!
102
103 Let's get started!
104
105 @node A Scheme Crash Course
106 @section A Scheme Crash Course
107
108 @cindex Scheme, crash course
109
110 Guix uses the Guile implementation of Scheme. To start playing with the
111 language, install it with @code{guix install guile} and start a
112 @uref{https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop,
113 REPL} by running @code{guile} from the command line.
114
115 Alternatively you can also run @code{guix environment --ad-hoc guile -- guile}
116 if you'd rather not have Guile installed in your user profile.
117
118 In the following examples, lines show what you would type at the REPL;
119 lines starting with ``@result{}'' show evaluation results, while lines
120 starting with ``@print{}'' show things that get printed. @xref{Using Guile
121 Interactively,,, guile, GNU Guile Reference Manual}), for more details on the
122 REPL.
123
124 @itemize
125 @item
126 Scheme syntax boils down to a tree of expressions (or @emph{s-expression} in
127 Lisp lingo). An expression can be a literal such as numbers and strings, or a
128 compound which is a parenthesized list of compounds and literals. @code{#t}
129 and @code{#f} stand for the Booleans ``true'' and ``false'', respectively.
130
131 Examples of valid expressions:
132
133 @lisp
134 "Hello World!"
135 @result{} "Hello World!"
136
137 17
138 @result{} 17
139
140 (display (string-append "Hello " "Guix" "\n"))
141 @print{} Hello Guix!
142 @result{} #<unspecified>
143 @end lisp
144
145 @item
146 This last example is a function call nested in another function call. When a
147 parenthesized expression is evaluated, the first term is the function and the
148 rest are the arguments passed to the function. Every function returns the
149 last evaluated expression as its return value.
150
151 @item
152 Anonymous functions are declared with the @code{lambda} term:
153
154 @lisp
155 (lambda (x) (* x x))
156 @result{} #<procedure 120e348 at <unknown port>:24:0 (x)>
157 @end lisp
158
159 The above procedure returns the square of its argument. Since everything is
160 an expression, the @code{lambda} expression returns an anonymous procedure,
161 which can in turn be applied to an argument:
162
163 @lisp
164 ((lambda (x) (* x x)) 3)
165 @result{} 9
166 @end lisp
167
168 @item
169 Anything can be assigned a global name with @code{define}:
170
171 @lisp
172 (define a 3)
173 (define square (lambda (x) (* x x)))
174 (square a)
175 @result{} 9
176 @end lisp
177
178 @item
179 Procedures can be defined more concisely with the following syntax:
180
181 @lisp
182 (define (square x) (* x x))
183 @end lisp
184
185 @item
186 A list structure can be created with the @code{list} procedure:
187
188 @lisp
189 (list 2 a 5 7)
190 @result{} (2 3 5 7)
191 @end lisp
192
193 @item
194 The @dfn{quote} disables evaluation of a parenthesized expression: the
195 first term is not called over the other terms (@pxref{Expression Syntax,
196 quote,, guile, GNU Guile Reference Manual}). Thus it effectively
197 returns a list of terms.
198
199 @lisp
200 '(display (string-append "Hello " "Guix" "\n"))
201 @result{} (display (string-append "Hello " "Guix" "\n"))
202
203 '(2 a 5 7)
204 @result{} (2 a 5 7)
205 @end lisp
206
207 @item
208 The @dfn{quasiquote} disables evaluation of a parenthesized expression
209 until @dfn{unquote} (a comma) re-enables it. Thus it provides us with
210 fine-grained control over what is evaluated and what is not.
211
212 @lisp
213 `(2 a 5 7 (2 ,a 5 ,(+ a 4)))
214 @result{} (2 a 5 7 (2 3 5 7))
215 @end lisp
216
217 Note that the above result is a list of mixed elements: numbers, symbols (here
218 @code{a}) and the last element is a list itself.
219
220 @item
221 Multiple variables can be named locally with @code{let} (@pxref{Local
222 Bindings,,, guile, GNU Guile Reference Manual}):
223
224 @lisp
225 (define x 10)
226 (let ((x 2)
227 (y 3))
228 (list x y))
229 @result{} (2 3)
230
231 x
232 @result{} 10
233
234 y
235 @error{} In procedure module-lookup: Unbound variable: y
236 @end lisp
237
238 Use @code{let*} to allow later variable declarations to refer to earlier
239 definitions.
240
241 @lisp
242 (let* ((x 2)
243 (y (* x 3)))
244 (list x y))
245 @result{} (2 6)
246 @end lisp
247
248 @item
249 The keyword syntax is @code{#:}; it is used to create unique identifiers.
250 @pxref{Keywords,,, guile, GNU Guile Reference Manual}.
251
252 @item
253 The percentage @code{%} is typically used for read-only global variables in
254 the build stage. Note that it is merely a convention, like @code{_} in C.
255 Scheme treats @code{%} exactly the same as any other letter.
256
257 @item
258 Modules are created with @code{define-module} (@pxref{Creating Guile
259 Modules,,, guile, GNU Guile Reference Manual}). For instance
260
261 @lisp
262 (define-module (guix build-system ruby)
263 #:use-module (guix store)
264 #:export (ruby-build
265 ruby-build-system))
266 @end lisp
267
268 defines the module @code{guix build-system ruby} which must be located in
269 @file{guix/build-system/ruby.scm} somewhere in the Guile load path. It
270 depends on the @code{(guix store)} module and it exports two variables,
271 @code{ruby-build} and @code{ruby-build-system}.
272 @end itemize
273
274 For a more detailed introduction, check out
275 @uref{http://www.troubleshooters.com/codecorn/scheme_guile/hello.htm, Scheme
276 at a Glance}, by Steve Litt.
277
278 One of the reference Scheme books is the seminal ``Structure and
279 Interpretation of Computer Programs'', by Harold Abelson and Gerald Jay
280 Sussman, with Julie Sussman. You'll find a
281 @uref{https://mitpress.mit.edu/sites/default/files/sicp/index.html, free copy
282 online}, together with
283 @uref{https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/,
284 videos of the lectures by the authors}. The book is available in Texinfo
285 format as the @code{sicp} Guix package. Go ahead, run @code{guix install
286 sicp} and start reading with @code{info sicp} (@pxref{,,, sicp, Structure and Interpretation of Computer Programs}).
287 An @uref{https://sarabander.github.io/sicp/, unofficial ebook is also
288 available}.
289
290 You'll find more books, tutorials and other resources at
291 @url{https://schemers.org/}.
292
293
294 @c *********************************************************************
295 @node Packaging
296 @chapter Packaging
297
298 @cindex packaging
299
300 This chapter is dedicated to teaching you how to add packages to the
301 collection of packages that come with GNU Guix. This involves writing package
302 definitions in Guile Scheme, organizing them in package modules, and building
303 them.
304
305 @menu
306 * Packaging Tutorial:: A tutorial on how to add packages to Guix.
307 @end menu
308
309 @node Packaging Tutorial
310 @section Packaging Tutorial
311
312 GNU Guix stands out as the @emph{hackable} package manager, mostly because it
313 uses @uref{https://www.gnu.org/software/guile/, GNU Guile}, a powerful
314 high-level programming language, one of the
315 @uref{https://en.wikipedia.org/wiki/Scheme_%28programming_language%29, Scheme}
316 dialects from the
317 @uref{https://en.wikipedia.org/wiki/Lisp_%28programming_language%29, Lisp family}.
318
319 Package definitions are also written in Scheme, which empowers Guix in some
320 very unique ways, unlike most other package managers that use shell scripts or
321 simple languages.
322
323 @itemize
324 @item
325 Use functions, structures, macros and all of Scheme expressiveness for your
326 package definitions.
327
328 @item
329 Inheritance makes it easy to customize a package by inheriting from it and
330 modifying only what is needed.
331
332 @item
333 Batch processing: the whole package collection can be parsed, filtered and
334 processed. Building a headless server with all graphical interfaces stripped
335 out? It's possible. Want to rebuild everything from source using specific
336 compiler optimization flags? Pass the @code{#:make-flags "..."} argument to
337 the list of packages. It wouldn't be a stretch to think
338 @uref{https://wiki.gentoo.org/wiki/USE_flag, Gentoo USE flags} here, but this
339 goes even further: the changes don't have to be thought out beforehand by the
340 packager, they can be @emph{programmed} by the user!
341 @end itemize
342
343 The following tutorial covers all the basics around package creation with Guix.
344 It does not assume much knowledge of the Guix system nor of the Lisp language.
345 The reader is only expected to be familiar with the command line and to have some
346 basic programming knowledge.
347
348 @node A ``Hello World'' package
349 @subsection A ``Hello World'' package
350
351 The ``Defining Packages'' section of the manual introduces the basics of Guix
352 packaging (@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}). In
353 the following section, we will partly go over those basics again.
354
355 GNU@tie{}Hello is a dummy project that serves as an idiomatic example for
356 packaging. It uses the GNU build system (@code{./configure && make && make
357 install}). Guix already provides a package definition which is a perfect
358 example to start with. You can look up its declaration with @code{guix edit
359 hello} from the command line. Let's see how it looks:
360
361 @lisp
362 (define-public hello
363 (package
364 (name "hello")
365 (version "2.10")
366 (source (origin
367 (method url-fetch)
368 (uri (string-append "mirror://gnu/hello/hello-" version
369 ".tar.gz"))
370 (sha256
371 (base32
372 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
373 (build-system gnu-build-system)
374 (synopsis "Hello, GNU world: An example GNU package")
375 (description
376 "GNU Hello prints the message \"Hello, world!\" and then exits. It
377 serves as an example of standard GNU coding practices. As such, it supports
378 command-line arguments, multiple languages, and so on.")
379 (home-page "https://www.gnu.org/software/hello/")
380 (license gpl3+)))
381 @end lisp
382
383 As you can see, most of it is rather straightforward. But let's review the
384 fields together:
385
386 @table @samp
387 @item name
388 The project name. Using Scheme conventions, we prefer to keep it
389 lower case, without underscore and using dash-separated words.
390
391 @item source
392 This field contains a description of the source code origin. The
393 @code{origin} record contains these fields:
394
395 @enumerate
396 @item The method, here @code{url-fetch} to download via HTTP/FTP, but other methods
397 exist, such as @code{git-fetch} for Git repositories.
398 @item The URI, which is typically some @code{https://} location for @code{url-fetch}. Here
399 the special `mirror://gnu` refers to a set of well known locations, all of
400 which can be used by Guix to fetch the source, should some of them fail.
401 @item The @code{sha256} checksum of the requested file. This is essential to ensure
402 the source is not corrupted. Note that Guix works with base32 strings,
403 hence the call to the @code{base32} function.
404 @end enumerate
405
406 @item build-system
407
408 This is where the power of abstraction provided by the Scheme language really
409 shines: in this case, the @code{gnu-build-system} abstracts away the famous
410 @code{./configure && make && make install} shell invocations. Other build
411 systems include the @code{trivial-build-system} which does not do anything and
412 requires from the packager to program all the build steps, the
413 @code{python-build-system}, the @code{emacs-build-system}, and many more
414 (@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
415
416 @item synopsis
417 It should be a concise summary of what the package does. For many packages a
418 tagline from the project's home page can be used as the synopsis.
419
420 @item description
421 Same as for the synopsis, it's fine to re-use the project description from the
422 homepage. Note that Guix uses Texinfo syntax.
423
424 @item home-page
425 Use HTTPS if available.
426
427 @item license
428 See @code{guix/licenses.scm} in the project source for a full list of
429 available licenses.
430 @end table
431
432 Time to build our first package! Nothing fancy here for now: we will stick to a
433 dummy @code{my-hello}, a copy of the above declaration.
434
435 As with the ritualistic ``Hello World'' taught with most programming languages,
436 this will possibly be the most ``manual'' approach. We will work out an ideal
437 setup later; for now we will go the simplest route.
438
439 Save the following to a file @file{my-hello.scm}.
440
441 @lisp
442 (use-modules (guix packages)
443 (guix download)
444 (guix build-system gnu)
445 (guix licenses))
446
447 (package
448 (name "my-hello")
449 (version "2.10")
450 (source (origin
451 (method url-fetch)
452 (uri (string-append "mirror://gnu/hello/hello-" version
453 ".tar.gz"))
454 (sha256
455 (base32
456 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
457 (build-system gnu-build-system)
458 (synopsis "Hello, Guix world: An example custom Guix package")
459 (description
460 "GNU Hello prints the message \"Hello, world!\" and then exits. It
461 serves as an example of standard GNU coding practices. As such, it supports
462 command-line arguments, multiple languages, and so on.")
463 (home-page "https://www.gnu.org/software/hello/")
464 (license gpl3+))
465 @end lisp
466
467 We will explain the extra code in a moment.
468
469 Feel free to play with the different values of the various fields. If you
470 change the source, you'll need to update the checksum. Indeed, Guix refuses to
471 build anything if the given checksum does not match the computed checksum of the
472 source code. To obtain the correct checksum of the package declaration, we
473 need to download the source, compute the sha256 checksum and convert it to
474 base32.
475
476 Thankfully, Guix can automate this task for us; all we need is to provide the
477 URI:
478
479 @c TRANSLATORS: This is example shell output.
480 @example sh
481 $ guix download mirror://gnu/hello/hello-2.10.tar.gz
482
483 Starting download of /tmp/guix-file.JLYgL7
484 From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
485 following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
486 …10.tar.gz 709KiB 2.5MiB/s 00:00 [##################] 100.0%
487 /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
488 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
489 @end example
490
491 In this specific case the output tells us which mirror was chosen.
492 If the result of the above command is not the same as in the above snippet,
493 update your @code{my-hello} declaration accordingly.
494
495 Note that GNU package tarballs come with an OpenPGP signature, so you
496 should definitely check the signature of this tarball with `gpg` to
497 authenticate it before going further:
498
499 @c TRANSLATORS: This is example shell output.
500 @example sh
501 $ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig
502
503 Starting download of /tmp/guix-file.03tFfb
504 From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
505 following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
506 ….tar.gz.sig 819B 1.2MiB/s 00:00 [##################] 100.0%
507 /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
508 0q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
509 $ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
510 gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
511 gpg: using RSA key A9553245FDE9B739
512 gpg: Good signature from "Sami Kerola <kerolasa@@iki.fi>" [unknown]
513 gpg: aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@@iki.fi>" [unknown]
514 gpg: WARNING: This key is not certified with a trusted signature!
515 gpg: There is no indication that the signature belongs to the owner.
516 Primary key fingerprint: 8ED3 96E3 7E38 D471 A005 30D3 A955 3245 FDE9 B739
517 @end example
518
519 You can then happily run
520
521 @c TRANSLATORS: Do not translate this command
522 @example sh
523 $ guix package --install-from-file=my-hello.scm
524 @end example
525
526 You should now have @code{my-hello} in your profile!
527
528 @c TRANSLATORS: Do not translate this command
529 @example sh
530 $ guix package --list-installed=my-hello
531 my-hello 2.10 out
532 /gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10
533 @end example
534
535 We've gone as far as we could without any knowledge of Scheme. Before moving
536 on to more complex packages, now is the right time to brush up on your Scheme
537 knowledge. @pxref{A Scheme Crash Course} to get up to speed.
538
539 @node Setup
540 @subsection Setup
541
542 In the rest of this chapter we will rely on some basic Scheme
543 programming knowledge. Now let's detail the different possible setups
544 for working on Guix packages.
545
546 There are several ways to set up a Guix packaging environment.
547
548 We recommend you work directly on the Guix source checkout since it makes it
549 easier for everyone to contribute to the project.
550
551 But first, let's look at other possibilities.
552
553 @node Local file
554 @subsubsection Local file
555
556 This is what we previously did with @samp{my-hello}. With the Scheme basics we've
557 covered, we are now able to explain the leading chunks. As stated in @code{guix
558 package --help}:
559
560 @example
561 -f, --install-from-file=FILE
562 install the package that the code within FILE
563 evaluates to
564 @end example
565
566 Thus the last expression @emph{must} return a package, which is the case in our
567 earlier example.
568
569 The @code{use-modules} expression tells which of the modules we need in the file.
570 Modules are a collection of values and procedures. They are commonly called
571 ``libraries'' or ``packages'' in other programming languages.
572
573 @node @samp{GUIX_PACKAGE_PATH}
574 @subsubsection @samp{GUIX_PACKAGE_PATH}
575
576 @emph{Note: Starting from Guix 0.16, the more flexible Guix @dfn{channels} are the
577 preferred way and supersede @samp{GUIX_PACKAGE_PATH}. See next section.}
578
579 It can be tedious to specify the file from the command line instead of simply
580 calling @code{guix package --install my-hello} as you would do with the official
581 packages.
582
583 Guix makes it possible to streamline the process by adding as many ``package
584 declaration directories'' as you want.
585
586 Create a directory, say @file{~./guix-packages} and add it to the @samp{GUIX_PACKAGE_PATH}
587 environment variable:
588
589 @example
590 $ mkdir ~/guix-packages
591 $ export GUIX_PACKAGE_PATH=~/guix-packages
592 @end example
593
594 To add several directories, separate them with a colon (@code{:}).
595
596 Our previous @samp{my-hello} needs some adjustments though:
597
598 @lisp
599 (define-module (my-hello)
600 #:use-module (guix licenses)
601 #:use-module (guix packages)
602 #:use-module (guix build-system gnu)
603 #:use-module (guix download))
604
605 (define-public my-hello
606 (package
607 (name "my-hello")
608 (version "2.10")
609 (source (origin
610 (method url-fetch)
611 (uri (string-append "mirror://gnu/hello/hello-" version
612 ".tar.gz"))
613 (sha256
614 (base32
615 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
616 (build-system gnu-build-system)
617 (synopsis "Hello, Guix world: An example custom Guix package")
618 (description
619 "GNU Hello prints the message \"Hello, world!\" and then exits. It
620 serves as an example of standard GNU coding practices. As such, it supports
621 command-line arguments, multiple languages, and so on.")
622 (home-page "https://www.gnu.org/software/hello/")
623 (license gpl3+)))
624 @end lisp
625
626 Note that we have assigned the package value to an exported variable name with
627 @code{define-public}. This is effectively assigning the package to the @code{my-hello}
628 variable so that it can be referenced, among other as dependency of other
629 packages.
630
631 If you use @code{guix package --install-from-file=my-hello.scm} on the above file, it
632 will fail because the last expression, @code{define-public}, does not return a
633 package. If you want to use @code{define-public} in this use-case nonetheless, make
634 sure the file ends with an evaluation of @code{my-hello}:
635
636 @lisp
637 ; ...
638 (define-public my-hello
639 ; ...
640 )
641
642 my-hello
643 @end lisp
644
645 This last example is not very typical.
646
647 Now @samp{my-hello} should be part of the package collection like all other official
648 packages. You can verify this with:
649
650 @example
651 $ guix package --show=my-hello
652 @end example
653
654 @node Guix channels
655 @subsubsection Guix channels
656
657 Guix 0.16 features channels, which is very similar to @samp{GUIX_PACKAGE_PATH} but
658 provides better integration and provenance tracking. Channels are not
659 necessarily local, they can be maintained as a public Git repository for
660 instance. Of course, several channels can be used at the same time.
661
662 @xref{Channels,,, guix, GNU Guix Reference Manual} for setup details.
663
664 @node Direct checkout hacking
665 @subsubsection Direct checkout hacking
666
667 Working directly on the Guix project is recommended: it reduces the friction
668 when the time comes to submit your changes upstream to let the community benefit
669 from your hard work!
670
671 Unlike most software distributions, the Guix repository holds in one place both
672 the tooling (including the package manager) and the package definitions. This
673 choice was made so that it would give developers the flexibility to modify the
674 API without breakage by updating all packages at the same time. This reduces
675 development inertia.
676
677 Check out the official @uref{https://git-scm.com/, Git} repository:
678
679 @example
680 $ git clone https://git.savannah.gnu.org/git/guix.git
681 @end example
682
683 In the rest of this article, we use @samp{$GUIX_CHECKOUT} to refer to the location of
684 the checkout.
685
686
687 Follow the instructions in the manual (@pxref{Contributing,,, guix, GNU Guix
688 Reference Manual}) to set up the repository environment.
689
690 Once ready, you should be able to use the package definitions from the
691 repository environment.
692
693 Feel free to edit package definitions found in @samp{$GUIX_CHECKOUT/gnu/packages}.
694
695 The @samp{$GUIX_CHECKOUT/pre-inst-env} script lets you use @samp{guix} over the package
696 collection of the repository (@pxref{Running Guix Before It Is
697 Installed,,, guix, GNU Guix Reference Manual}).
698
699 @itemize
700 @item
701 Search packages, such as Ruby:
702
703 @example
704 $ cd $GUIX_CHECKOUT
705 $ ./pre-inst-env guix package --list-available=ruby
706 ruby 1.8.7-p374 out gnu/packages/ruby.scm:119:2
707 ruby 2.1.6 out gnu/packages/ruby.scm:91:2
708 ruby 2.2.2 out gnu/packages/ruby.scm:39:2
709 @end example
710
711 @item
712 Build a package, here Ruby version 2.1:
713
714 @example
715 $ ./pre-inst-env guix build --keep-failed ruby@@2.1
716 /gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
717 @end example
718
719 @item
720 Install it to your user profile:
721
722 @example
723 $ ./pre-inst-env guix package --install ruby@@2.1
724 @end example
725
726 @item
727 Check for common mistakes:
728
729 @example
730 $ ./pre-inst-env guix lint ruby@@2.1
731 @end example
732 @end itemize
733
734 Guix strives at maintaining a high packaging standard; when contributing to the
735 Guix project, remember to
736
737 @itemize
738 @item
739 follow the coding style (@pxref{Coding Style,,, guix, GNU Guix Reference Manual}),
740 @item
741 and review the check list from the manual (@pxref{Submitting Patches,,, guix, GNU Guix Reference Manual}).
742 @end itemize
743
744 Once you are happy with the result, you are welcome to send your contribution to
745 make it part of Guix. This process is also detailed in the manual. (@pxref{Contributing,,, guix, GNU Guix Reference Manual})
746
747
748 It's a community effort so the more join in, the better Guix becomes!
749
750 @node Extended example
751 @subsection Extended example
752
753 The above ``Hello World'' example is as simple as it goes. Packages can be more
754 complex than that and Guix can handle more advanced scenarios. Let's look at
755 another, more sophisticated package (slightly modified from the source):
756
757 @lisp
758 (define-module (gnu packages version-control)
759 #:use-module ((guix licenses) #:prefix license:)
760 #:use-module (guix utils)
761 #:use-module (guix packages)
762 #:use-module (guix git-download)
763 #:use-module (guix build-system cmake)
764 #:use-module (gnu packages ssh)
765 #:use-module (gnu packages web)
766 #:use-module (gnu packages pkg-config)
767 #:use-module (gnu packages python)
768 #:use-module (gnu packages compression)
769 #:use-module (gnu packages tls))
770
771 (define-public my-libgit2
772 (let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
773 (revision "1"))
774 (package
775 (name "my-libgit2")
776 (version (git-version "0.26.6" revision commit))
777 (source (origin
778 (method git-fetch)
779 (uri (git-reference
780 (url "https://github.com/libgit2/libgit2/")
781 (commit commit)))
782 (file-name (git-file-name name version))
783 (sha256
784 (base32
785 "17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
786 (patches (search-patches "libgit2-mtime-0.patch"))
787 (modules '((guix build utils)))
788 (snippet '(begin
789 ;; Remove bundled software.
790 (delete-file-recursively "deps")
791 #t))))
792 (build-system cmake-build-system)
793 (outputs '("out" "debug"))
794 (arguments
795 `(#:tests? #t ; Run the test suite (this is the default)
796 #:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
797 #:phases
798 (modify-phases %standard-phases
799 (add-after 'unpack 'fix-hardcoded-paths
800 (lambda _
801 (substitute* "tests/repo/init.c"
802 (("#!/bin/sh") (string-append "#!" (which "sh"))))
803 (substitute* "tests/clar/fs.h"
804 (("/bin/cp") (which "cp"))
805 (("/bin/rm") (which "rm")))
806 #t))
807 ;; Run checks more verbosely.
808 (replace 'check
809 (lambda _ (invoke "./libgit2_clar" "-v" "-Q")))
810 (add-after 'unpack 'make-files-writable-for-tests
811 (lambda _ (for-each make-file-writable (find-files "." ".*")))))))
812 (inputs
813 `(("libssh2" ,libssh2)
814 ("http-parser" ,http-parser)
815 ("python" ,python-wrapper)))
816 (native-inputs
817 `(("pkg-config" ,pkg-config)))
818 (propagated-inputs
819 ;; These two libraries are in 'Requires.private' in libgit2.pc.
820 `(("openssl" ,openssl)
821 ("zlib" ,zlib)))
822 (home-page "https://libgit2.github.com/")
823 (synopsis "Library providing Git core methods")
824 (description
825 "Libgit2 is a portable, pure C implementation of the Git core methods
826 provided as a re-entrant linkable library with a solid API, allowing you to
827 write native speed custom Git applications in any language with bindings.")
828 ;; GPLv2 with linking exception
829 (license license:gpl2))))
830 @end lisp
831
832 (In those cases were you only want to tweak a few fields from a package
833 definition, you should rely on inheritance instead of copy-pasting everything.
834 See below.)
835
836 Let's discuss those fields in depth.
837
838 @subsubsection @code{git-fetch} method
839
840 Unlike the @code{url-fetch} method, @code{git-fetch} expects a @code{git-reference} which takes
841 a Git repository and a commit. The commit can be any Git reference such as
842 tags, so if the @code{version} is tagged, then it can be used directly. Sometimes
843 the tag is prefixed with a @code{v}, in which case you'd use @code{(commit (string-append
844 "v" version))}.
845
846 To ensure that the source code from the Git repository is stored in a unique
847 directory with a readable name we use @code{(file-name (git-file-name name
848 version))}.
849
850 Note that there is also a @code{git-version} procedure that can be used to derive the
851 version when packaging programs for a specific commit.
852
853 @subsubsection Snippets
854
855 Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching
856 the source. They are a Guix-y alternative to the traditional @file{.patch} files.
857 Because of the quote, the code in only evaluated when passed to the Guix daemon
858 for building. There can be as many snippets as needed.
859
860 Snippets might need additional Guile modules which can be imported from the
861 @code{modules} field.
862
863 @subsubsection Inputs
864
865 First, a syntactic comment: See the quasi-quote / comma syntax?
866
867 @lisp
868 (native-inputs
869 `(("pkg-config" ,pkg-config)))
870 @end lisp
871
872 is equivalent to
873
874 @lisp
875 (native-inputs
876 (list (list "pkg-config" pkg-config)))
877 @end lisp
878
879 You'll mostly see the former because it's shorter.
880
881 There are 3 different input types. In short:
882
883 @table @asis
884 @item native-inputs
885 Required for building but not runtime -- installing a package
886 through a substitute won't install these inputs.
887 @item inputs
888 Installed in the store but not in the profile, as well as being
889 present at build time.
890 @item propagated-inputs
891 Installed in the store and in the profile, as well as
892 being present at build time.
893 @end table
894
895 @xref{Package Reference,,, guix, GNU Guix Reference Manual} for more details.
896
897 The distinction between the various inputs is important: if a dependency can be
898 handled as an @emph{input} instead of a @emph{propagated input}, it should be done so, or
899 else it ``pollutes'' the user profile for no good reason.
900
901 For instance, a user installing a graphical program that depends on a
902 command line tool might only be interested in the graphical part, so there is no
903 need to force the command line tool into the user profile. The dependency is a
904 concern to the package, not to the user. @emph{Inputs} make it possible to handle
905 dependencies without bugging the user by adding undesired executable files (or
906 libraries) to their profile.
907
908 Same goes for @emph{native-inputs}: once the program is installed, build-time
909 dependencies can be safely garbage-collected.
910 It also matters when a substitute is available, in which case only the @emph{inputs}
911 and @emph{propagated inputs} will be fetched: the @emph{native inputs} are not required to
912 install a package from a substitute.
913
914 @subsubsection Outputs
915
916 Just like how a package can have multiple inputs, it can also produce multiple
917 outputs.
918
919 Each output corresponds to a separate directory in the store.
920
921 The user can choose which output to install; this is useful to save space or
922 to avoid polluting the user profile with unwanted executables or libraries.
923
924 Output separation is optional. When the @code{outputs} field is left out, the
925 default and only output (the complete package) is referred to as @code{"out"}.
926
927 Typical separate output names include @code{debug} and @code{doc}.
928
929 It's advised to separate outputs only when you've shown it's worth it: if the
930 output size is significant (compare with @code{guix size}) or in case the package is
931 modular.
932
933 @subsubsection Build system arguments
934
935 The @code{arguments} is a keyword-value list used to configure the build process.
936
937 The simplest argument @code{#:tests?} can be used to disable the test suite when
938 building the package. This is mostly useful when the package does not feature
939 any test suite. It's strongly recommended to keep the test suite on if there is
940 one.
941
942 Another common argument is @code{:make-flags}, which specifies a list of flags to
943 append when running make, as you would from the command line. For instance, the
944 following flags
945
946 @lisp
947 #:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
948 "CC=gcc")
949 @end lisp
950
951 translate into
952
953 @example
954 $ make CC=gcc prefix=/gnu/store/...-<out>
955 @end example
956
957 This sets the C compiler to @code{gcc} and the @code{prefix} variable (the installation
958 directory in Make parlance) to @code{(assoc-ref %outputs "out")}, which is a build-stage
959 global variable pointing to the destination directory in the store (something like
960 @file{/gnu/store/...-my-libgit2-20180408}).
961
962 Similarly, it's possible to set the configure flags:
963
964 @lisp
965 #:configure-flags '("-DUSE_SHA1DC=ON")
966 @end lisp
967
968 The @code{%build-inputs} variable is also generated in scope. It's an association
969 table that maps the input names to their store directories.
970
971 The @code{phases} keyword lists the sequential steps of the build system. Typically
972 phases include @code{unpack}, @code{configure}, @code{build}, @code{install} and @code{check}. To know
973 more about those phases, you need to work out the appropriate build system
974 definition in @samp{$GUIX_CHECKOUT/guix/build/gnu-build-system.scm}:
975
976 @lisp
977 (define %standard-phases
978 ;; Standard build phases, as a list of symbol/procedure pairs.
979 (let-syntax ((phases (syntax-rules ()
980 ((_ p ...) `((p . ,p) ...)))))
981 (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
982 bootstrap
983 patch-usr-bin-file
984 patch-source-shebangs configure patch-generated-file-shebangs
985 build check install
986 patch-shebangs strip
987 validate-runpath
988 validate-documentation-location
989 delete-info-dir-file
990 patch-dot-desktop-files
991 install-license-files
992 reset-gzip-timestamps
993 compress-documentation)))
994 @end lisp
995
996 Or from the REPL:
997
998 @lisp
999 (add-to-load-path "/path/to/guix/checkout")
1000 ,use (guix build gnu-build-system)
1001 (map first %standard-phases)
1002 @result{} (set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap patch-usr-bin-file patch-source-shebangs configure patch-generated-file-shebangs build check install patch-shebangs strip validate-runpath validate-documentation-location delete-info-dir-file patch-dot-desktop-files install-license-files reset-gzip-timestamps compress-documentation)
1003 @end lisp
1004
1005 If you want to know more about what happens during those phases, consult the
1006 associated procedures.
1007
1008 For instance, as of this writing the definition of @code{unpack} for the GNU build
1009 system is
1010
1011 @lisp
1012 (define* (unpack #:key source #:allow-other-keys)
1013 "Unpack SOURCE in the working directory, and change directory within the
1014 source. When SOURCE is a directory, copy it in a sub-directory of the current
1015 working directory."
1016 (if (file-is-directory? source)
1017 (begin
1018 (mkdir "source")
1019 (chdir "source")
1020
1021 ;; Preserve timestamps (set to the Epoch) on the copied tree so that
1022 ;; things work deterministically.
1023 (copy-recursively source "."
1024 #:keep-mtime? #t))
1025 (begin
1026 (if (string-suffix? ".zip" source)
1027 (invoke "unzip" source)
1028 (invoke "tar" "xvf" source))
1029 (chdir (first-subdirectory "."))))
1030 #t)
1031 @end lisp
1032
1033 Note the @code{chdir} call: it changes the working directory to where the source was
1034 unpacked.
1035 Thus every phase following the @code{unpack} will use the source as a working
1036 directory, which is why we can directly work on the source files.
1037 That is to say, unless a later phase changes the working directory to something
1038 else.
1039
1040 We modify the list of @code{%standard-phases} of the build system with the
1041 @code{modify-phases} macro as per the list of specified modifications, which may have
1042 the following forms:
1043
1044 @itemize
1045 @item
1046 @code{(add-before PHASE NEW-PHASE PROCEDURE)}: Run @code{PROCEDURE} named @code{NEW-PHASE} before @code{PHASE}.
1047 @item
1048 @code{(add-after PHASE NEW-PHASE PROCEDURE)}: Same, but afterwards.
1049 @item
1050 @code{(replace PHASE PROCEDURE)}.
1051 @item
1052 @code{(delete PHASE)}.
1053 @end itemize
1054
1055 The @code{PROCEDURE} supports the keyword arguments @code{inputs} and @code{outputs}. Each
1056 input (whether @emph{native}, @emph{propagated} or not) and output directory is referenced
1057 by their name in those variables. Thus @code{(assoc-ref outputs "out")} is the store
1058 directory of the main output of the package. A phase procedure may look like
1059 this:
1060
1061 @lisp
1062 (lambda* (#:key inputs outputs #:allow-other-keys)
1063 (let (((bash-directory (assoc-ref inputs "bash"))
1064 (output-directory (assoc-ref outputs "out"))
1065 (doc-directory (assoc-ref outputs "doc"))
1066 ; ...
1067 #t)
1068 @end lisp
1069
1070 The procedure must return @code{#t} on success. It's brittle to rely on the return
1071 value of the last expression used to tweak the phase because there is no
1072 guarantee it would be a @code{#t}. Hence the trailing @code{#t} to ensure the right value
1073 is returned on success.
1074
1075 @subsubsection Code staging
1076
1077 The astute reader may have noticed the quasi-quote and comma syntax in the
1078 argument field. Indeed, the build code in the package declaration should not be
1079 evaluated on the client side, but only when passed to the Guix daemon. This
1080 mechanism of passing code around two running processes is called @uref{https://arxiv.org/abs/1709.00833, code staging}.
1081
1082 @subsubsection Utility functions
1083
1084 When customizing @code{phases}, we often need to write code that mimics the
1085 equivalent system invocations (@code{make}, @code{mkdir}, @code{cp}, etc.)@: commonly used during
1086 regular ``Unix-style'' installations.
1087
1088 Some like @code{chmod} are native to Guile.
1089 @xref{,,, guile, Guile reference manual} for a complete list.
1090
1091 Guix provides additional helper functions which prove especially handy in the
1092 context of package management.
1093
1094 Some of those functions can be found in
1095 @samp{$GUIX_CHECKOUT/guix/guix/build/utils.scm}. Most of them mirror the behaviour
1096 of the traditional Unix system commands:
1097
1098 @table @asis
1099 @item which
1100 Like the @samp{which} system command.
1101 @item find-files
1102 Akin to the @samp{find} system command.
1103 @item mkdir-p
1104 Like @samp{mkdir -p}, which creates all parents as needed.
1105 @item install-file
1106 Similar to @samp{install} when installing a file to a (possibly
1107 non-existing) directory. Guile has @code{copy-file} which works
1108 like @samp{cp}.
1109 @item copy-recursively
1110 Like @samp{cp -r}.
1111 @item delete-file-recursively
1112 Like @samp{rm -rf}.
1113 @item invoke
1114 Run an executable. This should be used instead of @code{system*}.
1115 @item with-directory-excursion
1116 Run the body in a different working directory,
1117 then restore the previous working directory.
1118 @item substitute*
1119 A ``@command{sed}-like'' function.
1120 @end table
1121
1122 @subsubsection Module prefix
1123
1124 The license in our last example needs a prefix: this is because of how the
1125 @code{license} module was imported in the package, as @code{#:use-module ((guix licenses)
1126 #:prefix license:)}. The Guile module import mechanism
1127 (@pxref{Using Guile Modules,,, guile, Guile reference manual})
1128 gives the user full control over namespacing: this is needed to avoid
1129 clashes between, say, the
1130 @samp{zlib} variable from @samp{licenses.scm} (a @emph{license} value) and the @samp{zlib} variable
1131 from @samp{compression.scm} (a @emph{package} value).
1132
1133 @node Other build systems
1134 @subsection Other build systems
1135
1136 What we've seen so far covers the majority of packages using a build system
1137 other than the @code{trivial-build-system}. The latter does not automate anything
1138 and leaves you to build everything manually. This can be more demanding and we
1139 won't cover it here for now, but thankfully it is rarely necessary to fall back
1140 on this system.
1141
1142 For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the
1143 process is very similar to the GNU build system except for a few specialized
1144 arguments.
1145
1146 @xref{Build Systems,,, guix, GNU Guix Reference Manual}, for more
1147 information on build systems, or check the source code in the
1148 @samp{$GUIX_CHECKOUT/guix/build} and
1149 @samp{$GUIX_CHECKOUT/guix/build-system} directories.
1150
1151 @node Programmable and automated package definition
1152 @subsection Programmable and automated package definition
1153
1154 We can't repeat it enough: having a full-fledged programming language at hand
1155 empowers us in ways that reach far beyond traditional package management.
1156
1157 Let's illustrate this with some awesome features of Guix!
1158
1159 @node Recursive importers
1160 @subsubsection Recursive importers
1161
1162 You might find some build systems good enough that there is little to do at all
1163 to write a package, to the point that it becomes repetitive and tedious after a
1164 while. A @emph{raison d'être} of computers is to replace human beings at those
1165 boring tasks. So let's tell Guix to do this for us and create the package
1166 definition of an R package from CRAN (the output is trimmed for conciseness):
1167
1168 @example
1169 $ guix import cran --recursive walrus
1170
1171 (define-public r-mc2d
1172 ; ...
1173 (license gpl2+)))
1174
1175 (define-public r-jmvcore
1176 ; ...
1177 (license gpl2+)))
1178
1179 (define-public r-wrs2
1180 ; ...
1181 (license gpl3)))
1182
1183 (define-public r-walrus
1184 (package
1185 (name "r-walrus")
1186 (version "1.0.3")
1187 (source
1188 (origin
1189 (method url-fetch)
1190 (uri (cran-uri "walrus" version))
1191 (sha256
1192 (base32
1193 "1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
1194 (build-system r-build-system)
1195 (propagated-inputs
1196 `(("r-ggplot2" ,r-ggplot2)
1197 ("r-jmvcore" ,r-jmvcore)
1198 ("r-r6" ,r-r6)
1199 ("r-wrs2" ,r-wrs2)))
1200 (home-page "https://github.com/jamovi/walrus")
1201 (synopsis "Robust Statistical Methods")
1202 (description
1203 "This package provides a toolbox of common robust statistical
1204 tests, including robust descriptives, robust t-tests, and robust ANOVA.
1205 It is also available as a module for 'jamovi' (see
1206 <https://www.jamovi.org> for more information). Walrus is based on the
1207 WRS2 package by Patrick Mair, which is in turn based on the scripts and
1208 work of Rand Wilcox. These analyses are described in depth in the book
1209 'Introduction to Robust Estimation & Hypothesis Testing'.")
1210 (license gpl3)))
1211 @end example
1212
1213 The recursive importer won't import packages for which Guix already has package
1214 definitions, except for the very first.
1215
1216 Not all applications can be packaged this way, only those relying on a select
1217 number of supported systems. Read about the full list of importers in
1218 the guix import section of the manual
1219 (@pxref{Invoking guix import,,, guix, GNU Guix Reference Manual}).
1220
1221 @node Automatic update
1222 @subsubsection Automatic update
1223
1224 Guix can be smart enough to check for updates on systems it knows. It can
1225 report outdated package definitions with
1226
1227 @example
1228 $ guix refresh hello
1229 @end example
1230
1231 In most cases, updating a package to a newer version requires little more than
1232 changing the version number and the checksum. Guix can do that automatically as
1233 well:
1234
1235 @example
1236 $ guix refresh hello --update
1237 @end example
1238
1239 @node Inheritance
1240 @subsubsection Inheritance
1241
1242 If you've started browsing the existing package definitions, you might have
1243 noticed that a significant number of them have a @code{inherit} field:
1244
1245 @lisp
1246 (define-public adwaita-icon-theme
1247 (package (inherit gnome-icon-theme)
1248 (name "adwaita-icon-theme")
1249 (version "3.26.1")
1250 (source (origin
1251 (method url-fetch)
1252 (uri (string-append "mirror://gnome/sources/" name "/"
1253 (version-major+minor version) "/"
1254 name "-" version ".tar.xz"))
1255 (sha256
1256 (base32
1257 "17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
1258 (native-inputs
1259 `(("gtk-encode-symbolic-svg" ,gtk+ "bin")))))
1260 @end lisp
1261
1262 All unspecified fields are inherited from the parent package. This is very
1263 convenient to create alternative packages, for instance with different source,
1264 version or compilation options.
1265
1266 @node Getting help
1267 @subsection Getting help
1268
1269 Sadly, some applications can be tough to package. Sometimes they need a patch to
1270 work with the non-standard filesystem hierarchy enforced by the store.
1271 Sometimes the tests won't run properly. (They can be skipped but this is not
1272 recommended.) Other times the resulting package won't be reproducible.
1273
1274 Should you be stuck, unable to figure out how to fix any sort of packaging
1275 issue, don't hesitate to ask the community for help.
1276
1277 See the @uref{https://www.gnu.org/software/guix/contact/, Guix homepage} for information on the mailing lists, IRC, etc.
1278
1279 @node Conclusion
1280 @subsection Conclusion
1281
1282 This tutorial was a showcase of the sophisticated package management that Guix
1283 boasts. At this point we have mostly restricted this introduction to the
1284 @code{gnu-build-system} which is a core abstraction layer on which more advanced
1285 abstractions are based.
1286
1287 Where do we go from here? Next we ought to dissect the innards of the build
1288 system by removing all abstractions, using the @code{trivial-build-system}: this
1289 should give us a thorough understanding of the process before investigating some
1290 more advanced packaging techniques and edge cases.
1291
1292 Other features worth exploring are the interactive editing and debugging
1293 capabilities of Guix provided by the Guile REPL@.
1294
1295 Those fancy features are completely optional and can wait; now is a good time
1296 to take a well-deserved break. With what we've introduced here you should be
1297 well armed to package lots of programs. You can get started right away and
1298 hopefully we will see your contributions soon!
1299
1300 @node References
1301 @subsection References
1302
1303 @itemize
1304 @item
1305 The @uref{https://www.gnu.org/software/guix/manual/en/html_node/Defining-Packages.html, package reference in the manual}
1306
1307 @item
1308 @uref{https://gitlab.com/pjotrp/guix-notes/blob/master/HACKING.org, Pjotr’s hacking guide to GNU Guix}
1309
1310 @item
1311 @uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
1312 @end itemize
1313
1314 @c *********************************************************************
1315 @node System Configuration
1316 @chapter System Configuration
1317
1318 Guix offers a flexible language for declaratively configuring your Guix
1319 System. This flexibility can at times be overwhelming. The purpose of this
1320 chapter is to demonstrate some advanced configuration concepts.
1321
1322 @pxref{System Configuration,,, guix, GNU Guix Reference Manual} for a complete
1323 reference.
1324
1325 @menu
1326 * Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
1327 * Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
1328 * Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
1329 * Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
1330 * Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
1331 @end menu
1332
1333 @node Customizing the Kernel
1334 @section Customizing the Kernel
1335
1336 Guix is, at its core, a source based distribution with substitutes
1337 (@pxref{Substitutes,,, guix, GNU Guix Reference Manual}), and as such building
1338 packages from their source code is an expected part of regular package
1339 installations and upgrades. Given this starting point, it makes sense that
1340 efforts are made to reduce the amount of time spent compiling packages, and
1341 recent changes and upgrades to the building and distribution of substitutes
1342 continues to be a topic of discussion within Guix.
1343
1344 The kernel, while not requiring an overabundance of RAM to build, does take a
1345 rather long time on an average machine. The official kernel configuration, as
1346 is the case with many GNU/Linux distributions, errs on the side of
1347 inclusiveness, and this is really what causes the build to take such a long
1348 time when the kernel is built from source.
1349
1350 The Linux kernel, however, can also just be described as a regular old
1351 package, and as such can be customized just like any other package. The
1352 procedure is a little bit different, although this is primarily due to the
1353 nature of how the package definition is written.
1354
1355 The @code{linux-libre} kernel package definition is actually a procedure which
1356 creates a package.
1357
1358 @lisp
1359 (define* (make-linux-libre version hash supported-systems
1360 #:key
1361 ;; A function that takes an arch and a variant.
1362 ;; See kernel-config for an example.
1363 (extra-version #f)
1364 (configuration-file #f)
1365 (defconfig "defconfig")
1366 (extra-options %default-extra-linux-options)
1367 (patches (list %boot-logo-patch)))
1368 ...)
1369 @end lisp
1370
1371 The current @code{linux-libre} package is for the 5.1.x series, and is
1372 declared like this:
1373
1374 @lisp
1375 (define-public linux-libre
1376 (make-linux-libre %linux-libre-version
1377 %linux-libre-hash
1378 '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux")
1379 #:patches %linux-libre-5.1-patches
1380 #:configuration-file kernel-config))
1381 @end lisp
1382
1383 Any keys which are not assigned values inherit their default value from the
1384 @code{make-linux-libre} definition. When comparing the two snippets above,
1385 you may notice that the code comment in the first doesn't actually refer to
1386 the @code{#:extra-version} keyword; it is actually for
1387 @code{#:configuration-file}. Because of this, it is not actually easy to
1388 include a custom kernel configuration from the definition, but don't worry,
1389 there are other ways to work with what we do have.
1390
1391 There are two ways to create a kernel with a custom kernel configuration. The
1392 first is to provide a standard @file{.config} file during the build process by
1393 including an actual @file{.config} file as a native input to our custom
1394 kernel. The following is a snippet from the custom @code{'configure} phase of
1395 the @code{make-linux-libre} package definition:
1396
1397 @lisp
1398 (let ((build (assoc-ref %standard-phases 'build))
1399 (config (assoc-ref (or native-inputs inputs) "kconfig")))
1400
1401 ;; Use a custom kernel configuration file or a default
1402 ;; configuration file.
1403 (if config
1404 (begin
1405 (copy-file config ".config")
1406 (chmod ".config" #o666))
1407 (invoke "make" ,defconfig))
1408 @end lisp
1409
1410 Below is a sample kernel package. The @code{linux-libre} package is nothing
1411 special and can be inherited from and have its fields overridden like any
1412 other package:
1413
1414 @lisp
1415 (define-public linux-libre/E2140
1416 (package
1417 (inherit linux-libre)
1418 (native-inputs
1419 `(("kconfig" ,(local-file "E2140.config"))
1420 ,@@(alist-delete "kconfig"
1421 (package-native-inputs linux-libre))))))
1422 @end lisp
1423
1424 In the same directory as the file defining @code{linux-libre-E2140} is a file
1425 named @file{E2140.config}, which is an actual kernel configuration file. The
1426 @code{defconfig} keyword of @code{make-linux-libre} is left blank here, so the
1427 only kernel configuration in the package is the one which was included in the
1428 @code{native-inputs} field.
1429
1430 The second way to create a custom kernel is to pass a new value to the
1431 @code{extra-options} keyword of the @code{make-linux-libre} procedure. The
1432 @code{extra-options} keyword works with another function defined right below
1433 it:
1434
1435 @lisp
1436 (define %default-extra-linux-options
1437 `(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
1438 ("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #t)
1439 ;; Modules required for initrd:
1440 ("CONFIG_NET_9P" . m)
1441 ("CONFIG_NET_9P_VIRTIO" . m)
1442 ("CONFIG_VIRTIO_BLK" . m)
1443 ("CONFIG_VIRTIO_NET" . m)
1444 ("CONFIG_VIRTIO_PCI" . m)
1445 ("CONFIG_VIRTIO_BALLOON" . m)
1446 ("CONFIG_VIRTIO_MMIO" . m)
1447 ("CONFIG_FUSE_FS" . m)
1448 ("CONFIG_CIFS" . m)
1449 ("CONFIG_9P_FS" . m)))
1450
1451 (define (config->string options)
1452 (string-join (map (match-lambda
1453 ((option . 'm)
1454 (string-append option "=m"))
1455 ((option . #t)
1456 (string-append option "=y"))
1457 ((option . #f)
1458 (string-append option "=n")))
1459 options)
1460 "\n"))
1461 @end lisp
1462
1463 And in the custom configure script from the `make-linux-libre` package:
1464
1465 @lisp
1466 ;; Appending works even when the option wasn't in the
1467 ;; file. The last one prevails if duplicated.
1468 (let ((port (open-file ".config" "a"))
1469 (extra-configuration ,(config->string extra-options)))
1470 (display extra-configuration port)
1471 (close-port port))
1472
1473 (invoke "make" "oldconfig"))))
1474 @end lisp
1475
1476 So by not providing a configuration-file the @file{.config} starts blank, and
1477 then we write into it the collection of flags that we want. Here's another
1478 custom kernel:
1479
1480 @lisp
1481 (define %macbook41-full-config
1482 (append %macbook41-config-options
1483 %filesystems
1484 %efi-support
1485 %emulation
1486 (@@@@ (gnu packages linux) %default-extra-linux-options)))
1487
1488 (define-public linux-libre-macbook41
1489 ;; XXX: Access the internal 'make-linux-libre' procedure, which is
1490 ;; private and unexported, and is liable to change in the future.
1491 ((@@@@ (gnu packages linux) make-linux-libre) (@@@@ (gnu packages linux) %linux-libre-version)
1492 (@@@@ (gnu packages linux) %linux-libre-hash)
1493 '("x86_64-linux")
1494 #:extra-version "macbook41"
1495 #:patches (@@@@ (gnu packages linux) %linux-libre-5.1-patches)
1496 #:extra-options %macbook41-config-options))
1497 @end lisp
1498
1499 In the above example @code{%filesystems} is a collection of flags enabling
1500 different filesystem support, @code{%efi-support} enables EFI support and
1501 @code{%emulation} enables a x86_64-linux machine to act in 32-bit mode also.
1502 @code{%default-extra-linux-options} are the ones quoted above, which had to be
1503 added in since they were replaced in the @code{extra-options} keyword.
1504
1505 This all sounds like it should be doable, but how does one even know which
1506 modules are required for a particular system? Two places that can be helpful
1507 in trying to answer this question is the
1508 @uref{https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel, Gentoo
1509 Handbook} and the
1510 @uref{https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig,
1511 documentation from the kernel itself}. From the kernel documentation, it
1512 seems that @code{make localmodconfig} is the command we want.
1513
1514 In order to actually run @code{make localmodconfig} we first need to get and
1515 unpack the kernel source code:
1516
1517 @example shell
1518 tar xf $(guix build linux-libre --source)
1519 @end example
1520
1521 Once inside the directory containing the source code run @code{touch .config}
1522 to create an initial, empty @file{.config} to start with. @code{make
1523 localmodconfig} works by seeing what you already have in @file{.config} and
1524 letting you know what you're missing. If the file is blank then you're
1525 missing everything. The next step is to run:
1526
1527 @example shell
1528 guix environment linux-libre -- make localmodconfig
1529 @end example
1530
1531 and note the output. Do note that the @file{.config} file is still empty.
1532 The output generally contains two types of warnings. The first start with
1533 "WARNING" and can actually be ignored in our case. The second read:
1534
1535 @example shell
1536 module pcspkr did not have configs CONFIG_INPUT_PCSPKR
1537 @end example
1538
1539 For each of these lines, copy the @code{CONFIG_XXXX_XXXX} portion into the
1540 @file{.config} in the directory, and append @code{=m}, so in the end it looks
1541 like this:
1542
1543 @example shell
1544 CONFIG_INPUT_PCSPKR=m
1545 CONFIG_VIRTIO=m
1546 @end example
1547
1548 After copying all the configuration options, run @code{make localmodconfig}
1549 again to make sure that you don't have any output starting with ``module''.
1550 After all of these machine specific modules there are a couple more left that
1551 are also needed. @code{CONFIG_MODULES} is necessary so that you can build and
1552 load modules separately and not have everything built into the kernel.
1553 @code{CONFIG_BLK_DEV_SD} is required for reading from hard drives. It is
1554 possible that there are other modules which you will need.
1555
1556 This post does not aim to be a guide to configuring your own kernel however,
1557 so if you do decide to build a custom kernel you'll have to seek out other
1558 guides to create a kernel which is just right for your needs.
1559
1560 The second way to setup the kernel configuration makes more use of Guix's
1561 features and allows you to share configuration segments between different
1562 kernels. For example, all machines using EFI to boot have a number of EFI
1563 configuration flags that they need. It is likely that all the kernels will
1564 share a list of filesystems to support. By using variables it is easier to
1565 see at a glance what features are enabled and to make sure you don't have
1566 features in one kernel but missing in another.
1567
1568 Left undiscussed however, is Guix's initrd and its customization. It is
1569 likely that you'll need to modify the initrd on a machine using a custom
1570 kernel, since certain modules which are expected to be built may not be
1571 available for inclusion into the initrd.
1572
1573 @node Connecting to Wireguard VPN
1574 @section Connecting to Wireguard VPN
1575
1576 To connect to a Wireguard VPN server you need the kernel module to be
1577 loaded in memory and a package providing networking tools that support
1578 it (e.g. @code{wireguard-tools} or @code{network-manager}).
1579
1580 Here is a configuration example for Linux-Libre < 5.6, where the module
1581 is out of tree and need to be loaded manually---following revisions of
1582 the kernel have it built-in and so don't need such configuration:
1583
1584 @lisp
1585 (use-modules (gnu))
1586 (use-service-modules desktop)
1587 (use-package-modules vpn)
1588
1589 (operating-system
1590 ;; …
1591 (services (cons (simple-service 'wireguard-module
1592 kernel-module-loader-service-type
1593 '("wireguard"))
1594 %desktop-services))
1595 (packages (cons wireguard-tools %base-packages))
1596 (kernel-loadable-modules (list wireguard-linux-compat)))
1597 @end lisp
1598
1599 After reconfiguring and restarting your system you can either use
1600 Wireguard tools or NetworkManager to connect to a VPN server.
1601
1602 @subsection Using Wireguard tools
1603
1604 To test your Wireguard setup it is convenient to use @command{wg-quick}.
1605 Just give it a configuration file @command{wg-quick up ./wg0.conf}; or
1606 put that file in @file{/etc/wireguard} and run @command{wg-quick up wg0}
1607 instead.
1608
1609 @quotation Note
1610 Be warned that the author described this command as a: “[…] very quick
1611 and dirty bash script […]”.
1612 @end quotation
1613
1614 @subsection Using NetworkManager
1615
1616 Thanks to NetworkManager support for Wireguard we can connect to our VPN
1617 using @command{nmcli} command. Up to this point this guide assumes that
1618 you're using Network Manager service provided by
1619 @code{%desktop-services}. Ortherwise you need to adjust your services
1620 list to load @code{network-manager-service-type} and reconfigure your
1621 Guix system.
1622
1623 To import your VPN configuration execute nmcli import command:
1624
1625 @example shell
1626 # nmcli connection import type wireguard file wg0.conf
1627 Connection 'wg0' (edbee261-aa5a-42db-b032-6c7757c60fde) successfully added
1628 @end example
1629
1630 This will create a configuration file in
1631 @file{/etc/NetworkManager/wg0.nmconnection}. Next connect to the
1632 Wireguard server:
1633
1634 @example shell
1635 $ nmcli connection up wg0
1636 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
1637 @end example
1638
1639 By default NetworkManager will connect automatically on system boot. To
1640 change that behaviour you need to edit your config:
1641
1642 @example shell
1643 # nmcli connection modify wg0 connection.autoconnect no
1644 @end example
1645
1646 For more specific information about NetworkManager and wireguard
1647 @uref{https://blogs.gnome.org/thaller/2019/03/15/wireguard-in-networkmanager/,see
1648 this post by thaller}.
1649
1650 @node Customizing a Window Manager
1651 @section Customizing a Window Manager
1652 @cindex wm
1653
1654 @node StumpWM
1655 @subsection StumpWM
1656 @cindex stumpwm
1657
1658 You could install StumpWM with a Guix system by adding
1659 @code{stumpwm} and optionally @code{`(,stumpwm "lib")}
1660 packages to a system configuration file, e.g.@: @file{/etc/config.scm}.
1661
1662 An example configuration can look like this:
1663
1664 @lisp
1665 (use-modules (gnu))
1666 (use-package-modules wm)
1667
1668 (operating-system
1669 ;; …
1670 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
1671 %base-packages)))
1672 @end lisp
1673
1674 @cindex stumpwm fonts
1675 By default StumpWM uses X11 fonts, which could be small or pixelated on
1676 your system. You could fix this by installing StumpWM contrib Lisp
1677 module @code{sbcl-ttf-fonts}, adding it to Guix system packages:
1678
1679 @lisp
1680 (use-modules (gnu))
1681 (use-package-modules fonts wm)
1682
1683 (operating-system
1684 ;; …
1685 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
1686 sbcl-ttf-fonts font-dejavu %base-packages)))
1687 @end lisp
1688
1689 Then you need to add the following code to a StumpWM configuration file
1690 @file{~/.stumpwm.d/init.lisp}:
1691
1692 @lisp
1693 (require :ttf-fonts)
1694 (setf xft:*font-dirs* '("/run/current-system/profile/share/fonts/"))
1695 (setf clx-truetype:+font-cache-filename+ (concat (getenv "HOME") "/.fonts/font-cache.sexp"))
1696 (xft:cache-fonts)
1697 (set-font (make-instance 'xft:font :family "DejaVu Sans Mono" :subfamily "Book" :size 11))
1698 @end lisp
1699
1700 @node Session lock
1701 @subsection Session lock
1702 @cindex sessionlock
1703
1704 Depending on your environment, locking the screen of your session might come built in
1705 or it might be something you have to set up yourself. If you use a desktop environment
1706 like GNOME or KDE, it's usually built in. If you use a plain window manager like
1707 StumpWM or EXWM, you might have to set it up yourself.
1708
1709 @node Xorg
1710 @subsubsection Xorg
1711
1712 If you use Xorg, you can use the utility
1713 @uref{https://www.mankier.com/1/xss-lock, xss-lock} to lock the screen of your session.
1714 xss-lock is triggered by DPMS which since Xorg 1.8 is auto-detected and enabled if
1715 ACPI is also enabled at kernel runtime.
1716
1717 To use xss-lock, you can simple execute it and put it into the background before
1718 you start your window manager from e.g. your @file{~/.xsession}:
1719
1720 @example
1721 xss-lock -- slock &
1722 exec stumpwm
1723 @end example
1724
1725 In this example, xss-lock uses @code{slock} to do the actual locking of the screen when
1726 it determines it's appropriate, like when you suspend your device.
1727
1728 For slock to be allowed to be a screen locker for the graphical session, it needs to
1729 be made setuid-root so it can authenticate users, and it needs a PAM service. This
1730 can be achieved by adding the following service to your @file{config.scm}:
1731
1732 @lisp
1733 (screen-locker-service slock)
1734 @end lisp
1735
1736 If you manually lock your screen, e.g. by directly calling slock when you want to lock
1737 your screen but not suspend it, it's a good idea to notify xss-lock about this so no
1738 confusion occurs. This can be done by executing @code{xset s activate} immediately
1739 before you execute slock.
1740
1741 @node Setting up a bind mount
1742 @section Setting up a bind mount
1743
1744 To bind mount a file system, one must first set up some definitions
1745 before the @code{operating-system} section of the system definition. In
1746 this example we will bind mount a folder from a spinning disk drive to
1747 @file{/tmp}, to save wear and tear on the primary SSD, without
1748 dedicating an entire partition to be mounted as @file{/tmp}.
1749
1750 First, the source drive that hosts the folder we wish to bind mount
1751 should be defined, so that the bind mount can depend on it.
1752
1753 @lisp
1754 (define source-drive ;; "source-drive" can be named anything you want.
1755 (file-system
1756 (device (uuid "UUID goes here"))
1757 (mount-point "/path-to-spinning-disk-goes-here")
1758 (type "ext4"))) ;; Make sure to set this to the appropriate type for your drive.
1759 @end lisp
1760
1761 The source folder must also be defined, so that guix will know it's not
1762 a regular block device, but a folder.
1763 @lisp
1764 (define (%source-directory) "/path-to-spinning-disk-goes-here/tmp") ;; "source-directory" can be named any valid variable name.
1765 @end lisp
1766
1767 Finally, inside the @code{file-systems} definition, we must add the
1768 mount itself.
1769
1770 @lisp
1771 (file-systems (cons*
1772
1773 ...<other drives omitted for clarity>...
1774
1775 source-drive ;; Must match the name you gave the source drive in the earlier definition.
1776
1777 (file-system
1778 (device (%source-directory)) ;; Make sure "source-directory" matches your earlier definition.
1779 (mount-point "/tmp")
1780 (type "none") ;; We are mounting a folder, not a partition, so this type needs to be "none"
1781 (flags '(bind-mount))
1782 (dependencies (list source-drive)) ;; Ensure "source-drive" matches what you've named the variable for the drive.
1783 )
1784
1785 ...<other drives omitted for clarity>...
1786
1787 ))
1788 @end lisp
1789
1790 @node Getting substitutes from Tor
1791 @section Getting substitutes from Tor
1792
1793 Guix daemon can use a HTTP proxy to get substitutes, here we are
1794 configuring it to get them via Tor.
1795
1796 @quotation Warning
1797 @emph{Not all} Guix daemon's traffic will go through Tor! Only
1798 HTTP/HTTPS will get proxied; FTP, Git protocol, SSH, etc connections
1799 will still go through the clearnet. Again, this configuration isn't
1800 foolproof some of your traffic won't get routed by Tor at all. Use it
1801 at your own risk.
1802 @end quotation
1803
1804 Guix's substitute server is available as a Onion service, if you want
1805 to use it to get your substitutes from Tor configure your system as
1806 follow:
1807
1808 @lisp
1809 (use-modules (gnu))
1810 (use-service-module base networking)
1811
1812 (operating-system
1813
1814 (services
1815 (cons
1816 (service tor-service-type
1817 (tor-configuration
1818 (config-file (plain-file "tor-config"
1819 "HTTPTunnelPort 127.0.0.1:9250"))))
1820 (modify-services %base-services
1821 (guix-service-type
1822 config => (guix-configuration
1823 (inherit config)
1824 ;; ci.guix.gnu.org's Onion service
1825 (substitute-urls "https://bp7o7ckwlewr4slm.onion")
1826 (http-proxy "http://localhost:9250")))))))
1827 @end lisp
1828
1829 This will keep a tor process running that provides a HTTP CONNECT tunnel
1830 which will be used by @command{guix-daemon}. The daemon can use other
1831 protocols than HTTP(S) to get remote resources, request using those
1832 protocols won't go through Tor since we are only setting a HTTP tunnel
1833 here. Note that @code{substitutes-urls} is using HTTPS and not HTTP or
1834 it won't work, that's a limitation of Tor's tunnel; you may want to use
1835 @command{privoxy} instead to avoid such limitations.
1836
1837 If you don't want to always get substitutes through Tor but using it just
1838 some of the times, then skip the @code{guix-configuration}. When you
1839 want to get a substitute from the Tor tunnel run:
1840
1841 @example
1842 sudo herd set-http-proxy guix-daemon http://localhost:9250
1843 guix build --substitute-urls=https://bp7o7ckwlewr4slm.onion …
1844 @end example
1845
1846 @c *********************************************************************
1847 @node Advanced package management
1848 @chapter Advanced package management
1849
1850 Guix is a functional package manager that offers many features beyond
1851 what more traditional package managers can do. To the uninitiated,
1852 those features might not have obvious use cases at first. The purpose
1853 of this chapter is to demonstrate some advanced package management
1854 concepts.
1855
1856 @pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
1857 reference.
1858
1859 @menu
1860 * Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
1861 @end menu
1862
1863 @node Guix Profiles in Practice
1864 @section Guix Profiles in Practice
1865
1866 Guix provides a very useful feature that may be quite foreign to newcomers:
1867 @emph{profiles}. They are a way to group package installations together and all users
1868 on the same system are free to use as many profiles as they want.
1869
1870 Whether you're a developer or not, you may find that multiple profiles bring you
1871 great power and flexibility. While they shift the paradigm somewhat compared to
1872 @emph{traditional package managers}, they are very convenient to use once you've
1873 understood how to set them up.
1874
1875 If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
1876 kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
1877 just Python software. Furthermore, profiles are self-sufficient: they capture
1878 all the runtime dependencies which guarantees that all programs within a profile
1879 will always work at any point in time.
1880
1881 Multiple profiles have many benefits:
1882
1883 @itemize
1884 @item
1885 Clean semantic separation of the various packages a user needs for different contexts.
1886
1887 @item
1888 Multiple profiles can be made available into the environment either on login
1889 or within a dedicated shell.
1890
1891 @item
1892 Profiles can be loaded on demand. For instance, the user can use multiple
1893 shells, each of them running different profiles.
1894
1895 @item
1896 Isolation: Programs from one profile will not use programs from the other, and
1897 the user can even install different versions of the same programs to the two
1898 profiles without conflict.
1899
1900 @item
1901 Deduplication: Profiles share dependencies that happens to be the exact same.
1902 This makes multiple profiles storage-efficient.
1903
1904 @item
1905 Reproducible: when used with declarative manifests, a profile can be fully
1906 specified by the Guix commit that was active when it was set up. This means
1907 that the exact same profile can be
1908 @uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
1909 set up anywhere and anytime}, with just the commit information. See the
1910 section on @ref{Reproducible profiles}.
1911
1912 @item
1913 Easier upgrades and maintenance: Multiple profiles make it easy to keep
1914 package listings at hand and make upgrades completely friction-less.
1915 @end itemize
1916
1917 Concretely, here follows some typical profiles:
1918
1919 @itemize
1920 @item
1921 The dependencies of a project you are working on.
1922
1923 @item
1924 Your favourite programming language libraries.
1925
1926 @item
1927 Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
1928
1929 @item
1930 @TeX{}live (this one can be really useful when you need to install just one
1931 package for this one document you've just received over email).
1932
1933 @item
1934 Games.
1935 @end itemize
1936
1937 Let's dive in the set up!
1938
1939 @node Basic setup with manifests
1940 @subsection Basic setup with manifests
1941
1942 A Guix profile can be set up @emph{via} a so-called @emph{manifest specification} that looks like
1943 this:
1944
1945 @lisp
1946 (specifications->manifest
1947 '("package-1"
1948 ;; Version 1.3 of package-2.
1949 "package-2@@1.3"
1950 ;; The "lib" output of package-3.
1951 "package-3:lib"
1952 ; ...
1953 "package-N"))
1954 @end lisp
1955
1956 @pxref{Invoking guix package,,, guix, GNU Guix Reference Manual}, for
1957 the syntax details.
1958
1959 We can create a manifest specification per profile and install them this way:
1960
1961 @example
1962 GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
1963 mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
1964 guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
1965 @end example
1966
1967 Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
1968 where we will store our profiles in the rest of this article.
1969
1970 Placing all your profiles in a single directory, with each profile getting its
1971 own sub-directory, is somewhat cleaner. This way, each sub-directory will
1972 contain all the symlinks for precisely one profile. Besides, ``looping over
1973 profiles'' becomes obvious from any programming language (e.g.@: a shell script) by
1974 simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
1975
1976 Note that it's also possible to loop over the output of
1977
1978 @example
1979 guix package --list-profiles
1980 @end example
1981
1982 although you'll probably have to filter out @file{~/.config/guix/current}.
1983
1984 To enable all profiles on login, add this to your @file{~/.bash_profile} (or similar):
1985
1986 @example
1987 for i in $GUIX_EXTRA_PROFILES/*; do
1988 profile=$i/$(basename "$i")
1989 if [ -f "$profile"/etc/profile ]; then
1990 GUIX_PROFILE="$profile"
1991 . "$GUIX_PROFILE"/etc/profile
1992 fi
1993 unset profile
1994 done
1995 @end example
1996
1997 Note to Guix System users: the above reflects how your default profile
1998 @file{~/.guix-profile} is activated from @file{/etc/profile}, that latter being loaded by
1999 @file{~/.bashrc} by default.
2000
2001 You can obviously choose to only enable a subset of them:
2002
2003 @example
2004 for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
2005 profile=$i/$(basename "$i")
2006 if [ -f "$profile"/etc/profile ]; then
2007 GUIX_PROFILE="$profile"
2008 . "$GUIX_PROFILE"/etc/profile
2009 fi
2010 unset profile
2011 done
2012 @end example
2013
2014 When a profile is off, it's straightforward to enable it for an individual shell
2015 without "polluting" the rest of the user session:
2016
2017 @example
2018 GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
2019 @end example
2020
2021 The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. This file
2022 contains shell code that exports the right environment variables necessary to
2023 activate the software contained in the profile. It is built automatically by
2024 Guix and meant to be sourced.
2025 It contains the same variables you would get if you ran:
2026
2027 @example
2028 guix package --search-paths=prefix --profile=$my_profile"
2029 @end example
2030
2031 Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual})
2032 for the command line options.
2033
2034 To upgrade a profile, simply install the manifest again:
2035
2036 @example
2037 guix package -m /path/to/guix-my-project-manifest.scm -p "$GUIX_EXTRA_PROFILES"/my-project/my-project
2038 @end example
2039
2040 To upgrade all profiles, it's easy enough to loop over them. For instance,
2041 assuming your manifest specifications are stored in
2042 @file{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
2043 of the profile (e.g.@: "project1"), you could do the following in Bourne shell:
2044
2045 @example
2046 for profile in "$GUIX_EXTRA_PROFILES"/*; do
2047 guix package --profile="$profile" --manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
2048 done
2049 @end example
2050
2051 Each profile has its own generations:
2052
2053 @example
2054 guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
2055 @end example
2056
2057 You can roll-back to any generation of a given profile:
2058
2059 @example
2060 guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
2061 @end example
2062
2063 Finally, if you want to switch to a profile without inheriting from the
2064 current environment, you can activate it from an empty shell:
2065
2066 @example
2067 env -i $(which bash) --login --noprofile --norc
2068 . my-project/etc/profile
2069 @end example
2070
2071 @node Required packages
2072 @subsection Required packages
2073
2074 Activating a profile essentially boils down to exporting a bunch of
2075 environmental variables. This is the role of the @samp{etc/profile} within the
2076 profile.
2077
2078 @emph{Note: Only the environmental variables of the packages that consume them will
2079 be set.}
2080
2081 For instance, @samp{MANPATH} won't be set if there is no consumer application for man
2082 pages within the profile. So if you need to transparently access man pages once
2083 the profile is loaded, you've got two options:
2084
2085 @itemize
2086 @item
2087 Either export the variable manually, e.g.
2088 @example
2089 export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH
2090 @end example
2091
2092 @item
2093 Or include @samp{man-db} to the profile manifest.
2094 @end itemize
2095
2096 The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
2097 @samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
2098
2099 @node Default profile
2100 @subsection Default profile
2101
2102 What about the default profile that Guix keeps in @file{~/.guix-profile}?
2103
2104 You can assign it the role you want. Typically you would install the manifest
2105 of the packages you want to use all the time.
2106
2107 Alternatively, you could keep it ``manifest-less'' for throw-away packages
2108 that you would just use for a couple of days.
2109 This way makes it convenient to run
2110
2111 @example
2112 guix install package-foo
2113 guix upgrade package-bar
2114 @end example
2115
2116 without having to specify the path to a profile.
2117
2118 @node The benefits of manifests
2119 @subsection The benefits of manifests
2120
2121 Manifests are a convenient way to keep your package lists around and, say,
2122 to synchronize them across multiple machines using a version control system.
2123
2124 A common complaint about manifests is that they can be slow to install when they
2125 contain large number of packages. This is especially cumbersome when you just
2126 want get an upgrade for one package within a big manifest.
2127
2128 This is one more reason to use multiple profiles, which happen to be just
2129 perfect to break down manifests into multiple sets of semantically connected
2130 packages. Using multiple, small profiles provides more flexibility and
2131 usability.
2132
2133 Manifests come with multiple benefits. In particular, they ease maintenance:
2134
2135 @itemize
2136 @item
2137 When a profile is set up from a manifest, the manifest itself is
2138 self-sufficient to keep a ``package listing'' around and reinstall the profile
2139 later or on a different system. For ad-hoc profiles, we would need to
2140 generate a manifest specification manually and maintain the package versions
2141 for the packages that don't use the default version.
2142
2143 @item
2144 @code{guix package --upgrade} always tries to update the packages that have
2145 propagated inputs, even if there is nothing to do. Guix manifests remove this
2146 problem.
2147
2148 @item
2149 When partially upgrading a profile, conflicts may arise (due to diverging
2150 dependencies between the updated and the non-updated packages) and they can be
2151 annoying to resolve manually. Manifests remove this problem altogether since
2152 all packages are always upgraded at once.
2153
2154 @item
2155 As mentioned above, manifests allow for reproducible profiles, while the
2156 imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
2157 different profiles every time even when they hold the same packages. See
2158 @uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
2159
2160 @item
2161 Manifest specifications are usable by other @samp{guix} commands. For example, you
2162 can run @code{guix weather -m manifest.scm} to see how many substitutes are
2163 available, which can help you decide whether you want to try upgrading today
2164 or wait a while. Another example: you can run @code{guix pack -m manifest.scm} to
2165 create a pack containing all the packages in the manifest (and their
2166 transitive references).
2167
2168 @item
2169 Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
2170 They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
2171 @end itemize
2172
2173 It's important to understand that while manifests can be used to declare
2174 profiles, they are not strictly equivalent: profiles have the side effect that
2175 they ``pin'' packages in the store, which prevents them from being
2176 garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
2177 and ensures that they will still be available at any point in
2178 the future.
2179
2180 Let's take an example:
2181
2182 @enumerate
2183 @item
2184 We have an environment for hacking on a project for which there isn't a Guix
2185 package yet. We build the environment using a manifest, and then run @code{guix
2186 environment -m manifest.scm}. So far so good.
2187
2188 @item
2189 Many weeks pass and we have run a couple of @code{guix pull} in the mean time.
2190 Maybe a dependency from our manifest has been updated; or we may have run
2191 @code{guix gc} and some packages needed by our manifest have been
2192 garbage-collected.
2193
2194 @item
2195 Eventually, we set to work on that project again, so we run @code{guix environment
2196 -m manifest.scm}. But now we have to wait for Guix to build and install
2197 stuff!
2198 @end enumerate
2199
2200 Ideally, we could spare the rebuild time. And indeed we can, all we need is to
2201 install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
2202 . "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
2203 hacking environment will be available at all times.
2204
2205 @emph{Security warning:} While keeping old profiles around can be convenient, keep in
2206 mind that outdated packages may not have received the latest security fixes.
2207
2208 @node Reproducible profiles
2209 @subsection Reproducible profiles
2210
2211 To reproduce a profile bit-for-bit, we need two pieces of information:
2212
2213 @itemize
2214 @item
2215 a manifest,
2216 @item
2217 a Guix channel specification.
2218 @end itemize
2219
2220 Indeed, manifests alone might not be enough: different Guix versions (or
2221 different channels) can produce different outputs for a given manifest.
2222
2223 You can output the Guix channel specification with @samp{guix describe
2224 --format=channels}.
2225 Save this to a file, say @samp{channel-specs.scm}.
2226
2227 On another computer, you can use the channel specification file and the manifest
2228 to reproduce the exact same profile:
2229
2230 @example
2231 GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
2232 GUIX_EXTRA=$HOME/.guix-extra
2233
2234 mkdir "$GUIX_EXTRA"/my-project
2235 guix pull --channels=channel-specs.scm --profile "$GUIX_EXTRA/my-project/guix"
2236
2237 mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
2238 "$GUIX_EXTRA"/my-project/guix/bin/guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
2239 @end example
2240
2241 It's safe to delete the Guix channel profile you've just installed with the
2242 channel specification, the project profile does not depend on it.
2243
2244 @c *********************************************************************
2245 @node Acknowledgments
2246 @chapter Acknowledgments
2247
2248 Guix is based on the @uref{https://nixos.org/nix/, Nix package manager},
2249 which was designed and
2250 implemented by Eelco Dolstra, with contributions from other people (see
2251 the @file{nix/AUTHORS} file in Guix.) Nix pioneered functional package
2252 management, and promoted unprecedented features, such as transactional
2253 package upgrades and rollbacks, per-user profiles, and referentially
2254 transparent build processes. Without this work, Guix would not exist.
2255
2256 The Nix-based software distributions, Nixpkgs and NixOS, have also been
2257 an inspiration for Guix.
2258
2259 GNU@tie{}Guix itself is a collective work with contributions from a
2260 number of people. See the @file{AUTHORS} file in Guix for more
2261 information on these fine people. The @file{THANKS} file lists people
2262 who have helped by reporting bugs, taking care of the infrastructure,
2263 providing artwork and themes, making suggestions, and more---thank you!
2264
2265 This document includes adapted sections from articles that have previously
2266 been published on the Guix blog at @uref{https://guix.gnu.org/blog}.
2267
2268
2269 @c *********************************************************************
2270 @node GNU Free Documentation License
2271 @appendix GNU Free Documentation License
2272 @cindex license, GNU Free Documentation License
2273 @include fdl-1.3.texi
2274
2275 @c *********************************************************************
2276 @node Concept Index
2277 @unnumbered Concept Index
2278 @printindex cp
2279
2280 @bye
2281
2282 @c Local Variables:
2283 @c ispell-local-dictionary: "american";
2284 @c End: