doc: cookbook: Clarify 'git-fetch' conventions.
[jackhill/guix/guix.git] / doc / guix-cookbook.texi
1 \input texinfo
2 @c -*-texinfo-*-
3
4 @c %**start of header
5 @setfilename guix-cookbook.info
6 @documentencoding UTF-8
7 @settitle GNU Guix Cookbook
8 @c %**end of header
9
10 @copying
11 Copyright @copyright{} 2019 Ricardo Wurmus@*
12 Copyright @copyright{} 2019 Efraim Flashner@*
13 Copyright @copyright{} 2019 Pierre Neidhardt@*
14 Copyright @copyright{} 2020 Oleg Pykhalov@*
15 Copyright @copyright{} 2020 Matthew Brooks@*
16 Copyright @copyright{} 2020 Marcin Karpezo@*
17 Copyright @copyright{} 2020 Brice Waegeneire@*
18
19 Permission is granted to copy, distribute and/or modify this document
20 under the terms of the GNU Free Documentation License, Version 1.3 or
21 any later version published by the Free Software Foundation; with no
22 Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
23 copy of the license is included in the section entitled ``GNU Free
24 Documentation License''.
25 @end copying
26
27 @dircategory System administration
28 @direntry
29 * Guix cookbook: (guix-cookbook). Tutorials and examples for GNU Guix.
30 @end direntry
31
32 @titlepage
33 @title GNU Guix Cookbook
34 @subtitle Tutorials and examples for using the GNU Guix Functional Package Manager
35 @author The GNU Guix Developers
36
37 @page
38 @vskip 0pt plus 1filll
39
40 @insertcopying
41 @end titlepage
42
43 @contents
44
45 @c *********************************************************************
46 @node Top
47 @top GNU Guix Cookbook
48
49 This document presents tutorials and detailed examples for GNU@tie{}Guix, a
50 functional package management tool written for the GNU system. Please
51 @pxref{Top,,, guix, GNU Guix reference manual} for details about the system,
52 its API, and related concepts.
53
54 @c TRANSLATORS: You can replace the following paragraph with information on
55 @c how to join your own translation team and how to report issues with the
56 @c translation.
57 If you would like to translate this document in your native language, consider
58 joining the @uref{https://translationproject.org/domain/guix-cookbook.html,
59 Translation Project}.
60
61 @menu
62 * Scheme tutorials:: Meet your new favorite language!
63 * Packaging:: Packaging tutorials
64 * System Configuration:: Customizing the GNU System
65 * Advanced package management:: Power to the users!
66
67 * Acknowledgments:: Thanks!
68 * GNU Free Documentation License:: The license of this document.
69 * Concept Index:: Concepts.
70
71 @detailmenu
72 --- The Detailed Node Listing ---
73
74 Scheme tutorials
75
76 * A Scheme Crash Course:: Learn the basics of Scheme
77
78 Packaging
79
80 * Packaging Tutorial:: Let's add a package to Guix!
81
82 System Configuration
83
84 * Customizing the Kernel:: Creating and using a custom Linux kernel
85
86
87 @end detailmenu
88 @end menu
89
90 @c *********************************************************************
91 @node Scheme tutorials
92 @chapter Scheme tutorials
93
94 GNU@tie{}Guix is written in the general purpose programming language Scheme,
95 and many of its features can be accessed and manipulated programmatically.
96 You can use Scheme to generate package definitions, to modify them, to build
97 them, to deploy whole operating systems, etc.
98
99 Knowing the basics of how to program in Scheme will unlock many of the
100 advanced features Guix provides --- and you don't even need to be an
101 experienced programmer to use them!
102
103 Let's get started!
104
105 @node A Scheme Crash Course
106 @section A Scheme Crash Course
107
108 @cindex Scheme, crash course
109
110 Guix uses the Guile implementation of Scheme. To start playing with the
111 language, install it with @code{guix install guile} and start a
112 @uref{https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop,
113 REPL} by running @code{guile} from the command line.
114
115 Alternatively you can also run @code{guix environment --ad-hoc guile -- guile}
116 if you'd rather not have Guile installed in your user profile.
117
118 In the following examples, lines show what you would type at the REPL;
119 lines starting with ``@result{}'' show evaluation results, while lines
120 starting with ``@print{}'' show things that get printed. @xref{Using Guile
121 Interactively,,, guile, GNU Guile Reference Manual}), for more details on the
122 REPL.
123
124 @itemize
125 @item
126 Scheme syntax boils down to a tree of expressions (or @emph{s-expression} in
127 Lisp lingo). An expression can be a literal such as numbers and strings, or a
128 compound which is a parenthesized list of compounds and literals. @code{#t}
129 and @code{#f} stand for the Booleans ``true'' and ``false'', respectively.
130
131 Examples of valid expressions:
132
133 @lisp
134 "Hello World!"
135 @result{} "Hello World!"
136
137 17
138 @result{} 17
139
140 (display (string-append "Hello " "Guix" "\n"))
141 @print{} Hello Guix!
142 @result{} #<unspecified>
143 @end lisp
144
145 @item
146 This last example is a function call nested in another function call. When a
147 parenthesized expression is evaluated, the first term is the function and the
148 rest are the arguments passed to the function. Every function returns the
149 last evaluated expression as its return value.
150
151 @item
152 Anonymous functions are declared with the @code{lambda} term:
153
154 @lisp
155 (lambda (x) (* x x))
156 @result{} #<procedure 120e348 at <unknown port>:24:0 (x)>
157 @end lisp
158
159 The above procedure returns the square of its argument. Since everything is
160 an expression, the @code{lambda} expression returns an anonymous procedure,
161 which can in turn be applied to an argument:
162
163 @lisp
164 ((lambda (x) (* x x)) 3)
165 @result{} 9
166 @end lisp
167
168 @item
169 Anything can be assigned a global name with @code{define}:
170
171 @lisp
172 (define a 3)
173 (define square (lambda (x) (* x x)))
174 (square a)
175 @result{} 9
176 @end lisp
177
178 @item
179 Procedures can be defined more concisely with the following syntax:
180
181 @lisp
182 (define (square x) (* x x))
183 @end lisp
184
185 @item
186 A list structure can be created with the @code{list} procedure:
187
188 @lisp
189 (list 2 a 5 7)
190 @result{} (2 3 5 7)
191 @end lisp
192
193 @item
194 The @dfn{quote} disables evaluation of a parenthesized expression: the
195 first term is not called over the other terms (@pxref{Expression Syntax,
196 quote,, guile, GNU Guile Reference Manual}). Thus it effectively
197 returns a list of terms.
198
199 @lisp
200 '(display (string-append "Hello " "Guix" "\n"))
201 @result{} (display (string-append "Hello " "Guix" "\n"))
202
203 '(2 a 5 7)
204 @result{} (2 a 5 7)
205 @end lisp
206
207 @item
208 The @dfn{quasiquote} disables evaluation of a parenthesized expression
209 until @dfn{unquote} (a comma) re-enables it. Thus it provides us with
210 fine-grained control over what is evaluated and what is not.
211
212 @lisp
213 `(2 a 5 7 (2 ,a 5 ,(+ a 4)))
214 @result{} (2 a 5 7 (2 3 5 7))
215 @end lisp
216
217 Note that the above result is a list of mixed elements: numbers, symbols (here
218 @code{a}) and the last element is a list itself.
219
220 @item
221 Multiple variables can be named locally with @code{let} (@pxref{Local
222 Bindings,,, guile, GNU Guile Reference Manual}):
223
224 @lisp
225 (define x 10)
226 (let ((x 2)
227 (y 3))
228 (list x y))
229 @result{} (2 3)
230
231 x
232 @result{} 10
233
234 y
235 @error{} In procedure module-lookup: Unbound variable: y
236 @end lisp
237
238 Use @code{let*} to allow later variable declarations to refer to earlier
239 definitions.
240
241 @lisp
242 (let* ((x 2)
243 (y (* x 3)))
244 (list x y))
245 @result{} (2 6)
246 @end lisp
247
248 @item
249 The keyword syntax is @code{#:}; it is used to create unique identifiers.
250 @pxref{Keywords,,, guile, GNU Guile Reference Manual}.
251
252 @item
253 The percentage @code{%} is typically used for read-only global variables in
254 the build stage. Note that it is merely a convention, like @code{_} in C.
255 Scheme treats @code{%} exactly the same as any other letter.
256
257 @item
258 Modules are created with @code{define-module} (@pxref{Creating Guile
259 Modules,,, guile, GNU Guile Reference Manual}). For instance
260
261 @lisp
262 (define-module (guix build-system ruby)
263 #:use-module (guix store)
264 #:export (ruby-build
265 ruby-build-system))
266 @end lisp
267
268 defines the module @code{guix build-system ruby} which must be located in
269 @file{guix/build-system/ruby.scm} somewhere in the Guile load path. It
270 depends on the @code{(guix store)} module and it exports two variables,
271 @code{ruby-build} and @code{ruby-build-system}.
272 @end itemize
273
274 For a more detailed introduction, check out
275 @uref{http://www.troubleshooters.com/codecorn/scheme_guile/hello.htm, Scheme
276 at a Glance}, by Steve Litt.
277
278 One of the reference Scheme books is the seminal ``Structure and
279 Interpretation of Computer Programs'', by Harold Abelson and Gerald Jay
280 Sussman, with Julie Sussman. You'll find a
281 @uref{https://mitpress.mit.edu/sites/default/files/sicp/index.html, free copy
282 online}, together with
283 @uref{https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/,
284 videos of the lectures by the authors}. The book is available in Texinfo
285 format as the @code{sicp} Guix package. Go ahead, run @code{guix install
286 sicp} and start reading with @code{info sicp} (@pxref{,,, sicp, Structure and Interpretation of Computer Programs}).
287 An @uref{https://sarabander.github.io/sicp/, unofficial ebook is also
288 available}.
289
290 You'll find more books, tutorials and other resources at
291 @url{https://schemers.org/}.
292
293
294 @c *********************************************************************
295 @node Packaging
296 @chapter Packaging
297
298 @cindex packaging
299
300 This chapter is dedicated to teaching you how to add packages to the
301 collection of packages that come with GNU Guix. This involves writing package
302 definitions in Guile Scheme, organizing them in package modules, and building
303 them.
304
305 @menu
306 * Packaging Tutorial:: A tutorial on how to add packages to Guix.
307 @end menu
308
309 @node Packaging Tutorial
310 @section Packaging Tutorial
311
312 GNU Guix stands out as the @emph{hackable} package manager, mostly because it
313 uses @uref{https://www.gnu.org/software/guile/, GNU Guile}, a powerful
314 high-level programming language, one of the
315 @uref{https://en.wikipedia.org/wiki/Scheme_%28programming_language%29, Scheme}
316 dialects from the
317 @uref{https://en.wikipedia.org/wiki/Lisp_%28programming_language%29, Lisp family}.
318
319 Package definitions are also written in Scheme, which empowers Guix in some
320 very unique ways, unlike most other package managers that use shell scripts or
321 simple languages.
322
323 @itemize
324 @item
325 Use functions, structures, macros and all of Scheme expressiveness for your
326 package definitions.
327
328 @item
329 Inheritance makes it easy to customize a package by inheriting from it and
330 modifying only what is needed.
331
332 @item
333 Batch processing: the whole package collection can be parsed, filtered and
334 processed. Building a headless server with all graphical interfaces stripped
335 out? It's possible. Want to rebuild everything from source using specific
336 compiler optimization flags? Pass the @code{#:make-flags "..."} argument to
337 the list of packages. It wouldn't be a stretch to think
338 @uref{https://wiki.gentoo.org/wiki/USE_flag, Gentoo USE flags} here, but this
339 goes even further: the changes don't have to be thought out beforehand by the
340 packager, they can be @emph{programmed} by the user!
341 @end itemize
342
343 The following tutorial covers all the basics around package creation with Guix.
344 It does not assume much knowledge of the Guix system nor of the Lisp language.
345 The reader is only expected to be familiar with the command line and to have some
346 basic programming knowledge.
347
348 @node A ``Hello World'' package
349 @subsection A ``Hello World'' package
350
351 The ``Defining Packages'' section of the manual introduces the basics of Guix
352 packaging (@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}). In
353 the following section, we will partly go over those basics again.
354
355 GNU@tie{}Hello is a dummy project that serves as an idiomatic example for
356 packaging. It uses the GNU build system (@code{./configure && make && make
357 install}). Guix already provides a package definition which is a perfect
358 example to start with. You can look up its declaration with @code{guix edit
359 hello} from the command line. Let's see how it looks:
360
361 @lisp
362 (define-public hello
363 (package
364 (name "hello")
365 (version "2.10")
366 (source (origin
367 (method url-fetch)
368 (uri (string-append "mirror://gnu/hello/hello-" version
369 ".tar.gz"))
370 (sha256
371 (base32
372 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
373 (build-system gnu-build-system)
374 (synopsis "Hello, GNU world: An example GNU package")
375 (description
376 "GNU Hello prints the message \"Hello, world!\" and then exits. It
377 serves as an example of standard GNU coding practices. As such, it supports
378 command-line arguments, multiple languages, and so on.")
379 (home-page "https://www.gnu.org/software/hello/")
380 (license gpl3+)))
381 @end lisp
382
383 As you can see, most of it is rather straightforward. But let's review the
384 fields together:
385
386 @table @samp
387 @item name
388 The project name. Using Scheme conventions, we prefer to keep it
389 lower case, without underscore and using dash-separated words.
390
391 @item source
392 This field contains a description of the source code origin. The
393 @code{origin} record contains these fields:
394
395 @enumerate
396 @item The method, here @code{url-fetch} to download via HTTP/FTP, but other methods
397 exist, such as @code{git-fetch} for Git repositories.
398 @item The URI, which is typically some @code{https://} location for @code{url-fetch}. Here
399 the special `mirror://gnu` refers to a set of well known locations, all of
400 which can be used by Guix to fetch the source, should some of them fail.
401 @item The @code{sha256} checksum of the requested file. This is essential to ensure
402 the source is not corrupted. Note that Guix works with base32 strings,
403 hence the call to the @code{base32} function.
404 @end enumerate
405
406 @item build-system
407
408 This is where the power of abstraction provided by the Scheme language really
409 shines: in this case, the @code{gnu-build-system} abstracts away the famous
410 @code{./configure && make && make install} shell invocations. Other build
411 systems include the @code{trivial-build-system} which does not do anything and
412 requires from the packager to program all the build steps, the
413 @code{python-build-system}, the @code{emacs-build-system}, and many more
414 (@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
415
416 @item synopsis
417 It should be a concise summary of what the package does. For many packages a
418 tagline from the project's home page can be used as the synopsis.
419
420 @item description
421 Same as for the synopsis, it's fine to re-use the project description from the
422 homepage. Note that Guix uses Texinfo syntax.
423
424 @item home-page
425 Use HTTPS if available.
426
427 @item license
428 See @code{guix/licenses.scm} in the project source for a full list of
429 available licenses.
430 @end table
431
432 Time to build our first package! Nothing fancy here for now: we will stick to a
433 dummy @code{my-hello}, a copy of the above declaration.
434
435 As with the ritualistic ``Hello World'' taught with most programming languages,
436 this will possibly be the most ``manual'' approach. We will work out an ideal
437 setup later; for now we will go the simplest route.
438
439 Save the following to a file @file{my-hello.scm}.
440
441 @lisp
442 (use-modules (guix packages)
443 (guix download)
444 (guix build-system gnu)
445 (guix licenses))
446
447 (package
448 (name "my-hello")
449 (version "2.10")
450 (source (origin
451 (method url-fetch)
452 (uri (string-append "mirror://gnu/hello/hello-" version
453 ".tar.gz"))
454 (sha256
455 (base32
456 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
457 (build-system gnu-build-system)
458 (synopsis "Hello, Guix world: An example custom Guix package")
459 (description
460 "GNU Hello prints the message \"Hello, world!\" and then exits. It
461 serves as an example of standard GNU coding practices. As such, it supports
462 command-line arguments, multiple languages, and so on.")
463 (home-page "https://www.gnu.org/software/hello/")
464 (license gpl3+))
465 @end lisp
466
467 We will explain the extra code in a moment.
468
469 Feel free to play with the different values of the various fields. If you
470 change the source, you'll need to update the checksum. Indeed, Guix refuses to
471 build anything if the given checksum does not match the computed checksum of the
472 source code. To obtain the correct checksum of the package declaration, we
473 need to download the source, compute the sha256 checksum and convert it to
474 base32.
475
476 Thankfully, Guix can automate this task for us; all we need is to provide the
477 URI:
478
479 @c TRANSLATORS: This is example shell output.
480 @example sh
481 $ guix download mirror://gnu/hello/hello-2.10.tar.gz
482
483 Starting download of /tmp/guix-file.JLYgL7
484 From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
485 following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
486 …10.tar.gz 709KiB 2.5MiB/s 00:00 [##################] 100.0%
487 /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
488 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
489 @end example
490
491 In this specific case the output tells us which mirror was chosen.
492 If the result of the above command is not the same as in the above snippet,
493 update your @code{my-hello} declaration accordingly.
494
495 Note that GNU package tarballs come with an OpenPGP signature, so you
496 should definitely check the signature of this tarball with `gpg` to
497 authenticate it before going further:
498
499 @c TRANSLATORS: This is example shell output.
500 @example sh
501 $ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig
502
503 Starting download of /tmp/guix-file.03tFfb
504 From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
505 following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
506 ….tar.gz.sig 819B 1.2MiB/s 00:00 [##################] 100.0%
507 /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
508 0q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
509 $ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
510 gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
511 gpg: using RSA key A9553245FDE9B739
512 gpg: Good signature from "Sami Kerola <kerolasa@@iki.fi>" [unknown]
513 gpg: aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@@iki.fi>" [unknown]
514 gpg: WARNING: This key is not certified with a trusted signature!
515 gpg: There is no indication that the signature belongs to the owner.
516 Primary key fingerprint: 8ED3 96E3 7E38 D471 A005 30D3 A955 3245 FDE9 B739
517 @end example
518
519 You can then happily run
520
521 @c TRANSLATORS: Do not translate this command
522 @example sh
523 $ guix package --install-from-file=my-hello.scm
524 @end example
525
526 You should now have @code{my-hello} in your profile!
527
528 @c TRANSLATORS: Do not translate this command
529 @example sh
530 $ guix package --list-installed=my-hello
531 my-hello 2.10 out
532 /gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10
533 @end example
534
535 We've gone as far as we could without any knowledge of Scheme. Before moving
536 on to more complex packages, now is the right time to brush up on your Scheme
537 knowledge. @pxref{A Scheme Crash Course} to get up to speed.
538
539 @node Setup
540 @subsection Setup
541
542 In the rest of this chapter we will rely on some basic Scheme
543 programming knowledge. Now let's detail the different possible setups
544 for working on Guix packages.
545
546 There are several ways to set up a Guix packaging environment.
547
548 We recommend you work directly on the Guix source checkout since it makes it
549 easier for everyone to contribute to the project.
550
551 But first, let's look at other possibilities.
552
553 @node Local file
554 @subsubsection Local file
555
556 This is what we previously did with @samp{my-hello}. With the Scheme basics we've
557 covered, we are now able to explain the leading chunks. As stated in @code{guix
558 package --help}:
559
560 @example
561 -f, --install-from-file=FILE
562 install the package that the code within FILE
563 evaluates to
564 @end example
565
566 Thus the last expression @emph{must} return a package, which is the case in our
567 earlier example.
568
569 The @code{use-modules} expression tells which of the modules we need in the file.
570 Modules are a collection of values and procedures. They are commonly called
571 ``libraries'' or ``packages'' in other programming languages.
572
573 @node @samp{GUIX_PACKAGE_PATH}
574 @subsubsection @samp{GUIX_PACKAGE_PATH}
575
576 @emph{Note: Starting from Guix 0.16, the more flexible Guix @dfn{channels} are the
577 preferred way and supersede @samp{GUIX_PACKAGE_PATH}. See next section.}
578
579 It can be tedious to specify the file from the command line instead of simply
580 calling @code{guix package --install my-hello} as you would do with the official
581 packages.
582
583 Guix makes it possible to streamline the process by adding as many ``package
584 declaration directories'' as you want.
585
586 Create a directory, say @file{~./guix-packages} and add it to the @samp{GUIX_PACKAGE_PATH}
587 environment variable:
588
589 @example
590 $ mkdir ~/guix-packages
591 $ export GUIX_PACKAGE_PATH=~/guix-packages
592 @end example
593
594 To add several directories, separate them with a colon (@code{:}).
595
596 Our previous @samp{my-hello} needs some adjustments though:
597
598 @lisp
599 (define-module (my-hello)
600 #:use-module (guix licenses)
601 #:use-module (guix packages)
602 #:use-module (guix build-system gnu)
603 #:use-module (guix download))
604
605 (define-public my-hello
606 (package
607 (name "my-hello")
608 (version "2.10")
609 (source (origin
610 (method url-fetch)
611 (uri (string-append "mirror://gnu/hello/hello-" version
612 ".tar.gz"))
613 (sha256
614 (base32
615 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
616 (build-system gnu-build-system)
617 (synopsis "Hello, Guix world: An example custom Guix package")
618 (description
619 "GNU Hello prints the message \"Hello, world!\" and then exits. It
620 serves as an example of standard GNU coding practices. As such, it supports
621 command-line arguments, multiple languages, and so on.")
622 (home-page "https://www.gnu.org/software/hello/")
623 (license gpl3+)))
624 @end lisp
625
626 Note that we have assigned the package value to an exported variable name with
627 @code{define-public}. This is effectively assigning the package to the @code{my-hello}
628 variable so that it can be referenced, among other as dependency of other
629 packages.
630
631 If you use @code{guix package --install-from-file=my-hello.scm} on the above file, it
632 will fail because the last expression, @code{define-public}, does not return a
633 package. If you want to use @code{define-public} in this use-case nonetheless, make
634 sure the file ends with an evaluation of @code{my-hello}:
635
636 @lisp
637 ; ...
638 (define-public my-hello
639 ; ...
640 )
641
642 my-hello
643 @end lisp
644
645 This last example is not very typical.
646
647 Now @samp{my-hello} should be part of the package collection like all other official
648 packages. You can verify this with:
649
650 @example
651 $ guix package --show=my-hello
652 @end example
653
654 @node Guix channels
655 @subsubsection Guix channels
656
657 Guix 0.16 features channels, which is very similar to @samp{GUIX_PACKAGE_PATH} but
658 provides better integration and provenance tracking. Channels are not
659 necessarily local, they can be maintained as a public Git repository for
660 instance. Of course, several channels can be used at the same time.
661
662 @xref{Channels,,, guix, GNU Guix Reference Manual} for setup details.
663
664 @node Direct checkout hacking
665 @subsubsection Direct checkout hacking
666
667 Working directly on the Guix project is recommended: it reduces the friction
668 when the time comes to submit your changes upstream to let the community benefit
669 from your hard work!
670
671 Unlike most software distributions, the Guix repository holds in one place both
672 the tooling (including the package manager) and the package definitions. This
673 choice was made so that it would give developers the flexibility to modify the
674 API without breakage by updating all packages at the same time. This reduces
675 development inertia.
676
677 Check out the official @uref{https://git-scm.com/, Git} repository:
678
679 @example
680 $ git clone https://git.savannah.gnu.org/git/guix.git
681 @end example
682
683 In the rest of this article, we use @samp{$GUIX_CHECKOUT} to refer to the location of
684 the checkout.
685
686
687 Follow the instructions in the manual (@pxref{Contributing,,, guix, GNU Guix
688 Reference Manual}) to set up the repository environment.
689
690 Once ready, you should be able to use the package definitions from the
691 repository environment.
692
693 Feel free to edit package definitions found in @samp{$GUIX_CHECKOUT/gnu/packages}.
694
695 The @samp{$GUIX_CHECKOUT/pre-inst-env} script lets you use @samp{guix} over the package
696 collection of the repository (@pxref{Running Guix Before It Is
697 Installed,,, guix, GNU Guix Reference Manual}).
698
699 @itemize
700 @item
701 Search packages, such as Ruby:
702
703 @example
704 $ cd $GUIX_CHECKOUT
705 $ ./pre-inst-env guix package --list-available=ruby
706 ruby 1.8.7-p374 out gnu/packages/ruby.scm:119:2
707 ruby 2.1.6 out gnu/packages/ruby.scm:91:2
708 ruby 2.2.2 out gnu/packages/ruby.scm:39:2
709 @end example
710
711 @item
712 Build a package, here Ruby version 2.1:
713
714 @example
715 $ ./pre-inst-env guix build --keep-failed ruby@@2.1
716 /gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
717 @end example
718
719 @item
720 Install it to your user profile:
721
722 @example
723 $ ./pre-inst-env guix package --install ruby@@2.1
724 @end example
725
726 @item
727 Check for common mistakes:
728
729 @example
730 $ ./pre-inst-env guix lint ruby@@2.1
731 @end example
732 @end itemize
733
734 Guix strives at maintaining a high packaging standard; when contributing to the
735 Guix project, remember to
736
737 @itemize
738 @item
739 follow the coding style (@pxref{Coding Style,,, guix, GNU Guix Reference Manual}),
740 @item
741 and review the check list from the manual (@pxref{Submitting Patches,,, guix, GNU Guix Reference Manual}).
742 @end itemize
743
744 Once you are happy with the result, you are welcome to send your contribution to
745 make it part of Guix. This process is also detailed in the manual. (@pxref{Contributing,,, guix, GNU Guix Reference Manual})
746
747
748 It's a community effort so the more join in, the better Guix becomes!
749
750 @node Extended example
751 @subsection Extended example
752
753 The above ``Hello World'' example is as simple as it goes. Packages can be more
754 complex than that and Guix can handle more advanced scenarios. Let's look at
755 another, more sophisticated package (slightly modified from the source):
756
757 @lisp
758 (define-module (gnu packages version-control)
759 #:use-module ((guix licenses) #:prefix license:)
760 #:use-module (guix utils)
761 #:use-module (guix packages)
762 #:use-module (guix git-download)
763 #:use-module (guix build-system cmake)
764 #:use-module (gnu packages ssh)
765 #:use-module (gnu packages web)
766 #:use-module (gnu packages pkg-config)
767 #:use-module (gnu packages python)
768 #:use-module (gnu packages compression)
769 #:use-module (gnu packages tls))
770
771 (define-public my-libgit2
772 (let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
773 (revision "1"))
774 (package
775 (name "my-libgit2")
776 (version (git-version "0.26.6" revision commit))
777 (source (origin
778 (method git-fetch)
779 (uri (git-reference
780 (url "https://github.com/libgit2/libgit2/")
781 (commit commit)))
782 (file-name (git-file-name name version))
783 (sha256
784 (base32
785 "17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
786 (patches (search-patches "libgit2-mtime-0.patch"))
787 (modules '((guix build utils)))
788 (snippet '(begin
789 ;; Remove bundled software.
790 (delete-file-recursively "deps")
791 #t))))
792 (build-system cmake-build-system)
793 (outputs '("out" "debug"))
794 (arguments
795 `(#:tests? #t ; Run the test suite (this is the default)
796 #:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
797 #:phases
798 (modify-phases %standard-phases
799 (add-after 'unpack 'fix-hardcoded-paths
800 (lambda _
801 (substitute* "tests/repo/init.c"
802 (("#!/bin/sh") (string-append "#!" (which "sh"))))
803 (substitute* "tests/clar/fs.h"
804 (("/bin/cp") (which "cp"))
805 (("/bin/rm") (which "rm")))
806 #t))
807 ;; Run checks more verbosely.
808 (replace 'check
809 (lambda _ (invoke "./libgit2_clar" "-v" "-Q")))
810 (add-after 'unpack 'make-files-writable-for-tests
811 (lambda _ (for-each make-file-writable (find-files "." ".*")))))))
812 (inputs
813 `(("libssh2" ,libssh2)
814 ("http-parser" ,http-parser)
815 ("python" ,python-wrapper)))
816 (native-inputs
817 `(("pkg-config" ,pkg-config)))
818 (propagated-inputs
819 ;; These two libraries are in 'Requires.private' in libgit2.pc.
820 `(("openssl" ,openssl)
821 ("zlib" ,zlib)))
822 (home-page "https://libgit2.github.com/")
823 (synopsis "Library providing Git core methods")
824 (description
825 "Libgit2 is a portable, pure C implementation of the Git core methods
826 provided as a re-entrant linkable library with a solid API, allowing you to
827 write native speed custom Git applications in any language with bindings.")
828 ;; GPLv2 with linking exception
829 (license license:gpl2))))
830 @end lisp
831
832 (In those cases were you only want to tweak a few fields from a package
833 definition, you should rely on inheritance instead of copy-pasting everything.
834 See below.)
835
836 Let's discuss those fields in depth.
837
838 @subsubsection @code{git-fetch} method
839
840 Unlike the @code{url-fetch} method, @code{git-fetch} expects a @code{git-reference} which takes
841 a Git repository and a commit. The commit can be any Git reference such as
842 tags, so if the @code{version} is tagged, then it can be used directly. Sometimes
843 the tag is prefixed with a @code{v}, in which case you'd use @code{(commit (string-append
844 "v" version))}.
845
846 To ensure that the source code from the Git repository is stored in a
847 directory with a descriptive name, we use @code{(file-name (git-file-name name
848 version))}.
849
850 The @code{git-version} procedure that can be used to derive the
851 version when packaging programs for a specific commit, following the
852 Guix contributor guidelines (@pxref{Version Numbers,,, guix, GNU Guix
853 Reference Manual}).
854
855 @subsubsection Snippets
856
857 Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching
858 the source. They are a Guix-y alternative to the traditional @file{.patch} files.
859 Because of the quote, the code in only evaluated when passed to the Guix daemon
860 for building. There can be as many snippets as needed.
861
862 Snippets might need additional Guile modules which can be imported from the
863 @code{modules} field.
864
865 @subsubsection Inputs
866
867 First, a syntactic comment: See the quasi-quote / comma syntax?
868
869 @lisp
870 (native-inputs
871 `(("pkg-config" ,pkg-config)))
872 @end lisp
873
874 is equivalent to
875
876 @lisp
877 (native-inputs
878 (list (list "pkg-config" pkg-config)))
879 @end lisp
880
881 You'll mostly see the former because it's shorter.
882
883 There are 3 different input types. In short:
884
885 @table @asis
886 @item native-inputs
887 Required for building but not runtime -- installing a package
888 through a substitute won't install these inputs.
889 @item inputs
890 Installed in the store but not in the profile, as well as being
891 present at build time.
892 @item propagated-inputs
893 Installed in the store and in the profile, as well as
894 being present at build time.
895 @end table
896
897 @xref{Package Reference,,, guix, GNU Guix Reference Manual} for more details.
898
899 The distinction between the various inputs is important: if a dependency can be
900 handled as an @emph{input} instead of a @emph{propagated input}, it should be done so, or
901 else it ``pollutes'' the user profile for no good reason.
902
903 For instance, a user installing a graphical program that depends on a
904 command line tool might only be interested in the graphical part, so there is no
905 need to force the command line tool into the user profile. The dependency is a
906 concern to the package, not to the user. @emph{Inputs} make it possible to handle
907 dependencies without bugging the user by adding undesired executable files (or
908 libraries) to their profile.
909
910 Same goes for @emph{native-inputs}: once the program is installed, build-time
911 dependencies can be safely garbage-collected.
912 It also matters when a substitute is available, in which case only the @emph{inputs}
913 and @emph{propagated inputs} will be fetched: the @emph{native inputs} are not required to
914 install a package from a substitute.
915
916 @subsubsection Outputs
917
918 Just like how a package can have multiple inputs, it can also produce multiple
919 outputs.
920
921 Each output corresponds to a separate directory in the store.
922
923 The user can choose which output to install; this is useful to save space or
924 to avoid polluting the user profile with unwanted executables or libraries.
925
926 Output separation is optional. When the @code{outputs} field is left out, the
927 default and only output (the complete package) is referred to as @code{"out"}.
928
929 Typical separate output names include @code{debug} and @code{doc}.
930
931 It's advised to separate outputs only when you've shown it's worth it: if the
932 output size is significant (compare with @code{guix size}) or in case the package is
933 modular.
934
935 @subsubsection Build system arguments
936
937 The @code{arguments} is a keyword-value list used to configure the build process.
938
939 The simplest argument @code{#:tests?} can be used to disable the test suite when
940 building the package. This is mostly useful when the package does not feature
941 any test suite. It's strongly recommended to keep the test suite on if there is
942 one.
943
944 Another common argument is @code{:make-flags}, which specifies a list of flags to
945 append when running make, as you would from the command line. For instance, the
946 following flags
947
948 @lisp
949 #:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
950 "CC=gcc")
951 @end lisp
952
953 translate into
954
955 @example
956 $ make CC=gcc prefix=/gnu/store/...-<out>
957 @end example
958
959 This sets the C compiler to @code{gcc} and the @code{prefix} variable (the installation
960 directory in Make parlance) to @code{(assoc-ref %outputs "out")}, which is a build-stage
961 global variable pointing to the destination directory in the store (something like
962 @file{/gnu/store/...-my-libgit2-20180408}).
963
964 Similarly, it's possible to set the configure flags:
965
966 @lisp
967 #:configure-flags '("-DUSE_SHA1DC=ON")
968 @end lisp
969
970 The @code{%build-inputs} variable is also generated in scope. It's an association
971 table that maps the input names to their store directories.
972
973 The @code{phases} keyword lists the sequential steps of the build system. Typically
974 phases include @code{unpack}, @code{configure}, @code{build}, @code{install} and @code{check}. To know
975 more about those phases, you need to work out the appropriate build system
976 definition in @samp{$GUIX_CHECKOUT/guix/build/gnu-build-system.scm}:
977
978 @lisp
979 (define %standard-phases
980 ;; Standard build phases, as a list of symbol/procedure pairs.
981 (let-syntax ((phases (syntax-rules ()
982 ((_ p ...) `((p . ,p) ...)))))
983 (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
984 bootstrap
985 patch-usr-bin-file
986 patch-source-shebangs configure patch-generated-file-shebangs
987 build check install
988 patch-shebangs strip
989 validate-runpath
990 validate-documentation-location
991 delete-info-dir-file
992 patch-dot-desktop-files
993 install-license-files
994 reset-gzip-timestamps
995 compress-documentation)))
996 @end lisp
997
998 Or from the REPL:
999
1000 @lisp
1001 (add-to-load-path "/path/to/guix/checkout")
1002 ,use (guix build gnu-build-system)
1003 (map first %standard-phases)
1004 @result{} (set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap patch-usr-bin-file patch-source-shebangs configure patch-generated-file-shebangs build check install patch-shebangs strip validate-runpath validate-documentation-location delete-info-dir-file patch-dot-desktop-files install-license-files reset-gzip-timestamps compress-documentation)
1005 @end lisp
1006
1007 If you want to know more about what happens during those phases, consult the
1008 associated procedures.
1009
1010 For instance, as of this writing the definition of @code{unpack} for the GNU build
1011 system is
1012
1013 @lisp
1014 (define* (unpack #:key source #:allow-other-keys)
1015 "Unpack SOURCE in the working directory, and change directory within the
1016 source. When SOURCE is a directory, copy it in a sub-directory of the current
1017 working directory."
1018 (if (file-is-directory? source)
1019 (begin
1020 (mkdir "source")
1021 (chdir "source")
1022
1023 ;; Preserve timestamps (set to the Epoch) on the copied tree so that
1024 ;; things work deterministically.
1025 (copy-recursively source "."
1026 #:keep-mtime? #t))
1027 (begin
1028 (if (string-suffix? ".zip" source)
1029 (invoke "unzip" source)
1030 (invoke "tar" "xvf" source))
1031 (chdir (first-subdirectory "."))))
1032 #t)
1033 @end lisp
1034
1035 Note the @code{chdir} call: it changes the working directory to where the source was
1036 unpacked.
1037 Thus every phase following the @code{unpack} will use the source as a working
1038 directory, which is why we can directly work on the source files.
1039 That is to say, unless a later phase changes the working directory to something
1040 else.
1041
1042 We modify the list of @code{%standard-phases} of the build system with the
1043 @code{modify-phases} macro as per the list of specified modifications, which may have
1044 the following forms:
1045
1046 @itemize
1047 @item
1048 @code{(add-before PHASE NEW-PHASE PROCEDURE)}: Run @code{PROCEDURE} named @code{NEW-PHASE} before @code{PHASE}.
1049 @item
1050 @code{(add-after PHASE NEW-PHASE PROCEDURE)}: Same, but afterwards.
1051 @item
1052 @code{(replace PHASE PROCEDURE)}.
1053 @item
1054 @code{(delete PHASE)}.
1055 @end itemize
1056
1057 The @code{PROCEDURE} supports the keyword arguments @code{inputs} and @code{outputs}. Each
1058 input (whether @emph{native}, @emph{propagated} or not) and output directory is referenced
1059 by their name in those variables. Thus @code{(assoc-ref outputs "out")} is the store
1060 directory of the main output of the package. A phase procedure may look like
1061 this:
1062
1063 @lisp
1064 (lambda* (#:key inputs outputs #:allow-other-keys)
1065 (let (((bash-directory (assoc-ref inputs "bash"))
1066 (output-directory (assoc-ref outputs "out"))
1067 (doc-directory (assoc-ref outputs "doc"))
1068 ; ...
1069 #t)
1070 @end lisp
1071
1072 The procedure must return @code{#t} on success. It's brittle to rely on the return
1073 value of the last expression used to tweak the phase because there is no
1074 guarantee it would be a @code{#t}. Hence the trailing @code{#t} to ensure the right value
1075 is returned on success.
1076
1077 @subsubsection Code staging
1078
1079 The astute reader may have noticed the quasi-quote and comma syntax in the
1080 argument field. Indeed, the build code in the package declaration should not be
1081 evaluated on the client side, but only when passed to the Guix daemon. This
1082 mechanism of passing code around two running processes is called @uref{https://arxiv.org/abs/1709.00833, code staging}.
1083
1084 @subsubsection Utility functions
1085
1086 When customizing @code{phases}, we often need to write code that mimics the
1087 equivalent system invocations (@code{make}, @code{mkdir}, @code{cp}, etc.)@: commonly used during
1088 regular ``Unix-style'' installations.
1089
1090 Some like @code{chmod} are native to Guile.
1091 @xref{,,, guile, Guile reference manual} for a complete list.
1092
1093 Guix provides additional helper functions which prove especially handy in the
1094 context of package management.
1095
1096 Some of those functions can be found in
1097 @samp{$GUIX_CHECKOUT/guix/guix/build/utils.scm}. Most of them mirror the behaviour
1098 of the traditional Unix system commands:
1099
1100 @table @asis
1101 @item which
1102 Like the @samp{which} system command.
1103 @item find-files
1104 Akin to the @samp{find} system command.
1105 @item mkdir-p
1106 Like @samp{mkdir -p}, which creates all parents as needed.
1107 @item install-file
1108 Similar to @samp{install} when installing a file to a (possibly
1109 non-existing) directory. Guile has @code{copy-file} which works
1110 like @samp{cp}.
1111 @item copy-recursively
1112 Like @samp{cp -r}.
1113 @item delete-file-recursively
1114 Like @samp{rm -rf}.
1115 @item invoke
1116 Run an executable. This should be used instead of @code{system*}.
1117 @item with-directory-excursion
1118 Run the body in a different working directory,
1119 then restore the previous working directory.
1120 @item substitute*
1121 A ``@command{sed}-like'' function.
1122 @end table
1123
1124 @subsubsection Module prefix
1125
1126 The license in our last example needs a prefix: this is because of how the
1127 @code{license} module was imported in the package, as @code{#:use-module ((guix licenses)
1128 #:prefix license:)}. The Guile module import mechanism
1129 (@pxref{Using Guile Modules,,, guile, Guile reference manual})
1130 gives the user full control over namespacing: this is needed to avoid
1131 clashes between, say, the
1132 @samp{zlib} variable from @samp{licenses.scm} (a @emph{license} value) and the @samp{zlib} variable
1133 from @samp{compression.scm} (a @emph{package} value).
1134
1135 @node Other build systems
1136 @subsection Other build systems
1137
1138 What we've seen so far covers the majority of packages using a build system
1139 other than the @code{trivial-build-system}. The latter does not automate anything
1140 and leaves you to build everything manually. This can be more demanding and we
1141 won't cover it here for now, but thankfully it is rarely necessary to fall back
1142 on this system.
1143
1144 For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the
1145 process is very similar to the GNU build system except for a few specialized
1146 arguments.
1147
1148 @xref{Build Systems,,, guix, GNU Guix Reference Manual}, for more
1149 information on build systems, or check the source code in the
1150 @samp{$GUIX_CHECKOUT/guix/build} and
1151 @samp{$GUIX_CHECKOUT/guix/build-system} directories.
1152
1153 @node Programmable and automated package definition
1154 @subsection Programmable and automated package definition
1155
1156 We can't repeat it enough: having a full-fledged programming language at hand
1157 empowers us in ways that reach far beyond traditional package management.
1158
1159 Let's illustrate this with some awesome features of Guix!
1160
1161 @node Recursive importers
1162 @subsubsection Recursive importers
1163
1164 You might find some build systems good enough that there is little to do at all
1165 to write a package, to the point that it becomes repetitive and tedious after a
1166 while. A @emph{raison d'être} of computers is to replace human beings at those
1167 boring tasks. So let's tell Guix to do this for us and create the package
1168 definition of an R package from CRAN (the output is trimmed for conciseness):
1169
1170 @example
1171 $ guix import cran --recursive walrus
1172
1173 (define-public r-mc2d
1174 ; ...
1175 (license gpl2+)))
1176
1177 (define-public r-jmvcore
1178 ; ...
1179 (license gpl2+)))
1180
1181 (define-public r-wrs2
1182 ; ...
1183 (license gpl3)))
1184
1185 (define-public r-walrus
1186 (package
1187 (name "r-walrus")
1188 (version "1.0.3")
1189 (source
1190 (origin
1191 (method url-fetch)
1192 (uri (cran-uri "walrus" version))
1193 (sha256
1194 (base32
1195 "1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
1196 (build-system r-build-system)
1197 (propagated-inputs
1198 `(("r-ggplot2" ,r-ggplot2)
1199 ("r-jmvcore" ,r-jmvcore)
1200 ("r-r6" ,r-r6)
1201 ("r-wrs2" ,r-wrs2)))
1202 (home-page "https://github.com/jamovi/walrus")
1203 (synopsis "Robust Statistical Methods")
1204 (description
1205 "This package provides a toolbox of common robust statistical
1206 tests, including robust descriptives, robust t-tests, and robust ANOVA.
1207 It is also available as a module for 'jamovi' (see
1208 <https://www.jamovi.org> for more information). Walrus is based on the
1209 WRS2 package by Patrick Mair, which is in turn based on the scripts and
1210 work of Rand Wilcox. These analyses are described in depth in the book
1211 'Introduction to Robust Estimation & Hypothesis Testing'.")
1212 (license gpl3)))
1213 @end example
1214
1215 The recursive importer won't import packages for which Guix already has package
1216 definitions, except for the very first.
1217
1218 Not all applications can be packaged this way, only those relying on a select
1219 number of supported systems. Read about the full list of importers in
1220 the guix import section of the manual
1221 (@pxref{Invoking guix import,,, guix, GNU Guix Reference Manual}).
1222
1223 @node Automatic update
1224 @subsubsection Automatic update
1225
1226 Guix can be smart enough to check for updates on systems it knows. It can
1227 report outdated package definitions with
1228
1229 @example
1230 $ guix refresh hello
1231 @end example
1232
1233 In most cases, updating a package to a newer version requires little more than
1234 changing the version number and the checksum. Guix can do that automatically as
1235 well:
1236
1237 @example
1238 $ guix refresh hello --update
1239 @end example
1240
1241 @node Inheritance
1242 @subsubsection Inheritance
1243
1244 If you've started browsing the existing package definitions, you might have
1245 noticed that a significant number of them have a @code{inherit} field:
1246
1247 @lisp
1248 (define-public adwaita-icon-theme
1249 (package (inherit gnome-icon-theme)
1250 (name "adwaita-icon-theme")
1251 (version "3.26.1")
1252 (source (origin
1253 (method url-fetch)
1254 (uri (string-append "mirror://gnome/sources/" name "/"
1255 (version-major+minor version) "/"
1256 name "-" version ".tar.xz"))
1257 (sha256
1258 (base32
1259 "17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
1260 (native-inputs
1261 `(("gtk-encode-symbolic-svg" ,gtk+ "bin")))))
1262 @end lisp
1263
1264 All unspecified fields are inherited from the parent package. This is very
1265 convenient to create alternative packages, for instance with different source,
1266 version or compilation options.
1267
1268 @node Getting help
1269 @subsection Getting help
1270
1271 Sadly, some applications can be tough to package. Sometimes they need a patch to
1272 work with the non-standard filesystem hierarchy enforced by the store.
1273 Sometimes the tests won't run properly. (They can be skipped but this is not
1274 recommended.) Other times the resulting package won't be reproducible.
1275
1276 Should you be stuck, unable to figure out how to fix any sort of packaging
1277 issue, don't hesitate to ask the community for help.
1278
1279 See the @uref{https://www.gnu.org/software/guix/contact/, Guix homepage} for information on the mailing lists, IRC, etc.
1280
1281 @node Conclusion
1282 @subsection Conclusion
1283
1284 This tutorial was a showcase of the sophisticated package management that Guix
1285 boasts. At this point we have mostly restricted this introduction to the
1286 @code{gnu-build-system} which is a core abstraction layer on which more advanced
1287 abstractions are based.
1288
1289 Where do we go from here? Next we ought to dissect the innards of the build
1290 system by removing all abstractions, using the @code{trivial-build-system}: this
1291 should give us a thorough understanding of the process before investigating some
1292 more advanced packaging techniques and edge cases.
1293
1294 Other features worth exploring are the interactive editing and debugging
1295 capabilities of Guix provided by the Guile REPL@.
1296
1297 Those fancy features are completely optional and can wait; now is a good time
1298 to take a well-deserved break. With what we've introduced here you should be
1299 well armed to package lots of programs. You can get started right away and
1300 hopefully we will see your contributions soon!
1301
1302 @node References
1303 @subsection References
1304
1305 @itemize
1306 @item
1307 The @uref{https://www.gnu.org/software/guix/manual/en/html_node/Defining-Packages.html, package reference in the manual}
1308
1309 @item
1310 @uref{https://gitlab.com/pjotrp/guix-notes/blob/master/HACKING.org, Pjotr’s hacking guide to GNU Guix}
1311
1312 @item
1313 @uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
1314 @end itemize
1315
1316 @c *********************************************************************
1317 @node System Configuration
1318 @chapter System Configuration
1319
1320 Guix offers a flexible language for declaratively configuring your Guix
1321 System. This flexibility can at times be overwhelming. The purpose of this
1322 chapter is to demonstrate some advanced configuration concepts.
1323
1324 @pxref{System Configuration,,, guix, GNU Guix Reference Manual} for a complete
1325 reference.
1326
1327 @menu
1328 * Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
1329 * Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
1330 * Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
1331 * Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
1332 * Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
1333 @end menu
1334
1335 @node Customizing the Kernel
1336 @section Customizing the Kernel
1337
1338 Guix is, at its core, a source based distribution with substitutes
1339 (@pxref{Substitutes,,, guix, GNU Guix Reference Manual}), and as such building
1340 packages from their source code is an expected part of regular package
1341 installations and upgrades. Given this starting point, it makes sense that
1342 efforts are made to reduce the amount of time spent compiling packages, and
1343 recent changes and upgrades to the building and distribution of substitutes
1344 continues to be a topic of discussion within Guix.
1345
1346 The kernel, while not requiring an overabundance of RAM to build, does take a
1347 rather long time on an average machine. The official kernel configuration, as
1348 is the case with many GNU/Linux distributions, errs on the side of
1349 inclusiveness, and this is really what causes the build to take such a long
1350 time when the kernel is built from source.
1351
1352 The Linux kernel, however, can also just be described as a regular old
1353 package, and as such can be customized just like any other package. The
1354 procedure is a little bit different, although this is primarily due to the
1355 nature of how the package definition is written.
1356
1357 The @code{linux-libre} kernel package definition is actually a procedure which
1358 creates a package.
1359
1360 @lisp
1361 (define* (make-linux-libre version hash supported-systems
1362 #:key
1363 ;; A function that takes an arch and a variant.
1364 ;; See kernel-config for an example.
1365 (extra-version #f)
1366 (configuration-file #f)
1367 (defconfig "defconfig")
1368 (extra-options %default-extra-linux-options)
1369 (patches (list %boot-logo-patch)))
1370 ...)
1371 @end lisp
1372
1373 The current @code{linux-libre} package is for the 5.1.x series, and is
1374 declared like this:
1375
1376 @lisp
1377 (define-public linux-libre
1378 (make-linux-libre %linux-libre-version
1379 %linux-libre-hash
1380 '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux")
1381 #:patches %linux-libre-5.1-patches
1382 #:configuration-file kernel-config))
1383 @end lisp
1384
1385 Any keys which are not assigned values inherit their default value from the
1386 @code{make-linux-libre} definition. When comparing the two snippets above,
1387 you may notice that the code comment in the first doesn't actually refer to
1388 the @code{#:extra-version} keyword; it is actually for
1389 @code{#:configuration-file}. Because of this, it is not actually easy to
1390 include a custom kernel configuration from the definition, but don't worry,
1391 there are other ways to work with what we do have.
1392
1393 There are two ways to create a kernel with a custom kernel configuration. The
1394 first is to provide a standard @file{.config} file during the build process by
1395 including an actual @file{.config} file as a native input to our custom
1396 kernel. The following is a snippet from the custom @code{'configure} phase of
1397 the @code{make-linux-libre} package definition:
1398
1399 @lisp
1400 (let ((build (assoc-ref %standard-phases 'build))
1401 (config (assoc-ref (or native-inputs inputs) "kconfig")))
1402
1403 ;; Use a custom kernel configuration file or a default
1404 ;; configuration file.
1405 (if config
1406 (begin
1407 (copy-file config ".config")
1408 (chmod ".config" #o666))
1409 (invoke "make" ,defconfig))
1410 @end lisp
1411
1412 Below is a sample kernel package. The @code{linux-libre} package is nothing
1413 special and can be inherited from and have its fields overridden like any
1414 other package:
1415
1416 @lisp
1417 (define-public linux-libre/E2140
1418 (package
1419 (inherit linux-libre)
1420 (native-inputs
1421 `(("kconfig" ,(local-file "E2140.config"))
1422 ,@@(alist-delete "kconfig"
1423 (package-native-inputs linux-libre))))))
1424 @end lisp
1425
1426 In the same directory as the file defining @code{linux-libre-E2140} is a file
1427 named @file{E2140.config}, which is an actual kernel configuration file. The
1428 @code{defconfig} keyword of @code{make-linux-libre} is left blank here, so the
1429 only kernel configuration in the package is the one which was included in the
1430 @code{native-inputs} field.
1431
1432 The second way to create a custom kernel is to pass a new value to the
1433 @code{extra-options} keyword of the @code{make-linux-libre} procedure. The
1434 @code{extra-options} keyword works with another function defined right below
1435 it:
1436
1437 @lisp
1438 (define %default-extra-linux-options
1439 `(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
1440 ("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #t)
1441 ;; Modules required for initrd:
1442 ("CONFIG_NET_9P" . m)
1443 ("CONFIG_NET_9P_VIRTIO" . m)
1444 ("CONFIG_VIRTIO_BLK" . m)
1445 ("CONFIG_VIRTIO_NET" . m)
1446 ("CONFIG_VIRTIO_PCI" . m)
1447 ("CONFIG_VIRTIO_BALLOON" . m)
1448 ("CONFIG_VIRTIO_MMIO" . m)
1449 ("CONFIG_FUSE_FS" . m)
1450 ("CONFIG_CIFS" . m)
1451 ("CONFIG_9P_FS" . m)))
1452
1453 (define (config->string options)
1454 (string-join (map (match-lambda
1455 ((option . 'm)
1456 (string-append option "=m"))
1457 ((option . #t)
1458 (string-append option "=y"))
1459 ((option . #f)
1460 (string-append option "=n")))
1461 options)
1462 "\n"))
1463 @end lisp
1464
1465 And in the custom configure script from the `make-linux-libre` package:
1466
1467 @lisp
1468 ;; Appending works even when the option wasn't in the
1469 ;; file. The last one prevails if duplicated.
1470 (let ((port (open-file ".config" "a"))
1471 (extra-configuration ,(config->string extra-options)))
1472 (display extra-configuration port)
1473 (close-port port))
1474
1475 (invoke "make" "oldconfig"))))
1476 @end lisp
1477
1478 So by not providing a configuration-file the @file{.config} starts blank, and
1479 then we write into it the collection of flags that we want. Here's another
1480 custom kernel:
1481
1482 @lisp
1483 (define %macbook41-full-config
1484 (append %macbook41-config-options
1485 %filesystems
1486 %efi-support
1487 %emulation
1488 (@@@@ (gnu packages linux) %default-extra-linux-options)))
1489
1490 (define-public linux-libre-macbook41
1491 ;; XXX: Access the internal 'make-linux-libre' procedure, which is
1492 ;; private and unexported, and is liable to change in the future.
1493 ((@@@@ (gnu packages linux) make-linux-libre) (@@@@ (gnu packages linux) %linux-libre-version)
1494 (@@@@ (gnu packages linux) %linux-libre-hash)
1495 '("x86_64-linux")
1496 #:extra-version "macbook41"
1497 #:patches (@@@@ (gnu packages linux) %linux-libre-5.1-patches)
1498 #:extra-options %macbook41-config-options))
1499 @end lisp
1500
1501 In the above example @code{%filesystems} is a collection of flags enabling
1502 different filesystem support, @code{%efi-support} enables EFI support and
1503 @code{%emulation} enables a x86_64-linux machine to act in 32-bit mode also.
1504 @code{%default-extra-linux-options} are the ones quoted above, which had to be
1505 added in since they were replaced in the @code{extra-options} keyword.
1506
1507 This all sounds like it should be doable, but how does one even know which
1508 modules are required for a particular system? Two places that can be helpful
1509 in trying to answer this question is the
1510 @uref{https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel, Gentoo
1511 Handbook} and the
1512 @uref{https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig,
1513 documentation from the kernel itself}. From the kernel documentation, it
1514 seems that @code{make localmodconfig} is the command we want.
1515
1516 In order to actually run @code{make localmodconfig} we first need to get and
1517 unpack the kernel source code:
1518
1519 @example shell
1520 tar xf $(guix build linux-libre --source)
1521 @end example
1522
1523 Once inside the directory containing the source code run @code{touch .config}
1524 to create an initial, empty @file{.config} to start with. @code{make
1525 localmodconfig} works by seeing what you already have in @file{.config} and
1526 letting you know what you're missing. If the file is blank then you're
1527 missing everything. The next step is to run:
1528
1529 @example shell
1530 guix environment linux-libre -- make localmodconfig
1531 @end example
1532
1533 and note the output. Do note that the @file{.config} file is still empty.
1534 The output generally contains two types of warnings. The first start with
1535 "WARNING" and can actually be ignored in our case. The second read:
1536
1537 @example shell
1538 module pcspkr did not have configs CONFIG_INPUT_PCSPKR
1539 @end example
1540
1541 For each of these lines, copy the @code{CONFIG_XXXX_XXXX} portion into the
1542 @file{.config} in the directory, and append @code{=m}, so in the end it looks
1543 like this:
1544
1545 @example shell
1546 CONFIG_INPUT_PCSPKR=m
1547 CONFIG_VIRTIO=m
1548 @end example
1549
1550 After copying all the configuration options, run @code{make localmodconfig}
1551 again to make sure that you don't have any output starting with ``module''.
1552 After all of these machine specific modules there are a couple more left that
1553 are also needed. @code{CONFIG_MODULES} is necessary so that you can build and
1554 load modules separately and not have everything built into the kernel.
1555 @code{CONFIG_BLK_DEV_SD} is required for reading from hard drives. It is
1556 possible that there are other modules which you will need.
1557
1558 This post does not aim to be a guide to configuring your own kernel however,
1559 so if you do decide to build a custom kernel you'll have to seek out other
1560 guides to create a kernel which is just right for your needs.
1561
1562 The second way to setup the kernel configuration makes more use of Guix's
1563 features and allows you to share configuration segments between different
1564 kernels. For example, all machines using EFI to boot have a number of EFI
1565 configuration flags that they need. It is likely that all the kernels will
1566 share a list of filesystems to support. By using variables it is easier to
1567 see at a glance what features are enabled and to make sure you don't have
1568 features in one kernel but missing in another.
1569
1570 Left undiscussed however, is Guix's initrd and its customization. It is
1571 likely that you'll need to modify the initrd on a machine using a custom
1572 kernel, since certain modules which are expected to be built may not be
1573 available for inclusion into the initrd.
1574
1575 @node Connecting to Wireguard VPN
1576 @section Connecting to Wireguard VPN
1577
1578 To connect to a Wireguard VPN server you need the kernel module to be
1579 loaded in memory and a package providing networking tools that support
1580 it (e.g. @code{wireguard-tools} or @code{network-manager}).
1581
1582 Here is a configuration example for Linux-Libre < 5.6, where the module
1583 is out of tree and need to be loaded manually---following revisions of
1584 the kernel have it built-in and so don't need such configuration:
1585
1586 @lisp
1587 (use-modules (gnu))
1588 (use-service-modules desktop)
1589 (use-package-modules vpn)
1590
1591 (operating-system
1592 ;; …
1593 (services (cons (simple-service 'wireguard-module
1594 kernel-module-loader-service-type
1595 '("wireguard"))
1596 %desktop-services))
1597 (packages (cons wireguard-tools %base-packages))
1598 (kernel-loadable-modules (list wireguard-linux-compat)))
1599 @end lisp
1600
1601 After reconfiguring and restarting your system you can either use
1602 Wireguard tools or NetworkManager to connect to a VPN server.
1603
1604 @subsection Using Wireguard tools
1605
1606 To test your Wireguard setup it is convenient to use @command{wg-quick}.
1607 Just give it a configuration file @command{wg-quick up ./wg0.conf}; or
1608 put that file in @file{/etc/wireguard} and run @command{wg-quick up wg0}
1609 instead.
1610
1611 @quotation Note
1612 Be warned that the author described this command as a: “[…] very quick
1613 and dirty bash script […]”.
1614 @end quotation
1615
1616 @subsection Using NetworkManager
1617
1618 Thanks to NetworkManager support for Wireguard we can connect to our VPN
1619 using @command{nmcli} command. Up to this point this guide assumes that
1620 you're using Network Manager service provided by
1621 @code{%desktop-services}. Ortherwise you need to adjust your services
1622 list to load @code{network-manager-service-type} and reconfigure your
1623 Guix system.
1624
1625 To import your VPN configuration execute nmcli import command:
1626
1627 @example shell
1628 # nmcli connection import type wireguard file wg0.conf
1629 Connection 'wg0' (edbee261-aa5a-42db-b032-6c7757c60fde) successfully added
1630 @end example
1631
1632 This will create a configuration file in
1633 @file{/etc/NetworkManager/wg0.nmconnection}. Next connect to the
1634 Wireguard server:
1635
1636 @example shell
1637 $ nmcli connection up wg0
1638 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
1639 @end example
1640
1641 By default NetworkManager will connect automatically on system boot. To
1642 change that behaviour you need to edit your config:
1643
1644 @example shell
1645 # nmcli connection modify wg0 connection.autoconnect no
1646 @end example
1647
1648 For more specific information about NetworkManager and wireguard
1649 @uref{https://blogs.gnome.org/thaller/2019/03/15/wireguard-in-networkmanager/,see
1650 this post by thaller}.
1651
1652 @node Customizing a Window Manager
1653 @section Customizing a Window Manager
1654 @cindex wm
1655
1656 @node StumpWM
1657 @subsection StumpWM
1658 @cindex stumpwm
1659
1660 You could install StumpWM with a Guix system by adding
1661 @code{stumpwm} and optionally @code{`(,stumpwm "lib")}
1662 packages to a system configuration file, e.g.@: @file{/etc/config.scm}.
1663
1664 An example configuration can look like this:
1665
1666 @lisp
1667 (use-modules (gnu))
1668 (use-package-modules wm)
1669
1670 (operating-system
1671 ;; …
1672 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
1673 %base-packages)))
1674 @end lisp
1675
1676 @cindex stumpwm fonts
1677 By default StumpWM uses X11 fonts, which could be small or pixelated on
1678 your system. You could fix this by installing StumpWM contrib Lisp
1679 module @code{sbcl-ttf-fonts}, adding it to Guix system packages:
1680
1681 @lisp
1682 (use-modules (gnu))
1683 (use-package-modules fonts wm)
1684
1685 (operating-system
1686 ;; …
1687 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
1688 sbcl-ttf-fonts font-dejavu %base-packages)))
1689 @end lisp
1690
1691 Then you need to add the following code to a StumpWM configuration file
1692 @file{~/.stumpwm.d/init.lisp}:
1693
1694 @lisp
1695 (require :ttf-fonts)
1696 (setf xft:*font-dirs* '("/run/current-system/profile/share/fonts/"))
1697 (setf clx-truetype:+font-cache-filename+ (concat (getenv "HOME") "/.fonts/font-cache.sexp"))
1698 (xft:cache-fonts)
1699 (set-font (make-instance 'xft:font :family "DejaVu Sans Mono" :subfamily "Book" :size 11))
1700 @end lisp
1701
1702 @node Session lock
1703 @subsection Session lock
1704 @cindex sessionlock
1705
1706 Depending on your environment, locking the screen of your session might come built in
1707 or it might be something you have to set up yourself. If you use a desktop environment
1708 like GNOME or KDE, it's usually built in. If you use a plain window manager like
1709 StumpWM or EXWM, you might have to set it up yourself.
1710
1711 @node Xorg
1712 @subsubsection Xorg
1713
1714 If you use Xorg, you can use the utility
1715 @uref{https://www.mankier.com/1/xss-lock, xss-lock} to lock the screen of your session.
1716 xss-lock is triggered by DPMS which since Xorg 1.8 is auto-detected and enabled if
1717 ACPI is also enabled at kernel runtime.
1718
1719 To use xss-lock, you can simple execute it and put it into the background before
1720 you start your window manager from e.g. your @file{~/.xsession}:
1721
1722 @example
1723 xss-lock -- slock &
1724 exec stumpwm
1725 @end example
1726
1727 In this example, xss-lock uses @code{slock} to do the actual locking of the screen when
1728 it determines it's appropriate, like when you suspend your device.
1729
1730 For slock to be allowed to be a screen locker for the graphical session, it needs to
1731 be made setuid-root so it can authenticate users, and it needs a PAM service. This
1732 can be achieved by adding the following service to your @file{config.scm}:
1733
1734 @lisp
1735 (screen-locker-service slock)
1736 @end lisp
1737
1738 If you manually lock your screen, e.g. by directly calling slock when you want to lock
1739 your screen but not suspend it, it's a good idea to notify xss-lock about this so no
1740 confusion occurs. This can be done by executing @code{xset s activate} immediately
1741 before you execute slock.
1742
1743 @node Setting up a bind mount
1744 @section Setting up a bind mount
1745
1746 To bind mount a file system, one must first set up some definitions
1747 before the @code{operating-system} section of the system definition. In
1748 this example we will bind mount a folder from a spinning disk drive to
1749 @file{/tmp}, to save wear and tear on the primary SSD, without
1750 dedicating an entire partition to be mounted as @file{/tmp}.
1751
1752 First, the source drive that hosts the folder we wish to bind mount
1753 should be defined, so that the bind mount can depend on it.
1754
1755 @lisp
1756 (define source-drive ;; "source-drive" can be named anything you want.
1757 (file-system
1758 (device (uuid "UUID goes here"))
1759 (mount-point "/path-to-spinning-disk-goes-here")
1760 (type "ext4"))) ;; Make sure to set this to the appropriate type for your drive.
1761 @end lisp
1762
1763 The source folder must also be defined, so that guix will know it's not
1764 a regular block device, but a folder.
1765 @lisp
1766 (define (%source-directory) "/path-to-spinning-disk-goes-here/tmp") ;; "source-directory" can be named any valid variable name.
1767 @end lisp
1768
1769 Finally, inside the @code{file-systems} definition, we must add the
1770 mount itself.
1771
1772 @lisp
1773 (file-systems (cons*
1774
1775 ...<other drives omitted for clarity>...
1776
1777 source-drive ;; Must match the name you gave the source drive in the earlier definition.
1778
1779 (file-system
1780 (device (%source-directory)) ;; Make sure "source-directory" matches your earlier definition.
1781 (mount-point "/tmp")
1782 (type "none") ;; We are mounting a folder, not a partition, so this type needs to be "none"
1783 (flags '(bind-mount))
1784 (dependencies (list source-drive)) ;; Ensure "source-drive" matches what you've named the variable for the drive.
1785 )
1786
1787 ...<other drives omitted for clarity>...
1788
1789 ))
1790 @end lisp
1791
1792 @node Getting substitutes from Tor
1793 @section Getting substitutes from Tor
1794
1795 Guix daemon can use a HTTP proxy to get substitutes, here we are
1796 configuring it to get them via Tor.
1797
1798 @quotation Warning
1799 @emph{Not all} Guix daemon's traffic will go through Tor! Only
1800 HTTP/HTTPS will get proxied; FTP, Git protocol, SSH, etc connections
1801 will still go through the clearnet. Again, this configuration isn't
1802 foolproof some of your traffic won't get routed by Tor at all. Use it
1803 at your own risk.
1804 @end quotation
1805
1806 Guix's substitute server is available as a Onion service, if you want
1807 to use it to get your substitutes from Tor configure your system as
1808 follow:
1809
1810 @lisp
1811 (use-modules (gnu))
1812 (use-service-module base networking)
1813
1814 (operating-system
1815
1816 (services
1817 (cons
1818 (service tor-service-type
1819 (tor-configuration
1820 (config-file (plain-file "tor-config"
1821 "HTTPTunnelPort 127.0.0.1:9250"))))
1822 (modify-services %base-services
1823 (guix-service-type
1824 config => (guix-configuration
1825 (inherit config)
1826 ;; ci.guix.gnu.org's Onion service
1827 (substitute-urls "https://bp7o7ckwlewr4slm.onion")
1828 (http-proxy "http://localhost:9250")))))))
1829 @end lisp
1830
1831 This will keep a tor process running that provides a HTTP CONNECT tunnel
1832 which will be used by @command{guix-daemon}. The daemon can use other
1833 protocols than HTTP(S) to get remote resources, request using those
1834 protocols won't go through Tor since we are only setting a HTTP tunnel
1835 here. Note that @code{substitutes-urls} is using HTTPS and not HTTP or
1836 it won't work, that's a limitation of Tor's tunnel; you may want to use
1837 @command{privoxy} instead to avoid such limitations.
1838
1839 If you don't want to always get substitutes through Tor but using it just
1840 some of the times, then skip the @code{guix-configuration}. When you
1841 want to get a substitute from the Tor tunnel run:
1842
1843 @example
1844 sudo herd set-http-proxy guix-daemon http://localhost:9250
1845 guix build --substitute-urls=https://bp7o7ckwlewr4slm.onion …
1846 @end example
1847
1848 @c *********************************************************************
1849 @node Advanced package management
1850 @chapter Advanced package management
1851
1852 Guix is a functional package manager that offers many features beyond
1853 what more traditional package managers can do. To the uninitiated,
1854 those features might not have obvious use cases at first. The purpose
1855 of this chapter is to demonstrate some advanced package management
1856 concepts.
1857
1858 @pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
1859 reference.
1860
1861 @menu
1862 * Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
1863 @end menu
1864
1865 @node Guix Profiles in Practice
1866 @section Guix Profiles in Practice
1867
1868 Guix provides a very useful feature that may be quite foreign to newcomers:
1869 @emph{profiles}. They are a way to group package installations together and all users
1870 on the same system are free to use as many profiles as they want.
1871
1872 Whether you're a developer or not, you may find that multiple profiles bring you
1873 great power and flexibility. While they shift the paradigm somewhat compared to
1874 @emph{traditional package managers}, they are very convenient to use once you've
1875 understood how to set them up.
1876
1877 If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
1878 kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
1879 just Python software. Furthermore, profiles are self-sufficient: they capture
1880 all the runtime dependencies which guarantees that all programs within a profile
1881 will always work at any point in time.
1882
1883 Multiple profiles have many benefits:
1884
1885 @itemize
1886 @item
1887 Clean semantic separation of the various packages a user needs for different contexts.
1888
1889 @item
1890 Multiple profiles can be made available into the environment either on login
1891 or within a dedicated shell.
1892
1893 @item
1894 Profiles can be loaded on demand. For instance, the user can use multiple
1895 shells, each of them running different profiles.
1896
1897 @item
1898 Isolation: Programs from one profile will not use programs from the other, and
1899 the user can even install different versions of the same programs to the two
1900 profiles without conflict.
1901
1902 @item
1903 Deduplication: Profiles share dependencies that happens to be the exact same.
1904 This makes multiple profiles storage-efficient.
1905
1906 @item
1907 Reproducible: when used with declarative manifests, a profile can be fully
1908 specified by the Guix commit that was active when it was set up. This means
1909 that the exact same profile can be
1910 @uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
1911 set up anywhere and anytime}, with just the commit information. See the
1912 section on @ref{Reproducible profiles}.
1913
1914 @item
1915 Easier upgrades and maintenance: Multiple profiles make it easy to keep
1916 package listings at hand and make upgrades completely friction-less.
1917 @end itemize
1918
1919 Concretely, here follows some typical profiles:
1920
1921 @itemize
1922 @item
1923 The dependencies of a project you are working on.
1924
1925 @item
1926 Your favourite programming language libraries.
1927
1928 @item
1929 Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
1930
1931 @item
1932 @TeX{}live (this one can be really useful when you need to install just one
1933 package for this one document you've just received over email).
1934
1935 @item
1936 Games.
1937 @end itemize
1938
1939 Let's dive in the set up!
1940
1941 @node Basic setup with manifests
1942 @subsection Basic setup with manifests
1943
1944 A Guix profile can be set up @emph{via} a so-called @emph{manifest specification} that looks like
1945 this:
1946
1947 @lisp
1948 (specifications->manifest
1949 '("package-1"
1950 ;; Version 1.3 of package-2.
1951 "package-2@@1.3"
1952 ;; The "lib" output of package-3.
1953 "package-3:lib"
1954 ; ...
1955 "package-N"))
1956 @end lisp
1957
1958 @pxref{Invoking guix package,,, guix, GNU Guix Reference Manual}, for
1959 the syntax details.
1960
1961 We can create a manifest specification per profile and install them this way:
1962
1963 @example
1964 GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
1965 mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
1966 guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
1967 @end example
1968
1969 Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
1970 where we will store our profiles in the rest of this article.
1971
1972 Placing all your profiles in a single directory, with each profile getting its
1973 own sub-directory, is somewhat cleaner. This way, each sub-directory will
1974 contain all the symlinks for precisely one profile. Besides, ``looping over
1975 profiles'' becomes obvious from any programming language (e.g.@: a shell script) by
1976 simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
1977
1978 Note that it's also possible to loop over the output of
1979
1980 @example
1981 guix package --list-profiles
1982 @end example
1983
1984 although you'll probably have to filter out @file{~/.config/guix/current}.
1985
1986 To enable all profiles on login, add this to your @file{~/.bash_profile} (or similar):
1987
1988 @example
1989 for i in $GUIX_EXTRA_PROFILES/*; do
1990 profile=$i/$(basename "$i")
1991 if [ -f "$profile"/etc/profile ]; then
1992 GUIX_PROFILE="$profile"
1993 . "$GUIX_PROFILE"/etc/profile
1994 fi
1995 unset profile
1996 done
1997 @end example
1998
1999 Note to Guix System users: the above reflects how your default profile
2000 @file{~/.guix-profile} is activated from @file{/etc/profile}, that latter being loaded by
2001 @file{~/.bashrc} by default.
2002
2003 You can obviously choose to only enable a subset of them:
2004
2005 @example
2006 for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
2007 profile=$i/$(basename "$i")
2008 if [ -f "$profile"/etc/profile ]; then
2009 GUIX_PROFILE="$profile"
2010 . "$GUIX_PROFILE"/etc/profile
2011 fi
2012 unset profile
2013 done
2014 @end example
2015
2016 When a profile is off, it's straightforward to enable it for an individual shell
2017 without "polluting" the rest of the user session:
2018
2019 @example
2020 GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
2021 @end example
2022
2023 The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. This file
2024 contains shell code that exports the right environment variables necessary to
2025 activate the software contained in the profile. It is built automatically by
2026 Guix and meant to be sourced.
2027 It contains the same variables you would get if you ran:
2028
2029 @example
2030 guix package --search-paths=prefix --profile=$my_profile"
2031 @end example
2032
2033 Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual})
2034 for the command line options.
2035
2036 To upgrade a profile, simply install the manifest again:
2037
2038 @example
2039 guix package -m /path/to/guix-my-project-manifest.scm -p "$GUIX_EXTRA_PROFILES"/my-project/my-project
2040 @end example
2041
2042 To upgrade all profiles, it's easy enough to loop over them. For instance,
2043 assuming your manifest specifications are stored in
2044 @file{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
2045 of the profile (e.g.@: "project1"), you could do the following in Bourne shell:
2046
2047 @example
2048 for profile in "$GUIX_EXTRA_PROFILES"/*; do
2049 guix package --profile="$profile" --manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
2050 done
2051 @end example
2052
2053 Each profile has its own generations:
2054
2055 @example
2056 guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
2057 @end example
2058
2059 You can roll-back to any generation of a given profile:
2060
2061 @example
2062 guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
2063 @end example
2064
2065 Finally, if you want to switch to a profile without inheriting from the
2066 current environment, you can activate it from an empty shell:
2067
2068 @example
2069 env -i $(which bash) --login --noprofile --norc
2070 . my-project/etc/profile
2071 @end example
2072
2073 @node Required packages
2074 @subsection Required packages
2075
2076 Activating a profile essentially boils down to exporting a bunch of
2077 environmental variables. This is the role of the @samp{etc/profile} within the
2078 profile.
2079
2080 @emph{Note: Only the environmental variables of the packages that consume them will
2081 be set.}
2082
2083 For instance, @samp{MANPATH} won't be set if there is no consumer application for man
2084 pages within the profile. So if you need to transparently access man pages once
2085 the profile is loaded, you've got two options:
2086
2087 @itemize
2088 @item
2089 Either export the variable manually, e.g.
2090 @example
2091 export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH
2092 @end example
2093
2094 @item
2095 Or include @samp{man-db} to the profile manifest.
2096 @end itemize
2097
2098 The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
2099 @samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
2100
2101 @node Default profile
2102 @subsection Default profile
2103
2104 What about the default profile that Guix keeps in @file{~/.guix-profile}?
2105
2106 You can assign it the role you want. Typically you would install the manifest
2107 of the packages you want to use all the time.
2108
2109 Alternatively, you could keep it ``manifest-less'' for throw-away packages
2110 that you would just use for a couple of days.
2111 This way makes it convenient to run
2112
2113 @example
2114 guix install package-foo
2115 guix upgrade package-bar
2116 @end example
2117
2118 without having to specify the path to a profile.
2119
2120 @node The benefits of manifests
2121 @subsection The benefits of manifests
2122
2123 Manifests are a convenient way to keep your package lists around and, say,
2124 to synchronize them across multiple machines using a version control system.
2125
2126 A common complaint about manifests is that they can be slow to install when they
2127 contain large number of packages. This is especially cumbersome when you just
2128 want get an upgrade for one package within a big manifest.
2129
2130 This is one more reason to use multiple profiles, which happen to be just
2131 perfect to break down manifests into multiple sets of semantically connected
2132 packages. Using multiple, small profiles provides more flexibility and
2133 usability.
2134
2135 Manifests come with multiple benefits. In particular, they ease maintenance:
2136
2137 @itemize
2138 @item
2139 When a profile is set up from a manifest, the manifest itself is
2140 self-sufficient to keep a ``package listing'' around and reinstall the profile
2141 later or on a different system. For ad-hoc profiles, we would need to
2142 generate a manifest specification manually and maintain the package versions
2143 for the packages that don't use the default version.
2144
2145 @item
2146 @code{guix package --upgrade} always tries to update the packages that have
2147 propagated inputs, even if there is nothing to do. Guix manifests remove this
2148 problem.
2149
2150 @item
2151 When partially upgrading a profile, conflicts may arise (due to diverging
2152 dependencies between the updated and the non-updated packages) and they can be
2153 annoying to resolve manually. Manifests remove this problem altogether since
2154 all packages are always upgraded at once.
2155
2156 @item
2157 As mentioned above, manifests allow for reproducible profiles, while the
2158 imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
2159 different profiles every time even when they hold the same packages. See
2160 @uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
2161
2162 @item
2163 Manifest specifications are usable by other @samp{guix} commands. For example, you
2164 can run @code{guix weather -m manifest.scm} to see how many substitutes are
2165 available, which can help you decide whether you want to try upgrading today
2166 or wait a while. Another example: you can run @code{guix pack -m manifest.scm} to
2167 create a pack containing all the packages in the manifest (and their
2168 transitive references).
2169
2170 @item
2171 Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
2172 They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
2173 @end itemize
2174
2175 It's important to understand that while manifests can be used to declare
2176 profiles, they are not strictly equivalent: profiles have the side effect that
2177 they ``pin'' packages in the store, which prevents them from being
2178 garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
2179 and ensures that they will still be available at any point in
2180 the future.
2181
2182 Let's take an example:
2183
2184 @enumerate
2185 @item
2186 We have an environment for hacking on a project for which there isn't a Guix
2187 package yet. We build the environment using a manifest, and then run @code{guix
2188 environment -m manifest.scm}. So far so good.
2189
2190 @item
2191 Many weeks pass and we have run a couple of @code{guix pull} in the mean time.
2192 Maybe a dependency from our manifest has been updated; or we may have run
2193 @code{guix gc} and some packages needed by our manifest have been
2194 garbage-collected.
2195
2196 @item
2197 Eventually, we set to work on that project again, so we run @code{guix environment
2198 -m manifest.scm}. But now we have to wait for Guix to build and install
2199 stuff!
2200 @end enumerate
2201
2202 Ideally, we could spare the rebuild time. And indeed we can, all we need is to
2203 install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
2204 . "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
2205 hacking environment will be available at all times.
2206
2207 @emph{Security warning:} While keeping old profiles around can be convenient, keep in
2208 mind that outdated packages may not have received the latest security fixes.
2209
2210 @node Reproducible profiles
2211 @subsection Reproducible profiles
2212
2213 To reproduce a profile bit-for-bit, we need two pieces of information:
2214
2215 @itemize
2216 @item
2217 a manifest,
2218 @item
2219 a Guix channel specification.
2220 @end itemize
2221
2222 Indeed, manifests alone might not be enough: different Guix versions (or
2223 different channels) can produce different outputs for a given manifest.
2224
2225 You can output the Guix channel specification with @samp{guix describe
2226 --format=channels}.
2227 Save this to a file, say @samp{channel-specs.scm}.
2228
2229 On another computer, you can use the channel specification file and the manifest
2230 to reproduce the exact same profile:
2231
2232 @example
2233 GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
2234 GUIX_EXTRA=$HOME/.guix-extra
2235
2236 mkdir "$GUIX_EXTRA"/my-project
2237 guix pull --channels=channel-specs.scm --profile "$GUIX_EXTRA/my-project/guix"
2238
2239 mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
2240 "$GUIX_EXTRA"/my-project/guix/bin/guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
2241 @end example
2242
2243 It's safe to delete the Guix channel profile you've just installed with the
2244 channel specification, the project profile does not depend on it.
2245
2246 @c *********************************************************************
2247 @node Acknowledgments
2248 @chapter Acknowledgments
2249
2250 Guix is based on the @uref{https://nixos.org/nix/, Nix package manager},
2251 which was designed and
2252 implemented by Eelco Dolstra, with contributions from other people (see
2253 the @file{nix/AUTHORS} file in Guix.) Nix pioneered functional package
2254 management, and promoted unprecedented features, such as transactional
2255 package upgrades and rollbacks, per-user profiles, and referentially
2256 transparent build processes. Without this work, Guix would not exist.
2257
2258 The Nix-based software distributions, Nixpkgs and NixOS, have also been
2259 an inspiration for Guix.
2260
2261 GNU@tie{}Guix itself is a collective work with contributions from a
2262 number of people. See the @file{AUTHORS} file in Guix for more
2263 information on these fine people. The @file{THANKS} file lists people
2264 who have helped by reporting bugs, taking care of the infrastructure,
2265 providing artwork and themes, making suggestions, and more---thank you!
2266
2267 This document includes adapted sections from articles that have previously
2268 been published on the Guix blog at @uref{https://guix.gnu.org/blog}.
2269
2270
2271 @c *********************************************************************
2272 @node GNU Free Documentation License
2273 @appendix GNU Free Documentation License
2274 @cindex license, GNU Free Documentation License
2275 @include fdl-1.3.texi
2276
2277 @c *********************************************************************
2278 @node Concept Index
2279 @unnumbered Concept Index
2280 @printindex cp
2281
2282 @bye
2283
2284 @c Local Variables:
2285 @c ispell-local-dictionary: "american";
2286 @c End: