doc: cookbook: Clarify 'git-fetch' conventions.
[jackhill/guix/guix.git] / doc / guix-cookbook.texi
CommitLineData
7bc46ecc
RW
1\input texinfo
2@c -*-texinfo-*-
3
4@c %**start of header
5@setfilename guix-cookbook.info
6@documentencoding UTF-8
7@settitle GNU Guix Cookbook
8@c %**end of header
9
10@copying
11Copyright @copyright{} 2019 Ricardo Wurmus@*
12Copyright @copyright{} 2019 Efraim Flashner@*
13Copyright @copyright{} 2019 Pierre Neidhardt@*
65051fdc 14Copyright @copyright{} 2020 Oleg Pykhalov@*
60651dd9 15Copyright @copyright{} 2020 Matthew Brooks@*
145df67b 16Copyright @copyright{} 2020 Marcin Karpezo@*
c987b723 17Copyright @copyright{} 2020 Brice Waegeneire@*
7bc46ecc
RW
18
19Permission is granted to copy, distribute and/or modify this document
20under the terms of the GNU Free Documentation License, Version 1.3 or
21any later version published by the Free Software Foundation; with no
22Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
23copy of the license is included in the section entitled ``GNU Free
24Documentation License''.
25@end copying
26
27@dircategory System administration
28@direntry
29* Guix cookbook: (guix-cookbook). Tutorials and examples for GNU Guix.
30@end direntry
31
32@titlepage
33@title GNU Guix Cookbook
34@subtitle Tutorials and examples for using the GNU Guix Functional Package Manager
35@author The GNU Guix Developers
36
37@page
38@vskip 0pt plus 1filll
39
40@insertcopying
41@end titlepage
42
43@contents
44
45@c *********************************************************************
46@node Top
47@top GNU Guix Cookbook
48
49This document presents tutorials and detailed examples for GNU@tie{}Guix, a
50functional package management tool written for the GNU system. Please
51@pxref{Top,,, guix, GNU Guix reference manual} for details about the system,
52its API, and related concepts.
53
54@c TRANSLATORS: You can replace the following paragraph with information on
55@c how to join your own translation team and how to report issues with the
56@c translation.
57If you would like to translate this document in your native language, consider
58joining the @uref{https://translationproject.org/domain/guix-cookbook.html,
59Translation Project}.
60
61@menu
62* Scheme tutorials:: Meet your new favorite language!
63* Packaging:: Packaging tutorials
64* System Configuration:: Customizing the GNU System
4c463569 65* Advanced package management:: Power to the users!
7bc46ecc
RW
66
67* Acknowledgments:: Thanks!
68* GNU Free Documentation License:: The license of this document.
69* Concept Index:: Concepts.
70
71@detailmenu
72 --- The Detailed Node Listing ---
73
74Scheme tutorials
75
76* A Scheme Crash Course:: Learn the basics of Scheme
77
78Packaging
79
80* Packaging Tutorial:: Let's add a package to Guix!
81
82System Configuration
83
84* Customizing the Kernel:: Creating and using a custom Linux kernel
85
86
87@end detailmenu
88@end menu
89
90@c *********************************************************************
91@node Scheme tutorials
92@chapter Scheme tutorials
93
94GNU@tie{}Guix is written in the general purpose programming language Scheme,
95and many of its features can be accessed and manipulated programmatically.
96You can use Scheme to generate package definitions, to modify them, to build
97them, to deploy whole operating systems, etc.
98
99Knowing the basics of how to program in Scheme will unlock many of the
100advanced features Guix provides --- and you don't even need to be an
101experienced programmer to use them!
102
103Let's get started!
104
105@node A Scheme Crash Course
106@section A Scheme Crash Course
107
108@cindex Scheme, crash course
109
110Guix uses the Guile implementation of Scheme. To start playing with the
111language, install it with @code{guix install guile} and start a
112@uref{https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop,
113REPL} by running @code{guile} from the command line.
114
115Alternatively you can also run @code{guix environment --ad-hoc guile -- guile}
116if you'd rather not have Guile installed in your user profile.
117
cf1e6f5f
LC
118In the following examples, lines show what you would type at the REPL;
119lines starting with ``@result{}'' show evaluation results, while lines
120starting with ``@print{}'' show things that get printed. @xref{Using Guile
121Interactively,,, guile, GNU Guile Reference Manual}), for more details on the
7bc46ecc
RW
122REPL.
123
124@itemize
125@item
126Scheme syntax boils down to a tree of expressions (or @emph{s-expression} in
127Lisp lingo). An expression can be a literal such as numbers and strings, or a
128compound which is a parenthesized list of compounds and literals. @code{#t}
0cbef07b 129and @code{#f} stand for the Booleans ``true'' and ``false'', respectively.
7bc46ecc
RW
130
131Examples of valid expressions:
132
b1eecb5c 133@lisp
7bc46ecc 134"Hello World!"
cf1e6f5f
LC
135@result{} "Hello World!"
136
7bc46ecc 13717
cf1e6f5f
LC
138@result{} 17
139
140(display (string-append "Hello " "Guix" "\n"))
141@print{} Hello Guix!
142@result{} #<unspecified>
b1eecb5c 143@end lisp
7bc46ecc
RW
144
145@item
146This last example is a function call nested in another function call. When a
147parenthesized expression is evaluated, the first term is the function and the
148rest are the arguments passed to the function. Every function returns the
149last evaluated expression as its return value.
150
151@item
152Anonymous functions are declared with the @code{lambda} term:
153
b1eecb5c 154@lisp
cf1e6f5f
LC
155(lambda (x) (* x x))
156@result{} #<procedure 120e348 at <unknown port>:24:0 (x)>
b1eecb5c 157@end lisp
7bc46ecc
RW
158
159The above procedure returns the square of its argument. Since everything is
160an expression, the @code{lambda} expression returns an anonymous procedure,
161which can in turn be applied to an argument:
162
b1eecb5c 163@lisp
cf1e6f5f
LC
164((lambda (x) (* x x)) 3)
165@result{} 9
b1eecb5c 166@end lisp
7bc46ecc
RW
167
168@item
169Anything can be assigned a global name with @code{define}:
170
b1eecb5c 171@lisp
cf1e6f5f
LC
172(define a 3)
173(define square (lambda (x) (* x x)))
174(square a)
175@result{} 9
b1eecb5c 176@end lisp
7bc46ecc
RW
177
178@item
179Procedures can be defined more concisely with the following syntax:
180
b1eecb5c 181@lisp
7bc46ecc 182(define (square x) (* x x))
b1eecb5c 183@end lisp
7bc46ecc
RW
184
185@item
186A list structure can be created with the @code{list} procedure:
187
b1eecb5c 188@lisp
cf1e6f5f
LC
189(list 2 a 5 7)
190@result{} (2 3 5 7)
b1eecb5c 191@end lisp
7bc46ecc
RW
192
193@item
6a893fa8
LC
194The @dfn{quote} disables evaluation of a parenthesized expression: the
195first term is not called over the other terms (@pxref{Expression Syntax,
196quote,, guile, GNU Guile Reference Manual}). Thus it effectively
197returns a list of terms.
7bc46ecc 198
b1eecb5c 199@lisp
cf1e6f5f
LC
200'(display (string-append "Hello " "Guix" "\n"))
201@result{} (display (string-append "Hello " "Guix" "\n"))
202
203'(2 a 5 7)
204@result{} (2 a 5 7)
b1eecb5c 205@end lisp
7bc46ecc
RW
206
207@item
6a893fa8
LC
208The @dfn{quasiquote} disables evaluation of a parenthesized expression
209until @dfn{unquote} (a comma) re-enables it. Thus it provides us with
210fine-grained control over what is evaluated and what is not.
7bc46ecc 211
b1eecb5c 212@lisp
cf1e6f5f
LC
213`(2 a 5 7 (2 ,a 5 ,(+ a 4)))
214@result{} (2 a 5 7 (2 3 5 7))
b1eecb5c 215@end lisp
7bc46ecc
RW
216
217Note that the above result is a list of mixed elements: numbers, symbols (here
218@code{a}) and the last element is a list itself.
219
220@item
6a893fa8
LC
221Multiple variables can be named locally with @code{let} (@pxref{Local
222Bindings,,, guile, GNU Guile Reference Manual}):
7bc46ecc 223
b1eecb5c 224@lisp
cf1e6f5f
LC
225(define x 10)
226(let ((x 2)
227 (y 3))
228 (list x y))
229@result{} (2 3)
230
231x
232@result{} 10
233
234y
235@error{} In procedure module-lookup: Unbound variable: y
b1eecb5c 236@end lisp
7bc46ecc
RW
237
238Use @code{let*} to allow later variable declarations to refer to earlier
239definitions.
240
b1eecb5c 241@lisp
cf1e6f5f
LC
242(let* ((x 2)
243 (y (* x 3)))
244 (list x y))
245@result{} (2 6)
b1eecb5c 246@end lisp
7bc46ecc
RW
247
248@item
249The keyword syntax is @code{#:}; it is used to create unique identifiers.
250@pxref{Keywords,,, guile, GNU Guile Reference Manual}.
251
252@item
253The percentage @code{%} is typically used for read-only global variables in
254the build stage. Note that it is merely a convention, like @code{_} in C.
255Scheme treats @code{%} exactly the same as any other letter.
256
257@item
6a893fa8
LC
258Modules are created with @code{define-module} (@pxref{Creating Guile
259Modules,,, guile, GNU Guile Reference Manual}). For instance
7bc46ecc 260
b1eecb5c 261@lisp
7bc46ecc
RW
262(define-module (guix build-system ruby)
263 #:use-module (guix store)
264 #:export (ruby-build
265 ruby-build-system))
b1eecb5c 266@end lisp
7bc46ecc
RW
267
268defines the module @code{guix build-system ruby} which must be located in
269@file{guix/build-system/ruby.scm} somewhere in the Guile load path. It
270depends on the @code{(guix store)} module and it exports two variables,
271@code{ruby-build} and @code{ruby-build-system}.
272@end itemize
273
274For a more detailed introduction, check out
275@uref{http://www.troubleshooters.com/codecorn/scheme_guile/hello.htm, Scheme
276at a Glance}, by Steve Litt.
277
278One of the reference Scheme books is the seminal ``Structure and
279Interpretation of Computer Programs'', by Harold Abelson and Gerald Jay
280Sussman, with Julie Sussman. You'll find a
281@uref{https://mitpress.mit.edu/sites/default/files/sicp/index.html, free copy
282online}, together with
283@uref{https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/,
284videos of the lectures by the authors}. The book is available in Texinfo
285format as the @code{sicp} Guix package. Go ahead, run @code{guix install
ed79636c 286sicp} and start reading with @code{info sicp} (@pxref{,,, sicp, Structure and Interpretation of Computer Programs}).
7bc46ecc
RW
287An @uref{https://sarabander.github.io/sicp/, unofficial ebook is also
288available}.
289
290You'll find more books, tutorials and other resources at
291@url{https://schemers.org/}.
292
293
294@c *********************************************************************
295@node Packaging
296@chapter Packaging
297
298@cindex packaging
299
300This chapter is dedicated to teaching you how to add packages to the
301collection of packages that come with GNU Guix. This involves writing package
302definitions in Guile Scheme, organizing them in package modules, and building
303them.
304
305@menu
306* Packaging Tutorial:: A tutorial on how to add packages to Guix.
307@end menu
308
309@node Packaging Tutorial
310@section Packaging Tutorial
311
312GNU Guix stands out as the @emph{hackable} package manager, mostly because it
313uses @uref{https://www.gnu.org/software/guile/, GNU Guile}, a powerful
314high-level programming language, one of the
315@uref{https://en.wikipedia.org/wiki/Scheme_%28programming_language%29, Scheme}
316dialects from the
317@uref{https://en.wikipedia.org/wiki/Lisp_%28programming_language%29, Lisp family}.
318
319Package definitions are also written in Scheme, which empowers Guix in some
320very unique ways, unlike most other package managers that use shell scripts or
321simple languages.
322
323@itemize
324@item
325Use functions, structures, macros and all of Scheme expressiveness for your
326package definitions.
327
328@item
329Inheritance makes it easy to customize a package by inheriting from it and
330modifying only what is needed.
5c3e94cd 331
7bc46ecc
RW
332@item
333Batch processing: the whole package collection can be parsed, filtered and
334processed. Building a headless server with all graphical interfaces stripped
335out? It's possible. Want to rebuild everything from source using specific
336compiler optimization flags? Pass the @code{#:make-flags "..."} argument to
337the list of packages. It wouldn't be a stretch to think
338@uref{https://wiki.gentoo.org/wiki/USE_flag, Gentoo USE flags} here, but this
339goes even further: the changes don't have to be thought out beforehand by the
340packager, they can be @emph{programmed} by the user!
341@end itemize
342
343The following tutorial covers all the basics around package creation with Guix.
344It does not assume much knowledge of the Guix system nor of the Lisp language.
345The reader is only expected to be familiar with the command line and to have some
346basic programming knowledge.
347
0cbef07b
LC
348@node A ``Hello World'' package
349@subsection A ``Hello World'' package
7bc46ecc 350
0cbef07b 351The ``Defining Packages'' section of the manual introduces the basics of Guix
7bc46ecc
RW
352packaging (@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}). In
353the following section, we will partly go over those basics again.
354
0cbef07b 355GNU@tie{}Hello is a dummy project that serves as an idiomatic example for
7bc46ecc
RW
356packaging. It uses the GNU build system (@code{./configure && make && make
357install}). Guix already provides a package definition which is a perfect
358example to start with. You can look up its declaration with @code{guix edit
359hello} from the command line. Let's see how it looks:
360
b1eecb5c 361@lisp
7bc46ecc
RW
362(define-public hello
363 (package
364 (name "hello")
365 (version "2.10")
366 (source (origin
367 (method url-fetch)
368 (uri (string-append "mirror://gnu/hello/hello-" version
369 ".tar.gz"))
370 (sha256
371 (base32
372 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
373 (build-system gnu-build-system)
374 (synopsis "Hello, GNU world: An example GNU package")
375 (description
376 "GNU Hello prints the message \"Hello, world!\" and then exits. It
377serves as an example of standard GNU coding practices. As such, it supports
378command-line arguments, multiple languages, and so on.")
379 (home-page "https://www.gnu.org/software/hello/")
380 (license gpl3+)))
b1eecb5c 381@end lisp
7bc46ecc
RW
382
383As you can see, most of it is rather straightforward. But let's review the
384fields together:
385
386@table @samp
387@item name
388The project name. Using Scheme conventions, we prefer to keep it
389lower case, without underscore and using dash-separated words.
390
391@item source
392This field contains a description of the source code origin. The
393@code{origin} record contains these fields:
394
395@enumerate
396@item The method, here @code{url-fetch} to download via HTTP/FTP, but other methods
397 exist, such as @code{git-fetch} for Git repositories.
398@item The URI, which is typically some @code{https://} location for @code{url-fetch}. Here
399 the special `mirror://gnu` refers to a set of well known locations, all of
400 which can be used by Guix to fetch the source, should some of them fail.
401@item The @code{sha256} checksum of the requested file. This is essential to ensure
402 the source is not corrupted. Note that Guix works with base32 strings,
403 hence the call to the @code{base32} function.
404@end enumerate
405
406@item build-system
407
408This is where the power of abstraction provided by the Scheme language really
409shines: in this case, the @code{gnu-build-system} abstracts away the famous
410@code{./configure && make && make install} shell invocations. Other build
411systems include the @code{trivial-build-system} which does not do anything and
412requires from the packager to program all the build steps, the
413@code{python-build-system}, the @code{emacs-build-system}, and many more
414(@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
415
416@item synopsis
417It should be a concise summary of what the package does. For many packages a
418tagline from the project's home page can be used as the synopsis.
419
420@item description
421Same as for the synopsis, it's fine to re-use the project description from the
422homepage. Note that Guix uses Texinfo syntax.
423
424@item home-page
425Use HTTPS if available.
426
427@item license
428See @code{guix/licenses.scm} in the project source for a full list of
429available licenses.
430@end table
431
432Time to build our first package! Nothing fancy here for now: we will stick to a
0cbef07b 433dummy @code{my-hello}, a copy of the above declaration.
7bc46ecc 434
0cbef07b
LC
435As with the ritualistic ``Hello World'' taught with most programming languages,
436this will possibly be the most ``manual'' approach. We will work out an ideal
7bc46ecc
RW
437setup later; for now we will go the simplest route.
438
439Save the following to a file @file{my-hello.scm}.
440
b1eecb5c 441@lisp
7bc46ecc
RW
442(use-modules (guix packages)
443 (guix download)
444 (guix build-system gnu)
445 (guix licenses))
446
447(package
448 (name "my-hello")
449 (version "2.10")
450 (source (origin
451 (method url-fetch)
452 (uri (string-append "mirror://gnu/hello/hello-" version
453 ".tar.gz"))
454 (sha256
455 (base32
456 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
457 (build-system gnu-build-system)
458 (synopsis "Hello, Guix world: An example custom Guix package")
459 (description
460 "GNU Hello prints the message \"Hello, world!\" and then exits. It
461serves as an example of standard GNU coding practices. As such, it supports
462command-line arguments, multiple languages, and so on.")
463 (home-page "https://www.gnu.org/software/hello/")
464 (license gpl3+))
b1eecb5c 465@end lisp
7bc46ecc
RW
466
467We will explain the extra code in a moment.
468
469Feel free to play with the different values of the various fields. If you
470change the source, you'll need to update the checksum. Indeed, Guix refuses to
471build anything if the given checksum does not match the computed checksum of the
472source code. To obtain the correct checksum of the package declaration, we
473need to download the source, compute the sha256 checksum and convert it to
474base32.
475
476Thankfully, Guix can automate this task for us; all we need is to provide the
477URI:
478
479@c TRANSLATORS: This is example shell output.
480@example sh
481$ guix download mirror://gnu/hello/hello-2.10.tar.gz
482
483Starting download of /tmp/guix-file.JLYgL7
484From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
485following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
486 …10.tar.gz 709KiB 2.5MiB/s 00:00 [##################] 100.0%
487/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
4880ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
489@end example
490
491In this specific case the output tells us which mirror was chosen.
492If the result of the above command is not the same as in the above snippet,
493update your @code{my-hello} declaration accordingly.
494
495Note that GNU package tarballs come with an OpenPGP signature, so you
496should definitely check the signature of this tarball with `gpg` to
497authenticate it before going further:
498
499@c TRANSLATORS: This is example shell output.
500@example sh
501$ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig
502
503Starting download of /tmp/guix-file.03tFfb
504From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
505following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
506 ….tar.gz.sig 819B 1.2MiB/s 00:00 [##################] 100.0%
507/gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
5080q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
509$ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
510gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
511gpg: using RSA key A9553245FDE9B739
512gpg: Good signature from "Sami Kerola <kerolasa@@iki.fi>" [unknown]
513gpg: aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@@iki.fi>" [unknown]
514gpg: WARNING: This key is not certified with a trusted signature!
515gpg: There is no indication that the signature belongs to the owner.
516Primary key fingerprint: 8ED3 96E3 7E38 D471 A005 30D3 A955 3245 FDE9 B739
517@end example
518
519You can then happily run
520
521@c TRANSLATORS: Do not translate this command
522@example sh
523$ guix package --install-from-file=my-hello.scm
524@end example
525
526You should now have @code{my-hello} in your profile!
527
528@c TRANSLATORS: Do not translate this command
529@example sh
530$ guix package --list-installed=my-hello
531my-hello 2.10 out
532/gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10
533@end example
534
535We've gone as far as we could without any knowledge of Scheme. Before moving
536on to more complex packages, now is the right time to brush up on your Scheme
537knowledge. @pxref{A Scheme Crash Course} to get up to speed.
538
ffe059af
PN
539@node Setup
540@subsection Setup
7bc46ecc 541
ffe059af
PN
542In the rest of this chapter we will rely on some basic Scheme
543programming knowledge. Now let's detail the different possible setups
544for working on Guix packages.
545
546There are several ways to set up a Guix packaging environment.
547
548We recommend you work directly on the Guix source checkout since it makes it
549easier for everyone to contribute to the project.
550
551But first, let's look at other possibilities.
552
553@node Local file
554@subsubsection Local file
555
556This is what we previously did with @samp{my-hello}. With the Scheme basics we've
557covered, we are now able to explain the leading chunks. As stated in @code{guix
558package --help}:
559
560@example
561 -f, --install-from-file=FILE
562 install the package that the code within FILE
563 evaluates to
564@end example
565
566Thus the last expression @emph{must} return a package, which is the case in our
567earlier example.
568
569The @code{use-modules} expression tells which of the modules we need in the file.
570Modules are a collection of values and procedures. They are commonly called
0cbef07b 571``libraries'' or ``packages'' in other programming languages.
ffe059af
PN
572
573@node @samp{GUIX_PACKAGE_PATH}
574@subsubsection @samp{GUIX_PACKAGE_PATH}
575
0cbef07b 576@emph{Note: Starting from Guix 0.16, the more flexible Guix @dfn{channels} are the
ffe059af
PN
577preferred way and supersede @samp{GUIX_PACKAGE_PATH}. See next section.}
578
579It can be tedious to specify the file from the command line instead of simply
580calling @code{guix package --install my-hello} as you would do with the official
581packages.
582
0cbef07b
LC
583Guix makes it possible to streamline the process by adding as many ``package
584declaration directories'' as you want.
ffe059af 585
2c8305d1 586Create a directory, say @file{~./guix-packages} and add it to the @samp{GUIX_PACKAGE_PATH}
ffe059af
PN
587environment variable:
588
589@example
590$ mkdir ~/guix-packages
591$ export GUIX_PACKAGE_PATH=~/guix-packages
592@end example
593
594To add several directories, separate them with a colon (@code{:}).
595
596Our previous @samp{my-hello} needs some adjustments though:
597
d482e13f 598@lisp
ffe059af
PN
599(define-module (my-hello)
600 #:use-module (guix licenses)
601 #:use-module (guix packages)
602 #:use-module (guix build-system gnu)
603 #:use-module (guix download))
604
605(define-public my-hello
606 (package
607 (name "my-hello")
608 (version "2.10")
609 (source (origin
610 (method url-fetch)
611 (uri (string-append "mirror://gnu/hello/hello-" version
612 ".tar.gz"))
613 (sha256
614 (base32
615 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
616 (build-system gnu-build-system)
617 (synopsis "Hello, Guix world: An example custom Guix package")
618 (description
619 "GNU Hello prints the message \"Hello, world!\" and then exits. It
620serves as an example of standard GNU coding practices. As such, it supports
621command-line arguments, multiple languages, and so on.")
622 (home-page "https://www.gnu.org/software/hello/")
623 (license gpl3+)))
d482e13f 624@end lisp
ffe059af
PN
625
626Note that we have assigned the package value to an exported variable name with
627@code{define-public}. This is effectively assigning the package to the @code{my-hello}
628variable so that it can be referenced, among other as dependency of other
629packages.
630
631If you use @code{guix package --install-from-file=my-hello.scm} on the above file, it
632will fail because the last expression, @code{define-public}, does not return a
633package. If you want to use @code{define-public} in this use-case nonetheless, make
634sure the file ends with an evaluation of @code{my-hello}:
635
d482e13f 636@lisp
ffe059af
PN
637; ...
638(define-public my-hello
639 ; ...
640 )
641
642my-hello
d482e13f 643@end lisp
ffe059af
PN
644
645This last example is not very typical.
646
647Now @samp{my-hello} should be part of the package collection like all other official
648packages. You can verify this with:
649
650@example
651$ guix package --show=my-hello
652@end example
653
654@node Guix channels
655@subsubsection Guix channels
656
657Guix 0.16 features channels, which is very similar to @samp{GUIX_PACKAGE_PATH} but
658provides better integration and provenance tracking. Channels are not
659necessarily local, they can be maintained as a public Git repository for
660instance. Of course, several channels can be used at the same time.
661
662@xref{Channels,,, guix, GNU Guix Reference Manual} for setup details.
663
664@node Direct checkout hacking
665@subsubsection Direct checkout hacking
666
667Working directly on the Guix project is recommended: it reduces the friction
668when the time comes to submit your changes upstream to let the community benefit
669from your hard work!
670
671Unlike most software distributions, the Guix repository holds in one place both
672the tooling (including the package manager) and the package definitions. This
673choice was made so that it would give developers the flexibility to modify the
674API without breakage by updating all packages at the same time. This reduces
675development inertia.
676
677Check out the official @uref{https://git-scm.com/, Git} repository:
678
679@example
680$ git clone https://git.savannah.gnu.org/git/guix.git
681@end example
682
683In the rest of this article, we use @samp{$GUIX_CHECKOUT} to refer to the location of
684the checkout.
685
686
e97a4a29 687Follow the instructions in the manual (@pxref{Contributing,,, guix, GNU Guix
ffe059af
PN
688Reference Manual}) to set up the repository environment.
689
690Once ready, you should be able to use the package definitions from the
691repository environment.
692
693Feel free to edit package definitions found in @samp{$GUIX_CHECKOUT/gnu/packages}.
694
695The @samp{$GUIX_CHECKOUT/pre-inst-env} script lets you use @samp{guix} over the package
e97a4a29
LC
696collection of the repository (@pxref{Running Guix Before It Is
697Installed,,, guix, GNU Guix Reference Manual}).
ffe059af
PN
698
699@itemize
700@item
701Search packages, such as Ruby:
702
703@example
704 $ cd $GUIX_CHECKOUT
705 $ ./pre-inst-env guix package --list-available=ruby
706 ruby 1.8.7-p374 out gnu/packages/ruby.scm:119:2
707 ruby 2.1.6 out gnu/packages/ruby.scm:91:2
708 ruby 2.2.2 out gnu/packages/ruby.scm:39:2
709@end example
710
711@item
712Build a package, here Ruby version 2.1:
713
714@example
715 $ ./pre-inst-env guix build --keep-failed ruby@@2.1
716 /gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
717@end example
718
719@item
720Install it to your user profile:
721
722@example
723 $ ./pre-inst-env guix package --install ruby@@2.1
724@end example
725
726@item
727Check for common mistakes:
728
729@example
730 $ ./pre-inst-env guix lint ruby@@2.1
731@end example
732@end itemize
733
734Guix strives at maintaining a high packaging standard; when contributing to the
735Guix project, remember to
736
737@itemize
738@item
739follow the coding style (@pxref{Coding Style,,, guix, GNU Guix Reference Manual}),
740@item
741and review the check list from the manual (@pxref{Submitting Patches,,, guix, GNU Guix Reference Manual}).
742@end itemize
743
744Once you are happy with the result, you are welcome to send your contribution to
745make it part of Guix. This process is also detailed in the manual. (@pxref{Contributing,,, guix, GNU Guix Reference Manual})
746
747
748It's a community effort so the more join in, the better Guix becomes!
749
750@node Extended example
751@subsection Extended example
752
0cbef07b 753The above ``Hello World'' example is as simple as it goes. Packages can be more
ffe059af
PN
754complex than that and Guix can handle more advanced scenarios. Let's look at
755another, more sophisticated package (slightly modified from the source):
756
d482e13f 757@lisp
ffe059af
PN
758(define-module (gnu packages version-control)
759 #:use-module ((guix licenses) #:prefix license:)
760 #:use-module (guix utils)
761 #:use-module (guix packages)
762 #:use-module (guix git-download)
763 #:use-module (guix build-system cmake)
764 #:use-module (gnu packages ssh)
765 #:use-module (gnu packages web)
766 #:use-module (gnu packages pkg-config)
767 #:use-module (gnu packages python)
768 #:use-module (gnu packages compression)
769 #:use-module (gnu packages tls))
770
771(define-public my-libgit2
772 (let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
773 (revision "1"))
774 (package
775 (name "my-libgit2")
776 (version (git-version "0.26.6" revision commit))
777 (source (origin
778 (method git-fetch)
779 (uri (git-reference
780 (url "https://github.com/libgit2/libgit2/")
781 (commit commit)))
782 (file-name (git-file-name name version))
783 (sha256
784 (base32
785 "17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
786 (patches (search-patches "libgit2-mtime-0.patch"))
787 (modules '((guix build utils)))
788 (snippet '(begin
789 ;; Remove bundled software.
790 (delete-file-recursively "deps")
791 #t))))
792 (build-system cmake-build-system)
793 (outputs '("out" "debug"))
794 (arguments
795 `(#:tests? #t ; Run the test suite (this is the default)
796 #:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
797 #:phases
798 (modify-phases %standard-phases
799 (add-after 'unpack 'fix-hardcoded-paths
800 (lambda _
801 (substitute* "tests/repo/init.c"
802 (("#!/bin/sh") (string-append "#!" (which "sh"))))
803 (substitute* "tests/clar/fs.h"
804 (("/bin/cp") (which "cp"))
805 (("/bin/rm") (which "rm")))
806 #t))
807 ;; Run checks more verbosely.
808 (replace 'check
809 (lambda _ (invoke "./libgit2_clar" "-v" "-Q")))
810 (add-after 'unpack 'make-files-writable-for-tests
811 (lambda _ (for-each make-file-writable (find-files "." ".*")))))))
812 (inputs
813 `(("libssh2" ,libssh2)
814 ("http-parser" ,http-parser)
815 ("python" ,python-wrapper)))
816 (native-inputs
817 `(("pkg-config" ,pkg-config)))
818 (propagated-inputs
819 ;; These two libraries are in 'Requires.private' in libgit2.pc.
820 `(("openssl" ,openssl)
821 ("zlib" ,zlib)))
822 (home-page "https://libgit2.github.com/")
823 (synopsis "Library providing Git core methods")
824 (description
825 "Libgit2 is a portable, pure C implementation of the Git core methods
826provided as a re-entrant linkable library with a solid API, allowing you to
827write native speed custom Git applications in any language with bindings.")
828 ;; GPLv2 with linking exception
829 (license license:gpl2))))
d482e13f 830@end lisp
ffe059af
PN
831
832(In those cases were you only want to tweak a few fields from a package
833definition, you should rely on inheritance instead of copy-pasting everything.
834See below.)
835
836Let's discuss those fields in depth.
837
838@subsubsection @code{git-fetch} method
839
840Unlike the @code{url-fetch} method, @code{git-fetch} expects a @code{git-reference} which takes
841a Git repository and a commit. The commit can be any Git reference such as
842tags, so if the @code{version} is tagged, then it can be used directly. Sometimes
843the tag is prefixed with a @code{v}, in which case you'd use @code{(commit (string-append
844"v" version))}.
845
ab83e1f0
LC
846To ensure that the source code from the Git repository is stored in a
847directory with a descriptive name, we use @code{(file-name (git-file-name name
ffe059af
PN
848version))}.
849
ab83e1f0
LC
850The @code{git-version} procedure that can be used to derive the
851version when packaging programs for a specific commit, following the
852Guix contributor guidelines (@pxref{Version Numbers,,, guix, GNU Guix
853Reference Manual}).
ffe059af
PN
854
855@subsubsection Snippets
856
857Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching
2c8305d1 858the source. They are a Guix-y alternative to the traditional @file{.patch} files.
ffe059af 859Because of the quote, the code in only evaluated when passed to the Guix daemon
0cbef07b 860for building. There can be as many snippets as needed.
ffe059af
PN
861
862Snippets might need additional Guile modules which can be imported from the
863@code{modules} field.
864
865@subsubsection Inputs
866
867First, a syntactic comment: See the quasi-quote / comma syntax?
868
d482e13f 869@lisp
ffe059af
PN
870 (native-inputs
871 `(("pkg-config" ,pkg-config)))
d482e13f 872@end lisp
ffe059af
PN
873
874is equivalent to
875
d482e13f 876@lisp
ffe059af
PN
877 (native-inputs
878 (list (list "pkg-config" pkg-config)))
d482e13f 879@end lisp
ffe059af
PN
880
881You'll mostly see the former because it's shorter.
882
883There are 3 different input types. In short:
884
885@table @asis
886@item native-inputs
887Required for building but not runtime -- installing a package
888through a substitute won't install these inputs.
889@item inputs
890Installed in the store but not in the profile, as well as being
891present at build time.
892@item propagated-inputs
893Installed in the store and in the profile, as well as
894being present at build time.
895@end table
896
897@xref{Package Reference,,, guix, GNU Guix Reference Manual} for more details.
898
899The distinction between the various inputs is important: if a dependency can be
900handled as an @emph{input} instead of a @emph{propagated input}, it should be done so, or
0cbef07b 901else it ``pollutes'' the user profile for no good reason.
ffe059af
PN
902
903For instance, a user installing a graphical program that depends on a
904command line tool might only be interested in the graphical part, so there is no
905need to force the command line tool into the user profile. The dependency is a
906concern to the package, not to the user. @emph{Inputs} make it possible to handle
907dependencies without bugging the user by adding undesired executable files (or
908libraries) to their profile.
909
910Same goes for @emph{native-inputs}: once the program is installed, build-time
911dependencies can be safely garbage-collected.
912It also matters when a substitute is available, in which case only the @emph{inputs}
913and @emph{propagated inputs} will be fetched: the @emph{native inputs} are not required to
914install a package from a substitute.
915
916@subsubsection Outputs
917
918Just like how a package can have multiple inputs, it can also produce multiple
919outputs.
920
921Each output corresponds to a separate directory in the store.
922
923The user can choose which output to install; this is useful to save space or
924to avoid polluting the user profile with unwanted executables or libraries.
925
926Output separation is optional. When the @code{outputs} field is left out, the
927default and only output (the complete package) is referred to as @code{"out"}.
928
929Typical separate output names include @code{debug} and @code{doc}.
930
931It's advised to separate outputs only when you've shown it's worth it: if the
932output size is significant (compare with @code{guix size}) or in case the package is
933modular.
934
935@subsubsection Build system arguments
936
937The @code{arguments} is a keyword-value list used to configure the build process.
938
939The simplest argument @code{#:tests?} can be used to disable the test suite when
940building the package. This is mostly useful when the package does not feature
941any test suite. It's strongly recommended to keep the test suite on if there is
942one.
943
944Another common argument is @code{:make-flags}, which specifies a list of flags to
945append when running make, as you would from the command line. For instance, the
946following flags
947
d482e13f 948@lisp
ffe059af
PN
949#:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
950 "CC=gcc")
d482e13f 951@end lisp
ffe059af
PN
952
953translate into
954
955@example
956$ make CC=gcc prefix=/gnu/store/...-<out>
957@end example
958
959This sets the C compiler to @code{gcc} and the @code{prefix} variable (the installation
960directory in Make parlance) to @code{(assoc-ref %outputs "out")}, which is a build-stage
961global variable pointing to the destination directory in the store (something like
2c8305d1 962@file{/gnu/store/...-my-libgit2-20180408}).
ffe059af 963
0cbef07b 964Similarly, it's possible to set the configure flags:
ffe059af 965
d482e13f 966@lisp
ffe059af 967#:configure-flags '("-DUSE_SHA1DC=ON")
d482e13f 968@end lisp
ffe059af
PN
969
970The @code{%build-inputs} variable is also generated in scope. It's an association
971table that maps the input names to their store directories.
972
973The @code{phases} keyword lists the sequential steps of the build system. Typically
974phases include @code{unpack}, @code{configure}, @code{build}, @code{install} and @code{check}. To know
975more about those phases, you need to work out the appropriate build system
976definition in @samp{$GUIX_CHECKOUT/guix/build/gnu-build-system.scm}:
977
d482e13f 978@lisp
ffe059af
PN
979(define %standard-phases
980 ;; Standard build phases, as a list of symbol/procedure pairs.
981 (let-syntax ((phases (syntax-rules ()
982 ((_ p ...) `((p . ,p) ...)))))
983 (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
984 bootstrap
985 patch-usr-bin-file
986 patch-source-shebangs configure patch-generated-file-shebangs
987 build check install
988 patch-shebangs strip
989 validate-runpath
990 validate-documentation-location
991 delete-info-dir-file
992 patch-dot-desktop-files
993 install-license-files
994 reset-gzip-timestamps
995 compress-documentation)))
d482e13f 996@end lisp
ffe059af
PN
997
998Or from the REPL:
999
d482e13f 1000@lisp
cf1e6f5f
LC
1001(add-to-load-path "/path/to/guix/checkout")
1002,use (guix build gnu-build-system)
1003(map first %standard-phases)
1004@result{} (set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap patch-usr-bin-file patch-source-shebangs configure patch-generated-file-shebangs build check install patch-shebangs strip validate-runpath validate-documentation-location delete-info-dir-file patch-dot-desktop-files install-license-files reset-gzip-timestamps compress-documentation)
d482e13f 1005@end lisp
ffe059af
PN
1006
1007If you want to know more about what happens during those phases, consult the
1008associated procedures.
1009
1010For instance, as of this writing the definition of @code{unpack} for the GNU build
1011system is
1012
d482e13f 1013@lisp
ffe059af
PN
1014(define* (unpack #:key source #:allow-other-keys)
1015 "Unpack SOURCE in the working directory, and change directory within the
1016source. When SOURCE is a directory, copy it in a sub-directory of the current
1017working directory."
1018 (if (file-is-directory? source)
1019 (begin
1020 (mkdir "source")
1021 (chdir "source")
1022
1023 ;; Preserve timestamps (set to the Epoch) on the copied tree so that
1024 ;; things work deterministically.
1025 (copy-recursively source "."
1026 #:keep-mtime? #t))
1027 (begin
1028 (if (string-suffix? ".zip" source)
1029 (invoke "unzip" source)
1030 (invoke "tar" "xvf" source))
1031 (chdir (first-subdirectory "."))))
1032 #t)
d482e13f 1033@end lisp
ffe059af
PN
1034
1035Note the @code{chdir} call: it changes the working directory to where the source was
1036unpacked.
1037Thus every phase following the @code{unpack} will use the source as a working
1038directory, which is why we can directly work on the source files.
1039That is to say, unless a later phase changes the working directory to something
1040else.
1041
1042We modify the list of @code{%standard-phases} of the build system with the
1043@code{modify-phases} macro as per the list of specified modifications, which may have
1044the following forms:
1045
1046@itemize
1047@item
1048@code{(add-before PHASE NEW-PHASE PROCEDURE)}: Run @code{PROCEDURE} named @code{NEW-PHASE} before @code{PHASE}.
1049@item
1050@code{(add-after PHASE NEW-PHASE PROCEDURE)}: Same, but afterwards.
1051@item
1052@code{(replace PHASE PROCEDURE)}.
1053@item
1054@code{(delete PHASE)}.
1055@end itemize
1056
1057The @code{PROCEDURE} supports the keyword arguments @code{inputs} and @code{outputs}. Each
1058input (whether @emph{native}, @emph{propagated} or not) and output directory is referenced
1059by their name in those variables. Thus @code{(assoc-ref outputs "out")} is the store
1060directory of the main output of the package. A phase procedure may look like
1061this:
1062
d482e13f 1063@lisp
ffe059af
PN
1064(lambda* (#:key inputs outputs #:allow-other-keys)
1065 (let (((bash-directory (assoc-ref inputs "bash"))
1066 (output-directory (assoc-ref outputs "out"))
1067 (doc-directory (assoc-ref outputs "doc"))
1068 ; ...
1069 #t)
d482e13f 1070@end lisp
ffe059af
PN
1071
1072The procedure must return @code{#t} on success. It's brittle to rely on the return
1073value of the last expression used to tweak the phase because there is no
1074guarantee it would be a @code{#t}. Hence the trailing @code{#t} to ensure the right value
1075is returned on success.
1076
1077@subsubsection Code staging
1078
1079The astute reader may have noticed the quasi-quote and comma syntax in the
1080argument field. Indeed, the build code in the package declaration should not be
1081evaluated on the client side, but only when passed to the Guix daemon. This
1082mechanism of passing code around two running processes is called @uref{https://arxiv.org/abs/1709.00833, code staging}.
1083
0cbef07b 1084@subsubsection Utility functions
ffe059af
PN
1085
1086When customizing @code{phases}, we often need to write code that mimics the
cc765e47 1087equivalent system invocations (@code{make}, @code{mkdir}, @code{cp}, etc.)@: commonly used during
0cbef07b 1088regular ``Unix-style'' installations.
ffe059af
PN
1089
1090Some like @code{chmod} are native to Guile.
1091@xref{,,, guile, Guile reference manual} for a complete list.
1092
1093Guix provides additional helper functions which prove especially handy in the
1094context of package management.
1095
1096Some of those functions can be found in
1097@samp{$GUIX_CHECKOUT/guix/guix/build/utils.scm}. Most of them mirror the behaviour
1098of the traditional Unix system commands:
1099
1100@table @asis
1101@item which
1102Like the @samp{which} system command.
1103@item find-files
1104Akin to the @samp{find} system command.
1105@item mkdir-p
1106Like @samp{mkdir -p}, which creates all parents as needed.
1107@item install-file
1108Similar to @samp{install} when installing a file to a (possibly
1109non-existing) directory. Guile has @code{copy-file} which works
1110like @samp{cp}.
1111@item copy-recursively
1112Like @samp{cp -r}.
1113@item delete-file-recursively
1114Like @samp{rm -rf}.
1115@item invoke
1116Run an executable. This should be used instead of @code{system*}.
1117@item with-directory-excursion
1118Run the body in a different working directory,
1119then restore the previous working directory.
1120@item substitute*
0cbef07b 1121A ``@command{sed}-like'' function.
ffe059af
PN
1122@end table
1123
1124@subsubsection Module prefix
1125
1126The license in our last example needs a prefix: this is because of how the
1127@code{license} module was imported in the package, as @code{#:use-module ((guix licenses)
1128#:prefix license:)}. The Guile module import mechanism
1129(@pxref{Using Guile Modules,,, guile, Guile reference manual})
1130gives the user full control over namespacing: this is needed to avoid
1131clashes between, say, the
1132@samp{zlib} variable from @samp{licenses.scm} (a @emph{license} value) and the @samp{zlib} variable
1133from @samp{compression.scm} (a @emph{package} value).
1134
1135@node Other build systems
1136@subsection Other build systems
1137
1138What we've seen so far covers the majority of packages using a build system
1139other than the @code{trivial-build-system}. The latter does not automate anything
1140and leaves you to build everything manually. This can be more demanding and we
1141won't cover it here for now, but thankfully it is rarely necessary to fall back
1142on this system.
1143
1144For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the
1145process is very similar to the GNU build system except for a few specialized
1146arguments.
1147
a9e25512
LC
1148@xref{Build Systems,,, guix, GNU Guix Reference Manual}, for more
1149information on build systems, or check the source code in the
1150@samp{$GUIX_CHECKOUT/guix/build} and
ffe059af 1151@samp{$GUIX_CHECKOUT/guix/build-system} directories.
ffe059af
PN
1152
1153@node Programmable and automated package definition
1154@subsection Programmable and automated package definition
1155
1156We can't repeat it enough: having a full-fledged programming language at hand
1157empowers us in ways that reach far beyond traditional package management.
1158
1159Let's illustrate this with some awesome features of Guix!
1160
1161@node Recursive importers
1162@subsubsection Recursive importers
1163
1164You might find some build systems good enough that there is little to do at all
1165to write a package, to the point that it becomes repetitive and tedious after a
1166while. A @emph{raison d'être} of computers is to replace human beings at those
1167boring tasks. So let's tell Guix to do this for us and create the package
1168definition of an R package from CRAN (the output is trimmed for conciseness):
1169
1170@example
1171$ guix import cran --recursive walrus
1172
1173(define-public r-mc2d
1174 ; ...
1175 (license gpl2+)))
1176
1177(define-public r-jmvcore
1178 ; ...
1179 (license gpl2+)))
1180
1181(define-public r-wrs2
1182 ; ...
1183 (license gpl3)))
1184
1185(define-public r-walrus
1186 (package
1187 (name "r-walrus")
1188 (version "1.0.3")
1189 (source
1190 (origin
1191 (method url-fetch)
1192 (uri (cran-uri "walrus" version))
1193 (sha256
1194 (base32
1195 "1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
1196 (build-system r-build-system)
1197 (propagated-inputs
1198 `(("r-ggplot2" ,r-ggplot2)
1199 ("r-jmvcore" ,r-jmvcore)
1200 ("r-r6" ,r-r6)
1201 ("r-wrs2" ,r-wrs2)))
1202 (home-page "https://github.com/jamovi/walrus")
1203 (synopsis "Robust Statistical Methods")
1204 (description
1205 "This package provides a toolbox of common robust statistical
1206tests, including robust descriptives, robust t-tests, and robust ANOVA.
1207It is also available as a module for 'jamovi' (see
1208<https://www.jamovi.org> for more information). Walrus is based on the
1209WRS2 package by Patrick Mair, which is in turn based on the scripts and
1210work of Rand Wilcox. These analyses are described in depth in the book
1211'Introduction to Robust Estimation & Hypothesis Testing'.")
1212 (license gpl3)))
1213@end example
1214
1215The recursive importer won't import packages for which Guix already has package
1216definitions, except for the very first.
1217
1218Not all applications can be packaged this way, only those relying on a select
1219number of supported systems. Read about the full list of importers in
1220the guix import section of the manual
1221(@pxref{Invoking guix import,,, guix, GNU Guix Reference Manual}).
1222
1223@node Automatic update
1224@subsubsection Automatic update
1225
1226Guix can be smart enough to check for updates on systems it knows. It can
1227report outdated package definitions with
1228
1229@example
1230$ guix refresh hello
1231@end example
1232
1233In most cases, updating a package to a newer version requires little more than
1234changing the version number and the checksum. Guix can do that automatically as
1235well:
1236
1237@example
1238$ guix refresh hello --update
1239@end example
1240
1241@node Inheritance
1242@subsubsection Inheritance
1243
1244If you've started browsing the existing package definitions, you might have
1245noticed that a significant number of them have a @code{inherit} field:
1246
d482e13f 1247@lisp
ffe059af
PN
1248(define-public adwaita-icon-theme
1249 (package (inherit gnome-icon-theme)
1250 (name "adwaita-icon-theme")
1251 (version "3.26.1")
1252 (source (origin
1253 (method url-fetch)
1254 (uri (string-append "mirror://gnome/sources/" name "/"
1255 (version-major+minor version) "/"
1256 name "-" version ".tar.xz"))
1257 (sha256
1258 (base32
1259 "17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
1260 (native-inputs
1261 `(("gtk-encode-symbolic-svg" ,gtk+ "bin")))))
d482e13f 1262@end lisp
ffe059af
PN
1263
1264All unspecified fields are inherited from the parent package. This is very
1265convenient to create alternative packages, for instance with different source,
1266version or compilation options.
1267
1268@node Getting help
1269@subsection Getting help
1270
1271Sadly, some applications can be tough to package. Sometimes they need a patch to
1272work with the non-standard filesystem hierarchy enforced by the store.
1273Sometimes the tests won't run properly. (They can be skipped but this is not
1274recommended.) Other times the resulting package won't be reproducible.
1275
1276Should you be stuck, unable to figure out how to fix any sort of packaging
1277issue, don't hesitate to ask the community for help.
1278
1279See the @uref{https://www.gnu.org/software/guix/contact/, Guix homepage} for information on the mailing lists, IRC, etc.
1280
1281@node Conclusion
1282@subsection Conclusion
1283
1284This tutorial was a showcase of the sophisticated package management that Guix
1285boasts. At this point we have mostly restricted this introduction to the
1286@code{gnu-build-system} which is a core abstraction layer on which more advanced
1287abstractions are based.
1288
1289Where do we go from here? Next we ought to dissect the innards of the build
1290system by removing all abstractions, using the @code{trivial-build-system}: this
1291should give us a thorough understanding of the process before investigating some
1292more advanced packaging techniques and edge cases.
1293
1294Other features worth exploring are the interactive editing and debugging
1295capabilities of Guix provided by the Guile REPL@.
1296
1297Those fancy features are completely optional and can wait; now is a good time
1298to take a well-deserved break. With what we've introduced here you should be
1299well armed to package lots of programs. You can get started right away and
1300hopefully we will see your contributions soon!
1301
1302@node References
1303@subsection References
1304
1305@itemize
1306@item
1307The @uref{https://www.gnu.org/software/guix/manual/en/html_node/Defining-Packages.html, package reference in the manual}
1308
1309@item
1310@uref{https://gitlab.com/pjotrp/guix-notes/blob/master/HACKING.org, Pjotr’s hacking guide to GNU Guix}
1311
1312@item
0cbef07b 1313@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
ffe059af 1314@end itemize
7bc46ecc
RW
1315
1316@c *********************************************************************
1317@node System Configuration
1318@chapter System Configuration
1319
1320Guix offers a flexible language for declaratively configuring your Guix
1321System. This flexibility can at times be overwhelming. The purpose of this
1322chapter is to demonstrate some advanced configuration concepts.
1323
1324@pxref{System Configuration,,, guix, GNU Guix Reference Manual} for a complete
1325reference.
1326
1327@menu
65051fdc 1328* Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
5c3e94cd 1329* Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
65051fdc 1330* Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
60651dd9 1331* Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
c987b723 1332* Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
7bc46ecc
RW
1333@end menu
1334
1335@node Customizing the Kernel
1336@section Customizing the Kernel
1337
1338Guix is, at its core, a source based distribution with substitutes
1339(@pxref{Substitutes,,, guix, GNU Guix Reference Manual}), and as such building
1340packages from their source code is an expected part of regular package
1341installations and upgrades. Given this starting point, it makes sense that
1342efforts are made to reduce the amount of time spent compiling packages, and
1343recent changes and upgrades to the building and distribution of substitutes
1344continues to be a topic of discussion within Guix.
1345
1346The kernel, while not requiring an overabundance of RAM to build, does take a
1347rather long time on an average machine. The official kernel configuration, as
1348is the case with many GNU/Linux distributions, errs on the side of
1349inclusiveness, and this is really what causes the build to take such a long
1350time when the kernel is built from source.
1351
1352The Linux kernel, however, can also just be described as a regular old
1353package, and as such can be customized just like any other package. The
1354procedure is a little bit different, although this is primarily due to the
1355nature of how the package definition is written.
1356
1357The @code{linux-libre} kernel package definition is actually a procedure which
1358creates a package.
1359
b1eecb5c 1360@lisp
7bc46ecc
RW
1361(define* (make-linux-libre version hash supported-systems
1362 #:key
1363 ;; A function that takes an arch and a variant.
1364 ;; See kernel-config for an example.
1365 (extra-version #f)
1366 (configuration-file #f)
1367 (defconfig "defconfig")
1368 (extra-options %default-extra-linux-options)
1369 (patches (list %boot-logo-patch)))
1370 ...)
b1eecb5c 1371@end lisp
7bc46ecc
RW
1372
1373The current @code{linux-libre} package is for the 5.1.x series, and is
1374declared like this:
1375
b1eecb5c 1376@lisp
7bc46ecc
RW
1377(define-public linux-libre
1378 (make-linux-libre %linux-libre-version
1379 %linux-libre-hash
1380 '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux")
1381 #:patches %linux-libre-5.1-patches
1382 #:configuration-file kernel-config))
b1eecb5c 1383@end lisp
7bc46ecc
RW
1384
1385Any keys which are not assigned values inherit their default value from the
1386@code{make-linux-libre} definition. When comparing the two snippets above,
1387you may notice that the code comment in the first doesn't actually refer to
1388the @code{#:extra-version} keyword; it is actually for
1389@code{#:configuration-file}. Because of this, it is not actually easy to
1390include a custom kernel configuration from the definition, but don't worry,
1391there are other ways to work with what we do have.
1392
1393There are two ways to create a kernel with a custom kernel configuration. The
1394first is to provide a standard @file{.config} file during the build process by
1395including an actual @file{.config} file as a native input to our custom
1396kernel. The following is a snippet from the custom @code{'configure} phase of
1397the @code{make-linux-libre} package definition:
1398
b1eecb5c 1399@lisp
7bc46ecc
RW
1400(let ((build (assoc-ref %standard-phases 'build))
1401 (config (assoc-ref (or native-inputs inputs) "kconfig")))
1402
1403 ;; Use a custom kernel configuration file or a default
1404 ;; configuration file.
1405 (if config
1406 (begin
1407 (copy-file config ".config")
1408 (chmod ".config" #o666))
1409 (invoke "make" ,defconfig))
b1eecb5c 1410@end lisp
7bc46ecc
RW
1411
1412Below is a sample kernel package. The @code{linux-libre} package is nothing
1413special and can be inherited from and have its fields overridden like any
1414other package:
1415
b1eecb5c 1416@lisp
7bc46ecc
RW
1417(define-public linux-libre/E2140
1418 (package
1419 (inherit linux-libre)
1420 (native-inputs
1421 `(("kconfig" ,(local-file "E2140.config"))
1422 ,@@(alist-delete "kconfig"
1423 (package-native-inputs linux-libre))))))
b1eecb5c 1424@end lisp
7bc46ecc
RW
1425
1426In the same directory as the file defining @code{linux-libre-E2140} is a file
1427named @file{E2140.config}, which is an actual kernel configuration file. The
1428@code{defconfig} keyword of @code{make-linux-libre} is left blank here, so the
1429only kernel configuration in the package is the one which was included in the
1430@code{native-inputs} field.
1431
1432The second way to create a custom kernel is to pass a new value to the
1433@code{extra-options} keyword of the @code{make-linux-libre} procedure. The
1434@code{extra-options} keyword works with another function defined right below
1435it:
1436
b1eecb5c 1437@lisp
7bc46ecc
RW
1438(define %default-extra-linux-options
1439 `(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
1440 ("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #t)
1441 ;; Modules required for initrd:
1442 ("CONFIG_NET_9P" . m)
1443 ("CONFIG_NET_9P_VIRTIO" . m)
1444 ("CONFIG_VIRTIO_BLK" . m)
1445 ("CONFIG_VIRTIO_NET" . m)
1446 ("CONFIG_VIRTIO_PCI" . m)
1447 ("CONFIG_VIRTIO_BALLOON" . m)
1448 ("CONFIG_VIRTIO_MMIO" . m)
1449 ("CONFIG_FUSE_FS" . m)
1450 ("CONFIG_CIFS" . m)
1451 ("CONFIG_9P_FS" . m)))
1452
1453(define (config->string options)
1454 (string-join (map (match-lambda
1455 ((option . 'm)
1456 (string-append option "=m"))
1457 ((option . #t)
1458 (string-append option "=y"))
1459 ((option . #f)
1460 (string-append option "=n")))
1461 options)
1462 "\n"))
b1eecb5c 1463@end lisp
7bc46ecc
RW
1464
1465And in the custom configure script from the `make-linux-libre` package:
1466
b1eecb5c 1467@lisp
7bc46ecc
RW
1468;; Appending works even when the option wasn't in the
1469;; file. The last one prevails if duplicated.
1470(let ((port (open-file ".config" "a"))
1471 (extra-configuration ,(config->string extra-options)))
1472 (display extra-configuration port)
1473 (close-port port))
1474
1475(invoke "make" "oldconfig"))))
b1eecb5c 1476@end lisp
7bc46ecc
RW
1477
1478So by not providing a configuration-file the @file{.config} starts blank, and
1479then we write into it the collection of flags that we want. Here's another
1480custom kernel:
1481
b1eecb5c 1482@lisp
7bc46ecc
RW
1483(define %macbook41-full-config
1484 (append %macbook41-config-options
1485 %filesystems
1486 %efi-support
1487 %emulation
1488 (@@@@ (gnu packages linux) %default-extra-linux-options)))
1489
1490(define-public linux-libre-macbook41
1491 ;; XXX: Access the internal 'make-linux-libre' procedure, which is
1492 ;; private and unexported, and is liable to change in the future.
1493 ((@@@@ (gnu packages linux) make-linux-libre) (@@@@ (gnu packages linux) %linux-libre-version)
1494 (@@@@ (gnu packages linux) %linux-libre-hash)
1495 '("x86_64-linux")
1496 #:extra-version "macbook41"
1497 #:patches (@@@@ (gnu packages linux) %linux-libre-5.1-patches)
1498 #:extra-options %macbook41-config-options))
b1eecb5c 1499@end lisp
7bc46ecc
RW
1500
1501In the above example @code{%filesystems} is a collection of flags enabling
1502different filesystem support, @code{%efi-support} enables EFI support and
1503@code{%emulation} enables a x86_64-linux machine to act in 32-bit mode also.
1504@code{%default-extra-linux-options} are the ones quoted above, which had to be
1505added in since they were replaced in the @code{extra-options} keyword.
1506
1507This all sounds like it should be doable, but how does one even know which
1508modules are required for a particular system? Two places that can be helpful
1509in trying to answer this question is the
1510@uref{https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel, Gentoo
1511Handbook} and the
1512@uref{https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig,
1513documentation from the kernel itself}. From the kernel documentation, it
1514seems that @code{make localmodconfig} is the command we want.
1515
1516In order to actually run @code{make localmodconfig} we first need to get and
1517unpack the kernel source code:
1518
1519@example shell
1520tar xf $(guix build linux-libre --source)
1521@end example
1522
1523Once inside the directory containing the source code run @code{touch .config}
1524to create an initial, empty @file{.config} to start with. @code{make
1525localmodconfig} works by seeing what you already have in @file{.config} and
1526letting you know what you're missing. If the file is blank then you're
1527missing everything. The next step is to run:
1528
1529@example shell
1530guix environment linux-libre -- make localmodconfig
1531@end example
1532
1533and note the output. Do note that the @file{.config} file is still empty.
1534The output generally contains two types of warnings. The first start with
1535"WARNING" and can actually be ignored in our case. The second read:
1536
1537@example shell
1538module pcspkr did not have configs CONFIG_INPUT_PCSPKR
1539@end example
1540
1541For each of these lines, copy the @code{CONFIG_XXXX_XXXX} portion into the
1542@file{.config} in the directory, and append @code{=m}, so in the end it looks
1543like this:
1544
1545@example shell
1546CONFIG_INPUT_PCSPKR=m
1547CONFIG_VIRTIO=m
1548@end example
1549
1550After copying all the configuration options, run @code{make localmodconfig}
0cbef07b 1551again to make sure that you don't have any output starting with ``module''.
7bc46ecc
RW
1552After all of these machine specific modules there are a couple more left that
1553are also needed. @code{CONFIG_MODULES} is necessary so that you can build and
1554load modules separately and not have everything built into the kernel.
1555@code{CONFIG_BLK_DEV_SD} is required for reading from hard drives. It is
1556possible that there are other modules which you will need.
1557
1558This post does not aim to be a guide to configuring your own kernel however,
1559so if you do decide to build a custom kernel you'll have to seek out other
1560guides to create a kernel which is just right for your needs.
1561
1562The second way to setup the kernel configuration makes more use of Guix's
1563features and allows you to share configuration segments between different
1564kernels. For example, all machines using EFI to boot have a number of EFI
1565configuration flags that they need. It is likely that all the kernels will
1566share a list of filesystems to support. By using variables it is easier to
1567see at a glance what features are enabled and to make sure you don't have
1568features in one kernel but missing in another.
1569
1570Left undiscussed however, is Guix's initrd and its customization. It is
1571likely that you'll need to modify the initrd on a machine using a custom
1572kernel, since certain modules which are expected to be built may not be
1573available for inclusion into the initrd.
1574
5c3e94cd
MK
1575@node Connecting to Wireguard VPN
1576@section Connecting to Wireguard VPN
1577
1578To connect to a Wireguard VPN server you need the kernel module to be
1579loaded in memory and a package providing networking tools that support
1580it (e.g. @code{wireguard-tools} or @code{network-manager}).
1581
1582Here is a configuration example for Linux-Libre < 5.6, where the module
1583is out of tree and need to be loaded manually---following revisions of
1584the kernel have it built-in and so don't need such configuration:
1585
1586@lisp
1587(use-modules (gnu))
1588(use-service-modules desktop)
1589(use-package-modules vpn)
1590
1591(operating-system
1592 ;; …
1593 (services (cons (simple-service 'wireguard-module
1594 kernel-module-loader-service-type
1595 '("wireguard"))
1596 %desktop-services))
1597 (packages (cons wireguard-tools %base-packages))
1598 (kernel-loadable-modules (list wireguard-linux-compat)))
1599@end lisp
1600
1601After reconfiguring and restarting your system you can either use
1602Wireguard tools or NetworkManager to connect to a VPN server.
1603
1604@subsection Using Wireguard tools
1605
1606To test your Wireguard setup it is convenient to use @command{wg-quick}.
1607Just give it a configuration file @command{wg-quick up ./wg0.conf}; or
1608put that file in @file{/etc/wireguard} and run @command{wg-quick up wg0}
1609instead.
1610
1611@quotation Note
1612Be warned that the author described this command as a: “[…] very quick
1613and dirty bash script […]”.
1614@end quotation
1615
1616@subsection Using NetworkManager
1617
1618Thanks to NetworkManager support for Wireguard we can connect to our VPN
1619using @command{nmcli} command. Up to this point this guide assumes that
1620you're using Network Manager service provided by
1621@code{%desktop-services}. Ortherwise you need to adjust your services
1622list to load @code{network-manager-service-type} and reconfigure your
1623Guix system.
1624
1625To import your VPN configuration execute nmcli import command:
1626
1627@example shell
1628# nmcli connection import type wireguard file wg0.conf
1629Connection 'wg0' (edbee261-aa5a-42db-b032-6c7757c60fde) successfully added
1630@end example
1631
1632This will create a configuration file in
1633@file{/etc/NetworkManager/wg0.nmconnection}. Next connect to the
1634Wireguard server:
1635
1636@example shell
1637$ nmcli connection up wg0
1638Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
1639@end example
1640
1641By default NetworkManager will connect automatically on system boot. To
1642change that behaviour you need to edit your config:
1643
1644@example shell
1645# nmcli connection modify wg0 connection.autoconnect no
1646@end example
1647
1648For more specific information about NetworkManager and wireguard
1649@uref{https://blogs.gnome.org/thaller/2019/03/15/wireguard-in-networkmanager/,see
1650this post by thaller}.
1651
65051fdc
OP
1652@node Customizing a Window Manager
1653@section Customizing a Window Manager
1654@cindex wm
1655
1656@node StumpWM
1657@subsection StumpWM
1658@cindex stumpwm
1659
1660You could install StumpWM with a Guix system by adding
145df67b 1661@code{stumpwm} and optionally @code{`(,stumpwm "lib")}
cc765e47 1662packages to a system configuration file, e.g.@: @file{/etc/config.scm}.
65051fdc
OP
1663
1664An example configuration can look like this:
1665
1666@lisp
1667(use-modules (gnu))
1668(use-package-modules wm)
1669
1670(operating-system
1671 ;; …
145df67b 1672 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
65051fdc
OP
1673 %base-packages)))
1674@end lisp
1675
1676@cindex stumpwm fonts
1677By default StumpWM uses X11 fonts, which could be small or pixelated on
1678your system. You could fix this by installing StumpWM contrib Lisp
751d1f01 1679module @code{sbcl-ttf-fonts}, adding it to Guix system packages:
65051fdc
OP
1680
1681@lisp
1682(use-modules (gnu))
1683(use-package-modules fonts wm)
1684
1685(operating-system
1686 ;; …
145df67b 1687 (packages (append (list sbcl stumpwm `(,stumpwm "lib"))
751d1f01 1688 sbcl-ttf-fonts font-dejavu %base-packages)))
65051fdc
OP
1689@end lisp
1690
1691Then you need to add the following code to a StumpWM configuration file
1692@file{~/.stumpwm.d/init.lisp}:
1693
1694@lisp
1695(require :ttf-fonts)
1696(setf xft:*font-dirs* '("/run/current-system/profile/share/fonts/"))
1697(setf clx-truetype:+font-cache-filename+ (concat (getenv "HOME") "/.fonts/font-cache.sexp"))
1698(xft:cache-fonts)
1699(set-font (make-instance 'xft:font :family "DejaVu Sans Mono" :subfamily "Book" :size 11))
1700@end lisp
1701
3142191d
CB
1702@node Session lock
1703@subsection Session lock
1704@cindex sessionlock
1705
1706Depending on your environment, locking the screen of your session might come built in
1707or it might be something you have to set up yourself. If you use a desktop environment
1708like GNOME or KDE, it's usually built in. If you use a plain window manager like
1709StumpWM or EXWM, you might have to set it up yourself.
1710
1711@node Xorg
1712@subsubsection Xorg
1713
1714If you use Xorg, you can use the utility
1715@uref{https://www.mankier.com/1/xss-lock, xss-lock} to lock the screen of your session.
1716xss-lock is triggered by DPMS which since Xorg 1.8 is auto-detected and enabled if
1717ACPI is also enabled at kernel runtime.
1718
1719To use xss-lock, you can simple execute it and put it into the background before
1720you start your window manager from e.g. your @file{~/.xsession}:
1721
1722@example
1723xss-lock -- slock &
1724exec stumpwm
1725@end example
1726
1727In this example, xss-lock uses @code{slock} to do the actual locking of the screen when
1728it determines it's appropriate, like when you suspend your device.
1729
1730For slock to be allowed to be a screen locker for the graphical session, it needs to
1731be made setuid-root so it can authenticate users, and it needs a PAM service. This
1732can be achieved by adding the following service to your @file{config.scm}:
1733
1734@lisp
1735(screen-locker-service slock)
1736@end lisp
1737
1738If you manually lock your screen, e.g. by directly calling slock when you want to lock
1739your screen but not suspend it, it's a good idea to notify xss-lock about this so no
1740confusion occurs. This can be done by executing @code{xset s activate} immediately
1741before you execute slock.
1742
60651dd9
MB
1743@node Setting up a bind mount
1744@section Setting up a bind mount
1745
1746To bind mount a file system, one must first set up some definitions
cc765e47 1747before the @code{operating-system} section of the system definition. In
60651dd9 1748this example we will bind mount a folder from a spinning disk drive to
2c8305d1
NG
1749@file{/tmp}, to save wear and tear on the primary SSD, without
1750dedicating an entire partition to be mounted as @file{/tmp}.
60651dd9
MB
1751
1752First, the source drive that hosts the folder we wish to bind mount
1753should be defined, so that the bind mount can depend on it.
1754
1755@lisp
1756(define source-drive ;; "source-drive" can be named anything you want.
1757 (file-system
1758 (device (uuid "UUID goes here"))
1759 (mount-point "/path-to-spinning-disk-goes-here")
1760 (type "ext4"))) ;; Make sure to set this to the appropriate type for your drive.
1761@end lisp
1762
1763The source folder must also be defined, so that guix will know it's not
1764a regular block device, but a folder.
1765@lisp
1766(define (%source-directory) "/path-to-spinning-disk-goes-here/tmp") ;; "source-directory" can be named any valid variable name.
1767@end lisp
1768
1769Finally, inside the @code{file-systems} definition, we must add the
1770mount itself.
1771
1772@lisp
1773(file-systems (cons*
1774
1775 ...<other drives omitted for clarity>...
1776
1777 source-drive ;; Must match the name you gave the source drive in the earlier definition.
1778
1779 (file-system
1780 (device (%source-directory)) ;; Make sure "source-directory" matches your earlier definition.
1781 (mount-point "/tmp")
1782 (type "none") ;; We are mounting a folder, not a partition, so this type needs to be "none"
1783 (flags '(bind-mount))
1784 (dependencies (list source-drive)) ;; Ensure "source-drive" matches what you've named the variable for the drive.
1785 )
1786
1787 ...<other drives omitted for clarity>...
1788
1789 ))
1790@end lisp
1791
c987b723
BW
1792@node Getting substitutes from Tor
1793@section Getting substitutes from Tor
1794
1795Guix daemon can use a HTTP proxy to get substitutes, here we are
1796configuring it to get them via Tor.
1797
1798@quotation Warning
1799@emph{Not all} Guix daemon's traffic will go through Tor! Only
1800HTTP/HTTPS will get proxied; FTP, Git protocol, SSH, etc connections
1801will still go through the clearnet. Again, this configuration isn't
1802foolproof some of your traffic won't get routed by Tor at all. Use it
1803at your own risk.
1804@end quotation
1805
1806Guix's substitute server is available as a Onion service, if you want
1807to use it to get your substitutes from Tor configure your system as
1808follow:
1809
1810@lisp
1811(use-modules (gnu))
1812(use-service-module base networking)
1813
1814(operating-system
1815
1816 (services
1817 (cons
1818 (service tor-service-type
1819 (tor-configuration
1820 (config-file (plain-file "tor-config"
1821 "HTTPTunnelPort 127.0.0.1:9250"))))
1822 (modify-services %base-services
1823 (guix-service-type
1824 config => (guix-configuration
1825 (inherit config)
1826 ;; ci.guix.gnu.org's Onion service
1827 (substitute-urls "https://bp7o7ckwlewr4slm.onion")
1828 (http-proxy "http://localhost:9250")))))))
1829@end lisp
1830
1831This will keep a tor process running that provides a HTTP CONNECT tunnel
1832which will be used by @command{guix-daemon}. The daemon can use other
1833protocols than HTTP(S) to get remote resources, request using those
1834protocols won't go through Tor since we are only setting a HTTP tunnel
1835here. Note that @code{substitutes-urls} is using HTTPS and not HTTP or
1836it won't work, that's a limitation of Tor's tunnel; you may want to use
1837@command{privoxy} instead to avoid such limitations.
1838
1839If you don't want to always get substitutes through Tor but using it just
1840some of the times, then skip the @code{guix-configuration}. When you
1841want to get a substitute from the Tor tunnel run:
1842
1843@example
1844sudo herd set-http-proxy guix-daemon http://localhost:9250
1845guix build --substitute-urls=https://bp7o7ckwlewr4slm.onion …
1846@end example
1847
4c463569
PN
1848@c *********************************************************************
1849@node Advanced package management
1850@chapter Advanced package management
1851
1852Guix is a functional package manager that offers many features beyond
1853what more traditional package managers can do. To the uninitiated,
1854those features might not have obvious use cases at first. The purpose
1855of this chapter is to demonstrate some advanced package management
1856concepts.
1857
1858@pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
1859reference.
1860
1861@menu
1862* Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
1863@end menu
1864
1865@node Guix Profiles in Practice
1866@section Guix Profiles in Practice
1867
1868Guix provides a very useful feature that may be quite foreign to newcomers:
1869@emph{profiles}. They are a way to group package installations together and all users
f6c27c55 1870on the same system are free to use as many profiles as they want.
4c463569
PN
1871
1872Whether you're a developer or not, you may find that multiple profiles bring you
1873great power and flexibility. While they shift the paradigm somewhat compared to
1874@emph{traditional package managers}, they are very convenient to use once you've
1875understood how to set them up.
1876
1877If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
1878kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
1879just Python software. Furthermore, profiles are self-sufficient: they capture
1880all the runtime dependencies which guarantees that all programs within a profile
1881will always work at any point in time.
1882
1883Multiple profiles have many benefits:
1884
1885@itemize
1886@item
1887Clean semantic separation of the various packages a user needs for different contexts.
1888
1889@item
1890Multiple profiles can be made available into the environment either on login
1891or within a dedicated shell.
1892
1893@item
1894Profiles can be loaded on demand. For instance, the user can use multiple
1895shells, each of them running different profiles.
1896
1897@item
1898Isolation: Programs from one profile will not use programs from the other, and
f6c27c55 1899the user can even install different versions of the same programs to the two
4c463569
PN
1900profiles without conflict.
1901
1902@item
1903Deduplication: Profiles share dependencies that happens to be the exact same.
1904This makes multiple profiles storage-efficient.
1905
1906@item
1907Reproducible: when used with declarative manifests, a profile can be fully
1908specified by the Guix commit that was active when it was set up. This means
f6c27c55
PN
1909that the exact same profile can be
1910@uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
1911set up anywhere and anytime}, with just the commit information. See the
1912section on @ref{Reproducible profiles}.
4c463569
PN
1913
1914@item
1915Easier upgrades and maintenance: Multiple profiles make it easy to keep
1916package listings at hand and make upgrades completely friction-less.
1917@end itemize
1918
1919Concretely, here follows some typical profiles:
1920
1921@itemize
1922@item
1923The dependencies of a project you are working on.
1924
1925@item
1926Your favourite programming language libraries.
1927
1928@item
1929Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
1930
1931@item
1932@TeX{}live (this one can be really useful when you need to install just one
1933package for this one document you've just received over email).
1934
1935@item
1936Games.
1937@end itemize
1938
1939Let's dive in the set up!
1940
1941@node Basic setup with manifests
1942@subsection Basic setup with manifests
1943
1944A Guix profile can be set up @emph{via} a so-called @emph{manifest specification} that looks like
1945this:
1946
b1eecb5c 1947@lisp
4c463569
PN
1948(specifications->manifest
1949 '("package-1"
1950 ;; Version 1.3 of package-2.
1951 "package-2@@1.3"
1952 ;; The "lib" output of package-3.
1953 "package-3:lib"
1954 ; ...
1955 "package-N"))
b1eecb5c 1956@end lisp
4c463569 1957
b1eecb5c 1958@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual}, for
4c463569
PN
1959the syntax details.
1960
1961We can create a manifest specification per profile and install them this way:
1962
1963@example
1964GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
1965mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
1966guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
1967@end example
1968
1969Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
1970where we will store our profiles in the rest of this article.
1971
1972Placing all your profiles in a single directory, with each profile getting its
1973own sub-directory, is somewhat cleaner. This way, each sub-directory will
128b136e
NG
1974contain all the symlinks for precisely one profile. Besides, ``looping over
1975profiles'' becomes obvious from any programming language (e.g.@: a shell script) by
4c463569
PN
1976simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
1977
1978Note that it's also possible to loop over the output of
1979
1980@example
1981guix package --list-profiles
1982@end example
1983
2c8305d1 1984although you'll probably have to filter out @file{~/.config/guix/current}.
4c463569 1985
2c8305d1 1986To enable all profiles on login, add this to your @file{~/.bash_profile} (or similar):
4c463569
PN
1987
1988@example
1989for i in $GUIX_EXTRA_PROFILES/*; do
1990 profile=$i/$(basename "$i")
1991 if [ -f "$profile"/etc/profile ]; then
1992 GUIX_PROFILE="$profile"
1993 . "$GUIX_PROFILE"/etc/profile
1994 fi
1995 unset profile
1996done
1997@end example
1998
1999Note to Guix System users: the above reflects how your default profile
2c8305d1
NG
2000@file{~/.guix-profile} is activated from @file{/etc/profile}, that latter being loaded by
2001@file{~/.bashrc} by default.
4c463569
PN
2002
2003You can obviously choose to only enable a subset of them:
2004
2005@example
2006for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
2007 profile=$i/$(basename "$i")
2008 if [ -f "$profile"/etc/profile ]; then
2009 GUIX_PROFILE="$profile"
2010 . "$GUIX_PROFILE"/etc/profile
2011 fi
2012 unset profile
2013done
2014@end example
2015
2016When a profile is off, it's straightforward to enable it for an individual shell
2017without "polluting" the rest of the user session:
2018
2019@example
2020GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
2021@end example
2022
2023The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. This file
2024contains shell code that exports the right environment variables necessary to
2025activate the software contained in the profile. It is built automatically by
2026Guix and meant to be sourced.
2027It contains the same variables you would get if you ran:
2028
2029@example
2030guix package --search-paths=prefix --profile=$my_profile"
2031@end example
2032
2033Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual})
2034for the command line options.
2035
2036To upgrade a profile, simply install the manifest again:
2037
2038@example
2039guix package -m /path/to/guix-my-project-manifest.scm -p "$GUIX_EXTRA_PROFILES"/my-project/my-project
2040@end example
2041
2042To upgrade all profiles, it's easy enough to loop over them. For instance,
2043assuming your manifest specifications are stored in
2c8305d1 2044@file{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
cc765e47 2045of the profile (e.g.@: "project1"), you could do the following in Bourne shell:
4c463569
PN
2046
2047@example
2048for profile in "$GUIX_EXTRA_PROFILES"/*; do
2049 guix package --profile="$profile" --manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
2050done
2051@end example
2052
2053Each profile has its own generations:
2054
2055@example
2056guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
2057@end example
2058
2059You can roll-back to any generation of a given profile:
2060
2061@example
2062guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
2063@end example
cb7b501d
PN
2064
2065Finally, if you want to switch to a profile without inheriting from the
2066current environment, you can activate it from an empty shell:
2067
2068@example
2069env -i $(which bash) --login --noprofile --norc
2070. my-project/etc/profile
2071@end example
4c463569
PN
2072
2073@node Required packages
2074@subsection Required packages
2075
2076Activating a profile essentially boils down to exporting a bunch of
2077environmental variables. This is the role of the @samp{etc/profile} within the
2078profile.
2079
2080@emph{Note: Only the environmental variables of the packages that consume them will
2081be set.}
2082
2083For instance, @samp{MANPATH} won't be set if there is no consumer application for man
2084pages within the profile. So if you need to transparently access man pages once
2085the profile is loaded, you've got two options:
2086
2087@itemize
2088@item
2089Either export the variable manually, e.g.
2090@example
f6c27c55 2091export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH
4c463569
PN
2092@end example
2093
2094@item
2095Or include @samp{man-db} to the profile manifest.
2096@end itemize
2097
2098The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
2099@samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
2100
2101@node Default profile
2102@subsection Default profile
2103
2c8305d1 2104What about the default profile that Guix keeps in @file{~/.guix-profile}?
4c463569
PN
2105
2106You can assign it the role you want. Typically you would install the manifest
2107of the packages you want to use all the time.
2108
128b136e 2109Alternatively, you could keep it ``manifest-less'' for throw-away packages
4c463569
PN
2110that you would just use for a couple of days.
2111This way makes it convenient to run
2112
2113@example
2114guix install package-foo
2115guix upgrade package-bar
2116@end example
2117
2118without having to specify the path to a profile.
2119
2120@node The benefits of manifests
2121@subsection The benefits of manifests
2122
2123Manifests are a convenient way to keep your package lists around and, say,
2124to synchronize them across multiple machines using a version control system.
2125
2126A common complaint about manifests is that they can be slow to install when they
2127contain large number of packages. This is especially cumbersome when you just
2128want get an upgrade for one package within a big manifest.
2129
2130This is one more reason to use multiple profiles, which happen to be just
2131perfect to break down manifests into multiple sets of semantically connected
2132packages. Using multiple, small profiles provides more flexibility and
2133usability.
2134
2135Manifests come with multiple benefits. In particular, they ease maintenance:
2136
2137@itemize
2138@item
2139When a profile is set up from a manifest, the manifest itself is
128b136e 2140self-sufficient to keep a ``package listing'' around and reinstall the profile
4c463569
PN
2141later or on a different system. For ad-hoc profiles, we would need to
2142generate a manifest specification manually and maintain the package versions
2143for the packages that don't use the default version.
2144
2145@item
2146@code{guix package --upgrade} always tries to update the packages that have
2147propagated inputs, even if there is nothing to do. Guix manifests remove this
2148problem.
2149
2150@item
2151When partially upgrading a profile, conflicts may arise (due to diverging
2152dependencies between the updated and the non-updated packages) and they can be
2153annoying to resolve manually. Manifests remove this problem altogether since
2154all packages are always upgraded at once.
2155
2156@item
2157As mentioned above, manifests allow for reproducible profiles, while the
2158imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
2159different profiles every time even when they hold the same packages. See
2160@uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
2161
2162@item
2163Manifest specifications are usable by other @samp{guix} commands. For example, you
2164can run @code{guix weather -m manifest.scm} to see how many substitutes are
2165available, which can help you decide whether you want to try upgrading today
2166or wait a while. Another example: you can run @code{guix pack -m manifest.scm} to
2167create a pack containing all the packages in the manifest (and their
2168transitive references).
2169
2170@item
2171Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
2172They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
2173@end itemize
2174
2175It's important to understand that while manifests can be used to declare
2176profiles, they are not strictly equivalent: profiles have the side effect that
128b136e 2177they ``pin'' packages in the store, which prevents them from being
4c463569
PN
2178garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
2179and ensures that they will still be available at any point in
2180the future.
2181
2182Let's take an example:
2183
2184@enumerate
2185@item
2186We have an environment for hacking on a project for which there isn't a Guix
2187package yet. We build the environment using a manifest, and then run @code{guix
2188 environment -m manifest.scm}. So far so good.
2189
2190@item
2191Many weeks pass and we have run a couple of @code{guix pull} in the mean time.
2192Maybe a dependency from our manifest has been updated; or we may have run
2193@code{guix gc} and some packages needed by our manifest have been
2194garbage-collected.
2195
2196@item
2197Eventually, we set to work on that project again, so we run @code{guix environment
2198 -m manifest.scm}. But now we have to wait for Guix to build and install
2199stuff!
2200@end enumerate
2201
2202Ideally, we could spare the rebuild time. And indeed we can, all we need is to
2203install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
2204. "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
2205hacking environment will be available at all times.
2206
2207@emph{Security warning:} While keeping old profiles around can be convenient, keep in
2208mind that outdated packages may not have received the latest security fixes.
2209
2210@node Reproducible profiles
2211@subsection Reproducible profiles
2212
2213To reproduce a profile bit-for-bit, we need two pieces of information:
2214
2215@itemize
2216@item
2217a manifest,
2218@item
2219a Guix channel specification.
2220@end itemize
2221
2222Indeed, manifests alone might not be enough: different Guix versions (or
2223different channels) can produce different outputs for a given manifest.
2224
2225You can output the Guix channel specification with @samp{guix describe
2226--format=channels}.
2227Save this to a file, say @samp{channel-specs.scm}.
2228
2229On another computer, you can use the channel specification file and the manifest
2230to reproduce the exact same profile:
2231
2232@example
2233GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
2234GUIX_EXTRA=$HOME/.guix-extra
2235
2236mkdir "$GUIX_EXTRA"/my-project
2237guix pull --channels=channel-specs.scm --profile "$GUIX_EXTRA/my-project/guix"
2238
2239mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
2240"$GUIX_EXTRA"/my-project/guix/bin/guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
2241@end example
2242
2243It's safe to delete the Guix channel profile you've just installed with the
2244channel specification, the project profile does not depend on it.
2245
7bc46ecc
RW
2246@c *********************************************************************
2247@node Acknowledgments
2248@chapter Acknowledgments
2249
2250Guix is based on the @uref{https://nixos.org/nix/, Nix package manager},
2251which was designed and
2252implemented by Eelco Dolstra, with contributions from other people (see
2253the @file{nix/AUTHORS} file in Guix.) Nix pioneered functional package
2254management, and promoted unprecedented features, such as transactional
2255package upgrades and rollbacks, per-user profiles, and referentially
2256transparent build processes. Without this work, Guix would not exist.
2257
2258The Nix-based software distributions, Nixpkgs and NixOS, have also been
2259an inspiration for Guix.
2260
2261GNU@tie{}Guix itself is a collective work with contributions from a
2262number of people. See the @file{AUTHORS} file in Guix for more
2263information on these fine people. The @file{THANKS} file lists people
2264who have helped by reporting bugs, taking care of the infrastructure,
2265providing artwork and themes, making suggestions, and more---thank you!
2266
2267This document includes adapted sections from articles that have previously
2268been published on the Guix blog at @uref{https://guix.gnu.org/blog}.
2269
2270
2271@c *********************************************************************
2272@node GNU Free Documentation License
2273@appendix GNU Free Documentation License
2274@cindex license, GNU Free Documentation License
2275@include fdl-1.3.texi
2276
2277@c *********************************************************************
2278@node Concept Index
2279@unnumbered Concept Index
2280@printindex cp
2281
2282@bye
2283
2284@c Local Variables:
2285@c ispell-local-dictionary: "american";
2286@c End: