Merge branch 'master' into core-updates
[jackhill/guix/guix.git] / doc / contributing.texi
1 @node Contributing
2 @chapter Contributing
3
4 This project is a cooperative effort, and we need your help to make it
5 grow! Please get in touch with us on @email{guix-devel@@gnu.org} and
6 @code{#guix} on the Freenode IRC network. We welcome ideas, bug
7 reports, patches, and anything that may be helpful to the project. We
8 particularly welcome help on packaging (@pxref{Packaging Guidelines}).
9
10 @cindex code of conduct, of contributors
11 @cindex contributor covenant
12 We want to provide a warm, friendly, and harassment-free environment, so
13 that anyone can contribute to the best of their abilities. To this end
14 our project uses a ``Contributor Covenant'', which was adapted from
15 @url{http://contributor-covenant.org/}. You can find a local version in
16 the @file{CODE-OF-CONDUCT} file in the source tree.
17
18 Contributors are not required to use their legal name in patches and
19 on-line communication; they can use any name or pseudonym of their
20 choice.
21
22 @menu
23 * Building from Git:: The latest and greatest.
24 * Running Guix Before It Is Installed:: Hacker tricks.
25 * The Perfect Setup:: The right tools.
26 * Coding Style:: Hygiene of the contributor.
27 * Submitting Patches:: Share your work.
28 @end menu
29
30 @node Building from Git
31 @section Building from Git
32
33 If you want to hack Guix itself, it is recommended to use the latest
34 version from the Git repository:
35
36 @example
37 git clone https://git.savannah.gnu.org/git/guix.git
38 @end example
39
40 When building Guix from a checkout,
41 the following packages are required in addition to those mentioned in
42 the installation instructions (@pxref{Requirements}).
43
44 @itemize
45 @item @url{http://gnu.org/software/autoconf/, GNU Autoconf};
46 @item @url{http://gnu.org/software/automake/, GNU Automake};
47 @item @url{http://gnu.org/software/gettext/, GNU Gettext};
48 @item @url{http://gnu.org/software/texinfo/, GNU Texinfo};
49 @item @url{http://www.graphviz.org/, Graphviz};
50 @item @url{http://www.gnu.org/software/help2man/, GNU Help2man (optional)}.
51 @end itemize
52
53 The easiest way to set up a development environment for Guix is, of
54 course, by using Guix! The following command starts a new shell where
55 all the dependencies and appropriate environment variables are set up to
56 hack on Guix:
57
58 @example
59 guix environment guix
60 @end example
61
62 @xref{Invoking guix environment}, for more information on that command.
63 Extra dependencies can be added with @option{--ad-hoc}:
64
65 @example
66 guix environment guix --ad-hoc help2man git strace
67 @end example
68
69 Run @command{./bootstrap} to generate the build system infrastructure
70 using Autoconf and Automake. If you get an error like this one:
71
72 @example
73 configure.ac:46: error: possibly undefined macro: PKG_CHECK_MODULES
74 @end example
75
76 @noindent
77 it probably means that Autoconf couldn’t find @file{pkg.m4}, which is
78 provided by pkg-config. Make sure that @file{pkg.m4} is available. The
79 same holds for the @file{guile.m4} set of macros provided by Guile. For
80 instance, if you installed Automake in @file{/usr/local}, it wouldn’t
81 look for @file{.m4} files in @file{/usr/share}. In that case, you have
82 to invoke the following command:
83
84 @example
85 export ACLOCAL_PATH=/usr/share/aclocal
86 @end example
87
88 @xref{Macro Search Path,,, automake, The GNU Automake Manual}, for
89 more information.
90
91 Then, run @command{./configure} as usual. Make sure to pass
92 @code{--localstatedir=@var{directory}} where @var{directory} is the
93 @code{localstatedir} value used by your current installation (@pxref{The
94 Store}, for information about this).
95
96 Finally, you have to invoke @code{make check} to run tests
97 (@pxref{Running the Test Suite}). If anything
98 fails, take a look at installation instructions (@pxref{Installation})
99 or send a message to the @email{guix-devel@@gnu.org, mailing list}.
100
101
102 @node Running Guix Before It Is Installed
103 @section Running Guix Before It Is Installed
104
105 In order to keep a sane working environment, you will find it useful to
106 test the changes made in your local source tree checkout without
107 actually installing them. So that you can distinguish between your
108 ``end-user'' hat and your ``motley'' costume.
109
110 To that end, all the command-line tools can be used even if you have not
111 run @code{make install}. To do that, prefix each command with
112 @command{./pre-inst-env} (the @file{pre-inst-env} script lives in the
113 top build tree of Guix), as in@footnote{The @option{-E} flag to
114 @command{sudo} guarantees that @code{GUILE_LOAD_PATH} is correctly set
115 such that @command{guix-daemon} and the tools it uses can find the Guile
116 modules they need.}:
117
118 @example
119 $ sudo -E ./pre-inst-env guix-daemon --build-users-group=guixbuild
120 $ ./pre-inst-env guix build hello
121 @end example
122
123 @noindent
124 Similarly, for a Guile session using the Guix modules:
125
126 @example
127 $ ./pre-inst-env guile -c '(use-modules (guix utils)) (pk (%current-system))'
128
129 ;;; ("x86_64-linux")
130 @end example
131
132 @noindent
133 @cindex REPL
134 @cindex read-eval-print loop
135 @dots{} and for a REPL (@pxref{Using Guile Interactively,,, guile, Guile
136 Reference Manual}):
137
138 @example
139 $ ./pre-inst-env guile
140 scheme@@(guile-user)> ,use(guix)
141 scheme@@(guile-user)> ,use(gnu)
142 scheme@@(guile-user)> (define snakes
143 (fold-packages
144 (lambda (package lst)
145 (if (string-prefix? "python"
146 (package-name package))
147 (cons package lst)
148 lst))
149 '()))
150 scheme@@(guile-user)> (length snakes)
151 $1 = 361
152 @end example
153
154 The @command{pre-inst-env} script sets up all the environment variables
155 necessary to support this, including @env{PATH} and @env{GUILE_LOAD_PATH}.
156
157 Note that @command{./pre-inst-env guix pull} does @emph{not} upgrade the
158 local source tree; it simply updates the @file{~/.config/guix/current}
159 symlink (@pxref{Invoking guix pull}). Run @command{git pull} instead if
160 you want to upgrade your local source tree.
161
162
163 @node The Perfect Setup
164 @section The Perfect Setup
165
166 The Perfect Setup to hack on Guix is basically the perfect setup used
167 for Guile hacking (@pxref{Using Guile in Emacs,,, guile, Guile Reference
168 Manual}). First, you need more than an editor, you need
169 @url{http://www.gnu.org/software/emacs, Emacs}, empowered by the
170 wonderful @url{http://nongnu.org/geiser/, Geiser}.
171
172 Geiser allows for interactive and incremental development from within
173 Emacs: code compilation and evaluation from within buffers, access to
174 on-line documentation (docstrings), context-sensitive completion,
175 @kbd{M-.} to jump to an object definition, a REPL to try out your code,
176 and more (@pxref{Introduction,,, geiser, Geiser User Manual}). For
177 convenient Guix development, make sure to augment Guile’s load path so
178 that it finds source files from your checkout:
179
180 @lisp
181 ;; @r{Assuming the Guix checkout is in ~/src/guix.}
182 (with-eval-after-load 'geiser-guile
183 (add-to-list 'geiser-guile-load-path "~/src/guix"))
184 @end lisp
185
186 To actually edit the code, Emacs already has a neat Scheme mode. But in
187 addition to that, you must not miss
188 @url{http://www.emacswiki.org/emacs/ParEdit, Paredit}. It provides
189 facilities to directly operate on the syntax tree, such as raising an
190 s-expression or wrapping it, swallowing or rejecting the following
191 s-expression, etc.
192
193 @cindex code snippets
194 @cindex templates
195 @cindex reducing boilerplate
196 We also provide templates for common git commit messages and package
197 definitions in the @file{etc/snippets} directory. These templates can
198 be used with @url{http://joaotavora.github.io/yasnippet/, YASnippet} to
199 expand short trigger strings to interactive text snippets. You may want
200 to add the snippets directory to the @var{yas-snippet-dirs} variable in
201 Emacs.
202
203 @lisp
204 ;; @r{Assuming the Guix checkout is in ~/src/guix.}
205 (with-eval-after-load 'yasnippet
206 (add-to-list 'yas-snippet-dirs "~/src/guix/etc/snippets"))
207 @end lisp
208
209 The commit message snippets depend on @url{https://magit.vc/, Magit} to
210 display staged files. When editing a commit message type @code{add}
211 followed by @kbd{TAB} to insert a commit message template for adding a
212 package; type @code{update} followed by @kbd{TAB} to insert a template
213 for updating a package.
214
215 The main snippet for @code{scheme-mode} is triggered by typing
216 @code{package...} followed by @kbd{TAB}. This snippet also inserts the
217 trigger string @code{origin...}, which can be expanded further. The
218 @code{origin} snippet in turn may insert other trigger strings ending on
219 @code{...}, which also can be expanded further.
220
221
222 @node Coding Style
223 @section Coding Style
224
225 In general our code follows the GNU Coding Standards (@pxref{Top,,,
226 standards, GNU Coding Standards}). However, they do not say much about
227 Scheme, so here are some additional rules.
228
229 @menu
230 * Programming Paradigm:: How to compose your elements.
231 * Modules:: Where to store your code?
232 * Data Types and Pattern Matching:: Implementing data structures.
233 * Formatting Code:: Writing conventions.
234 @end menu
235
236 @node Programming Paradigm
237 @subsection Programming Paradigm
238
239 Scheme code in Guix is written in a purely functional style. One
240 exception is code that involves input/output, and procedures that
241 implement low-level concepts, such as the @code{memoize} procedure.
242
243 @node Modules
244 @subsection Modules
245
246 Guile modules that are meant to be used on the builder side must live in
247 the @code{(guix build @dots{})} name space. They must not refer to
248 other Guix or GNU modules. However, it is OK for a ``host-side'' module
249 to use a build-side module.
250
251 Modules that deal with the broader GNU system should be in the
252 @code{(gnu @dots{})} name space rather than @code{(guix @dots{})}.
253
254 @node Data Types and Pattern Matching
255 @subsection Data Types and Pattern Matching
256
257 The tendency in classical Lisp is to use lists to represent everything,
258 and then to browse them ``by hand'' using @code{car}, @code{cdr},
259 @code{cadr}, and co. There are several problems with that style,
260 notably the fact that it is hard to read, error-prone, and a hindrance
261 to proper type error reports.
262
263 Guix code should define appropriate data types (for instance, using
264 @code{define-record-type*}) rather than abuse lists. In addition, it
265 should use pattern matching, via Guile’s @code{(ice-9 match)} module,
266 especially when matching lists.
267
268 @node Formatting Code
269 @subsection Formatting Code
270
271 @cindex formatting code
272 @cindex coding style
273 When writing Scheme code, we follow common wisdom among Scheme
274 programmers. In general, we follow the
275 @url{http://mumble.net/~campbell/scheme/style.txt, Riastradh's Lisp
276 Style Rules}. This document happens to describe the conventions mostly
277 used in Guile’s code too. It is very thoughtful and well written, so
278 please do read it.
279
280 Some special forms introduced in Guix, such as the @code{substitute*}
281 macro, have special indentation rules. These are defined in the
282 @file{.dir-locals.el} file, which Emacs automatically uses. Also note
283 that Emacs-Guix provides @code{guix-devel-mode} mode that indents and
284 highlights Guix code properly (@pxref{Development,,, emacs-guix, The
285 Emacs-Guix Reference Manual}).
286
287 @cindex indentation, of code
288 @cindex formatting, of code
289 If you do not use Emacs, please make sure to let your editor knows these
290 rules. To automatically indent a package definition, you can also run:
291
292 @example
293 ./etc/indent-code.el gnu/packages/@var{file}.scm @var{package}
294 @end example
295
296 @noindent
297 This automatically indents the definition of @var{package} in
298 @file{gnu/packages/@var{file}.scm} by running Emacs in batch mode. To
299 indent a whole file, omit the second argument:
300
301 @example
302 ./etc/indent-code.el gnu/services/@var{file}.scm
303 @end example
304
305 @cindex Vim, Scheme code editing
306 If you are editing code with Vim, we recommend that you run @code{:set
307 autoindent} so that your code is automatically indented as you type.
308 Additionally,
309 @uref{https://www.vim.org/scripts/script.php?script_id=3998,
310 @code{paredit.vim}} may help you deal with all these parentheses.
311
312 We require all top-level procedures to carry a docstring. This
313 requirement can be relaxed for simple private procedures in the
314 @code{(guix build @dots{})} name space, though.
315
316 Procedures should not have more than four positional parameters. Use
317 keyword parameters for procedures that take more than four parameters.
318
319
320 @node Submitting Patches
321 @section Submitting Patches
322
323 Development is done using the Git distributed version control system.
324 Thus, access to the repository is not strictly necessary. We welcome
325 contributions in the form of patches as produced by @code{git
326 format-patch} sent to the @email{guix-patches@@gnu.org} mailing list.
327
328 This mailing list is backed by a Debbugs instance accessible at
329 @uref{https://bugs.gnu.org/guix-patches}, which allows us to keep track
330 of submissions. Each message sent to that mailing list gets a new
331 tracking number assigned; people can then follow up on the submission by
332 sending email to @code{@var{NNN}@@debbugs.gnu.org}, where @var{NNN} is
333 the tracking number (@pxref{Sending a Patch Series}).
334
335 Please write commit logs in the ChangeLog format (@pxref{Change Logs,,,
336 standards, GNU Coding Standards}); you can check the commit history for
337 examples.
338
339 Before submitting a patch that adds or modifies a package definition,
340 please run through this check list:
341
342 @enumerate
343 @item
344 If the authors of the packaged software provide a cryptographic
345 signature for the release tarball, make an effort to verify the
346 authenticity of the archive. For a detached GPG signature file this
347 would be done with the @code{gpg --verify} command.
348
349 @item
350 Take some time to provide an adequate synopsis and description for the
351 package. @xref{Synopses and Descriptions}, for some guidelines.
352
353 @item
354 Run @code{guix lint @var{package}}, where @var{package} is the
355 name of the new or modified package, and fix any errors it reports
356 (@pxref{Invoking guix lint}).
357
358 @item
359 Make sure the package builds on your platform, using @code{guix build
360 @var{package}}.
361
362 @item
363 @cindex bundling
364 Make sure the package does not use bundled copies of software already
365 available as separate packages.
366
367 Sometimes, packages include copies of the source code of their
368 dependencies as a convenience for users. However, as a distribution, we
369 want to make sure that such packages end up using the copy we already
370 have in the distribution, if there is one. This improves resource usage
371 (the dependency is built and stored only once), and allows the
372 distribution to make transverse changes such as applying security
373 updates for a given software package in a single place and have them
374 affect the whole system---something that bundled copies prevent.
375
376 @item
377 Take a look at the profile reported by @command{guix size}
378 (@pxref{Invoking guix size}). This will allow you to notice references
379 to other packages unwillingly retained. It may also help determine
380 whether to split the package (@pxref{Packages with Multiple Outputs}),
381 and which optional dependencies should be used.
382
383 @item
384 For important changes, check that dependent package (if applicable) are
385 not affected by the change; @code{guix refresh --list-dependent
386 @var{package}} will help you do that (@pxref{Invoking guix refresh}).
387
388 @c See <https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00933.html>.
389 @cindex branching strategy
390 @cindex rebuild scheduling strategy
391 Depending on the number of dependent packages and thus the amount of
392 rebuilding induced, commits go to different branches, along these lines:
393
394 @table @asis
395 @item 300 dependent packages or less
396 @code{master} branch (non-disruptive changes).
397
398 @item between 300 and 1,200 dependent packages
399 @code{staging} branch (non-disruptive changes). This branch is intended
400 to be merged in @code{master} every 3 weeks or so. Topical changes
401 (e.g., an update of the GNOME stack) can instead go to a specific branch
402 (say, @code{gnome-updates}).
403
404 @item more than 1,200 dependent packages
405 @code{core-updates} branch (may include major and potentially disruptive
406 changes). This branch is intended to be merged in @code{master} every
407 2.5 months or so.
408 @end table
409
410 All these branches are @uref{https://hydra.gnu.org/project/gnu,
411 tracked by our build farm} and merged into @code{master} once
412 everything has been successfully built. This allows us to fix issues
413 before they hit users, and to reduce the window during which pre-built
414 binaries are not available.
415
416 Generally, branches other than @code{master} are considered
417 @emph{frozen} if there has been a recent evaluation, or there is a
418 corresponding @code{-next} branch. Please ask on the mailing list or
419 IRC if unsure where to place a patch.
420 @c TODO: It would be good with badges on the website that tracks these
421 @c branches. Or maybe even a status page.
422
423 @item
424 @cindex determinism, of build processes
425 @cindex reproducible builds, checking
426 Check whether the package's build process is deterministic. This
427 typically means checking whether an independent build of the package
428 yields the exact same result that you obtained, bit for bit.
429
430 A simple way to do that is by building the same package several times in
431 a row on your machine (@pxref{Invoking guix build}):
432
433 @example
434 guix build --rounds=2 my-package
435 @end example
436
437 This is enough to catch a class of common non-determinism issues, such
438 as timestamps or randomly-generated output in the build result.
439
440 Another option is to use @command{guix challenge} (@pxref{Invoking guix
441 challenge}). You may run it once the package has been committed and
442 built by @code{hydra.gnu.org} to check whether it obtains the same
443 result as you did. Better yet: Find another machine that can build it
444 and run @command{guix publish}. Since the remote build machine is
445 likely different from yours, this can catch non-determinism issues
446 related to the hardware---e.g., use of different instruction set
447 extensions---or to the operating system kernel---e.g., reliance on
448 @code{uname} or @file{/proc} files.
449
450 @item
451 When writing documentation, please use gender-neutral wording when
452 referring to people, such as
453 @uref{https://en.wikipedia.org/wiki/Singular_they, singular
454 ``they''@comma{} ``their''@comma{} ``them''}, and so forth.
455
456 @item
457 Verify that your patch contains only one set of related changes.
458 Bundling unrelated changes together makes reviewing harder and slower.
459
460 Examples of unrelated changes include the addition of several packages,
461 or a package update along with fixes to that package.
462
463 @item
464 Please follow our code formatting rules, possibly running the
465 @command{etc/indent-code.el} script to do that automatically for you
466 (@pxref{Formatting Code}).
467
468 @end enumerate
469
470 When posting a patch to the mailing list, use @samp{[PATCH] @dots{}} as
471 a subject. You may use your email client or the @command{git
472 send-email} command (@pxref{Sending a Patch Series}). We prefer to get
473 patches in plain text messages, either inline or as MIME attachments.
474 You are advised to pay attention if your email client changes anything
475 like line breaks or indentation which could potentially break the
476 patches.
477
478 When a bug is resolved, please close the thread by sending an email to
479 @email{@var{NNN}-done@@debbugs.gnu.org}.
480
481 @unnumberedsubsec Sending a Patch Series
482 @anchor{Sending a Patch Series}
483 @cindex patch series
484 @cindex @code{git send-email}
485 @cindex @code{git-send-email}
486
487 When sending a patch series (e.g., using @code{git send-email}), please
488 first send one message to @email{guix-patches@@gnu.org}, and then send
489 subsequent patches to @email{@var{NNN}@@debbugs.gnu.org} to make sure
490 they are kept together. See
491 @uref{https://debbugs.gnu.org/Advanced.html, the Debbugs documentation}
492 for more information.
493 @c Debbugs bug: https://debbugs.gnu.org/db/15/15361.html