Commit | Line | Data |
---|---|---|
64b9637c PJ |
1 | Here are the guidelines for being an Emacs pretester. |
2 | If you would like to do this, say so, and I'll add you to | |
3 | the pretest list. | |
4 | ||
5 | ||
6 | Information for Emacs Pretesters | |
7 | ||
8 | The purpose of Emacs pretesting is to verify that the new Emacs | |
9 | distribution, about to be released, works properly on your system *with | |
10 | no change whatever*, when installed following the precise | |
11 | recommendations that come with the Emacs distribution. | |
12 | ||
13 | Here are some guidelines on how to do pretesting so as to make it | |
14 | helpful. All of them follow from common sense together with the | |
15 | nature of the purpose and the situation. | |
16 | ||
17 | Please save this file, and reread it when a new series of pretests | |
18 | starts. | |
19 | ||
edc54035 TTN |
20 | * Get the pretest from gnu/emacs/pretest/emacs-MM.0.NN.tar.gz |
21 | on alpha.gnu.org. | |
64b9637c PJ |
22 | |
23 | * After a few days of testing, if there are no problems, please report | |
24 | that Emacs works for you and what configuration you are testing it on. | |
25 | ||
26 | * If you want to communicate with other pretesters, send mail to | |
27 | emacs-pretesters@gnu.org. I don't use that mailing list when I send | |
28 | to you because I've found that mailing lists tend to amplify random | |
29 | noise into long discussions or even arguments, and that can waste a | |
30 | lot of time. But when you have a reason to ask other pretesters for | |
31 | help, you can do it that way. | |
32 | ||
6f08f980 RS |
33 | * It is absolutely vital that you report even the smallest change or |
34 | departure from the standard sources and procedure. | |
64b9637c | 35 | |
6f08f980 | 36 | Otherwise, you are not testing the same program that we asked you to |
64b9637c | 37 | test. Testing a different program is usually of no use whatever. It |
6f08f980 RS |
38 | can even cause trouble, if you fail to tell us that you tested some |
39 | other program instead of what we are about to release. We might think | |
64b9637c PJ |
40 | that Emacs works, when in fact it has not even been tried, and might |
41 | have a glaring fault. | |
42 | ||
43 | * Don't use a site-load.el file or a site-init.el file when you pretest. | |
44 | Using either of those files means you are not testing Emacs as a typical | |
45 | site would use it. | |
46 | ||
47 | Actually, it does no harm to test Emacs with such customizations *as | |
48 | well as* testing it "out of the box". Anything you do that could find | |
6f08f980 RS |
49 | a bug is useful, as long as you make sure we know exactly what you |
50 | did. The important point is that testing with local changes is no | |
64b9637c PJ |
51 | substitute for testing Emacs exactly as it is distributed. |
52 | ||
53 | * Even changing the compilation options counts as a change in the | |
54 | program. The Emacs sources specify which compilation options to use. | |
55 | Some of them are specified in makefiles, and some in machine-specific | |
56 | configuration files. They also give you ways to override this--but if | |
57 | you do, then you are not testing what ordinary users will do. | |
58 | Therefore, when pretesting, it is vital to test with the default | |
59 | compilation options. | |
60 | ||
61 | (Testing with a different set of options can be useful *in addition*, | |
62 | but not *instead of* the default options.) | |
63 | ||
64 | * The machine and system configuration files of Emacs are parts of | |
65 | Emacs. So when you test Emacs, you need to do it with the | |
66 | configuration files that come with Emacs. | |
67 | ||
68 | If Emacs does not come with configuration files for a certain machine, | |
69 | and you test it with configuration files that don't come with Emacs, | |
70 | this is effectively changing Emacs. Because the crucial fact about | |
71 | the planned release is that, without changes, it doesn't work on that | |
72 | machine. | |
73 | ||
6f08f980 | 74 | To make Emacs work on that machine, we would need to install new |
64b9637c PJ |
75 | configuration files. That is not out of the question, since it is |
76 | safe--it certainly won't break any other machines that already work. | |
6f08f980 | 77 | But you will have to rush in the legal papers to give the FSF |
64b9637c PJ |
78 | permission to use such a large piece of text. |
79 | ||
80 | * Look in the etc/MACHINES file. | |
81 | ||
82 | The etc/MACHINES file says which configuration files to use for your | |
83 | machine, so use the ones that are recommended. If you guess, you might | |
84 | guess wrong and encounter spurious difficulties. What's more, if you | |
85 | don't follow etc/MACHINES then you aren't helping to test that its | |
86 | recommendations are valid. | |
87 | ||
88 | The etc/MACHINES file may describe other things that you need to do | |
89 | to make Emacs work on your machine. If so, you should follow these | |
90 | recommendations also, for the same reason. | |
91 | ||
92 | * Send your problem reports to emacs-pretest-bug@gnu.org, not | |
93 | bug-gnu-emacs. | |
94 | ||
6f08f980 RS |
95 | Sometimes we won't know what to do about a system-dependent issue, and |
96 | we may need people to say what happens if you try a certain thing on a | |
97 | certain system. When this happens, we'll send out a query. | |
64b9637c PJ |
98 | |
99 | * Don't delay sending information. | |
100 | ||
6f08f980 RS |
101 | When you test on a system and encounter no problems, please report it |
102 | right away. That way, we will know that someone has tested Emacs on | |
103 | that kind of system. | |
64b9637c PJ |
104 | |
105 | Please don't wait for several days "to see if it really works before | |
6f08f980 RS |
106 | you say anything." Tell us right away that Emacs seems basically to |
107 | work; then, if you notice a problem a few days later, tell us | |
64b9637c PJ |
108 | immediately about that when you see it. |
109 | ||
110 | It is okay if you double check things before reporting a problem, such | |
111 | as to see if you can easily fix it. But don't wait very long. A good | |
6f08f980 RS |
112 | rule to use in pretesting is always to report every problem on the |
113 | same day you encounter it, even if that means you can't find a | |
64b9637c PJ |
114 | solution before you report the problem. |
115 | ||
116 | I'd much rather hear about a problem today and a solution tomorrow | |
117 | than get both of them tomorrow at the same time. | |
118 | ||
119 | * Make each bug report self-contained. | |
120 | ||
121 | If you refer back to another message, whether from you or from someone | |
122 | else, then it will be necessary for anyone who wants to investigate | |
123 | the bug to find the other message. This may be difficult, it is | |
124 | probably time-consuming. | |
125 | ||
6f08f980 | 126 | To help save our time, simply copy the relevant parts of any previous |
64b9637c PJ |
127 | messages into your own bug report. |
128 | ||
6f08f980 | 129 | In particular, if we ask you for more information because a bug report |
64b9637c PJ |
130 | was incomplete, it is best to send me the *entire* collection of |
131 | relevant information, all together. If you send just the additional | |
6f08f980 RS |
132 | information, that makes extra work for us. There is even a risk that |
133 | we won't remember what question you are sending the answer to. | |
64b9637c PJ |
134 | |
135 | * When you encounter a bug that manifests itself as a Lisp error, | |
136 | try setting debug-on-error to t and making the bug happen again. | |
137 | Then you will get a Lisp backtrace. Including that in your bug report | |
138 | is very useful. | |
139 | ||
6f08f980 RS |
140 | * For advice on debugging, see etc/DEBUG. |
141 | ||
64b9637c PJ |
142 | * Debugging optimized code is possible, if you compile with GCC, but |
143 | in some cases the optimized code can be confusing. If you are not | |
144 | accustomed to that, recompile Emacs without -O. One way to do this is | |
145 | ||
146 | make clean | |
147 | make CFLAGS=-g | |
148 | ||
64b9637c PJ |
149 | * Configure tries to figure out what kind of system you have by |
150 | compiling and linking programs which calls various functions and looks | |
151 | at whether that succeeds. The file config.log contains any messages | |
152 | produced by compilers while running configure, to aid debugging if | |
153 | configure makes a mistake. But note that config.cache reads: | |
154 | ||
155 | # Giving --cache-file=/dev/null disables caching, for debugging configure. | |
156 | ||
177c0ea7 | 157 | or more simply, |
64b9637c PJ |
158 | |
159 | rm config.cache | |
160 | ./configure | |
161 | ||
6f08f980 RS |
162 | * Don't try changing Emacs *in any way* during pretest unless it fails |
163 | to work unchanged. | |
164 | ||
64b9637c PJ |
165 | * Always be precise when talking about changes you have made. Show |
166 | things rather than describing them. Use exact filenames (relative to | |
167 | the main directory of the distribution), not partial ones. For | |
168 | example, say "I changed Makefile" rather than "I changed the | |
169 | makefile". Instead of saying "I defined the MUMBLE macro", send a | |
170 | diff. | |
171 | ||
172 | * Always use `diff -c' to make diffs. If you don't include context, it | |
6f08f980 RS |
173 | may be hard for us to figure out where you propose to make the |
174 | changes. So we might ignore your patch. | |
64b9637c | 175 | |
6f08f980 | 176 | * When you write a fix, keep in mind that we can't install a change |
64b9637c PJ |
177 | that *might* break other systems without the risk that it will fail to |
178 | work and therefore require an additional cycle of pretesting. | |
179 | ||
180 | People often suggest fixing a problem by changing config.h or | |
181 | src/ymakefile or even src/Makefile to do something special that a | |
182 | particular system needs. Sometimes it is totally obvious that such | |
6f08f980 RS |
183 | changes would break Emacs for almost all users. We can't possibly |
184 | make a change like that. All we can do is ask you to find a fix that | |
185 | is safe to install. | |
64b9637c PJ |
186 | |
187 | Sometimes people send fixes that *might* be an improvement in | |
188 | general--but it is hard to be sure of this. I can install such | |
189 | changes some of the time, but not during pretest, when I am trying to | |
190 | get a new version to work reliably as quickly as possible. | |
191 | ||
6f08f980 RS |
192 | The safest changes for us to install are changes to the s- and m- |
193 | files. At least those can't break other systems. | |
64b9637c PJ |
194 | |
195 | Another safe kind of change is one that uses a conditional to make | |
196 | sure it will apply only to a particular kind of system. Ordinarily, | |
197 | that is a bad way to solve a problem, and I would want to find a | |
198 | cleaner alternative. But the virtue of safety can make it superior at | |
199 | pretest time. | |
200 | ||
6f08f980 RS |
201 | * Don't suggest changes during pretest to add features or make |
202 | something cleaner. Every change risks introducing a bug, so I won't | |
203 | install a change during pretest unless it is *necessary*. | |
64b9637c PJ |
204 | |
205 | * If you would like to suggest changes for purposes other than fixing | |
206 | user-visible bugs, don't wait till pretest time. Instead, send them | |
6f08f980 RS |
207 | after we have made a release that proves to be stable. That is the |
208 | easiest time to consider such suggestions. If you send them at | |
209 | pretest time, we will have to defer them till later, and that might | |
210 | mean we forget all about them. | |
64b9637c PJ |
211 | |
212 | * In some cases, if you don't follow these guidelines, your | |
6f08f980 RS |
213 | information might still be useful, but we would have to do more work |
214 | to make use of it. That might cause it to fall by the wayside. | |
64b9637c PJ |
215 | \f |
216 | Local Variables: | |
217 | mode: text | |
218 | End: | |
ab5796a9 MB |
219 | |
220 | # arch-tag: caf47b2c-b56b-44f7-a760-b5bfbed15fd3 |