Commit | Line | Data |
---|---|---|
64b9637c PJ |
1 | Here are the guidelines for being an Emacs pretester. |
2 | If you would like to do this, say so, and I'll add you to | |
3 | the pretest list. | |
4 | ||
5 | ||
6 | Information for Emacs Pretesters | |
7 | ||
8 | The purpose of Emacs pretesting is to verify that the new Emacs | |
9 | distribution, about to be released, works properly on your system *with | |
10 | no change whatever*, when installed following the precise | |
11 | recommendations that come with the Emacs distribution. | |
12 | ||
13 | Here are some guidelines on how to do pretesting so as to make it | |
14 | helpful. All of them follow from common sense together with the | |
15 | nature of the purpose and the situation. | |
16 | ||
17 | Please save this file, and reread it when a new series of pretests | |
18 | starts. | |
19 | ||
20 | * Get the pretest from gnu/emacs/emacs-MM.NN.tar.gz and | |
21 | gnu/emacs/leim-MM.NN.tar.gz on alpha.gnu.org. | |
22 | ||
23 | * After a few days of testing, if there are no problems, please report | |
24 | that Emacs works for you and what configuration you are testing it on. | |
25 | ||
26 | * If you want to communicate with other pretesters, send mail to | |
27 | emacs-pretesters@gnu.org. I don't use that mailing list when I send | |
28 | to you because I've found that mailing lists tend to amplify random | |
29 | noise into long discussions or even arguments, and that can waste a | |
30 | lot of time. But when you have a reason to ask other pretesters for | |
31 | help, you can do it that way. | |
32 | ||
33 | * It is absolutely vital that you tell me about even the smallest | |
34 | change or departure from the standard sources and procedure. | |
35 | ||
36 | Otherwise, you are not testing the same program that I asked you to | |
37 | test. Testing a different program is usually of no use whatever. It | |
38 | can even cause trouble if you fail to tell me that you tested some | |
39 | other program instead of what I am about to release. I might think | |
40 | that Emacs works, when in fact it has not even been tried, and might | |
41 | have a glaring fault. | |
42 | ||
43 | * Don't use a site-load.el file or a site-init.el file when you pretest. | |
44 | Using either of those files means you are not testing Emacs as a typical | |
45 | site would use it. | |
46 | ||
47 | Actually, it does no harm to test Emacs with such customizations *as | |
48 | well as* testing it "out of the box". Anything you do that could find | |
49 | a bug is useful, as long as you make sure I know exactly what you did. | |
50 | The important point is that testing with local changes is no | |
51 | substitute for testing Emacs exactly as it is distributed. | |
52 | ||
53 | * Even changing the compilation options counts as a change in the | |
54 | program. The Emacs sources specify which compilation options to use. | |
55 | Some of them are specified in makefiles, and some in machine-specific | |
56 | configuration files. They also give you ways to override this--but if | |
57 | you do, then you are not testing what ordinary users will do. | |
58 | Therefore, when pretesting, it is vital to test with the default | |
59 | compilation options. | |
60 | ||
61 | (Testing with a different set of options can be useful *in addition*, | |
62 | but not *instead of* the default options.) | |
63 | ||
64 | * The machine and system configuration files of Emacs are parts of | |
65 | Emacs. So when you test Emacs, you need to do it with the | |
66 | configuration files that come with Emacs. | |
67 | ||
68 | If Emacs does not come with configuration files for a certain machine, | |
69 | and you test it with configuration files that don't come with Emacs, | |
70 | this is effectively changing Emacs. Because the crucial fact about | |
71 | the planned release is that, without changes, it doesn't work on that | |
72 | machine. | |
73 | ||
74 | To make Emacs work on that machine, I would need to install new | |
75 | configuration files. That is not out of the question, since it is | |
76 | safe--it certainly won't break any other machines that already work. | |
77 | But you will have to rush me the legal papers to give the FSF | |
78 | permission to use such a large piece of text. | |
79 | ||
80 | * Look in the etc/MACHINES file. | |
81 | ||
82 | The etc/MACHINES file says which configuration files to use for your | |
83 | machine, so use the ones that are recommended. If you guess, you might | |
84 | guess wrong and encounter spurious difficulties. What's more, if you | |
85 | don't follow etc/MACHINES then you aren't helping to test that its | |
86 | recommendations are valid. | |
87 | ||
88 | The etc/MACHINES file may describe other things that you need to do | |
89 | to make Emacs work on your machine. If so, you should follow these | |
90 | recommendations also, for the same reason. | |
91 | ||
92 | * Send your problem reports to emacs-pretest-bug@gnu.org, not | |
93 | bug-gnu-emacs. | |
94 | ||
95 | Sometimes I won't know what to do about a system-dependent issue, and | |
96 | I may need people to tell me what happens if you try a certain thing | |
97 | on a certain system. When this happens, I'll send out a query. | |
98 | ||
99 | * Don't delay sending information. | |
100 | ||
101 | When you test on a system and encounter no problems, please tell me | |
102 | about it right away. That way, I will know that someone has tested | |
103 | Emacs on that kind of system. | |
104 | ||
105 | Please don't wait for several days "to see if it really works before | |
106 | you say anything." Tell me right away that Emacs seems basically to | |
107 | work; then, if you notice a problem a few days later, tell me | |
108 | immediately about that when you see it. | |
109 | ||
110 | It is okay if you double check things before reporting a problem, such | |
111 | as to see if you can easily fix it. But don't wait very long. A good | |
112 | rule to use in pretesting is always to tell me about every problem on | |
113 | the same day you encounter it, even if that means you can't find a | |
114 | solution before you report the problem. | |
115 | ||
116 | I'd much rather hear about a problem today and a solution tomorrow | |
117 | than get both of them tomorrow at the same time. | |
118 | ||
119 | * Make each bug report self-contained. | |
120 | ||
121 | If you refer back to another message, whether from you or from someone | |
122 | else, then it will be necessary for anyone who wants to investigate | |
123 | the bug to find the other message. This may be difficult, it is | |
124 | probably time-consuming. | |
125 | ||
126 | To help me save time, simply copy the relevant parts of any previous | |
127 | messages into your own bug report. | |
128 | ||
129 | In particular, if I ask you for more information because a bug report | |
130 | was incomplete, it is best to send me the *entire* collection of | |
131 | relevant information, all together. If you send just the additional | |
132 | information, that makes me do extra work. There is even a risk that | |
133 | I won't remember what question you are sending me the answer to. | |
134 | ||
135 | * When you encounter a bug that manifests itself as a Lisp error, | |
136 | try setting debug-on-error to t and making the bug happen again. | |
137 | Then you will get a Lisp backtrace. Including that in your bug report | |
138 | is very useful. | |
139 | ||
140 | * Debugging optimized code is possible, if you compile with GCC, but | |
141 | in some cases the optimized code can be confusing. If you are not | |
142 | accustomed to that, recompile Emacs without -O. One way to do this is | |
143 | ||
144 | make clean | |
145 | make CFLAGS=-g | |
146 | ||
147 | * If you use X windows, it is a good idea to run Emacs under GDB (or | |
148 | some other suitable debugger) *all the time*, at least while | |
149 | pretesting. | |
150 | ||
151 | Then, when Emacs crashes, you will be able to debug the live process, | |
152 | not just a core dump. The `pr' command defined in src/.gdbinit is very | |
153 | useful in this case for examining Lisp_Object values as they would | |
154 | appear in Lisp. | |
155 | ||
156 | If you can't use `pr' because Emacs has got a fault already, or | |
157 | because you have only a core dump, you can use `xtype' to look at the | |
158 | type of a value, and then choose one of the other commands `xsymbol', | |
159 | `xstring', `xcons', `xvector' and so on to examine the contents. | |
160 | ||
161 | I myself *always* run Emacs under GDB so that I can debug conveniently | |
162 | if the occasion arises. | |
163 | ||
164 | * To get Lisp-level backtrace information within GDB, | |
165 | look for stack frames that call Ffuncall. Select them one by one in GDB | |
166 | and type this: | |
167 | ||
168 | p *args | |
169 | pr | |
170 | ||
171 | This will print the name of the Lisp function called by that level | |
172 | of function calling. | |
173 | ||
174 | By printing the remaining elements of args, you can see the argument | |
175 | values. Here's how to print the first argument: | |
176 | ||
177 | p args[1] | |
178 | pr | |
179 | ||
180 | If you do not have a live process, you can use xtype and the other | |
181 | x... commands such as xsymbol to get such information, albeit less | |
182 | conveniently. | |
183 | ||
184 | * Even with a live process, these x... commands are useful for | |
185 | examining the fields in a buffer, window, process, frame or marker. | |
186 | Here's an example using concepts explained in the node "Value History" | |
187 | of the GDB manual to print the variable frame from this line in | |
188 | xmenu.c: | |
189 | ||
190 | buf.frame_or_window = Fcons (frame, prefix); | |
191 | ||
192 | First, use these commands: | |
193 | ||
194 | cd src | |
195 | gdb emacs | |
196 | b xmenu.c:1209 | |
197 | r -q | |
198 | ||
199 | Then type C-x 5 2 to create a new frame, and it hits the breakpoint: | |
200 | ||
201 | (gdb) p frame | |
202 | $1 = 1077872640 | |
203 | (gdb) xtype | |
204 | Lisp_Vectorlike | |
205 | PVEC_FRAME | |
206 | (gdb) xframe | |
207 | $2 = (struct frame *) 0x3f0800 | |
208 | (gdb) p *$ | |
209 | $3 = { | |
210 | size = 536871989, | |
211 | next = 0x366240, | |
212 | name = 809661752, | |
213 | [...] | |
214 | } | |
215 | (gdb) p $3->name | |
216 | $4 = 809661752 | |
217 | ||
218 | Now we can use `pr' to print the name of the frame: | |
219 | ||
220 | (gdb) pr | |
221 | "emacs@steenrod.math.nwu.edu" | |
222 | ||
223 | * The Emacs C code heavily uses macros defined in lisp.h. So suppose | |
224 | we want the address of the l-value expression near the bottom of | |
225 | `kbd_buffer_store_event' from keyboard.c: | |
226 | ||
227 | XVECTOR (kbd_buffer_frame_or_window)->contents[kbd_store_ptr | |
228 | - kbd_buffer] | |
229 | = event->frame_or_window); | |
230 | ||
231 | XVECTOR is a macro, and therefore GDB does not know about it. | |
232 | GDB cannot evaluate p XVECTOR (kbd_buffer_frame_or_window). | |
233 | ||
234 | However, you can use the xvector command in GDB to get the same | |
235 | result. Here is how: | |
236 | ||
237 | (gdb) p kbd_buffer_frame_or_window | |
238 | $1 = 1078005760 | |
239 | (gdb) xvector | |
240 | $2 = (struct Lisp_Vector *) 0x411000 | |
241 | 0 | |
242 | (gdb) p $->contents[kbd_store_ptr - kbd_buffer] | |
243 | $3 = 1077872640 | |
244 | (gdb) p &$ | |
245 | $4 = (int *) 0x411008 | |
246 | ||
247 | * Here's a related example of macros and the GDB `define' command. | |
248 | There are many Lisp vectors such as `recent_keys', which contains the | |
249 | last 100 keystrokes. We can print this Lisp vector | |
250 | ||
251 | p recent_keys | |
252 | pr | |
253 | ||
254 | But this may be inconvenient, since `recent_keys' is much more verbose | |
255 | than `C-h l'. We might want to print only the last 10 elements of | |
256 | this vector. `recent_keys' is updated in keyboard.c by the command | |
257 | ||
258 | XVECTOR (recent_keys)->contents[recent_keys_index] = c; | |
259 | ||
260 | So we define a GDB command `xvector-elts', so the last 10 keystrokes | |
261 | are printed by | |
262 | ||
263 | xvector-elts recent_keys recent_keys_index 10 | |
264 | ||
265 | where you can define xvector-elts as follows: | |
266 | ||
267 | define xvector-elts | |
268 | set $i = 0 | |
269 | p $arg0 | |
270 | xvector | |
271 | set $foo = $ | |
272 | while $i < $arg2 | |
273 | p $foo->contents[$arg1-($i++)] | |
274 | pr | |
275 | end | |
276 | document xvector-elts | |
277 | Prints a range of elements of a Lisp vector. | |
278 | xvector-elts v n i | |
279 | prints `i' elements of the vector `v' ending at the index `n'. | |
280 | end | |
281 | ||
282 | * To debug what happens while preloading and dumping Emacs, | |
283 | do `gdb temacs' and start it with `r -batch -l loadup dump'. | |
284 | ||
285 | If temacs actually succeeds when running under GDB in this way, do not | |
286 | try to run the dumped Emacs, because it was dumped with the GDB | |
287 | breakpoints in it. | |
288 | ||
289 | * If you encounter X protocol errors, try evaluating (x-synchronize t). | |
290 | That puts Emacs into synchronous mode, where each Xlib call checks for | |
291 | errors before it returns. This mode is much slower, but when you get | |
292 | an error, you will see exactly which call really caused the error. | |
293 | ||
294 | * If the symptom of the bug is that Emacs fails to respond, don't | |
295 | assume Emacs is `hung'--it may instead be in an infinite loop. To | |
296 | find out which, make the problem happen under GDB and stop Emacs once | |
297 | it is not responding. (If Emacs is using X Windows directly, you can | |
298 | stop Emacs by typing C-z at the GDB job.) Then try stepping with | |
299 | `step'. If Emacs is hung, the `step' command won't return. If it is | |
300 | looping, `step' will return. | |
301 | ||
302 | If this shows Emacs is hung in a system call, stop it again and | |
303 | examine the arguments of the call. In your bug report, state exactly | |
304 | where in the source the system call is, and what the arguments are. | |
305 | ||
306 | If Emacs is in an infinite loop, please determine where the loop | |
307 | starts and ends. The easiest way to do this is to use the GDB command | |
308 | `finish'. Each time you use it, Emacs resumes execution until it | |
309 | exits one stack frame. Keep typing `finish' until it doesn't | |
310 | return--that means the infinite loop is in the stack frame which you | |
311 | just tried to finish. | |
312 | ||
313 | Stop Emacs again, and use `finish' repeatedly again until you get back | |
314 | to that frame. Then use `next' to step through that frame. By | |
315 | stepping, you will see where the loop starts and ends. Also please | |
316 | examine the data being used in the loop and try to determine why the | |
317 | loop does not exit when it should. Include all of this information in | |
318 | your bug report. | |
319 | ||
320 | * If certain operations in Emacs are slower than they used to be, here | |
321 | is some advice for how to find out why. | |
322 | ||
323 | Stop Emacs repeatedly during the slow operation, and make a backtrace | |
324 | each time. Compare the backtraces looking for a pattern--a specific | |
325 | function that shows up more often than you'd expect. | |
326 | ||
327 | If you don't see a pattern in the C backtraces, get some Lisp | |
328 | backtrace information by looking at Ffuncall frames (see above), and | |
329 | again look for a pattern. | |
330 | ||
331 | When using X, you can stop Emacs at any time by typing C-z at GDB. | |
332 | When not using X, you can do this with C-g. | |
333 | ||
334 | * Configure tries to figure out what kind of system you have by | |
335 | compiling and linking programs which calls various functions and looks | |
336 | at whether that succeeds. The file config.log contains any messages | |
337 | produced by compilers while running configure, to aid debugging if | |
338 | configure makes a mistake. But note that config.cache reads: | |
339 | ||
340 | # Giving --cache-file=/dev/null disables caching, for debugging configure. | |
341 | ||
342 | or more simply, | |
343 | ||
344 | rm config.cache | |
345 | ./configure | |
346 | ||
347 | * Always be precise when talking about changes you have made. Show | |
348 | things rather than describing them. Use exact filenames (relative to | |
349 | the main directory of the distribution), not partial ones. For | |
350 | example, say "I changed Makefile" rather than "I changed the | |
351 | makefile". Instead of saying "I defined the MUMBLE macro", send a | |
352 | diff. | |
353 | ||
354 | * Always use `diff -c' to make diffs. If you don't include context, it | |
355 | may be hard for me to figure out where you propose to make the | |
356 | changes. So I might have to ignore your patch. | |
357 | ||
358 | * When you write a fix, keep in mind that I can't install a change | |
359 | that *might* break other systems without the risk that it will fail to | |
360 | work and therefore require an additional cycle of pretesting. | |
361 | ||
362 | People often suggest fixing a problem by changing config.h or | |
363 | src/ymakefile or even src/Makefile to do something special that a | |
364 | particular system needs. Sometimes it is totally obvious that such | |
365 | changes would break Emacs for almost all users. I can't possibly make | |
366 | a change like that. All I can do is send it back to you and ask you | |
367 | to find a fix that is safe to install. | |
368 | ||
369 | Sometimes people send fixes that *might* be an improvement in | |
370 | general--but it is hard to be sure of this. I can install such | |
371 | changes some of the time, but not during pretest, when I am trying to | |
372 | get a new version to work reliably as quickly as possible. | |
373 | ||
374 | The safest changes for me to install are changes to the s- and m- | |
375 | files. At least I know those can't affect most systems. | |
376 | ||
377 | Another safe kind of change is one that uses a conditional to make | |
378 | sure it will apply only to a particular kind of system. Ordinarily, | |
379 | that is a bad way to solve a problem, and I would want to find a | |
380 | cleaner alternative. But the virtue of safety can make it superior at | |
381 | pretest time. | |
382 | ||
383 | * Don't try changing Emacs *in any way* unless it fails to work unchanged. | |
384 | ||
385 | * Don't even suggest changes to add features or make something | |
386 | cleaner. Every change I install could introduce a bug, so I won't | |
387 | install a change during pretest unless I see it is *necessary*. | |
388 | ||
389 | * If you would like to suggest changes for purposes other than fixing | |
390 | user-visible bugs, don't wait till pretest time. Instead, send them | |
391 | after I have made a release that proves to be stable. Then I can give | |
392 | your suggestions proper consideration. If you send them at pretest | |
393 | time, I will have to defer them till later, and that might mean I | |
394 | forget all about them. | |
395 | ||
396 | * In some cases, if you don't follow these guidelines, your | |
397 | information might still be useful, but I might have to do more work to | |
398 | make use of it. Unfortunately, I am so far behind in my work that I | |
399 | just can't keep up unless you help me to do it efficiently. | |
400 | \f | |
401 | Some suggestions for debugging on MS Windows: | |
402 | ||
403 | Marc Fleischeuers, Geoff Voelker and Andrew Innes | |
404 | ||
405 | To debug emacs with Microsoft Visual C++, you either start emacs from | |
406 | the debugger or attach the debugger to a running emacs process. To | |
407 | start emacs from the debugger, you can use the file bin/debug.bat. The | |
408 | Microsoft Developer studio will start and under Project, Settings, | |
409 | Debug, General you can set the command-line arguments and emacs' | |
410 | startup directory. Set breakpoints (Edit, Breakpoints) at Fsignal and | |
411 | other functions that you want to examine. Run the program (Build, | |
412 | Start debug). Emacs will start and the debugger will take control as | |
413 | soon as a breakpoint is hit. | |
414 | ||
415 | You can also attach the debugger to an already running emacs process. | |
416 | To do this, start up the Microsoft Developer studio and select Build, | |
417 | Start debug, Attach to process. Choose the emacs process from the | |
418 | list. Send a break to the running process (Debug, Break) and you will | |
419 | find that execution is halted somewhere in user32.dll. Open the stack | |
420 | trace window and go up the stack to w32_msg_pump. Now you can set | |
421 | breakpoints in emacs (Edit, Breakpoints). Continue the running emacs | |
422 | process (Debug, Step out) and control will return to emacs, until a | |
423 | breakpoint is hit. | |
424 | ||
425 | To examine the contents of a lisp variable, you can use the function | |
426 | 'debug_print'. Right-click on a variable, select QuickWatch, and | |
427 | place 'debug_print(' and ')' around the expression. Press | |
428 | 'Recalculate' and the output is sent to the 'Debug' pane in the Output | |
429 | window. If emacs was started from the debugger, a console window was | |
430 | opened at emacs' startup; this console window also shows the output of | |
431 | 'debug_print'. It is also possible to keep appropriately masked and | |
432 | typecast lisp symbols in the Watch window, this is more convenient | |
433 | when steeping though the code. For instance, on entering | |
434 | apply_lambda, you can watch (struct Lisp_Symbol *) (0xfffffff & | |
435 | args[0]). | |
436 | ||
437 | \f | |
438 | Local Variables: | |
439 | mode: text | |
440 | End: |