| 1 | Here are the guidelines for being an Emacs pretester. |
| 2 | If you would like to do this, say so, and I'll add you to |
| 3 | the pretest list. |
| 4 | |
| 5 | |
| 6 | Information for Emacs Pretesters |
| 7 | |
| 8 | The purpose of Emacs pretesting is to verify that the new Emacs |
| 9 | distribution, about to be released, works properly on your system *with |
| 10 | no change whatever*, when installed following the precise |
| 11 | recommendations that come with the Emacs distribution. |
| 12 | |
| 13 | Here are some guidelines on how to do pretesting so as to make it |
| 14 | helpful. All of them follow from common sense together with the |
| 15 | nature of the purpose and the situation. |
| 16 | |
| 17 | Please save this file, and reread it when a new series of pretests |
| 18 | starts. |
| 19 | |
| 20 | * Get the pretest from gnu/emacs/pretest/emacs-MM.0.NN.tar.gz |
| 21 | on alpha.gnu.org. |
| 22 | |
| 23 | * After a few days of testing, if there are no problems, please report |
| 24 | that Emacs works for you and what configuration you are testing it on. |
| 25 | |
| 26 | * If you want to communicate with other pretesters, send mail to |
| 27 | emacs-pretesters@gnu.org. I don't use that mailing list when I send |
| 28 | to you because I've found that mailing lists tend to amplify random |
| 29 | noise into long discussions or even arguments, and that can waste a |
| 30 | lot of time. But when you have a reason to ask other pretesters for |
| 31 | help, you can do it that way. |
| 32 | |
| 33 | * It is absolutely vital that you report even the smallest change or |
| 34 | departure from the standard sources and procedure. |
| 35 | |
| 36 | Otherwise, you are not testing the same program that we asked you to |
| 37 | test. Testing a different program is usually of no use whatever. It |
| 38 | can even cause trouble, if you fail to tell us that you tested some |
| 39 | other program instead of what we are about to release. We might think |
| 40 | that Emacs works, when in fact it has not even been tried, and might |
| 41 | have a glaring fault. |
| 42 | |
| 43 | * Don't use a site-load.el file or a site-init.el file when you pretest. |
| 44 | Using either of those files means you are not testing Emacs as a typical |
| 45 | site would use it. |
| 46 | |
| 47 | Actually, it does no harm to test Emacs with such customizations *as |
| 48 | well as* testing it "out of the box". Anything you do that could find |
| 49 | a bug is useful, as long as you make sure we know exactly what you |
| 50 | did. The important point is that testing with local changes is no |
| 51 | substitute for testing Emacs exactly as it is distributed. |
| 52 | |
| 53 | * Even changing the compilation options counts as a change in the |
| 54 | program. The Emacs sources specify which compilation options to use. |
| 55 | Some of them are specified in makefiles, and some in machine-specific |
| 56 | configuration files. They also give you ways to override this--but if |
| 57 | you do, then you are not testing what ordinary users will do. |
| 58 | Therefore, when pretesting, it is vital to test with the default |
| 59 | compilation options. |
| 60 | |
| 61 | (Testing with a different set of options can be useful *in addition*, |
| 62 | but not *instead of* the default options.) |
| 63 | |
| 64 | * The machine and system configuration files of Emacs are parts of |
| 65 | Emacs. So when you test Emacs, you need to do it with the |
| 66 | configuration files that come with Emacs. |
| 67 | |
| 68 | If Emacs does not come with configuration files for a certain machine, |
| 69 | and you test it with configuration files that don't come with Emacs, |
| 70 | this is effectively changing Emacs. Because the crucial fact about |
| 71 | the planned release is that, without changes, it doesn't work on that |
| 72 | machine. |
| 73 | |
| 74 | To make Emacs work on that machine, we would need to install new |
| 75 | configuration files. That is not out of the question, since it is |
| 76 | safe--it certainly won't break any other machines that already work. |
| 77 | But you will have to rush in the legal papers to give the FSF |
| 78 | permission to use such a large piece of text. |
| 79 | |
| 80 | * Look in the etc/MACHINES file. |
| 81 | |
| 82 | The etc/MACHINES file says which configuration files to use for your |
| 83 | machine, so use the ones that are recommended. If you guess, you might |
| 84 | guess wrong and encounter spurious difficulties. What's more, if you |
| 85 | don't follow etc/MACHINES then you aren't helping to test that its |
| 86 | recommendations are valid. |
| 87 | |
| 88 | The etc/MACHINES file may describe other things that you need to do |
| 89 | to make Emacs work on your machine. If so, you should follow these |
| 90 | recommendations also, for the same reason. |
| 91 | |
| 92 | * Send your problem reports to bug-gnu-emacs@gnu.org. |
| 93 | |
| 94 | Sometimes we won't know what to do about a system-dependent issue, and |
| 95 | we may need people to say what happens if you try a certain thing on a |
| 96 | certain system. When this happens, we'll send out a query. |
| 97 | |
| 98 | * Don't delay sending information. |
| 99 | |
| 100 | When you test on a system and encounter no problems, please report it |
| 101 | right away. That way, we will know that someone has tested Emacs on |
| 102 | that kind of system. |
| 103 | |
| 104 | Please don't wait for several days "to see if it really works before |
| 105 | you say anything." Tell us right away that Emacs seems basically to |
| 106 | work; then, if you notice a problem a few days later, tell us |
| 107 | immediately about that when you see it. |
| 108 | |
| 109 | It is okay if you double check things before reporting a problem, such |
| 110 | as to see if you can easily fix it. But don't wait very long. A good |
| 111 | rule to use in pretesting is always to report every problem on the |
| 112 | same day you encounter it, even if that means you can't find a |
| 113 | solution before you report the problem. |
| 114 | |
| 115 | I'd much rather hear about a problem today and a solution tomorrow |
| 116 | than get both of them tomorrow at the same time. |
| 117 | |
| 118 | * Make each bug report self-contained. |
| 119 | |
| 120 | If you refer back to another message, whether from you or from someone |
| 121 | else, then it will be necessary for anyone who wants to investigate |
| 122 | the bug to find the other message. This may be difficult, it is |
| 123 | probably time-consuming. |
| 124 | |
| 125 | To help save our time, simply copy the relevant parts of any previous |
| 126 | messages into your own bug report. |
| 127 | |
| 128 | In particular, if we ask you for more information because a bug report |
| 129 | was incomplete, it is best to send me the *entire* collection of |
| 130 | relevant information, all together. If you send just the additional |
| 131 | information, that makes extra work for us. There is even a risk that |
| 132 | we won't remember what question you are sending the answer to. |
| 133 | |
| 134 | * When you encounter a bug that manifests itself as a Lisp error, |
| 135 | try setting debug-on-error to t and making the bug happen again. |
| 136 | Then you will get a Lisp backtrace. Including that in your bug report |
| 137 | is very useful. |
| 138 | |
| 139 | * For advice on debugging, see etc/DEBUG. |
| 140 | |
| 141 | * Debugging optimized code is possible, if you compile with GCC, but |
| 142 | in some cases the optimized code can be confusing. If you are not |
| 143 | accustomed to that, recompile Emacs without -O. One way to do this is |
| 144 | |
| 145 | make clean |
| 146 | make CFLAGS=-g |
| 147 | |
| 148 | * Configure tries to figure out what kind of system you have by |
| 149 | compiling and linking programs which calls various functions and looks |
| 150 | at whether that succeeds. The file config.log contains any messages |
| 151 | produced by compilers while running configure, to aid debugging if |
| 152 | configure makes a mistake. But note that config.cache reads: |
| 153 | |
| 154 | # Giving --cache-file=/dev/null disables caching, for debugging configure. |
| 155 | |
| 156 | or more simply, |
| 157 | |
| 158 | rm config.cache |
| 159 | ./configure |
| 160 | |
| 161 | * Don't try changing Emacs *in any way* during pretest unless it fails |
| 162 | to work unchanged. |
| 163 | |
| 164 | * Always be precise when talking about changes you have made. Show |
| 165 | things rather than describing them. Use exact filenames (relative to |
| 166 | the main directory of the distribution), not partial ones. For |
| 167 | example, say "I changed Makefile" rather than "I changed the |
| 168 | makefile". Instead of saying "I defined the MUMBLE macro", send a |
| 169 | diff. |
| 170 | |
| 171 | * Always use `diff -c' to make diffs. If you don't include context, it |
| 172 | may be hard for us to figure out where you propose to make the |
| 173 | changes. So we might ignore your patch. |
| 174 | |
| 175 | * When you write a fix, keep in mind that we can't install a change |
| 176 | that *might* break other systems without the risk that it will fail to |
| 177 | work and therefore require an additional cycle of pretesting. |
| 178 | |
| 179 | People often suggest fixing a problem by changing config.h or |
| 180 | src/Makefile to do something special that a particular system needs. |
| 181 | Sometimes it is totally obvious that such changes would break Emacs |
| 182 | for almost all users. We can't possibly make a change like that. All |
| 183 | we can do is ask you to find a fix that is safe to install. |
| 184 | |
| 185 | Sometimes people send fixes that *might* be an improvement in |
| 186 | general--but it is hard to be sure of this. I can install such |
| 187 | changes some of the time, but not during pretest, when I am trying to |
| 188 | get a new version to work reliably as quickly as possible. |
| 189 | |
| 190 | The safest changes for us to install are changes to the s- and m- |
| 191 | files. At least those can't break other systems. |
| 192 | |
| 193 | Another safe kind of change is one that uses a conditional to make |
| 194 | sure it will apply only to a particular kind of system. Ordinarily, |
| 195 | that is a bad way to solve a problem, and I would want to find a |
| 196 | cleaner alternative. But the virtue of safety can make it superior at |
| 197 | pretest time. |
| 198 | |
| 199 | * Don't suggest changes during pretest to add features or make |
| 200 | something cleaner. Every change risks introducing a bug, so I won't |
| 201 | install a change during pretest unless it is *necessary*. |
| 202 | |
| 203 | * If you would like to suggest changes for purposes other than fixing |
| 204 | user-visible bugs, don't wait till pretest time. Instead, send them |
| 205 | after we have made a release that proves to be stable. That is the |
| 206 | easiest time to consider such suggestions. If you send them at |
| 207 | pretest time, we will have to defer them till later, and that might |
| 208 | mean we forget all about them. |
| 209 | |
| 210 | * In some cases, if you don't follow these guidelines, your |
| 211 | information might still be useful, but we would have to do more work |
| 212 | to make use of it. That might cause it to fall by the wayside. |
| 213 | \f |
| 214 | Local Variables: |
| 215 | mode: text |
| 216 | End: |
| 217 | |