(send_process_trap): Unblock SIGPIPE.
[bpt/emacs.git] / man / mule.texi
1 @c This is part of the Emacs manual.
2 @c Copyright (C) 1997, 1999, 2000, 2001 Free Software Foundation, Inc.
3 @c See file emacs.texi for copying conditions.
4 @node International, Major Modes, Frames, Top
5 @chapter International Character Set Support
6 @cindex MULE
7 @cindex international scripts
8 @cindex multibyte characters
9 @cindex encoding of characters
10
11 @cindex Celtic
12 @cindex Chinese
13 @cindex Cyrillic
14 @cindex Czech
15 @cindex Devanagari
16 @cindex Hindi
17 @cindex Marathi
18 @cindex Ethiopic
19 @cindex German
20 @cindex Greek
21 @cindex Hebrew
22 @cindex IPA
23 @cindex Japanese
24 @cindex Korean
25 @cindex Lao
26 @cindex Latin
27 @cindex Polish
28 @cindex Romanian
29 @cindex Slovak
30 @cindex Slovenian
31 @cindex Thai
32 @cindex Tibetan
33 @cindex Turkish
34 @cindex Vietnamese
35 @cindex Dutch
36 @cindex Spanish
37 Emacs supports a wide variety of international character sets,
38 including European variants of the Latin alphabet, as well as Chinese,
39 Cyrillic, Devanagari (Hindi and Marathi), Ethiopic, Greek, Hebrew, IPA,
40 Japanese, Korean, Lao, Thai, Tibetan, and Vietnamese scripts. These features
41 have been merged from the modified version of Emacs known as MULE (for
42 ``MULti-lingual Enhancement to GNU Emacs'')
43
44 Emacs also supports various encodings of these characters used by
45 other internationalized software, such as word processors and mailers.
46
47 Emacs allows editing text with international characters by supporting
48 all the related activities:
49
50 @itemize @bullet
51 @item
52 You can visit files with non-@acronym{ASCII} characters, save non-@acronym{ASCII} text, and
53 pass non-@acronym{ASCII} text between Emacs and programs it invokes (such as
54 compilers, spell-checkers, and mailers). Setting your language
55 environment (@pxref{Language Environments}) takes care of setting up the
56 coding systems and other options for a specific language or culture.
57 Alternatively, you can specify how Emacs should encode or decode text
58 for each command; see @ref{Specify Coding}.
59
60 @item
61 You can display non-@acronym{ASCII} characters encoded by the various scripts.
62 This works by using appropriate fonts on X and similar graphics
63 displays (@pxref{Defining Fontsets}), and by sending special codes to
64 text-only displays (@pxref{Specify Coding}). If some characters are
65 displayed incorrectly, refer to @ref{Undisplayable Characters}, which
66 describes possible problems and explains how to solve them.
67
68 @item
69 You can insert non-@acronym{ASCII} characters or search for them. To do that,
70 you can specify an input method (@pxref{Select Input Method}) suitable
71 for your language, or use the default input method set up when you set
72 your language environment. (Emacs input methods are part of the Leim
73 package, which must be installed for you to be able to use them.) If
74 your keyboard can produce non-@acronym{ASCII} characters, you can select an
75 appropriate keyboard coding system (@pxref{Specify Coding}), and Emacs
76 will accept those characters. Latin-1 characters can also be input by
77 using the @kbd{C-x 8} prefix, see @ref{Single-Byte Character Support,
78 C-x 8}. On X Window systems, your locale should be set to an
79 appropriate value to make sure Emacs interprets keyboard input
80 correctly; see @ref{Language Environments, locales}.
81 @end itemize
82
83 The rest of this chapter describes these issues in detail.
84
85 @menu
86 * International Chars:: Basic concepts of multibyte characters.
87 * Enabling Multibyte:: Controlling whether to use multibyte characters.
88 * Language Environments:: Setting things up for the language you use.
89 * Input Methods:: Entering text characters not on your keyboard.
90 * Select Input Method:: Specifying your choice of input methods.
91 * Multibyte Conversion:: How single-byte characters convert to multibyte.
92 * Coding Systems:: Character set conversion when you read and
93 write files, and so on.
94 * Recognize Coding:: How Emacs figures out which conversion to use.
95 * Specify Coding:: Various ways to choose which conversion to use.
96 * Fontsets:: Fontsets are collections of fonts
97 that cover the whole spectrum of characters.
98 * Defining Fontsets:: Defining a new fontset.
99 * Undisplayable Characters:: When characters don't display.
100 * Single-Byte Character Support::
101 You can pick one European character set
102 to use without multibyte characters.
103 * Charsets:: How Emacs groups its internal character codes.
104 @end menu
105
106 @node International Chars
107 @section Introduction to International Character Sets
108
109 The users of international character sets and scripts have established
110 many more-or-less standard coding systems for storing files. Emacs
111 internally uses a single multibyte character encoding, so that it can
112 intermix characters from all these scripts in a single buffer or string.
113 This encoding represents each non-@acronym{ASCII} character as a sequence of bytes
114 in the range 0200 through 0377. Emacs translates between the multibyte
115 character encoding and various other coding systems when reading and
116 writing files, when exchanging data with subprocesses, and (in some
117 cases) in the @kbd{C-q} command (@pxref{Multibyte Conversion}).
118
119 @kindex C-h h
120 @findex view-hello-file
121 @cindex undisplayable characters
122 @cindex @samp{?} in display
123 The command @kbd{C-h h} (@code{view-hello-file}) displays the file
124 @file{etc/HELLO}, which shows how to say ``hello'' in many languages.
125 This illustrates various scripts. If some characters can't be
126 displayed on your terminal, they appear as @samp{?} or as hollow boxes
127 (@pxref{Undisplayable Characters}).
128
129 Keyboards, even in the countries where these character sets are used,
130 generally don't have keys for all the characters in them. So Emacs
131 supports various @dfn{input methods}, typically one for each script or
132 language, to make it convenient to type them.
133
134 @kindex C-x RET
135 The prefix key @kbd{C-x @key{RET}} is used for commands that pertain
136 to multibyte characters, coding systems, and input methods.
137
138 @node Enabling Multibyte
139 @section Enabling Multibyte Characters
140
141 @cindex turn multibyte support on or off
142 You can enable or disable multibyte character support, either for
143 Emacs as a whole, or for a single buffer. When multibyte characters are
144 disabled in a buffer, then each byte in that buffer represents a
145 character, even codes 0200 through 0377. The old features for
146 supporting the European character sets, ISO Latin-1 and ISO Latin-2,
147 work as they did in Emacs 19 and also work for the other ISO 8859
148 character sets.
149
150 However, there is no need to turn off multibyte character support to
151 use ISO Latin; the Emacs multibyte character set includes all the
152 characters in these character sets, and Emacs can translate
153 automatically to and from the ISO codes.
154
155 By default, Emacs starts in multibyte mode, because that allows you to
156 use all the supported languages and scripts without limitations.
157
158 To edit a particular file in unibyte representation, visit it using
159 @code{find-file-literally}. @xref{Visiting}. To convert a buffer in
160 multibyte representation into a single-byte representation of the same
161 characters, the easiest way is to save the contents in a file, kill the
162 buffer, and find the file again with @code{find-file-literally}. You
163 can also use @kbd{C-x @key{RET} c}
164 (@code{universal-coding-system-argument}) and specify @samp{raw-text} as
165 the coding system with which to find or save a file. @xref{Specify
166 Coding}. Finding a file as @samp{raw-text} doesn't disable format
167 conversion, uncompression and auto mode selection as
168 @code{find-file-literally} does.
169
170 @vindex enable-multibyte-characters
171 @vindex default-enable-multibyte-characters
172 To turn off multibyte character support by default, start Emacs with
173 the @samp{--unibyte} option (@pxref{Initial Options}), or set the
174 environment variable @env{EMACS_UNIBYTE}. You can also customize
175 @code{enable-multibyte-characters} or, equivalently, directly set the
176 variable @code{default-enable-multibyte-characters} to @code{nil} in
177 your init file to have basically the same effect as @samp{--unibyte}.
178
179 @findex toggle-enable-multibyte-characters
180 To convert a unibyte session to a multibyte session, set
181 @code{default-enable-multibyte-characters} to @code{t}. Buffers which
182 were created in the unibyte session before you turn on multibyte support
183 will stay unibyte. You can turn on multibyte support in a specific
184 buffer by invoking the command @code{toggle-enable-multibyte-characters}
185 in that buffer.
186
187 @cindex Lisp files, and multibyte operation
188 @cindex multibyte operation, and Lisp files
189 @cindex unibyte operation, and Lisp files
190 @cindex init file, and non-@acronym{ASCII} characters
191 @cindex environment variables, and non-@acronym{ASCII} characters
192 With @samp{--unibyte}, multibyte strings are not created during
193 initialization from the values of environment variables,
194 @file{/etc/passwd} entries etc.@: that contain non-@acronym{ASCII} 8-bit
195 characters.
196
197 Emacs normally loads Lisp files as multibyte, regardless of whether
198 you used @samp{--unibyte}. This includes the Emacs initialization file,
199 @file{.emacs}, and the initialization files of Emacs packages such as
200 Gnus. However, you can specify unibyte loading for a particular Lisp
201 file, by putting @w{@samp{-*-unibyte: t;-*-}} in a comment on the first
202 line (@pxref{File Variables}). Then that file is always loaded as
203 unibyte text, even if you did not start Emacs with @samp{--unibyte}.
204 The motivation for these conventions is that it is more reliable to
205 always load any particular Lisp file in the same way. However, you can
206 load a Lisp file as unibyte, on any one occasion, by typing @kbd{C-x
207 @key{RET} c raw-text @key{RET}} immediately before loading it.
208
209 The mode line indicates whether multibyte character support is enabled
210 in the current buffer. If it is, there are two or more characters (most
211 often two dashes) before the colon near the beginning of the mode line.
212 When multibyte characters are not enabled, just one dash precedes the
213 colon.
214
215 @node Language Environments
216 @section Language Environments
217 @cindex language environments
218
219 All supported character sets are supported in Emacs buffers whenever
220 multibyte characters are enabled; there is no need to select a
221 particular language in order to display its characters in an Emacs
222 buffer. However, it is important to select a @dfn{language environment}
223 in order to set various defaults. The language environment really
224 represents a choice of preferred script (more or less) rather than a
225 choice of language.
226
227 The language environment controls which coding systems to recognize
228 when reading text (@pxref{Recognize Coding}). This applies to files,
229 incoming mail, netnews, and any other text you read into Emacs. It may
230 also specify the default coding system to use when you create a file.
231 Each language environment also specifies a default input method.
232
233 @findex set-language-environment
234 @vindex current-language-environment
235 To select a language environment, you can customize the variable
236 @code{current-language-environment} or use the command @kbd{M-x
237 set-language-environment}. It makes no difference which buffer is
238 current when you use this command, because the effects apply globally to
239 the Emacs session. The supported language environments include:
240
241 @cindex Euro sign
242 @cindex UTF-8
243 @quotation
244 Chinese-BIG5, Chinese-CNS, Chinese-GB, Cyrillic-ALT, Cyrillic-ISO,
245 Cyrillic-KOI8, Czech, Devanagari, Dutch, English, Ethiopic, German,
246 Greek, Hebrew, IPA, Japanese, Korean, Lao, Latin-1, Latin-2, Latin-3,
247 Latin-4, Latin-5, Latin-8 (Celtic), Latin-9 (updated Latin-1, with the
248 Euro sign), Polish, Romanian, Slovak, Slovenian, Spanish, Thai, Tibetan,
249 Turkish, UTF-8 (for a setup which prefers Unicode characters and files
250 encoded in UTF-8), and Vietnamese.
251 @end quotation
252
253 @cindex fonts for various scripts
254 @cindex Intlfonts package, installation
255 To display the script(s) used by your language environment on a
256 graphical display, you need to have a suitable font. If some of the
257 characters appear as empty boxes, you should install the GNU Intlfonts
258 package, which includes fonts for all supported scripts.@footnote{If
259 you run Emacs on X, you need to inform the X server about the location
260 of the newly installed fonts with the following commands:
261
262 @example
263 xset fp+ /usr/local/share/emacs/fonts
264 xset fp rehash
265 @end example
266 }
267 @xref{Fontsets}, for more details about setting up your fonts.
268
269 @findex set-locale-environment
270 @vindex locale-language-names
271 @vindex locale-charset-language-names
272 @cindex locales
273 Some operating systems let you specify the character-set locale you
274 are using by setting the locale environment variables @env{LC_ALL},
275 @env{LC_CTYPE}, or @env{LANG}.@footnote{If more than one of these is
276 set, the first one that is nonempty specifies your locale for this
277 purpose.} During startup, Emacs looks up your character-set locale's
278 name in the system locale alias table, matches its canonical name
279 against entries in the value of the variables
280 @code{locale-charset-language-names} and @code{locale-language-names},
281 and selects the corresponding language environment if a match is found.
282 (The former variable overrides the latter.) It also adjusts the display
283 table and terminal coding system, the locale coding system, the
284 preferred coding system as needed for the locale, and---last but not
285 least---the way Emacs decodes non-@acronym{ASCII} characters sent by your keyboard.
286
287 If you modify the @env{LC_ALL}, @env{LC_CTYPE}, or @env{LANG}
288 environment variables while running Emacs, you may want to invoke the
289 @code{set-locale-environment} function afterwards to readjust the
290 language environment from the new locale.
291
292 @vindex locale-preferred-coding-systems
293 The @code{set-locale-environment} function normally uses the preferred
294 coding system established by the language environment to decode system
295 messages. But if your locale matches an entry in the variable
296 @code{locale-preferred-coding-systems}, Emacs uses the corresponding
297 coding system instead. For example, if the locale @samp{ja_JP.PCK}
298 matches @code{japanese-shift-jis} in
299 @code{locale-preferred-coding-systems}, Emacs uses that encoding even
300 though it might normally use @code{japanese-iso-8bit}.
301
302 You can override the language environment chosen at startup with
303 explicit use of the command @code{set-language-environment}, or with
304 customization of @code{current-language-environment} in your init
305 file.
306
307 @kindex C-h L
308 @findex describe-language-environment
309 To display information about the effects of a certain language
310 environment @var{lang-env}, use the command @kbd{C-h L @var{lang-env}
311 @key{RET}} (@code{describe-language-environment}). This tells you which
312 languages this language environment is useful for, and lists the
313 character sets, coding systems, and input methods that go with it. It
314 also shows some sample text to illustrate scripts used in this language
315 environment. By default, this command describes the chosen language
316 environment.
317
318 @vindex set-language-environment-hook
319 You can customize any language environment with the normal hook
320 @code{set-language-environment-hook}. The command
321 @code{set-language-environment} runs that hook after setting up the new
322 language environment. The hook functions can test for a specific
323 language environment by checking the variable
324 @code{current-language-environment}. This hook is where you should
325 put non-default settings for specific language environment, such as
326 coding systems for keyboard input and terminal output, the default
327 input method, etc.
328
329 @vindex exit-language-environment-hook
330 Before it starts to set up the new language environment,
331 @code{set-language-environment} first runs the hook
332 @code{exit-language-environment-hook}. This hook is useful for undoing
333 customizations that were made with @code{set-language-environment-hook}.
334 For instance, if you set up a special key binding in a specific language
335 environment using @code{set-language-environment-hook}, you should set
336 up @code{exit-language-environment-hook} to restore the normal binding
337 for that key.
338
339 @node Input Methods
340 @section Input Methods
341
342 @cindex input methods
343 An @dfn{input method} is a kind of character conversion designed
344 specifically for interactive input. In Emacs, typically each language
345 has its own input method; sometimes several languages which use the same
346 characters can share one input method. A few languages support several
347 input methods.
348
349 The simplest kind of input method works by mapping @acronym{ASCII} letters
350 into another alphabet; this allows you to use one other alphabet
351 instead of @acronym{ASCII}. The Greek and Russian input methods
352 work this way.
353
354 A more powerful technique is composition: converting sequences of
355 characters into one letter. Many European input methods use composition
356 to produce a single non-@acronym{ASCII} letter from a sequence that consists of a
357 letter followed by accent characters (or vice versa). For example, some
358 methods convert the sequence @kbd{a'} into a single accented letter.
359 These input methods have no special commands of their own; all they do
360 is compose sequences of printing characters.
361
362 The input methods for syllabic scripts typically use mapping followed
363 by composition. The input methods for Thai and Korean work this way.
364 First, letters are mapped into symbols for particular sounds or tone
365 marks; then, sequences of these which make up a whole syllable are
366 mapped into one syllable sign.
367
368 Chinese and Japanese require more complex methods. In Chinese input
369 methods, first you enter the phonetic spelling of a Chinese word (in
370 input method @code{chinese-py}, among others), or a sequence of
371 portions of the character (input methods @code{chinese-4corner} and
372 @code{chinese-sw}, and others). One input sequence typically
373 corresponds to many possible Chinese characters. You select the one
374 you mean using keys such as @kbd{C-f}, @kbd{C-b}, @kbd{C-n},
375 @kbd{C-p}, and digits, which have special meanings in this situation.
376
377 The possible characters are conceptually arranged in several rows,
378 with each row holding up to 10 alternatives. Normally, Emacs displays
379 just one row at a time, in the echo area; @code{(@var{i}/@var{j})}
380 appears at the beginning, to indicate that this is the @var{i}th row
381 out of a total of @var{j} rows. Type @kbd{C-n} or @kbd{C-p} to
382 display the next row or the previous row.
383
384 Type @kbd{C-f} and @kbd{C-b} to move forward and backward among
385 the alternatives in the current row. As you do this, Emacs highlights
386 the current alternative with a special color; type @code{C-@key{SPC}}
387 to select the current alternative and use it as input. The
388 alternatives in the row are also numbered; the number appears before
389 the alternative. Typing a digit @var{n} selects the @var{n}th
390 alternative of the current row and uses it as input.
391
392 @key{TAB} in these Chinese input methods displays a buffer showing
393 all the possible characters at once; then clicking @kbd{Mouse-2} on
394 one of them selects that alternative. The keys @kbd{C-f}, @kbd{C-b},
395 @kbd{C-n}, @kbd{C-p}, and digits continue to work as usual, but they
396 do the highlighting in the buffer showing the possible characters,
397 rather than in the echo area.
398
399 In Japanese input methods, first you input a whole word using
400 phonetic spelling; then, after the word is in the buffer, Emacs
401 converts it into one or more characters using a large dictionary. One
402 phonetic spelling corresponds to a number of different Japanese words;
403 to select one of them, use @kbd{C-n} and @kbd{C-p} to cycle through
404 the alternatives.
405
406 Sometimes it is useful to cut off input method processing so that the
407 characters you have just entered will not combine with subsequent
408 characters. For example, in input method @code{latin-1-postfix}, the
409 sequence @kbd{e '} combines to form an @samp{e} with an accent. What if
410 you want to enter them as separate characters?
411
412 One way is to type the accent twice; this is a special feature for
413 entering the separate letter and accent. For example, @kbd{e ' '} gives
414 you the two characters @samp{e'}. Another way is to type another letter
415 after the @kbd{e}---something that won't combine with that---and
416 immediately delete it. For example, you could type @kbd{e e @key{DEL}
417 '} to get separate @samp{e} and @samp{'}.
418
419 Another method, more general but not quite as easy to type, is to use
420 @kbd{C-\ C-\} between two characters to stop them from combining. This
421 is the command @kbd{C-\} (@code{toggle-input-method}) used twice.
422 @ifinfo
423 @xref{Select Input Method}.
424 @end ifinfo
425
426 @cindex incremental search, input method interference
427 @kbd{C-\ C-\} is especially useful inside an incremental search,
428 because it stops waiting for more characters to combine, and starts
429 searching for what you have already entered.
430
431 To find out how to input the character after point using the current
432 input method, type @kbd{C-u C-x =}. @xref{Position Info}.
433
434 @vindex input-method-verbose-flag
435 @vindex input-method-highlight-flag
436 The variables @code{input-method-highlight-flag} and
437 @code{input-method-verbose-flag} control how input methods explain
438 what is happening. If @code{input-method-highlight-flag} is
439 non-@code{nil}, the partial sequence is highlighted in the buffer (for
440 most input methods---some disable this feature). If
441 @code{input-method-verbose-flag} is non-@code{nil}, the list of
442 possible characters to type next is displayed in the echo area (but
443 not when you are in the minibuffer).
444
445 @node Select Input Method
446 @section Selecting an Input Method
447
448 @table @kbd
449 @item C-\
450 Enable or disable use of the selected input method.
451
452 @item C-x @key{RET} C-\ @var{method} @key{RET}
453 Select a new input method for the current buffer.
454
455 @item C-h I @var{method} @key{RET}
456 @itemx C-h C-\ @var{method} @key{RET}
457 @findex describe-input-method
458 @kindex C-h I
459 @kindex C-h C-\
460 Describe the input method @var{method} (@code{describe-input-method}).
461 By default, it describes the current input method (if any). This
462 description should give you the full details of how to use any
463 particular input method.
464
465 @item M-x list-input-methods
466 Display a list of all the supported input methods.
467 @end table
468
469 @findex set-input-method
470 @vindex current-input-method
471 @kindex C-x RET C-\
472 To choose an input method for the current buffer, use @kbd{C-x
473 @key{RET} C-\} (@code{set-input-method}). This command reads the
474 input method name from the minibuffer; the name normally starts with the
475 language environment that it is meant to be used with. The variable
476 @code{current-input-method} records which input method is selected.
477
478 @findex toggle-input-method
479 @kindex C-\
480 Input methods use various sequences of @acronym{ASCII} characters to stand for
481 non-@acronym{ASCII} characters. Sometimes it is useful to turn off the input
482 method temporarily. To do this, type @kbd{C-\}
483 (@code{toggle-input-method}). To reenable the input method, type
484 @kbd{C-\} again.
485
486 If you type @kbd{C-\} and you have not yet selected an input method,
487 it prompts for you to specify one. This has the same effect as using
488 @kbd{C-x @key{RET} C-\} to specify an input method.
489
490 When invoked with a numeric argument, as in @kbd{C-u C-\},
491 @code{toggle-input-method} always prompts you for an input method,
492 suggesting the most recently selected one as the default.
493
494 @vindex default-input-method
495 Selecting a language environment specifies a default input method for
496 use in various buffers. When you have a default input method, you can
497 select it in the current buffer by typing @kbd{C-\}. The variable
498 @code{default-input-method} specifies the default input method
499 (@code{nil} means there is none).
500
501 In some language environments, which support several different input
502 methods, you might want to use an input method different from the
503 default chosen by @code{set-language-environment}. You can instruct
504 Emacs to select a different default input method for a certain
505 language environment, if you wish, by using
506 @code{set-language-environment-hook} (@pxref{Language Environments,
507 set-language-environment-hook}). For example:
508
509 @lisp
510 (defun my-chinese-setup ()
511 "Set up my private Chinese environment."
512 (if (equal current-language-environment "Chinese-GB")
513 (setq default-input-method "chinese-tonepy")))
514 (add-hook 'set-language-environment-hook 'my-chinese-setup)
515 @end lisp
516
517 @noindent
518 This sets the default input method to be @code{chinese-tonepy}
519 whenever you choose a Chinese-GB language environment.
520
521 @findex quail-set-keyboard-layout
522 Some input methods for alphabetic scripts work by (in effect)
523 remapping the keyboard to emulate various keyboard layouts commonly used
524 for those scripts. How to do this remapping properly depends on your
525 actual keyboard layout. To specify which layout your keyboard has, use
526 the command @kbd{M-x quail-set-keyboard-layout}.
527
528 @findex quail-show-key
529 You can use the command @kbd{M-x quail-show-key} to show what key
530 (or key sequence) to type in order to input the character following
531 point, using the selected keyboard layout.
532
533 @findex list-input-methods
534 To display a list of all the supported input methods, type @kbd{M-x
535 list-input-methods}. The list gives information about each input
536 method, including the string that stands for it in the mode line.
537
538 @node Multibyte Conversion
539 @section Unibyte and Multibyte Non-@acronym{ASCII} characters
540
541 When multibyte characters are enabled, character codes 0240 (octal)
542 through 0377 (octal) are not really legitimate in the buffer. The valid
543 non-@acronym{ASCII} printing characters have codes that start from 0400.
544
545 If you type a self-inserting character in the range 0240 through
546 0377, or if you use @kbd{C-q} to insert one, Emacs assumes you
547 intended to use one of the ISO Latin-@var{n} character sets, and
548 converts it to the Emacs code representing that Latin-@var{n}
549 character. You select @emph{which} ISO Latin character set to use
550 through your choice of language environment
551 @iftex
552 (see above).
553 @end iftex
554 @ifinfo
555 (@pxref{Language Environments}).
556 @end ifinfo
557 If you do not specify a choice, the default is Latin-1.
558
559 If you insert a character in the range 0200 through 0237, which
560 forms the @code{eight-bit-control} character set, it is inserted
561 literally. You should normally avoid doing this since buffers
562 containing such characters have to be written out in either the
563 @code{emacs-mule} or @code{raw-text} coding system, which is usually
564 not what you want.
565
566 @node Coding Systems
567 @section Coding Systems
568 @cindex coding systems
569
570 Users of various languages have established many more-or-less standard
571 coding systems for representing them. Emacs does not use these coding
572 systems internally; instead, it converts from various coding systems to
573 its own system when reading data, and converts the internal coding
574 system to other coding systems when writing data. Conversion is
575 possible in reading or writing files, in sending or receiving from the
576 terminal, and in exchanging data with subprocesses.
577
578 Emacs assigns a name to each coding system. Most coding systems are
579 used for one language, and the name of the coding system starts with the
580 language name. Some coding systems are used for several languages;
581 their names usually start with @samp{iso}. There are also special
582 coding systems @code{no-conversion}, @code{raw-text} and
583 @code{emacs-mule} which do not convert printing characters at all.
584
585 @cindex international files from DOS/Windows systems
586 A special class of coding systems, collectively known as
587 @dfn{codepages}, is designed to support text encoded by MS-Windows and
588 MS-DOS software. To use any of these systems, you need to create it
589 with @kbd{M-x codepage-setup}. @xref{MS-DOS and MULE}. After
590 creating the coding system for the codepage, you can use it as any
591 other coding system. For example, to visit a file encoded in codepage
592 850, type @kbd{C-x @key{RET} c cp850 @key{RET} C-x C-f @var{filename}
593 @key{RET}}.
594
595 In addition to converting various representations of non-@acronym{ASCII}
596 characters, a coding system can perform end-of-line conversion. Emacs
597 handles three different conventions for how to separate lines in a file:
598 newline, carriage-return linefeed, and just carriage-return.
599
600 @table @kbd
601 @item C-h C @var{coding} @key{RET}
602 Describe coding system @var{coding}.
603
604 @item C-h C @key{RET}
605 Describe the coding systems currently in use.
606
607 @item M-x list-coding-systems
608 Display a list of all the supported coding systems.
609 @end table
610
611 @kindex C-h C
612 @findex describe-coding-system
613 The command @kbd{C-h C} (@code{describe-coding-system}) displays
614 information about particular coding systems. You can specify a coding
615 system name as the argument; alternatively, with an empty argument, it
616 describes the coding systems currently selected for various purposes,
617 both in the current buffer and as the defaults, and the priority list
618 for recognizing coding systems (@pxref{Recognize Coding}).
619
620 @findex list-coding-systems
621 To display a list of all the supported coding systems, type @kbd{M-x
622 list-coding-systems}. The list gives information about each coding
623 system, including the letter that stands for it in the mode line
624 (@pxref{Mode Line}).
625
626 @cindex end-of-line conversion
627 @cindex MS-DOS end-of-line conversion
628 @cindex Macintosh end-of-line conversion
629 Each of the coding systems that appear in this list---except for
630 @code{no-conversion}, which means no conversion of any kind---specifies
631 how and whether to convert printing characters, but leaves the choice of
632 end-of-line conversion to be decided based on the contents of each file.
633 For example, if the file appears to use the sequence carriage-return
634 linefeed to separate lines, DOS end-of-line conversion will be used.
635
636 Each of the listed coding systems has three variants which specify
637 exactly what to do for end-of-line conversion:
638
639 @table @code
640 @item @dots{}-unix
641 Don't do any end-of-line conversion; assume the file uses
642 newline to separate lines. (This is the convention normally used
643 on Unix and GNU systems.)
644
645 @item @dots{}-dos
646 Assume the file uses carriage-return linefeed to separate lines, and do
647 the appropriate conversion. (This is the convention normally used on
648 Microsoft systems.@footnote{It is also specified for MIME @samp{text/*}
649 bodies and in other network transport contexts. It is different
650 from the SGML reference syntax record-start/record-end format which
651 Emacs doesn't support directly.})
652
653 @item @dots{}-mac
654 Assume the file uses carriage-return to separate lines, and do the
655 appropriate conversion. (This is the convention normally used on the
656 Macintosh system.)
657 @end table
658
659 These variant coding systems are omitted from the
660 @code{list-coding-systems} display for brevity, since they are entirely
661 predictable. For example, the coding system @code{iso-latin-1} has
662 variants @code{iso-latin-1-unix}, @code{iso-latin-1-dos} and
663 @code{iso-latin-1-mac}.
664
665 The coding system @code{raw-text} is good for a file which is mainly
666 @acronym{ASCII} text, but may contain byte values above 127 which are not meant to
667 encode non-@acronym{ASCII} characters. With @code{raw-text}, Emacs copies those
668 byte values unchanged, and sets @code{enable-multibyte-characters} to
669 @code{nil} in the current buffer so that they will be interpreted
670 properly. @code{raw-text} handles end-of-line conversion in the usual
671 way, based on the data encountered, and has the usual three variants to
672 specify the kind of end-of-line conversion to use.
673
674 In contrast, the coding system @code{no-conversion} specifies no
675 character code conversion at all---none for non-@acronym{ASCII} byte values and
676 none for end of line. This is useful for reading or writing binary
677 files, tar files, and other files that must be examined verbatim. It,
678 too, sets @code{enable-multibyte-characters} to @code{nil}.
679
680 The easiest way to edit a file with no conversion of any kind is with
681 the @kbd{M-x find-file-literally} command. This uses
682 @code{no-conversion}, and also suppresses other Emacs features that
683 might convert the file contents before you see them. @xref{Visiting}.
684
685 The coding system @code{emacs-mule} means that the file contains
686 non-@acronym{ASCII} characters stored with the internal Emacs encoding. It
687 handles end-of-line conversion based on the data encountered, and has
688 the usual three variants to specify the kind of end-of-line conversion.
689
690 @node Recognize Coding
691 @section Recognizing Coding Systems
692
693 Emacs tries to recognize which coding system to use for a given text
694 as an integral part of reading that text. (This applies to files
695 being read, output from subprocesses, text from X selections, etc.)
696 Emacs can select the right coding system automatically most of the
697 time---once you have specified your preferences.
698
699 Some coding systems can be recognized or distinguished by which byte
700 sequences appear in the data. However, there are coding systems that
701 cannot be distinguished, not even potentially. For example, there is no
702 way to distinguish between Latin-1 and Latin-2; they use the same byte
703 values with different meanings.
704
705 Emacs handles this situation by means of a priority list of coding
706 systems. Whenever Emacs reads a file, if you do not specify the coding
707 system to use, Emacs checks the data against each coding system,
708 starting with the first in priority and working down the list, until it
709 finds a coding system that fits the data. Then it converts the file
710 contents assuming that they are represented in this coding system.
711
712 The priority list of coding systems depends on the selected language
713 environment (@pxref{Language Environments}). For example, if you use
714 French, you probably want Emacs to prefer Latin-1 to Latin-2; if you use
715 Czech, you probably want Latin-2 to be preferred. This is one of the
716 reasons to specify a language environment.
717
718 @findex prefer-coding-system
719 However, you can alter the coding system priority list in detail
720 with the command @kbd{M-x prefer-coding-system}. This command reads
721 the name of a coding system from the minibuffer, and adds it to the
722 front of the priority list, so that it is preferred to all others. If
723 you use this command several times, each use adds one element to the
724 front of the priority list.
725
726 If you use a coding system that specifies the end-of-line conversion
727 type, such as @code{iso-8859-1-dos}, what this means is that Emacs
728 should attempt to recognize @code{iso-8859-1} with priority, and should
729 use DOS end-of-line conversion when it does recognize @code{iso-8859-1}.
730
731 @vindex file-coding-system-alist
732 Sometimes a file name indicates which coding system to use for the
733 file. The variable @code{file-coding-system-alist} specifies this
734 correspondence. There is a special function
735 @code{modify-coding-system-alist} for adding elements to this list. For
736 example, to read and write all @samp{.txt} files using the coding system
737 @code{china-iso-8bit}, you can execute this Lisp expression:
738
739 @smallexample
740 (modify-coding-system-alist 'file "\\.txt\\'" 'china-iso-8bit)
741 @end smallexample
742
743 @noindent
744 The first argument should be @code{file}, the second argument should be
745 a regular expression that determines which files this applies to, and
746 the third argument says which coding system to use for these files.
747
748 @vindex inhibit-eol-conversion
749 @cindex DOS-style end-of-line display
750 Emacs recognizes which kind of end-of-line conversion to use based on
751 the contents of the file: if it sees only carriage-returns, or only
752 carriage-return linefeed sequences, then it chooses the end-of-line
753 conversion accordingly. You can inhibit the automatic use of
754 end-of-line conversion by setting the variable @code{inhibit-eol-conversion}
755 to non-@code{nil}. If you do that, DOS-style files will be displayed
756 with the @samp{^M} characters visible in the buffer; some people
757 prefer this to the more subtle @samp{(DOS)} end-of-line type
758 indication near the left edge of the mode line (@pxref{Mode Line,
759 eol-mnemonic}).
760
761 @vindex inhibit-iso-escape-detection
762 @cindex escape sequences in files
763 By default, the automatic detection of coding system is sensitive to
764 escape sequences. If Emacs sees a sequence of characters that begin
765 with an escape character, and the sequence is valid as an ISO-2022
766 code, that tells Emacs to use one of the ISO-2022 encodings to decode
767 the file.
768
769 However, there may be cases that you want to read escape sequences
770 in a file as is. In such a case, you can set the variable
771 @code{inhibit-iso-escape-detection} to non-@code{nil}. Then the code
772 detection ignores any escape sequences, and never uses an ISO-2022
773 encoding. The result is that all escape sequences become visible in
774 the buffer.
775
776 The default value of @code{inhibit-iso-escape-detection} is
777 @code{nil}. We recommend that you not change it permanently, only for
778 one specific operation. That's because many Emacs Lisp source files
779 in the Emacs distribution contain non-@acronym{ASCII} characters encoded in the
780 coding system @code{iso-2022-7bit}, and they won't be
781 decoded correctly when you visit those files if you suppress the
782 escape sequence detection.
783
784 @vindex coding
785 You can specify the coding system for a particular file using the
786 @w{@samp{-*-@dots{}-*-}} construct at the beginning of a file, or a
787 local variables list at the end (@pxref{File Variables}). You do this
788 by defining a value for the ``variable'' named @code{coding}. Emacs
789 does not really have a variable @code{coding}; instead of setting a
790 variable, this uses the specified coding system for the file. For
791 example, @samp{-*-mode: C; coding: latin-1;-*-} specifies use of the
792 Latin-1 coding system, as well as C mode. When you specify the coding
793 explicitly in the file, that overrides
794 @code{file-coding-system-alist}.
795
796 @vindex auto-coding-alist
797 @vindex auto-coding-regexp-alist
798 @vindex auto-coding-functions
799 The variables @code{auto-coding-alist},
800 @code{auto-coding-regexp-alist} and @code{auto-coding-functions} are
801 the strongest way to specify the coding system for certain patterns of
802 file names, or for files containing certain patterns; these variables
803 even override @samp{-*-coding:-*-} tags in the file itself. Emacs
804 uses @code{auto-coding-alist} for tar and archive files, to prevent it
805 from being confused by a @samp{-*-coding:-*-} tag in a member of the
806 archive and thinking it applies to the archive file as a whole.
807 Likewise, Emacs uses @code{auto-coding-regexp-alist} to ensure that
808 RMAIL files, whose names in general don't match any particular
809 pattern, are decoded correctly. One of the builtin
810 @code{auto-coding-functions} detects the encoding for XML files.
811
812 If Emacs recognizes the encoding of a file incorrectly, you can
813 reread the file using the correct coding system by typing @kbd{C-x
814 @key{RET} c @var{coding-system} @key{RET} M-x revert-buffer
815 @key{RET}}. To see what coding system Emacs actually used to decode
816 the file, look at the coding system mnemonic letter near the left edge
817 of the mode line (@pxref{Mode Line}), or type @kbd{C-h C @key{RET}}.
818
819 @findex unify-8859-on-decoding-mode
820 The command @code{unify-8859-on-decoding-mode} enables a mode that
821 ``unifies'' the Latin alphabets when decoding text. This works by
822 converting all non-@acronym{ASCII} Latin-@var{n} characters to either Latin-1 or
823 Unicode characters. This way it is easier to use various
824 Latin-@var{n} alphabets together. In a future Emacs version we hope
825 to move towards full Unicode support and complete unification of
826 character sets.
827
828 @vindex buffer-file-coding-system
829 Once Emacs has chosen a coding system for a buffer, it stores that
830 coding system in @code{buffer-file-coding-system} and uses that coding
831 system, by default, for operations that write from this buffer into a
832 file. This includes the commands @code{save-buffer} and
833 @code{write-region}. If you want to write files from this buffer using
834 a different coding system, you can specify a different coding system for
835 the buffer using @code{set-buffer-file-coding-system} (@pxref{Specify
836 Coding}).
837
838 You can insert any possible character into any Emacs buffer, but
839 most coding systems can only handle some of the possible characters.
840 This means that it is possible for you to insert characters that
841 cannot be encoded with the coding system that will be used to save the
842 buffer. For example, you could start with an @acronym{ASCII} file and insert a
843 few Latin-1 characters into it, or you could edit a text file in
844 Polish encoded in @code{iso-8859-2} and add some Russian words to it.
845 When you save the buffer, Emacs cannot use the current value of
846 @code{buffer-file-coding-system}, because the characters you added
847 cannot be encoded by that coding system.
848
849 When that happens, Emacs tries the most-preferred coding system (set
850 by @kbd{M-x prefer-coding-system} or @kbd{M-x
851 set-language-environment}), and if that coding system can safely
852 encode all of the characters in the buffer, Emacs uses it, and stores
853 its value in @code{buffer-file-coding-system}. Otherwise, Emacs
854 displays a list of coding systems suitable for encoding the buffer's
855 contents, and asks you to choose one of those coding systems.
856
857 If you insert the unsuitable characters in a mail message, Emacs
858 behaves a bit differently. It additionally checks whether the
859 most-preferred coding system is recommended for use in MIME messages;
860 if not, Emacs tells you that the most-preferred coding system is
861 not recommended and prompts you for another coding system. This is so
862 you won't inadvertently send a message encoded in a way that your
863 recipient's mail software will have difficulty decoding. (If you do
864 want to use the most-preferred coding system, you can still type its
865 name in response to the question.)
866
867 @vindex sendmail-coding-system
868 When you send a message with Mail mode (@pxref{Sending Mail}), Emacs has
869 four different ways to determine the coding system to use for encoding
870 the message text. It tries the buffer's own value of
871 @code{buffer-file-coding-system}, if that is non-@code{nil}. Otherwise,
872 it uses the value of @code{sendmail-coding-system}, if that is
873 non-@code{nil}. The third way is to use the default coding system for
874 new files, which is controlled by your choice of language environment,
875 if that is non-@code{nil}. If all of these three values are @code{nil},
876 Emacs encodes outgoing mail using the Latin-1 coding system.
877
878 @vindex rmail-decode-mime-charset
879 When you get new mail in Rmail, each message is translated
880 automatically from the coding system it is written in, as if it were a
881 separate file. This uses the priority list of coding systems that you
882 have specified. If a MIME message specifies a character set, Rmail
883 obeys that specification, unless @code{rmail-decode-mime-charset} is
884 @code{nil}.
885
886 @vindex rmail-file-coding-system
887 For reading and saving Rmail files themselves, Emacs uses the coding
888 system specified by the variable @code{rmail-file-coding-system}. The
889 default value is @code{nil}, which means that Rmail files are not
890 translated (they are read and written in the Emacs internal character
891 code).
892
893 @node Specify Coding
894 @section Specifying a Coding System
895
896 In cases where Emacs does not automatically choose the right coding
897 system, you can use these commands to specify one:
898
899 @table @kbd
900 @item C-x @key{RET} f @var{coding} @key{RET}
901 Use coding system @var{coding} for saving or revisiting the visited
902 file in the current buffer.
903
904 @item C-x @key{RET} c @var{coding} @key{RET}
905 Specify coding system @var{coding} for the immediately following
906 command.
907
908 @item C-x @key{RET} r @var{coding} @key{RET}
909 Revisit the current file using the coding system @var{coding}.
910
911 @item C-x @key{RET} k @var{coding} @key{RET}
912 Use coding system @var{coding} for keyboard input.
913
914 @item C-x @key{RET} t @var{coding} @key{RET}
915 Use coding system @var{coding} for terminal output.
916
917 @item C-x @key{RET} p @var{input-coding} @key{RET} @var{output-coding} @key{RET}
918 Use coding systems @var{input-coding} and @var{output-coding} for
919 subprocess input and output in the current buffer.
920
921 @item C-x @key{RET} x @var{coding} @key{RET}
922 Use coding system @var{coding} for transferring selections to and from
923 other programs through the window system.
924
925 @item C-x @key{RET} F @var{coding} @key{RET}
926 Use coding system @var{coding} for encoding and decoding file
927 @emph{names}. This affects the use of non-ASCII characters in file
928 names. It has no effect on reading and writing the @emph{contents} of
929 files.
930
931 @item C-x @key{RET} X @var{coding} @key{RET}
932 Use coding system @var{coding} for transferring @emph{one}
933 selection---the next one---to or from the window system.
934
935 @item M-x recode-region
936 Convert the region from a previous coding system to a new one.
937 @end table
938
939 @kindex C-x RET f
940 @findex set-buffer-file-coding-system
941 The command @kbd{C-x @key{RET} f}
942 (@code{set-buffer-file-coding-system}) sets the file coding system for
943 the current buffer---in other words, it says which coding system to
944 use when saving or reverting the visited file. You specify which
945 coding system using the minibuffer. If you specify a coding system
946 that cannot handle all of the characters in the buffer, Emacs warns
947 you about the troublesome characters when you actually save the
948 buffer.
949
950 @kindex C-x RET c
951 @findex universal-coding-system-argument
952 Another way to specify the coding system for a file is when you visit
953 the file. First use the command @kbd{C-x @key{RET} c}
954 (@code{universal-coding-system-argument}); this command uses the
955 minibuffer to read a coding system name. After you exit the minibuffer,
956 the specified coding system is used for @emph{the immediately following
957 command}.
958
959 So if the immediately following command is @kbd{C-x C-f}, for example,
960 it reads the file using that coding system (and records the coding
961 system for when you later save the file). Or if the immediately following
962 command is @kbd{C-x C-w}, it writes the file using that coding system.
963 When you specify the coding system for saving in this way, instead
964 of with @kbd{C-x @key{RET} f}, there is no warning if the buffer
965 contains characters that the coding system cannot handle.
966
967 Other file commands affected by a specified coding system include
968 @kbd{C-x C-i} and @kbd{C-x C-v}, as well as the other-window variants
969 of @kbd{C-x C-f}. @kbd{C-x @key{RET} c} also affects commands that
970 start subprocesses, including @kbd{M-x shell} (@pxref{Shell}).
971
972 If the immediately following command does not use the coding system,
973 then @kbd{C-x @key{RET} c} ultimately has no effect.
974
975 An easy way to visit a file with no conversion is with the @kbd{M-x
976 find-file-literally} command. @xref{Visiting}.
977
978 @vindex default-buffer-file-coding-system
979 The variable @code{default-buffer-file-coding-system} specifies the
980 choice of coding system to use when you create a new file. It applies
981 when you find a new file, and when you create a buffer and then save it
982 in a file. Selecting a language environment typically sets this
983 variable to a good choice of default coding system for that language
984 environment.
985
986 @kindex C-x RET r
987 @findex revert-buffer-with-coding-system
988 If you visit a file with a wrong coding system, you can correct this
989 with @kbd{C-x @key{RET} r} (@code{revert-buffer-with-coding-system}).
990 This visits the current file again, using a coding system you specify.
991
992 @kindex C-x RET t
993 @findex set-terminal-coding-system
994 The command @kbd{C-x @key{RET} t} (@code{set-terminal-coding-system})
995 specifies the coding system for terminal output. If you specify a
996 character code for terminal output, all characters output to the
997 terminal are translated into that coding system.
998
999 This feature is useful for certain character-only terminals built to
1000 support specific languages or character sets---for example, European
1001 terminals that support one of the ISO Latin character sets. You need to
1002 specify the terminal coding system when using multibyte text, so that
1003 Emacs knows which characters the terminal can actually handle.
1004
1005 By default, output to the terminal is not translated at all, unless
1006 Emacs can deduce the proper coding system from your terminal type or
1007 your locale specification (@pxref{Language Environments}).
1008
1009 @kindex C-x RET k
1010 @findex set-keyboard-coding-system
1011 @vindex keyboard-coding-system
1012 The command @kbd{C-x @key{RET} k} (@code{set-keyboard-coding-system})
1013 or the variable @code{keyboard-coding-system} specifies the coding
1014 system for keyboard input. Character-code translation of keyboard
1015 input is useful for terminals with keys that send non-@acronym{ASCII}
1016 graphic characters---for example, some terminals designed for ISO
1017 Latin-1 or subsets of it.
1018
1019 By default, keyboard input is translated based on your system locale
1020 setting. If your terminal does not really support the encoding
1021 implied by your locale (for example, if you find it inserts a
1022 non-@acronym{ASCII} character if you type @kbd{M-i}), you will need to set
1023 @code{keyboard-coding-system} to @code{nil} to turn off encoding.
1024 You can do this by putting
1025
1026 @lisp
1027 (set-keyboard-coding-system nil)
1028 @end lisp
1029
1030 @noindent
1031 in your @file{~/.emacs} file.
1032
1033 There is a similarity between using a coding system translation for
1034 keyboard input, and using an input method: both define sequences of
1035 keyboard input that translate into single characters. However, input
1036 methods are designed to be convenient for interactive use by humans, and
1037 the sequences that are translated are typically sequences of @acronym{ASCII}
1038 printing characters. Coding systems typically translate sequences of
1039 non-graphic characters.
1040
1041 @kindex C-x RET x
1042 @kindex C-x RET X
1043 @findex set-selection-coding-system
1044 @findex set-next-selection-coding-system
1045 The command @kbd{C-x @key{RET} x} (@code{set-selection-coding-system})
1046 specifies the coding system for sending selected text to the window
1047 system, and for receiving the text of selections made in other
1048 applications. This command applies to all subsequent selections, until
1049 you override it by using the command again. The command @kbd{C-x
1050 @key{RET} X} (@code{set-next-selection-coding-system}) specifies the
1051 coding system for the next selection made in Emacs or read by Emacs.
1052
1053 @kindex C-x RET p
1054 @findex set-buffer-process-coding-system
1055 The command @kbd{C-x @key{RET} p} (@code{set-buffer-process-coding-system})
1056 specifies the coding system for input and output to a subprocess. This
1057 command applies to the current buffer; normally, each subprocess has its
1058 own buffer, and thus you can use this command to specify translation to
1059 and from a particular subprocess by giving the command in the
1060 corresponding buffer.
1061
1062 The default for translation of process input and output depends on the
1063 current language environment.
1064
1065 @findex recode-region
1066 If a piece of text has already been inserted into a buffer using the
1067 wrong coding system, you can decode it again using @kbd{M-x
1068 recode-region}. This prompts you for the old coding system and the
1069 desired coding system, and acts on the text in the region.
1070
1071 @vindex file-name-coding-system
1072 @cindex file names with non-@acronym{ASCII} characters
1073 @findex set-file-name-coding-system
1074 @kindex C-x @key{RET} F
1075 The variable @code{file-name-coding-system} specifies a coding
1076 system to use for encoding file names. If you set the variable to a
1077 coding system name (as a Lisp symbol or a string), Emacs encodes file
1078 names using that coding system for all file operations. This makes it
1079 possible to use non-@acronym{ASCII} characters in file names---or, at
1080 least, those non-@acronym{ASCII} characters which the specified coding
1081 system can encode. Use @kbd{C-x @key{RET} F}
1082 (@code{set-file-name-coding-system}) to specify this interactively.
1083
1084 If @code{file-name-coding-system} is @code{nil}, Emacs uses a default
1085 coding system determined by the selected language environment. In the
1086 default language environment, any non-@acronym{ASCII} characters in file names are
1087 not encoded specially; they appear in the file system using the internal
1088 Emacs representation.
1089
1090 @strong{Warning:} if you change @code{file-name-coding-system} (or the
1091 language environment) in the middle of an Emacs session, problems can
1092 result if you have already visited files whose names were encoded using
1093 the earlier coding system and cannot be encoded (or are encoded
1094 differently) under the new coding system. If you try to save one of
1095 these buffers under the visited file name, saving may use the wrong file
1096 name, or it may get an error. If such a problem happens, use @kbd{C-x
1097 C-w} to specify a new file name for that buffer.
1098
1099 @findex recode-file-name
1100 If a mistake occurs when encoding a file name, use the command
1101 command @kbd{M-x recode-file-name} to change the file name's coding
1102 system. This prompts for an existing file name, its old coding
1103 system, and the coding system to which you wish to convert.
1104
1105 @vindex locale-coding-system
1106 @cindex decoding non-@acronym{ASCII} keyboard input on X
1107 The variable @code{locale-coding-system} specifies a coding system
1108 to use when encoding and decoding system strings such as system error
1109 messages and @code{format-time-string} formats and time stamps. That
1110 coding system is also used for decoding non-@acronym{ASCII} keyboard input on X
1111 Window systems. You should choose a coding system that is compatible
1112 with the underlying system's text representation, which is normally
1113 specified by one of the environment variables @env{LC_ALL},
1114 @env{LC_CTYPE}, and @env{LANG}. (The first one, in the order
1115 specified above, whose value is nonempty is the one that determines
1116 the text representation.)
1117
1118 @node Fontsets
1119 @section Fontsets
1120 @cindex fontsets
1121
1122 A font for X typically defines shapes for a single alphabet or script.
1123 Therefore, displaying the entire range of scripts that Emacs supports
1124 requires a collection of many fonts. In Emacs, such a collection is
1125 called a @dfn{fontset}. A fontset is defined by a list of fonts, each
1126 assigned to handle a range of character codes.
1127
1128 Each fontset has a name, like a font. The available X fonts are
1129 defined by the X server; fontsets, however, are defined within Emacs
1130 itself. Once you have defined a fontset, you can use it within Emacs by
1131 specifying its name, anywhere that you could use a single font. Of
1132 course, Emacs fontsets can use only the fonts that the X server
1133 supports; if certain characters appear on the screen as hollow boxes,
1134 this means that the fontset in use for them has no font for those
1135 characters.@footnote{The Emacs installation instructions have information on
1136 additional font support.}
1137
1138 Emacs creates two fontsets automatically: the @dfn{standard fontset}
1139 and the @dfn{startup fontset}. The standard fontset is most likely to
1140 have fonts for a wide variety of non-@acronym{ASCII} characters; however, this is
1141 not the default for Emacs to use. (By default, Emacs tries to find a
1142 font that has bold and italic variants.) You can specify use of the
1143 standard fontset with the @samp{-fn} option, or with the @samp{Font} X
1144 resource (@pxref{Font X}). For example,
1145
1146 @example
1147 emacs -fn fontset-standard
1148 @end example
1149
1150 A fontset does not necessarily specify a font for every character
1151 code. If a fontset specifies no font for a certain character, or if it
1152 specifies a font that does not exist on your system, then it cannot
1153 display that character properly. It will display that character as an
1154 empty box instead.
1155
1156 @node Defining Fontsets
1157 @section Defining fontsets
1158
1159 @vindex standard-fontset-spec
1160 @cindex standard fontset
1161 Emacs creates a standard fontset automatically according to the value
1162 of @code{standard-fontset-spec}. This fontset's name is
1163
1164 @example
1165 -*-fixed-medium-r-normal-*-16-*-*-*-*-*-fontset-standard
1166 @end example
1167
1168 @noindent
1169 or just @samp{fontset-standard} for short.
1170
1171 Bold, italic, and bold-italic variants of the standard fontset are
1172 created automatically. Their names have @samp{bold} instead of
1173 @samp{medium}, or @samp{i} instead of @samp{r}, or both.
1174
1175 @cindex startup fontset
1176 If you specify a default @acronym{ASCII} font with the @samp{Font} resource or
1177 the @samp{-fn} argument, Emacs generates a fontset from it
1178 automatically. This is the @dfn{startup fontset} and its name is
1179 @code{fontset-startup}. It does this by replacing the @var{foundry},
1180 @var{family}, @var{add_style}, and @var{average_width} fields of the
1181 font name with @samp{*}, replacing @var{charset_registry} field with
1182 @samp{fontset}, and replacing @var{charset_encoding} field with
1183 @samp{startup}, then using the resulting string to specify a fontset.
1184
1185 For instance, if you start Emacs this way,
1186
1187 @example
1188 emacs -fn "*courier-medium-r-normal--14-140-*-iso8859-1"
1189 @end example
1190
1191 @noindent
1192 Emacs generates the following fontset and uses it for the initial X
1193 window frame:
1194
1195 @example
1196 -*-*-medium-r-normal-*-14-140-*-*-*-*-fontset-startup
1197 @end example
1198
1199 With the X resource @samp{Emacs.Font}, you can specify a fontset name
1200 just like an actual font name. But be careful not to specify a fontset
1201 name in a wildcard resource like @samp{Emacs*Font}---that wildcard
1202 specification matches various other resources, such as for menus, and
1203 menus cannot handle fontsets.
1204
1205 You can specify additional fontsets using X resources named
1206 @samp{Fontset-@var{n}}, where @var{n} is an integer starting from 0.
1207 The resource value should have this form:
1208
1209 @smallexample
1210 @var{fontpattern}, @r{[}@var{charsetname}:@var{fontname}@r{]@dots{}}
1211 @end smallexample
1212
1213 @noindent
1214 @var{fontpattern} should have the form of a standard X font name, except
1215 for the last two fields. They should have the form
1216 @samp{fontset-@var{alias}}.
1217
1218 The fontset has two names, one long and one short. The long name is
1219 @var{fontpattern}. The short name is @samp{fontset-@var{alias}}. You
1220 can refer to the fontset by either name.
1221
1222 The construct @samp{@var{charset}:@var{font}} specifies which font to
1223 use (in this fontset) for one particular character set. Here,
1224 @var{charset} is the name of a character set, and @var{font} is the
1225 font to use for that character set. You can use this construct any
1226 number of times in defining one fontset.
1227
1228 For the other character sets, Emacs chooses a font based on
1229 @var{fontpattern}. It replaces @samp{fontset-@var{alias}} with values
1230 that describe the character set. For the @acronym{ASCII} character font,
1231 @samp{fontset-@var{alias}} is replaced with @samp{ISO8859-1}.
1232
1233 In addition, when several consecutive fields are wildcards, Emacs
1234 collapses them into a single wildcard. This is to prevent use of
1235 auto-scaled fonts. Fonts made by scaling larger fonts are not usable
1236 for editing, and scaling a smaller font is not useful because it is
1237 better to use the smaller font in its own size, which is what Emacs
1238 does.
1239
1240 Thus if @var{fontpattern} is this,
1241
1242 @example
1243 -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24
1244 @end example
1245
1246 @noindent
1247 the font specification for @acronym{ASCII} characters would be this:
1248
1249 @example
1250 -*-fixed-medium-r-normal-*-24-*-ISO8859-1
1251 @end example
1252
1253 @noindent
1254 and the font specification for Chinese GB2312 characters would be this:
1255
1256 @example
1257 -*-fixed-medium-r-normal-*-24-*-gb2312*-*
1258 @end example
1259
1260 You may not have any Chinese font matching the above font
1261 specification. Most X distributions include only Chinese fonts that
1262 have @samp{song ti} or @samp{fangsong ti} in @var{family} field. In
1263 such a case, @samp{Fontset-@var{n}} can be specified as below:
1264
1265 @smallexample
1266 Emacs.Fontset-0: -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24,\
1267 chinese-gb2312:-*-*-medium-r-normal-*-24-*-gb2312*-*
1268 @end smallexample
1269
1270 @noindent
1271 Then, the font specifications for all but Chinese GB2312 characters have
1272 @samp{fixed} in the @var{family} field, and the font specification for
1273 Chinese GB2312 characters has a wild card @samp{*} in the @var{family}
1274 field.
1275
1276 @findex create-fontset-from-fontset-spec
1277 The function that processes the fontset resource value to create the
1278 fontset is called @code{create-fontset-from-fontset-spec}. You can also
1279 call this function explicitly to create a fontset.
1280
1281 @xref{Font X}, for more information about font naming in X.
1282
1283 @node Undisplayable Characters
1284 @section Undisplayable Characters
1285
1286 There may be a some non-@acronym{ASCII} characters that your terminal cannot
1287 display. Most non-windowing terminals support just a single character
1288 set (use the variable @code{default-terminal-coding-system}
1289 (@pxref{Specify Coding}) to tell Emacs which one); characters which
1290 can't be encoded in that coding system are displayed as @samp{?} by
1291 default.
1292
1293 Windowing terminals can display a broader range of characters, but
1294 you may not have fonts installed for all of them; characters that have
1295 no font appear as a hollow box.
1296
1297 If you use Latin-1 characters but your terminal can't display
1298 Latin-1, you can arrange to display mnemonic @acronym{ASCII} sequences
1299 instead, e.g.@: @samp{"o} for o-umlaut. Load the library
1300 @file{iso-ascii} to do this.
1301
1302 @vindex latin1-display
1303 If your terminal can display Latin-1, you can display characters
1304 from other European character sets using a mixture of equivalent
1305 Latin-1 characters and @acronym{ASCII} mnemonics. Customize the variable
1306 @code{latin1-display} to enable this. The mnemonic @acronym{ASCII}
1307 sequences mostly correspond to those of the prefix input methods.
1308
1309 @node Single-Byte Character Support
1310 @section Single-byte Character Set Support
1311
1312 @cindex European character sets
1313 @cindex accented characters
1314 @cindex ISO Latin character sets
1315 @cindex Unibyte operation
1316 The ISO 8859 Latin-@var{n} character sets define character codes in
1317 the range 0240 to 0377 octal (160 to 255 decimal) to handle the
1318 accented letters and punctuation needed by various European languages
1319 (and some non-European ones). If you disable multibyte characters,
1320 Emacs can still handle @emph{one} of these character codes at a time.
1321 To specify @emph{which} of these codes to use, invoke @kbd{M-x
1322 set-language-environment} and specify a suitable language environment
1323 such as @samp{Latin-@var{n}}.
1324
1325 For more information about unibyte operation, see @ref{Enabling
1326 Multibyte}. Note particularly that you probably want to ensure that
1327 your initialization files are read as unibyte if they contain non-@acronym{ASCII}
1328 characters.
1329
1330 @vindex unibyte-display-via-language-environment
1331 Emacs can also display those characters, provided the terminal or font
1332 in use supports them. This works automatically. Alternatively, if you
1333 are using a window system, Emacs can also display single-byte characters
1334 through fontsets, in effect by displaying the equivalent multibyte
1335 characters according to the current language environment. To request
1336 this, set the variable @code{unibyte-display-via-language-environment}
1337 to a non-@code{nil} value.
1338
1339 @cindex @code{iso-ascii} library
1340 If your terminal does not support display of the Latin-1 character
1341 set, Emacs can display these characters as @acronym{ASCII} sequences which at
1342 least give you a clear idea of what the characters are. To do this,
1343 load the library @code{iso-ascii}. Similar libraries for other
1344 Latin-@var{n} character sets could be implemented, but we don't have
1345 them yet.
1346
1347 @findex standard-display-8bit
1348 @cindex 8-bit display
1349 Normally non-ISO-8859 characters (decimal codes between 128 and 159
1350 inclusive) are displayed as octal escapes. You can change this for
1351 non-standard ``extended'' versions of ISO-8859 character sets by using the
1352 function @code{standard-display-8bit} in the @code{disp-table} library.
1353
1354 There are two ways to input single-byte non-@acronym{ASCII}
1355 characters:
1356
1357 @itemize @bullet
1358 @cindex 8-bit input
1359 @item
1360 You can use an input method for the selected language environment.
1361 @xref{Input Methods}. When you use an input method in a unibyte buffer,
1362 the non-@acronym{ASCII} character you specify with it is converted to unibyte.
1363
1364 @item
1365 If your keyboard can generate character codes 128 (decimal) and up,
1366 representing non-@acronym{ASCII} characters, you can type those character codes
1367 directly.
1368
1369 On a window system, you should not need to do anything special to use
1370 these keys; they should simply work. On a text-only terminal, you
1371 should use the command @code{M-x set-keyboard-coding-system} or the
1372 variable @code{keyboard-coding-system} to specify which coding system
1373 your keyboard uses (@pxref{Specify Coding}). Enabling this feature
1374 will probably require you to use @kbd{ESC} to type Meta characters;
1375 however, on a console terminal or in @code{xterm}, you can arrange for
1376 Meta to be converted to @kbd{ESC} and still be able type 8-bit
1377 characters present directly on the keyboard or using @kbd{Compose} or
1378 @kbd{AltGr} keys. @xref{User Input}.
1379
1380 @kindex C-x 8
1381 @cindex @code{iso-transl} library
1382 @cindex compose character
1383 @cindex dead character
1384 @item
1385 For Latin-1 only, you can use the key @kbd{C-x 8} as a ``compose
1386 character'' prefix for entry of non-@acronym{ASCII} Latin-1 printing
1387 characters. @kbd{C-x 8} is good for insertion (in the minibuffer as
1388 well as other buffers), for searching, and in any other context where
1389 a key sequence is allowed.
1390
1391 @kbd{C-x 8} works by loading the @code{iso-transl} library. Once that
1392 library is loaded, the @key{ALT} modifier key, if the keyboard has
1393 one, serves the same purpose as @kbd{C-x 8}: use @key{ALT} together
1394 with an accent character to modify the following letter. In addition,
1395 if the keyboard has keys for the Latin-1 ``dead accent characters,''
1396 they too are defined to compose with the following character, once
1397 @code{iso-transl} is loaded.
1398
1399 Use @kbd{C-x 8 C-h} to list all the available @kbd{C-x 8} translations.
1400 @end itemize
1401
1402 @node Charsets
1403 @section Charsets
1404 @cindex charsets
1405
1406 Emacs groups all supported characters into disjoint @dfn{charsets}.
1407 Each character code belongs to one and only one charset. For
1408 historical reasons, Emacs typically divides an 8-bit character code
1409 for an extended version of @acronym{ASCII} into two charsets: @acronym{ASCII}, which
1410 covers the codes 0 through 127, plus another charset which covers the
1411 ``right-hand part'' (the codes 128 and up). For instance, the
1412 characters of Latin-1 include the Emacs charset @code{ascii} plus the
1413 Emacs charset @code{latin-iso8859-1}.
1414
1415 Emacs characters belonging to different charsets may look the same,
1416 but they are still different characters. For example, the letter
1417 @samp{o} with acute accent in charset @code{latin-iso8859-1}, used for
1418 Latin-1, is different from the letter @samp{o} with acute accent in
1419 charset @code{latin-iso8859-2}, used for Latin-2.
1420
1421 @findex list-charset-chars
1422 @cindex characters in a certain charset
1423 @findex describe-character-set
1424 There are two commands for obtaining information about Emacs
1425 charsets. The command @kbd{M-x list-charset-chars} prompts for a name
1426 of a character set, and displays all the characters in that character
1427 set. The command @kbd{M-x describe-character-set} prompts for a
1428 charset name and displays information about that charset, including
1429 its internal representation within Emacs.
1430
1431 To find out which charset a character in the buffer belongs to,
1432 put point before it and type @kbd{C-u C-x =}.
1433
1434 @ignore
1435 arch-tag: 310ba60d-31ef-4ce7-91f1-f282dd57b6b3
1436 @end ignore