(list-buffers-noselect): Append the buffer's process status to its mode name.
[bpt/emacs.git] / lispref / nonascii.texi
1 @c -*-texinfo-*-
2 @c This is part of the GNU Emacs Lisp Reference Manual.
3 @c Copyright (C) 1998, 1999 Free Software Foundation, Inc.
4 @c See the file elisp.texi for copying conditions.
5 @setfilename ../info/characters
6 @node Non-ASCII Characters, Searching and Matching, Text, Top
7 @chapter Non-@acronym{ASCII} Characters
8 @cindex multibyte characters
9 @cindex non-@acronym{ASCII} characters
10
11 This chapter covers the special issues relating to non-@acronym{ASCII}
12 characters and how they are stored in strings and buffers.
13
14 @menu
15 * Text Representations:: Unibyte and multibyte representations
16 * Converting Representations:: Converting unibyte to multibyte and vice versa.
17 * Selecting a Representation:: Treating a byte sequence as unibyte or multi.
18 * Character Codes:: How unibyte and multibyte relate to
19 codes of individual characters.
20 * Character Sets:: The space of possible character codes
21 is divided into various character sets.
22 * Chars and Bytes:: More information about multibyte encodings.
23 * Splitting Characters:: Converting a character to its byte sequence.
24 * Scanning Charsets:: Which character sets are used in a buffer?
25 * Translation of Characters:: Translation tables are used for conversion.
26 * Coding Systems:: Coding systems are conversions for saving files.
27 * Input Methods:: Input methods allow users to enter various
28 non-ASCII characters without special keyboards.
29 * Locales:: Interacting with the POSIX locale.
30 @end menu
31
32 @node Text Representations
33 @section Text Representations
34 @cindex text representations
35
36 Emacs has two @dfn{text representations}---two ways to represent text
37 in a string or buffer. These are called @dfn{unibyte} and
38 @dfn{multibyte}. Each string, and each buffer, uses one of these two
39 representations. For most purposes, you can ignore the issue of
40 representations, because Emacs converts text between them as
41 appropriate. Occasionally in Lisp programming you will need to pay
42 attention to the difference.
43
44 @cindex unibyte text
45 In unibyte representation, each character occupies one byte and
46 therefore the possible character codes range from 0 to 255. Codes 0
47 through 127 are @acronym{ASCII} characters; the codes from 128 through 255
48 are used for one non-@acronym{ASCII} character set (you can choose which
49 character set by setting the variable @code{nonascii-insert-offset}).
50
51 @cindex leading code
52 @cindex multibyte text
53 @cindex trailing codes
54 In multibyte representation, a character may occupy more than one
55 byte, and as a result, the full range of Emacs character codes can be
56 stored. The first byte of a multibyte character is always in the range
57 128 through 159 (octal 0200 through 0237). These values are called
58 @dfn{leading codes}. The second and subsequent bytes of a multibyte
59 character are always in the range 160 through 255 (octal 0240 through
60 0377); these values are @dfn{trailing codes}.
61
62 Some sequences of bytes are not valid in multibyte text: for example,
63 a single isolated byte in the range 128 through 159 is not allowed. But
64 character codes 128 through 159 can appear in multibyte text,
65 represented as two-byte sequences. All the character codes 128 through
66 255 are possible (though slightly abnormal) in multibyte text; they
67 appear in multibyte buffers and strings when you do explicit encoding
68 and decoding (@pxref{Explicit Encoding}).
69
70 In a buffer, the buffer-local value of the variable
71 @code{enable-multibyte-characters} specifies the representation used.
72 The representation for a string is determined and recorded in the string
73 when the string is constructed.
74
75 @defvar enable-multibyte-characters
76 This variable specifies the current buffer's text representation.
77 If it is non-@code{nil}, the buffer contains multibyte text; otherwise,
78 it contains unibyte text.
79
80 You cannot set this variable directly; instead, use the function
81 @code{set-buffer-multibyte} to change a buffer's representation.
82 @end defvar
83
84 @defvar default-enable-multibyte-characters
85 This variable's value is entirely equivalent to @code{(default-value
86 'enable-multibyte-characters)}, and setting this variable changes that
87 default value. Setting the local binding of
88 @code{enable-multibyte-characters} in a specific buffer is not allowed,
89 but changing the default value is supported, and it is a reasonable
90 thing to do, because it has no effect on existing buffers.
91
92 The @samp{--unibyte} command line option does its job by setting the
93 default value to @code{nil} early in startup.
94 @end defvar
95
96 @defun position-bytes position
97 @tindex position-bytes
98 Return the byte-position corresponding to buffer position @var{position}
99 in the current buffer. If @var{position} is out of range, the value
100 is @code{nil}.
101 @end defun
102
103 @defun byte-to-position byte-position
104 @tindex byte-to-position
105 Return the buffer position corresponding to byte-position
106 @var{byte-position} in the current buffer. If @var{byte-position} is
107 out of range, the value is @code{nil}.
108 @end defun
109
110 @defun multibyte-string-p string
111 Return @code{t} if @var{string} is a multibyte string.
112 @end defun
113
114 @node Converting Representations
115 @section Converting Text Representations
116
117 Emacs can convert unibyte text to multibyte; it can also convert
118 multibyte text to unibyte, though this conversion loses information. In
119 general these conversions happen when inserting text into a buffer, or
120 when putting text from several strings together in one string. You can
121 also explicitly convert a string's contents to either representation.
122
123 Emacs chooses the representation for a string based on the text that
124 it is constructed from. The general rule is to convert unibyte text to
125 multibyte text when combining it with other multibyte text, because the
126 multibyte representation is more general and can hold whatever
127 characters the unibyte text has.
128
129 When inserting text into a buffer, Emacs converts the text to the
130 buffer's representation, as specified by
131 @code{enable-multibyte-characters} in that buffer. In particular, when
132 you insert multibyte text into a unibyte buffer, Emacs converts the text
133 to unibyte, even though this conversion cannot in general preserve all
134 the characters that might be in the multibyte text. The other natural
135 alternative, to convert the buffer contents to multibyte, is not
136 acceptable because the buffer's representation is a choice made by the
137 user that cannot be overridden automatically.
138
139 Converting unibyte text to multibyte text leaves @acronym{ASCII} characters
140 unchanged, and likewise character codes 128 through 159. It converts
141 the non-@acronym{ASCII} codes 160 through 255 by adding the value
142 @code{nonascii-insert-offset} to each character code. By setting this
143 variable, you specify which character set the unibyte characters
144 correspond to (@pxref{Character Sets}). For example, if
145 @code{nonascii-insert-offset} is 2048, which is @code{(- (make-char
146 'latin-iso8859-1) 128)}, then the unibyte non-@acronym{ASCII} characters
147 correspond to Latin 1. If it is 2688, which is @code{(- (make-char
148 'greek-iso8859-7) 128)}, then they correspond to Greek letters.
149
150 Converting multibyte text to unibyte is simpler: it discards all but
151 the low 8 bits of each character code. If @code{nonascii-insert-offset}
152 has a reasonable value, corresponding to the beginning of some character
153 set, this conversion is the inverse of the other: converting unibyte
154 text to multibyte and back to unibyte reproduces the original unibyte
155 text.
156
157 @defvar nonascii-insert-offset
158 This variable specifies the amount to add to a non-@acronym{ASCII} character
159 when converting unibyte text to multibyte. It also applies when
160 @code{self-insert-command} inserts a character in the unibyte
161 non-@acronym{ASCII} range, 128 through 255. However, the functions
162 @code{insert} and @code{insert-char} do not perform this conversion.
163
164 The right value to use to select character set @var{cs} is @code{(-
165 (make-char @var{cs}) 128)}. If the value of
166 @code{nonascii-insert-offset} is zero, then conversion actually uses the
167 value for the Latin 1 character set, rather than zero.
168 @end defvar
169
170 @defvar nonascii-translation-table
171 This variable provides a more general alternative to
172 @code{nonascii-insert-offset}. You can use it to specify independently
173 how to translate each code in the range of 128 through 255 into a
174 multibyte character. The value should be a char-table, or @code{nil}.
175 If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}.
176 @end defvar
177
178 The next three functions either return the argument @var{string}, or a
179 newly created string with no text properties.
180
181 @defun string-make-unibyte string
182 This function converts the text of @var{string} to unibyte
183 representation, if it isn't already, and returns the result. If
184 @var{string} is a unibyte string, it is returned unchanged. Multibyte
185 character codes are converted to unibyte according to
186 @code{nonascii-translation-table} or, if that is @code{nil}, using
187 @code{nonascii-insert-offset}. If the lookup in the translation table
188 fails, this function takes just the low 8 bits of each character.
189 @end defun
190
191 @defun string-make-multibyte string
192 This function converts the text of @var{string} to multibyte
193 representation, if it isn't already, and returns the result. If
194 @var{string} is a multibyte string or consists entirely of
195 @acronym{ASCII} characters, it is returned unchanged. In particular,
196 if @var{string} is unibyte and entirely @acronym{ASCII}, the returned
197 string is unibyte. (When the characters are all @acronym{ASCII},
198 Emacs primitives will treat the string the same way whether it is
199 unibyte or multibyte.) If @var{string} is unibyte and contains
200 non-@acronym{ASCII} characters, the function
201 @code{unibyte-char-to-multibyte} is used to convert each unibyte
202 character to a multibyte character.
203 @end defun
204
205 @defun string-to-multibyte string
206 This function returns a multibyte string containing the same sequence
207 of character codes as @var{string}. Unlike
208 @code{string-make-multibyte}, this function unconditionally returns a
209 multibyte string. If @var{string} is a multibyte string, it is
210 returned unchanged.
211 @end defun
212
213 @defun multibyte-char-to-unibyte char
214 This convert the multibyte character @var{char} to a unibyte
215 character, based on @code{nonascii-translation-table} and
216 @code{nonascii-insert-offset}.
217 @end defun
218
219 @defun unibyte-char-to-multibyte char
220 This convert the unibyte character @var{char} to a multibyte
221 character, based on @code{nonascii-translation-table} and
222 @code{nonascii-insert-offset}.
223 @end defun
224
225 @node Selecting a Representation
226 @section Selecting a Representation
227
228 Sometimes it is useful to examine an existing buffer or string as
229 multibyte when it was unibyte, or vice versa.
230
231 @defun set-buffer-multibyte multibyte
232 Set the representation type of the current buffer. If @var{multibyte}
233 is non-@code{nil}, the buffer becomes multibyte. If @var{multibyte}
234 is @code{nil}, the buffer becomes unibyte.
235
236 This function leaves the buffer contents unchanged when viewed as a
237 sequence of bytes. As a consequence, it can change the contents viewed
238 as characters; a sequence of two bytes which is treated as one character
239 in multibyte representation will count as two characters in unibyte
240 representation. Character codes 128 through 159 are an exception. They
241 are represented by one byte in a unibyte buffer, but when the buffer is
242 set to multibyte, they are converted to two-byte sequences, and vice
243 versa.
244
245 This function sets @code{enable-multibyte-characters} to record which
246 representation is in use. It also adjusts various data in the buffer
247 (including overlays, text properties and markers) so that they cover the
248 same text as they did before.
249
250 You cannot use @code{set-buffer-multibyte} on an indirect buffer,
251 because indirect buffers always inherit the representation of the
252 base buffer.
253 @end defun
254
255 @defun string-as-unibyte string
256 This function returns a string with the same bytes as @var{string} but
257 treating each byte as a character. This means that the value may have
258 more characters than @var{string} has.
259
260 If @var{string} is already a unibyte string, then the value is
261 @var{string} itself. Otherwise it is a newly created string, with no
262 text properties. If @var{string} is multibyte, any characters it
263 contains of charset @code{eight-bit-control} or @code{eight-bit-graphic}
264 are converted to the corresponding single byte.
265 @end defun
266
267 @defun string-as-multibyte string
268 This function returns a string with the same bytes as @var{string} but
269 treating each multibyte sequence as one character. This means that the
270 value may have fewer characters than @var{string} has.
271
272 If @var{string} is already a multibyte string, then the value is
273 @var{string} itself. Otherwise it is a newly created string, with no
274 text properties. If @var{string} is unibyte and contains any individual
275 8-bit bytes (i.e.@: not part of a multibyte form), they are converted to
276 the corresponding multibyte character of charset @code{eight-bit-control}
277 or @code{eight-bit-graphic}.
278 @end defun
279
280 @node Character Codes
281 @section Character Codes
282 @cindex character codes
283
284 The unibyte and multibyte text representations use different character
285 codes. The valid character codes for unibyte representation range from
286 0 to 255---the values that can fit in one byte. The valid character
287 codes for multibyte representation range from 0 to 524287, but not all
288 values in that range are valid. The values 128 through 255 are not
289 entirely proper in multibyte text, but they can occur if you do explicit
290 encoding and decoding (@pxref{Explicit Encoding}). Some other character
291 codes cannot occur at all in multibyte text. Only the @acronym{ASCII} codes
292 0 through 127 are completely legitimate in both representations.
293
294 @defun char-valid-p charcode &optional genericp
295 This returns @code{t} if @var{charcode} is valid for either one of the two
296 text representations.
297
298 @example
299 (char-valid-p 65)
300 @result{} t
301 (char-valid-p 256)
302 @result{} nil
303 (char-valid-p 2248)
304 @result{} t
305 @end example
306
307 If the optional argument @var{genericp} is non-@code{nil}, this
308 function also returns @code{t} if @var{charcode} is a generic
309 character (@pxref{Splitting Characters}).
310 @end defun
311
312 @node Character Sets
313 @section Character Sets
314 @cindex character sets
315
316 Emacs classifies characters into various @dfn{character sets}, each of
317 which has a name which is a symbol. Each character belongs to one and
318 only one character set.
319
320 In general, there is one character set for each distinct script. For
321 example, @code{latin-iso8859-1} is one character set,
322 @code{greek-iso8859-7} is another, and @code{ascii} is another. An
323 Emacs character set can hold at most 9025 characters; therefore, in some
324 cases, characters that would logically be grouped together are split
325 into several character sets. For example, one set of Chinese
326 characters, generally known as Big 5, is divided into two Emacs
327 character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
328
329 @acronym{ASCII} characters are in character set @code{ascii}. The
330 non-@acronym{ASCII} characters 128 through 159 are in character set
331 @code{eight-bit-control}, and codes 160 through 255 are in character set
332 @code{eight-bit-graphic}.
333
334 @defun charsetp object
335 Returns @code{t} if @var{object} is a symbol that names a character set,
336 @code{nil} otherwise.
337 @end defun
338
339 @defvar charset-list
340 The value is a list of all defined character set names.
341 @end defvar
342
343 @defun charset-list
344 This function returns the value of @code{charset-list}. It is only
345 provided for backward compatibility.
346 @end defun
347
348 @defun char-charset character
349 This function returns the name of the character set that @var{character}
350 belongs to, or the symbol @code{unknown} if @var{character} is not a
351 valid character.
352 @end defun
353
354 @defun charset-plist charset
355 @tindex charset-plist
356 This function returns the charset property list of the character set
357 @var{charset}. Although @var{charset} is a symbol, this is not the same
358 as the property list of that symbol. Charset properties are used for
359 special purposes within Emacs.
360 @end defun
361
362 @node Chars and Bytes
363 @section Characters and Bytes
364 @cindex bytes and characters
365
366 @cindex introduction sequence
367 @cindex dimension (of character set)
368 In multibyte representation, each character occupies one or more
369 bytes. Each character set has an @dfn{introduction sequence}, which is
370 normally one or two bytes long. (Exception: the @code{ascii} character
371 set and the @code{eight-bit-graphic} character set have a zero-length
372 introduction sequence.) The introduction sequence is the beginning of
373 the byte sequence for any character in the character set. The rest of
374 the character's bytes distinguish it from the other characters in the
375 same character set. Depending on the character set, there are either
376 one or two distinguishing bytes; the number of such bytes is called the
377 @dfn{dimension} of the character set.
378
379 @defun charset-dimension charset
380 This function returns the dimension of @var{charset}; at present, the
381 dimension is always 1 or 2.
382 @end defun
383
384 @defun charset-bytes charset
385 @tindex charset-bytes
386 This function returns the number of bytes used to represent a character
387 in character set @var{charset}.
388 @end defun
389
390 This is the simplest way to determine the byte length of a character
391 set's introduction sequence:
392
393 @example
394 (- (charset-bytes @var{charset})
395 (charset-dimension @var{charset}))
396 @end example
397
398 @node Splitting Characters
399 @section Splitting Characters
400
401 The functions in this section convert between characters and the byte
402 values used to represent them. For most purposes, there is no need to
403 be concerned with the sequence of bytes used to represent a character,
404 because Emacs translates automatically when necessary.
405
406 @defun split-char character
407 Return a list containing the name of the character set of
408 @var{character}, followed by one or two byte values (integers) which
409 identify @var{character} within that character set. The number of byte
410 values is the character set's dimension.
411
412 If @var{character} is invalid as a character code, @code{split-char}
413 returns a list consisting of the symbol @code{unknown} and @var{character}.
414
415 @example
416 (split-char 2248)
417 @result{} (latin-iso8859-1 72)
418 (split-char 65)
419 @result{} (ascii 65)
420 (split-char 128)
421 @result{} (eight-bit-control 128)
422 @end example
423 @end defun
424
425 @defun make-char charset &optional code1 code2
426 This function returns the character in character set @var{charset} whose
427 position codes are @var{code1} and @var{code2}. This is roughly the
428 inverse of @code{split-char}. Normally, you should specify either one
429 or both of @var{code1} and @var{code2} according to the dimension of
430 @var{charset}. For example,
431
432 @example
433 (make-char 'latin-iso8859-1 72)
434 @result{} 2248
435 @end example
436
437 Actually, the eighth bit of both @var{code1} and @var{code2} is zeroed
438 before they are used to index @var{charset}. Thus you may use, for
439 instance, an ISO 8859 character code rather than subtracting 128, as
440 is necessary to index the corresponding Emacs charset.
441 @end defun
442
443 @cindex generic characters
444 If you call @code{make-char} with no @var{byte-values}, the result is
445 a @dfn{generic character} which stands for @var{charset}. A generic
446 character is an integer, but it is @emph{not} valid for insertion in the
447 buffer as a character. It can be used in @code{char-table-range} to
448 refer to the whole character set (@pxref{Char-Tables}).
449 @code{char-valid-p} returns @code{nil} for generic characters.
450 For example:
451
452 @example
453 (make-char 'latin-iso8859-1)
454 @result{} 2176
455 (char-valid-p 2176)
456 @result{} nil
457 (char-valid-p 2176 t)
458 @result{} t
459 (split-char 2176)
460 @result{} (latin-iso8859-1 0)
461 @end example
462
463 The character sets @code{ascii}, @code{eight-bit-control}, and
464 @code{eight-bit-graphic} don't have corresponding generic characters. If
465 @var{charset} is one of them and you don't supply @var{code1},
466 @code{make-char} returns the character code corresponding to the
467 smallest code in @var{charset}.
468
469 @node Scanning Charsets
470 @section Scanning for Character Sets
471
472 Sometimes it is useful to find out which character sets appear in a
473 part of a buffer or a string. One use for this is in determining which
474 coding systems (@pxref{Coding Systems}) are capable of representing all
475 of the text in question.
476
477 @defun find-charset-region beg end &optional translation
478 This function returns a list of the character sets that appear in the
479 current buffer between positions @var{beg} and @var{end}.
480
481 The optional argument @var{translation} specifies a translation table to
482 be used in scanning the text (@pxref{Translation of Characters}). If it
483 is non-@code{nil}, then each character in the region is translated
484 through this table, and the value returned describes the translated
485 characters instead of the characters actually in the buffer.
486 @end defun
487
488 @defun find-charset-string string &optional translation
489 This function returns a list of the character sets that appear in the
490 string @var{string}. It is just like @code{find-charset-region}, except
491 that it applies to the contents of @var{string} instead of part of the
492 current buffer.
493 @end defun
494
495 @node Translation of Characters
496 @section Translation of Characters
497 @cindex character translation tables
498 @cindex translation tables
499
500 A @dfn{translation table} is a char-table that specifies a mapping
501 of characters into characters. These tables are used in encoding and
502 decoding, and for other purposes. Some coding systems specify their
503 own particular translation tables; there are also default translation
504 tables which apply to all other coding systems.
505
506 For instance, the coding-system @code{utf-8} has a translation table
507 that maps characters of various charsets (e.g.,
508 @code{latin-iso8859-@var{x}}) into Unicode character sets. This way,
509 it can encode Latin-2 characters into UTF-8. Meanwhile,
510 @code{unify-8859-on-decoding-mode} operates by specifying
511 @code{standard-translation-table-for-decode} to translate
512 Latin-@var{x} characters into corresponding Unicode characters.
513
514 @defun make-translation-table &rest translations
515 This function returns a translation table based on the argument
516 @var{translations}. Each element of @var{translations} should be a
517 list of elements of the form @code{(@var{from} . @var{to})}; this says
518 to translate the character @var{from} into @var{to}.
519
520 The arguments and the forms in each argument are processed in order,
521 and if a previous form already translates @var{to} to some other
522 character, say @var{to-alt}, @var{from} is also translated to
523 @var{to-alt}.
524
525 You can also map one whole character set into another character set with
526 the same dimension. To do this, you specify a generic character (which
527 designates a character set) for @var{from} (@pxref{Splitting Characters}).
528 In this case, if @var{to} is also a generic character, its character
529 set should have the same dimension as @var{from}'s. Then the
530 translation table translates each character of @var{from}'s character
531 set into the corresponding character of @var{to}'s character set. If
532 @var{from} is a generic character and @var{to} is an ordinary
533 character, then the translation table translates every character of
534 @var{from}'s character set into @var{to}.
535 @end defun
536
537 In decoding, the translation table's translations are applied to the
538 characters that result from ordinary decoding. If a coding system has
539 property @code{translation-table-for-decode}, that specifies the
540 translation table to use. (This is a property of the coding system,
541 as returned by @code{coding-system-get}, not a property of the symbol
542 that is the coding system's name. @xref{Coding System Basics,, Basic
543 Concepts of Coding Systems}.) Otherwise, if
544 @code{standard-translation-table-for-decode} is non-@code{nil},
545 decoding uses that table.
546
547 In encoding, the translation table's translations are applied to the
548 characters in the buffer, and the result of translation is actually
549 encoded. If a coding system has property
550 @code{translation-table-for-encode}, that specifies the translation
551 table to use. Otherwise the variable
552 @code{standard-translation-table-for-encode} specifies the translation
553 table.
554
555 @defvar standard-translation-table-for-decode
556 This is the default translation table for decoding, for
557 coding systems that don't specify any other translation table.
558 @end defvar
559
560 @defvar standard-translation-table-for-encode
561 This is the default translation table for encoding, for
562 coding systems that don't specify any other translation table.
563 @end defvar
564
565 @defvar translation-table-for-input
566 Self-inserting characters are translated through this translation
567 table before they are inserted. This variable automatically becomes
568 buffer-local when set.
569
570 @code{set-buffer-file-coding-system} sets this variable so that your
571 keyboard input gets translated into the character sets that the buffer
572 is likely to contain.
573 @end defvar
574
575 @node Coding Systems
576 @section Coding Systems
577
578 @cindex coding system
579 When Emacs reads or writes a file, and when Emacs sends text to a
580 subprocess or receives text from a subprocess, it normally performs
581 character code conversion and end-of-line conversion as specified
582 by a particular @dfn{coding system}.
583
584 How to define a coding system is an arcane matter, and is not
585 documented here.
586
587 @menu
588 * Coding System Basics:: Basic concepts.
589 * Encoding and I/O:: How file I/O functions handle coding systems.
590 * Lisp and Coding Systems:: Functions to operate on coding system names.
591 * User-Chosen Coding Systems:: Asking the user to choose a coding system.
592 * Default Coding Systems:: Controlling the default choices.
593 * Specifying Coding Systems:: Requesting a particular coding system
594 for a single file operation.
595 * Explicit Encoding:: Encoding or decoding text without doing I/O.
596 * Terminal I/O Encoding:: Use of encoding for terminal I/O.
597 * MS-DOS File Types:: How DOS "text" and "binary" files
598 relate to coding systems.
599 @end menu
600
601 @node Coding System Basics
602 @subsection Basic Concepts of Coding Systems
603
604 @cindex character code conversion
605 @dfn{Character code conversion} involves conversion between the encoding
606 used inside Emacs and some other encoding. Emacs supports many
607 different encodings, in that it can convert to and from them. For
608 example, it can convert text to or from encodings such as Latin 1, Latin
609 2, Latin 3, Latin 4, Latin 5, and several variants of ISO 2022. In some
610 cases, Emacs supports several alternative encodings for the same
611 characters; for example, there are three coding systems for the Cyrillic
612 (Russian) alphabet: ISO, Alternativnyj, and KOI8.
613
614 Most coding systems specify a particular character code for
615 conversion, but some of them leave the choice unspecified---to be chosen
616 heuristically for each file, based on the data.
617
618 @cindex end of line conversion
619 @dfn{End of line conversion} handles three different conventions used
620 on various systems for representing end of line in files. The Unix
621 convention is to use the linefeed character (also called newline). The
622 DOS convention is to use a carriage-return and a linefeed at the end of
623 a line. The Mac convention is to use just carriage-return.
624
625 @cindex base coding system
626 @cindex variant coding system
627 @dfn{Base coding systems} such as @code{latin-1} leave the end-of-line
628 conversion unspecified, to be chosen based on the data. @dfn{Variant
629 coding systems} such as @code{latin-1-unix}, @code{latin-1-dos} and
630 @code{latin-1-mac} specify the end-of-line conversion explicitly as
631 well. Most base coding systems have three corresponding variants whose
632 names are formed by adding @samp{-unix}, @samp{-dos} and @samp{-mac}.
633
634 The coding system @code{raw-text} is special in that it prevents
635 character code conversion, and causes the buffer visited with that
636 coding system to be a unibyte buffer. It does not specify the
637 end-of-line conversion, allowing that to be determined as usual by the
638 data, and has the usual three variants which specify the end-of-line
639 conversion. @code{no-conversion} is equivalent to @code{raw-text-unix}:
640 it specifies no conversion of either character codes or end-of-line.
641
642 The coding system @code{emacs-mule} specifies that the data is
643 represented in the internal Emacs encoding. This is like
644 @code{raw-text} in that no code conversion happens, but different in
645 that the result is multibyte data.
646
647 @defun coding-system-get coding-system property
648 This function returns the specified property of the coding system
649 @var{coding-system}. Most coding system properties exist for internal
650 purposes, but one that you might find useful is @code{mime-charset}.
651 That property's value is the name used in MIME for the character coding
652 which this coding system can read and write. Examples:
653
654 @example
655 (coding-system-get 'iso-latin-1 'mime-charset)
656 @result{} iso-8859-1
657 (coding-system-get 'iso-2022-cn 'mime-charset)
658 @result{} iso-2022-cn
659 (coding-system-get 'cyrillic-koi8 'mime-charset)
660 @result{} koi8-r
661 @end example
662
663 The value of the @code{mime-charset} property is also defined
664 as an alias for the coding system.
665 @end defun
666
667 @node Encoding and I/O
668 @subsection Encoding and I/O
669
670 The principal purpose of coding systems is for use in reading and
671 writing files. The function @code{insert-file-contents} uses
672 a coding system for decoding the file data, and @code{write-region}
673 uses one to encode the buffer contents.
674
675 You can specify the coding system to use either explicitly
676 (@pxref{Specifying Coding Systems}), or implicitly using the defaulting
677 mechanism (@pxref{Default Coding Systems}). But these methods may not
678 completely specify what to do. For example, they may choose a coding
679 system such as @code{undefined} which leaves the character code
680 conversion to be determined from the data. In these cases, the I/O
681 operation finishes the job of choosing a coding system. Very often
682 you will want to find out afterwards which coding system was chosen.
683
684 @defvar buffer-file-coding-system
685 This variable records the coding system that was used for visiting the
686 current buffer. It is used for saving the buffer, and for writing part
687 of the buffer with @code{write-region}. If the text to be written
688 cannot be safely encoded using the coding system specified by this
689 variable, these operations select an alternative encoding by calling
690 the function @code{select-safe-coding-system} (@pxref{User-Chosen
691 Coding Systems}). If selecting a different encoding requires to ask
692 the user to specify a coding system, @code{buffer-file-coding-system}
693 is updated to the newly selected coding system.
694
695 @code{buffer-file-coding-system} does @emph{not} affect sending text
696 to a subprocess.
697 @end defvar
698
699 @defvar save-buffer-coding-system
700 This variable specifies the coding system for saving the buffer (by
701 overriding @code{buffer-file-coding-system}). Note that it is not used
702 for @code{write-region}.
703
704 When a command to save the buffer starts out to use
705 @code{buffer-file-coding-system} (or @code{save-buffer-coding-system}),
706 and that coding system cannot handle
707 the actual text in the buffer, the command asks the user to choose
708 another coding system (by calling @code{select-safe-coding-system}).
709 After that happens, the command also updates
710 @code{buffer-file-coding-system} to represent the coding system that
711 the user specified.
712 @end defvar
713
714 @defvar last-coding-system-used
715 I/O operations for files and subprocesses set this variable to the
716 coding system name that was used. The explicit encoding and decoding
717 functions (@pxref{Explicit Encoding}) set it too.
718
719 @strong{Warning:} Since receiving subprocess output sets this variable,
720 it can change whenever Emacs waits; therefore, you should copy the
721 value shortly after the function call that stores the value you are
722 interested in.
723 @end defvar
724
725 The variable @code{selection-coding-system} specifies how to encode
726 selections for the window system. @xref{Window System Selections}.
727
728 @defvar file-name-coding-system
729 The variable @code{file-name-coding-system} specifies the coding
730 system to use for encoding file names. Emacs encodes file names using
731 that coding system for all file operations. If
732 @code{file-name-coding-system} is @code{nil}, Emacs uses a default
733 coding system determined by the selected language environment. In the
734 default language environment, any non-@acronym{ASCII} characters in
735 file names are not encoded specially; they appear in the file system
736 using the internal Emacs representation.
737 @end defvar
738
739 @strong{Warning:} if you change @code{file-name-coding-system} (or
740 the language environment) in the middle of an Emacs session, problems
741 can result if you have already visited files whose names were encoded
742 using the earlier coding system and are handled differently under the
743 new coding system. If you try to save one of these buffers under the
744 visited file name, saving may use the wrong file name, or it may get
745 an error. If such a problem happens, use @kbd{C-x C-w} to specify a
746 new file name for that buffer.
747
748 @node Lisp and Coding Systems
749 @subsection Coding Systems in Lisp
750
751 Here are the Lisp facilities for working with coding systems:
752
753 @defun coding-system-list &optional base-only
754 This function returns a list of all coding system names (symbols). If
755 @var{base-only} is non-@code{nil}, the value includes only the
756 base coding systems. Otherwise, it includes alias and variant coding
757 systems as well.
758 @end defun
759
760 @defun coding-system-p object
761 This function returns @code{t} if @var{object} is a coding system
762 name or @code{nil}.
763 @end defun
764
765 @defun check-coding-system coding-system
766 This function checks the validity of @var{coding-system}.
767 If that is valid, it returns @var{coding-system}.
768 Otherwise it signals an error with condition @code{coding-system-error}.
769 @end defun
770
771 @defun coding-system-change-eol-conversion coding-system eol-type
772 This function returns a coding system which is like @var{coding-system}
773 except for its eol conversion, which is specified by @code{eol-type}.
774 @var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or
775 @code{nil}. If it is @code{nil}, the returned coding system determines
776 the end-of-line conversion from the data.
777
778 @var{eol-type} may also be 0, 1 or 2, standing for @code{unix},
779 @code{dos} and @code{mac}, respectively.
780 @end defun
781
782 @defun coding-system-change-text-conversion eol-coding text-coding
783 This function returns a coding system which uses the end-of-line
784 conversion of @var{eol-coding}, and the text conversion of
785 @var{text-coding}. If @var{text-coding} is @code{nil}, it returns
786 @code{undecided}, or one of its variants according to @var{eol-coding}.
787 @end defun
788
789 @defun find-coding-systems-region from to
790 This function returns a list of coding systems that could be used to
791 encode a text between @var{from} and @var{to}. All coding systems in
792 the list can safely encode any multibyte characters in that portion of
793 the text.
794
795 If the text contains no multibyte characters, the function returns the
796 list @code{(undecided)}.
797 @end defun
798
799 @defun find-coding-systems-string string
800 This function returns a list of coding systems that could be used to
801 encode the text of @var{string}. All coding systems in the list can
802 safely encode any multibyte characters in @var{string}. If the text
803 contains no multibyte characters, this returns the list
804 @code{(undecided)}.
805 @end defun
806
807 @defun find-coding-systems-for-charsets charsets
808 This function returns a list of coding systems that could be used to
809 encode all the character sets in the list @var{charsets}.
810 @end defun
811
812 @defun detect-coding-region start end &optional highest
813 This function chooses a plausible coding system for decoding the text
814 from @var{start} to @var{end}. This text should be a byte sequence
815 (@pxref{Explicit Encoding}).
816
817 Normally this function returns a list of coding systems that could
818 handle decoding the text that was scanned. They are listed in order of
819 decreasing priority. But if @var{highest} is non-@code{nil}, then the
820 return value is just one coding system, the one that is highest in
821 priority.
822
823 If the region contains only @acronym{ASCII} characters, the value
824 is @code{undecided} or @code{(undecided)}, or a variant specifying
825 end-of-line conversion, if that can be deduced from the text.
826 @end defun
827
828 @defun detect-coding-string string &optional highest
829 This function is like @code{detect-coding-region} except that it
830 operates on the contents of @var{string} instead of bytes in the buffer.
831 @end defun
832
833 @xref{Coding systems for a subprocess,, Process Information}, in
834 particular the description of the functions
835 @code{process-coding-system} and @code{set-process-coding-system}, for
836 how to examine or set the coding systems used for I/O to a subprocess.
837
838 @node User-Chosen Coding Systems
839 @subsection User-Chosen Coding Systems
840
841 @cindex select safe coding system
842 @defun select-safe-coding-system from to &optional default-coding-system accept-default-p file
843 This function selects a coding system for encoding specified text,
844 asking the user to choose if necessary. Normally the specified text
845 is the text in the current buffer between @var{from} and @var{to}. If
846 @var{from} is a string, the string specifies the text to encode, and
847 @var{to} is ignored.
848
849 If @var{default-coding-system} is non-@code{nil}, that is the first
850 coding system to try; if that can handle the text,
851 @code{select-safe-coding-system} returns that coding system. It can
852 also be a list of coding systems; then the function tries each of them
853 one by one. After trying all of them, it next tries the current
854 buffer's value of @code{buffer-file-coding-system} (if it is not
855 @code{undecided}), then the value of
856 @code{default-buffer-file-coding-system} and finally the user's most
857 preferred coding system, which the user can set using the command
858 @code{prefer-coding-system} (@pxref{Recognize Coding,, Recognizing
859 Coding Systems, emacs, The GNU Emacs Manual}).
860
861 If one of those coding systems can safely encode all the specified
862 text, @code{select-safe-coding-system} chooses it and returns it.
863 Otherwise, it asks the user to choose from a list of coding systems
864 which can encode all the text, and returns the user's choice.
865
866 @var{default-coding-system} can also be a list whose first element is
867 t and whose other elements are coding systems. Then, if no coding
868 system in the list can handle the text, @code{select-safe-coding-system}
869 queries the user immediately, without trying any of the three
870 alternatives described above.
871
872 The optional argument @var{accept-default-p}, if non-@code{nil},
873 should be a function to determine whether a coding system selected
874 without user interaction is acceptable. @code{select-safe-coding-system}
875 calls this function with one argument, the base coding system of the
876 selected coding system. If @var{accept-default-p} returns @code{nil},
877 @code{select-safe-coding-system} rejects the silently selected coding
878 system, and asks the user to select a coding system from a list of
879 possible candidates.
880
881 @vindex select-safe-coding-system-accept-default-p
882 If the variable @code{select-safe-coding-system-accept-default-p} is
883 non-@code{nil}, its value overrides the value of
884 @var{accept-default-p}.
885
886 As a final step, before returning the chosen coding system,
887 @code{select-safe-coding-system} checks whether that coding system is
888 consistent with what would be selected if the contents of the region
889 were read from a file. (If not, this could lead to data corruption in
890 a file subsequently re-visited and edited.) Normally,
891 @code{select-safe-coding-system} uses @code{buffer-file-name} as the
892 file for this purpose, but if @var{file} is non-@code{nil}, it uses
893 that file instead (this can be relevant for @code{write-region} and
894 similar functions). If it detects an apparent inconsistency,
895 @code{select-safe-coding-system} queries the user before selecting the
896 coding system.
897 @end defun
898
899 Here are two functions you can use to let the user specify a coding
900 system, with completion. @xref{Completion}.
901
902 @defun read-coding-system prompt &optional default
903 This function reads a coding system using the minibuffer, prompting with
904 string @var{prompt}, and returns the coding system name as a symbol. If
905 the user enters null input, @var{default} specifies which coding system
906 to return. It should be a symbol or a string.
907 @end defun
908
909 @defun read-non-nil-coding-system prompt
910 This function reads a coding system using the minibuffer, prompting with
911 string @var{prompt}, and returns the coding system name as a symbol. If
912 the user tries to enter null input, it asks the user to try again.
913 @xref{Coding Systems}.
914 @end defun
915
916 @node Default Coding Systems
917 @subsection Default Coding Systems
918
919 This section describes variables that specify the default coding
920 system for certain files or when running certain subprograms, and the
921 function that I/O operations use to access them.
922
923 The idea of these variables is that you set them once and for all to the
924 defaults you want, and then do not change them again. To specify a
925 particular coding system for a particular operation in a Lisp program,
926 don't change these variables; instead, override them using
927 @code{coding-system-for-read} and @code{coding-system-for-write}
928 (@pxref{Specifying Coding Systems}).
929
930 @defvar auto-coding-regexp-alist
931 This variable is an alist of text patterns and corresponding coding
932 systems. Each element has the form @code{(@var{regexp}
933 . @var{coding-system})}; a file whose first few kilobytes match
934 @var{regexp} is decoded with @var{coding-system} when its contents are
935 read into a buffer. The settings in this alist take priority over
936 @code{coding:} tags in the files and the contents of
937 @code{file-coding-system-alist} (see below). The default value is set
938 so that Emacs automatically recognizes mail files in Babyl format and
939 reads them with no code conversions.
940 @end defvar
941
942 @defvar file-coding-system-alist
943 This variable is an alist that specifies the coding systems to use for
944 reading and writing particular files. Each element has the form
945 @code{(@var{pattern} . @var{coding})}, where @var{pattern} is a regular
946 expression that matches certain file names. The element applies to file
947 names that match @var{pattern}.
948
949 The @sc{cdr} of the element, @var{coding}, should be either a coding
950 system, a cons cell containing two coding systems, or a function name (a
951 symbol with a function definition). If @var{coding} is a coding system,
952 that coding system is used for both reading the file and writing it. If
953 @var{coding} is a cons cell containing two coding systems, its @sc{car}
954 specifies the coding system for decoding, and its @sc{cdr} specifies the
955 coding system for encoding.
956
957 If @var{coding} is a function name, the function should take one
958 argument, a list of all arguments passed to
959 @code{find-operation-coding-system}. It must return a coding system
960 or a cons cell containing two coding systems. This value has the same
961 meaning as described above.
962 @end defvar
963
964 @defvar process-coding-system-alist
965 This variable is an alist specifying which coding systems to use for a
966 subprocess, depending on which program is running in the subprocess. It
967 works like @code{file-coding-system-alist}, except that @var{pattern} is
968 matched against the program name used to start the subprocess. The coding
969 system or systems specified in this alist are used to initialize the
970 coding systems used for I/O to the subprocess, but you can specify
971 other coding systems later using @code{set-process-coding-system}.
972 @end defvar
973
974 @strong{Warning:} Coding systems such as @code{undecided}, which
975 determine the coding system from the data, do not work entirely reliably
976 with asynchronous subprocess output. This is because Emacs handles
977 asynchronous subprocess output in batches, as it arrives. If the coding
978 system leaves the character code conversion unspecified, or leaves the
979 end-of-line conversion unspecified, Emacs must try to detect the proper
980 conversion from one batch at a time, and this does not always work.
981
982 Therefore, with an asynchronous subprocess, if at all possible, use a
983 coding system which determines both the character code conversion and
984 the end of line conversion---that is, one like @code{latin-1-unix},
985 rather than @code{undecided} or @code{latin-1}.
986
987 @defvar network-coding-system-alist
988 This variable is an alist that specifies the coding system to use for
989 network streams. It works much like @code{file-coding-system-alist},
990 with the difference that the @var{pattern} in an element may be either a
991 port number or a regular expression. If it is a regular expression, it
992 is matched against the network service name used to open the network
993 stream.
994 @end defvar
995
996 @defvar default-process-coding-system
997 This variable specifies the coding systems to use for subprocess (and
998 network stream) input and output, when nothing else specifies what to
999 do.
1000
1001 The value should be a cons cell of the form @code{(@var{input-coding}
1002 . @var{output-coding})}. Here @var{input-coding} applies to input from
1003 the subprocess, and @var{output-coding} applies to output to it.
1004 @end defvar
1005
1006 @defvar auto-coding-functions
1007 This variable holds a list of functions that try to determine a
1008 coding system for a file based on its undecoded contents.
1009
1010 Each function in this list should be written to look at text in the
1011 current buffer, but should not modify it in any way. The buffer will
1012 contain undecoded text of parts of the file. Each function should
1013 take one argument, @var{size}, which tells it how many characters to
1014 look at, starting from point. If the function succeeds in determining
1015 a coding system for the file, it should return that coding system.
1016 Otherwise, it should return @code{nil}.
1017
1018 If a file has a @samp{coding:} tag, that takes precedence, so these
1019 functions won't be called.
1020 @end defvar
1021
1022 @defun find-operation-coding-system operation &rest arguments
1023 This function returns the coding system to use (by default) for
1024 performing @var{operation} with @var{arguments}. The value has this
1025 form:
1026
1027 @example
1028 (@var{decoding-system} . @var{encoding-system})
1029 @end example
1030
1031 The first element, @var{decoding-system}, is the coding system to use
1032 for decoding (in case @var{operation} does decoding), and
1033 @var{encoding-system} is the coding system for encoding (in case
1034 @var{operation} does encoding).
1035
1036 The argument @var{operation} should be a symbol, one of
1037 @code{insert-file-contents}, @code{write-region}, @code{call-process},
1038 @code{call-process-region}, @code{start-process}, or
1039 @code{open-network-stream}. These are the names of the Emacs I/O primitives
1040 that can do coding system conversion.
1041
1042 The remaining arguments should be the same arguments that might be given
1043 to that I/O primitive. Depending on the primitive, one of those
1044 arguments is selected as the @dfn{target}. For example, if
1045 @var{operation} does file I/O, whichever argument specifies the file
1046 name is the target. For subprocess primitives, the process name is the
1047 target. For @code{open-network-stream}, the target is the service name
1048 or port number.
1049
1050 This function looks up the target in @code{file-coding-system-alist},
1051 @code{process-coding-system-alist}, or
1052 @code{network-coding-system-alist}, depending on @var{operation}.
1053 @end defun
1054
1055 @node Specifying Coding Systems
1056 @subsection Specifying a Coding System for One Operation
1057
1058 You can specify the coding system for a specific operation by binding
1059 the variables @code{coding-system-for-read} and/or
1060 @code{coding-system-for-write}.
1061
1062 @defvar coding-system-for-read
1063 If this variable is non-@code{nil}, it specifies the coding system to
1064 use for reading a file, or for input from a synchronous subprocess.
1065
1066 It also applies to any asynchronous subprocess or network stream, but in
1067 a different way: the value of @code{coding-system-for-read} when you
1068 start the subprocess or open the network stream specifies the input
1069 decoding method for that subprocess or network stream. It remains in
1070 use for that subprocess or network stream unless and until overridden.
1071
1072 The right way to use this variable is to bind it with @code{let} for a
1073 specific I/O operation. Its global value is normally @code{nil}, and
1074 you should not globally set it to any other value. Here is an example
1075 of the right way to use the variable:
1076
1077 @example
1078 ;; @r{Read the file with no character code conversion.}
1079 ;; @r{Assume @acronym{crlf} represents end-of-line.}
1080 (let ((coding-system-for-read 'emacs-mule-dos))
1081 (insert-file-contents filename))
1082 @end example
1083
1084 When its value is non-@code{nil}, @code{coding-system-for-read} takes
1085 precedence over all other methods of specifying a coding system to use for
1086 input, including @code{file-coding-system-alist},
1087 @code{process-coding-system-alist} and
1088 @code{network-coding-system-alist}.
1089 @end defvar
1090
1091 @defvar coding-system-for-write
1092 This works much like @code{coding-system-for-read}, except that it
1093 applies to output rather than input. It affects writing to files,
1094 as well as sending output to subprocesses and net connections.
1095
1096 When a single operation does both input and output, as do
1097 @code{call-process-region} and @code{start-process}, both
1098 @code{coding-system-for-read} and @code{coding-system-for-write}
1099 affect it.
1100 @end defvar
1101
1102 @defvar inhibit-eol-conversion
1103 When this variable is non-@code{nil}, no end-of-line conversion is done,
1104 no matter which coding system is specified. This applies to all the
1105 Emacs I/O and subprocess primitives, and to the explicit encoding and
1106 decoding functions (@pxref{Explicit Encoding}).
1107 @end defvar
1108
1109 @node Explicit Encoding
1110 @subsection Explicit Encoding and Decoding
1111 @cindex encoding text
1112 @cindex decoding text
1113
1114 All the operations that transfer text in and out of Emacs have the
1115 ability to use a coding system to encode or decode the text.
1116 You can also explicitly encode and decode text using the functions
1117 in this section.
1118
1119 The result of encoding, and the input to decoding, are not ordinary
1120 text. They logically consist of a series of byte values; that is, a
1121 series of characters whose codes are in the range 0 through 255. In a
1122 multibyte buffer or string, character codes 128 through 159 are
1123 represented by multibyte sequences, but this is invisible to Lisp
1124 programs.
1125
1126 The usual way to read a file into a buffer as a sequence of bytes, so
1127 you can decode the contents explicitly, is with
1128 @code{insert-file-contents-literally} (@pxref{Reading from Files});
1129 alternatively, specify a non-@code{nil} @var{rawfile} argument when
1130 visiting a file with @code{find-file-noselect}. These methods result in
1131 a unibyte buffer.
1132
1133 The usual way to use the byte sequence that results from explicitly
1134 encoding text is to copy it to a file or process---for example, to write
1135 it with @code{write-region} (@pxref{Writing to Files}), and suppress
1136 encoding by binding @code{coding-system-for-write} to
1137 @code{no-conversion}.
1138
1139 Here are the functions to perform explicit encoding or decoding. The
1140 decoding functions produce sequences of bytes; the encoding functions
1141 are meant to operate on sequences of bytes. All of these functions
1142 discard text properties.
1143
1144 @deffn Command encode-coding-region start end coding-system
1145 This command encodes the text from @var{start} to @var{end} according
1146 to coding system @var{coding-system}. The encoded text replaces the
1147 original text in the buffer. The result of encoding is logically a
1148 sequence of bytes, but the buffer remains multibyte if it was multibyte
1149 before.
1150
1151 This command returns the length of the encoded text.
1152 @end deffn
1153
1154 @defun encode-coding-string string coding-system &optional nocopy
1155 This function encodes the text in @var{string} according to coding
1156 system @var{coding-system}. It returns a new string containing the
1157 encoded text, except when @var{nocopy} is non-@code{nil}, in which
1158 case the function may return @var{string} itself if the encoding
1159 operation is trivial. The result of encoding is a unibyte string.
1160 @end defun
1161
1162 @deffn Command decode-coding-region start end coding-system
1163 This command decodes the text from @var{start} to @var{end} according
1164 to coding system @var{coding-system}. The decoded text replaces the
1165 original text in the buffer. To make explicit decoding useful, the text
1166 before decoding ought to be a sequence of byte values, but both
1167 multibyte and unibyte buffers are acceptable.
1168
1169 This command returns the length of the decoded text.
1170 @end deffn
1171
1172 @defun decode-coding-string string coding-system &optional nocopy
1173 This function decodes the text in @var{string} according to coding
1174 system @var{coding-system}. It returns a new string containing the
1175 decoded text, except when @var{nocopy} is non-@code{nil}, in which
1176 case the function may return @var{string} itself if the decoding
1177 operation is trivial. To make explicit decoding useful, the contents
1178 of @var{string} ought to be a sequence of byte values, but a multibyte
1179 string is acceptable.
1180 @end defun
1181
1182 @defun decode-coding-inserted-region from to filename &optional visit beg end replace
1183 This function decodes the text from @var{from} to @var{to} as if
1184 it were being read from file @var{filename} using @code{insert-file-contents}
1185 using the rest of the arguments provided.
1186
1187 The normal way to use this function is after reading text from a file
1188 without decoding, if you decide you would rather have decoded it.
1189 Instead of deleting the text and reading it again, this time with
1190 decoding, you can call this function.
1191 @end defun
1192
1193 @node Terminal I/O Encoding
1194 @subsection Terminal I/O Encoding
1195
1196 Emacs can decode keyboard input using a coding system, and encode
1197 terminal output. This is useful for terminals that transmit or display
1198 text using a particular encoding such as Latin-1. Emacs does not set
1199 @code{last-coding-system-used} for encoding or decoding for the
1200 terminal.
1201
1202 @defun keyboard-coding-system
1203 This function returns the coding system that is in use for decoding
1204 keyboard input---or @code{nil} if no coding system is to be used.
1205 @end defun
1206
1207 @deffn Command set-keyboard-coding-system coding-system
1208 This command specifies @var{coding-system} as the coding system to
1209 use for decoding keyboard input. If @var{coding-system} is @code{nil},
1210 that means do not decode keyboard input.
1211 @end deffn
1212
1213 @defun terminal-coding-system
1214 This function returns the coding system that is in use for encoding
1215 terminal output---or @code{nil} for no encoding.
1216 @end defun
1217
1218 @deffn Command set-terminal-coding-system coding-system
1219 This command specifies @var{coding-system} as the coding system to use
1220 for encoding terminal output. If @var{coding-system} is @code{nil},
1221 that means do not encode terminal output.
1222 @end deffn
1223
1224 @node MS-DOS File Types
1225 @subsection MS-DOS File Types
1226 @cindex DOS file types
1227 @cindex MS-DOS file types
1228 @cindex Windows file types
1229 @cindex file types on MS-DOS and Windows
1230 @cindex text files and binary files
1231 @cindex binary files and text files
1232
1233 On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
1234 end-of-line conversion for a file by looking at the file's name. This
1235 feature classifies files as @dfn{text files} and @dfn{binary files}. By
1236 ``binary file'' we mean a file of literal byte values that are not
1237 necessarily meant to be characters; Emacs does no end-of-line conversion
1238 and no character code conversion for them. On the other hand, the bytes
1239 in a text file are intended to represent characters; when you create a
1240 new file whose name implies that it is a text file, Emacs uses DOS
1241 end-of-line conversion.
1242
1243 @defvar buffer-file-type
1244 This variable, automatically buffer-local in each buffer, records the
1245 file type of the buffer's visited file. When a buffer does not specify
1246 a coding system with @code{buffer-file-coding-system}, this variable is
1247 used to determine which coding system to use when writing the contents
1248 of the buffer. It should be @code{nil} for text, @code{t} for binary.
1249 If it is @code{t}, the coding system is @code{no-conversion}.
1250 Otherwise, @code{undecided-dos} is used.
1251
1252 Normally this variable is set by visiting a file; it is set to
1253 @code{nil} if the file was visited without any actual conversion.
1254 @end defvar
1255
1256 @defopt file-name-buffer-file-type-alist
1257 This variable holds an alist for recognizing text and binary files.
1258 Each element has the form (@var{regexp} . @var{type}), where
1259 @var{regexp} is matched against the file name, and @var{type} may be
1260 @code{nil} for text, @code{t} for binary, or a function to call to
1261 compute which. If it is a function, then it is called with a single
1262 argument (the file name) and should return @code{t} or @code{nil}.
1263
1264 When running on MS-DOS or MS-Windows, Emacs checks this alist to decide
1265 which coding system to use when reading a file. For a text file,
1266 @code{undecided-dos} is used. For a binary file, @code{no-conversion}
1267 is used.
1268
1269 If no element in this alist matches a given file name, then
1270 @code{default-buffer-file-type} says how to treat the file.
1271 @end defopt
1272
1273 @defopt default-buffer-file-type
1274 This variable says how to handle files for which
1275 @code{file-name-buffer-file-type-alist} says nothing about the type.
1276
1277 If this variable is non-@code{nil}, then these files are treated as
1278 binary: the coding system @code{no-conversion} is used. Otherwise,
1279 nothing special is done for them---the coding system is deduced solely
1280 from the file contents, in the usual Emacs fashion.
1281 @end defopt
1282
1283 @node Input Methods
1284 @section Input Methods
1285 @cindex input methods
1286
1287 @dfn{Input methods} provide convenient ways of entering non-@acronym{ASCII}
1288 characters from the keyboard. Unlike coding systems, which translate
1289 non-@acronym{ASCII} characters to and from encodings meant to be read by
1290 programs, input methods provide human-friendly commands. (@xref{Input
1291 Methods,,, emacs, The GNU Emacs Manual}, for information on how users
1292 use input methods to enter text.) How to define input methods is not
1293 yet documented in this manual, but here we describe how to use them.
1294
1295 Each input method has a name, which is currently a string;
1296 in the future, symbols may also be usable as input method names.
1297
1298 @defvar current-input-method
1299 This variable holds the name of the input method now active in the
1300 current buffer. (It automatically becomes local in each buffer when set
1301 in any fashion.) It is @code{nil} if no input method is active in the
1302 buffer now.
1303 @end defvar
1304
1305 @defopt default-input-method
1306 This variable holds the default input method for commands that choose an
1307 input method. Unlike @code{current-input-method}, this variable is
1308 normally global.
1309 @end defopt
1310
1311 @deffn Command set-input-method input-method
1312 This command activates input method @var{input-method} for the current
1313 buffer. It also sets @code{default-input-method} to @var{input-method}.
1314 If @var{input-method} is @code{nil}, this command deactivates any input
1315 method for the current buffer.
1316 @end deffn
1317
1318 @defun read-input-method-name prompt &optional default inhibit-null
1319 This function reads an input method name with the minibuffer, prompting
1320 with @var{prompt}. If @var{default} is non-@code{nil}, that is returned
1321 by default, if the user enters empty input. However, if
1322 @var{inhibit-null} is non-@code{nil}, empty input signals an error.
1323
1324 The returned value is a string.
1325 @end defun
1326
1327 @defvar input-method-alist
1328 This variable defines all the supported input methods.
1329 Each element defines one input method, and should have the form:
1330
1331 @example
1332 (@var{input-method} @var{language-env} @var{activate-func}
1333 @var{title} @var{description} @var{args}...)
1334 @end example
1335
1336 Here @var{input-method} is the input method name, a string;
1337 @var{language-env} is another string, the name of the language
1338 environment this input method is recommended for. (That serves only for
1339 documentation purposes.)
1340
1341 @var{activate-func} is a function to call to activate this method. The
1342 @var{args}, if any, are passed as arguments to @var{activate-func}. All
1343 told, the arguments to @var{activate-func} are @var{input-method} and
1344 the @var{args}.
1345
1346 @var{title} is a string to display in the mode line while this method is
1347 active. @var{description} is a string describing this method and what
1348 it is good for.
1349 @end defvar
1350
1351 The fundamental interface to input methods is through the
1352 variable @code{input-method-function}. @xref{Reading One Event},
1353 and @ref{Invoking the Input Method}.
1354
1355 @node Locales
1356 @section Locales
1357 @cindex locale
1358
1359 POSIX defines a concept of ``locales'' which control which language
1360 to use in language-related features. These Emacs variables control
1361 how Emacs interacts with these features.
1362
1363 @defvar locale-coding-system
1364 @tindex locale-coding-system
1365 @cindex keyboard input decoding on X
1366 This variable specifies the coding system to use for decoding system
1367 error messages and---on X Window system only---keyboard input, for
1368 encoding the format argument to @code{format-time-string}, and for
1369 decoding the return value of @code{format-time-string}.
1370 @end defvar
1371
1372 @defvar system-messages-locale
1373 @tindex system-messages-locale
1374 This variable specifies the locale to use for generating system error
1375 messages. Changing the locale can cause messages to come out in a
1376 different language or in a different orthography. If the variable is
1377 @code{nil}, the locale is specified by environment variables in the
1378 usual POSIX fashion.
1379 @end defvar
1380
1381 @defvar system-time-locale
1382 @tindex system-time-locale
1383 This variable specifies the locale to use for formatting time values.
1384 Changing the locale can cause messages to appear according to the
1385 conventions of a different language. If the variable is @code{nil}, the
1386 locale is specified by environment variables in the usual POSIX fashion.
1387 @end defvar
1388
1389 @defun locale-info item
1390 This function returns locale data @var{item} for the current POSIX
1391 locale, if available. @var{item} should be one of these symbols:
1392
1393 @table @code
1394 @item codeset
1395 Return the character set as a string (locale item @code{CODESET}).
1396
1397 @item days
1398 Return a 7-element vector of day names (locale items
1399 @code{DAY_1} through @code{DAY_7});
1400
1401 @item months
1402 Return a 12-element vector of month names (locale items @code{MON_1}
1403 through @code{MON_12}).
1404
1405 @item paper
1406 Return a list @code{(@var{width} @var{height})} for the default paper
1407 size measured in millimeters (locale items @code{PAPER_WIDTH} and
1408 @code{PAPER_HEIGHT}).
1409 @end table
1410
1411 If the system can't provide the requested information, or if
1412 @var{item} is not one of those symbols, the value is @code{nil}. All
1413 strings in the return value are decoded using
1414 @code{locale-coding-system}. @xref{Locales,,, libc, The GNU Libc Manual},
1415 for more information about locales and locale items.
1416 @end defun
1417
1418 @ignore
1419 arch-tag: be705bf8-941b-4c35-84fc-ad7d20ddb7cb
1420 @end ignore