Commit | Line | Data |
---|---|---|
cc6d0d2c RS |
1 | @c -*-texinfo-*- |
2 | @c This is part of the GNU Emacs Lisp Reference Manual. | |
3 | @c Copyright (C) 1998 Free Software Foundation, Inc. | |
4 | @c See the file elisp.texi for copying conditions. | |
5 | @setfilename ../info/characters | |
6 | @node Non-ASCII Characters, Searching and Matching, Text, Top | |
7 | @chapter Non-ASCII Characters | |
8 | @cindex multibyte characters | |
9 | @cindex non-ASCII characters | |
10 | ||
11 | This chapter covers the special issues relating to non-@sc{ASCII} | |
12 | characters and how they are stored in strings and buffers. | |
13 | ||
14 | @menu | |
15 | * Text Representations:: | |
16 | * Converting Representations:: | |
17 | * Selecting a Representation:: | |
18 | * Character Codes:: | |
19 | * Character Sets:: | |
cc6d0d2c | 20 | * Chars and Bytes:: |
a9f0a989 RS |
21 | * Splitting Characters:: |
22 | * Scanning Charsets:: | |
23 | * Translation of Characters:: | |
cc6d0d2c | 24 | * Coding Systems:: |
a9f0a989 | 25 | * Input Methods:: |
cc6d0d2c RS |
26 | @end menu |
27 | ||
28 | @node Text Representations | |
29 | @section Text Representations | |
30 | @cindex text representations | |
31 | ||
32 | Emacs has two @dfn{text representations}---two ways to represent text | |
33 | in a string or buffer. These are called @dfn{unibyte} and | |
34 | @dfn{multibyte}. Each string, and each buffer, uses one of these two | |
35 | representations. For most purposes, you can ignore the issue of | |
36 | representations, because Emacs converts text between them as | |
37 | appropriate. Occasionally in Lisp programming you will need to pay | |
38 | attention to the difference. | |
39 | ||
40 | @cindex unibyte text | |
41 | In unibyte representation, each character occupies one byte and | |
42 | therefore the possible character codes range from 0 to 255. Codes 0 | |
43 | through 127 are @sc{ASCII} characters; the codes from 128 through 255 | |
969fe9b5 RS |
44 | are used for one non-@sc{ASCII} character set (you can choose which |
45 | character set by setting the variable @code{nonascii-insert-offset}). | |
cc6d0d2c RS |
46 | |
47 | @cindex leading code | |
48 | @cindex multibyte text | |
1911e6e5 | 49 | @cindex trailing codes |
cc6d0d2c RS |
50 | In multibyte representation, a character may occupy more than one |
51 | byte, and as a result, the full range of Emacs character codes can be | |
52 | stored. The first byte of a multibyte character is always in the range | |
53 | 128 through 159 (octal 0200 through 0237). These values are called | |
a9f0a989 RS |
54 | @dfn{leading codes}. The second and subsequent bytes of a multibyte |
55 | character are always in the range 160 through 255 (octal 0240 through | |
1911e6e5 | 56 | 0377); these values are @dfn{trailing codes}. |
cc6d0d2c RS |
57 | |
58 | In a buffer, the buffer-local value of the variable | |
59 | @code{enable-multibyte-characters} specifies the representation used. | |
60 | The representation for a string is determined based on the string | |
61 | contents when the string is constructed. | |
62 | ||
cc6d0d2c | 63 | @defvar enable-multibyte-characters |
a9f0a989 | 64 | @tindex enable-multibyte-characters |
cc6d0d2c RS |
65 | This variable specifies the current buffer's text representation. |
66 | If it is non-@code{nil}, the buffer contains multibyte text; otherwise, | |
67 | it contains unibyte text. | |
68 | ||
969fe9b5 RS |
69 | You cannot set this variable directly; instead, use the function |
70 | @code{set-buffer-multibyte} to change a buffer's representation. | |
cc6d0d2c RS |
71 | @end defvar |
72 | ||
cc6d0d2c | 73 | @defvar default-enable-multibyte-characters |
a9f0a989 RS |
74 | @tindex default-enable-multibyte-characters |
75 | This variable's value is entirely equivalent to @code{(default-value | |
cc6d0d2c | 76 | 'enable-multibyte-characters)}, and setting this variable changes that |
a9f0a989 RS |
77 | default value. Setting the local binding of |
78 | @code{enable-multibyte-characters} in a specific buffer is not allowed, | |
79 | but changing the default value is supported, and it is a reasonable | |
80 | thing to do, because it has no effect on existing buffers. | |
cc6d0d2c RS |
81 | |
82 | The @samp{--unibyte} command line option does its job by setting the | |
83 | default value to @code{nil} early in startup. | |
84 | @end defvar | |
85 | ||
cc6d0d2c | 86 | @defun multibyte-string-p string |
a9f0a989 | 87 | @tindex multibyte-string-p |
cc6d0d2c RS |
88 | Return @code{t} if @var{string} contains multibyte characters. |
89 | @end defun | |
90 | ||
91 | @node Converting Representations | |
92 | @section Converting Text Representations | |
93 | ||
94 | Emacs can convert unibyte text to multibyte; it can also convert | |
95 | multibyte text to unibyte, though this conversion loses information. In | |
96 | general these conversions happen when inserting text into a buffer, or | |
97 | when putting text from several strings together in one string. You can | |
98 | also explicitly convert a string's contents to either representation. | |
99 | ||
100 | Emacs chooses the representation for a string based on the text that | |
101 | it is constructed from. The general rule is to convert unibyte text to | |
102 | multibyte text when combining it with other multibyte text, because the | |
103 | multibyte representation is more general and can hold whatever | |
104 | characters the unibyte text has. | |
105 | ||
106 | When inserting text into a buffer, Emacs converts the text to the | |
107 | buffer's representation, as specified by | |
108 | @code{enable-multibyte-characters} in that buffer. In particular, when | |
109 | you insert multibyte text into a unibyte buffer, Emacs converts the text | |
110 | to unibyte, even though this conversion cannot in general preserve all | |
111 | the characters that might be in the multibyte text. The other natural | |
112 | alternative, to convert the buffer contents to multibyte, is not | |
113 | acceptable because the buffer's representation is a choice made by the | |
969fe9b5 | 114 | user that cannot be overridden automatically. |
cc6d0d2c RS |
115 | |
116 | Converting unibyte text to multibyte text leaves @sc{ASCII} characters | |
969fe9b5 RS |
117 | unchanged, and likewise 128 through 159. It converts the non-@sc{ASCII} |
118 | codes 160 through 255 by adding the value @code{nonascii-insert-offset} | |
119 | to each character code. By setting this variable, you specify which | |
a9f0a989 RS |
120 | character set the unibyte characters correspond to (@pxref{Character |
121 | Sets}). For example, if @code{nonascii-insert-offset} is 2048, which is | |
122 | @code{(- (make-char 'latin-iso8859-1) 128)}, then the unibyte | |
123 | non-@sc{ASCII} characters correspond to Latin 1. If it is 2688, which | |
124 | is @code{(- (make-char 'greek-iso8859-7) 128)}, then they correspond to | |
125 | Greek letters. | |
cc6d0d2c RS |
126 | |
127 | Converting multibyte text to unibyte is simpler: it performs | |
128 | logical-and of each character code with 255. If | |
129 | @code{nonascii-insert-offset} has a reasonable value, corresponding to | |
130 | the beginning of some character set, this conversion is the inverse of | |
131 | the other: converting unibyte text to multibyte and back to unibyte | |
132 | reproduces the original unibyte text. | |
133 | ||
cc6d0d2c | 134 | @defvar nonascii-insert-offset |
a9f0a989 | 135 | @tindex nonascii-insert-offset |
cc6d0d2c RS |
136 | This variable specifies the amount to add to a non-@sc{ASCII} character |
137 | when converting unibyte text to multibyte. It also applies when | |
a9f0a989 RS |
138 | @code{self-insert-command} inserts a character in the unibyte |
139 | non-@sc{ASCII} range, 128 through 255. However, the function | |
140 | @code{insert-char} does not perform this conversion. | |
cc6d0d2c RS |
141 | |
142 | The right value to use to select character set @var{cs} is @code{(- | |
a9f0a989 | 143 | (make-char @var{cs}) 128)}. If the value of |
cc6d0d2c RS |
144 | @code{nonascii-insert-offset} is zero, then conversion actually uses the |
145 | value for the Latin 1 character set, rather than zero. | |
146 | @end defvar | |
147 | ||
a9f0a989 RS |
148 | @defvar nonascii-translation-table |
149 | @tindex nonascii-translation-table | |
cc6d0d2c RS |
150 | This variable provides a more general alternative to |
151 | @code{nonascii-insert-offset}. You can use it to specify independently | |
152 | how to translate each code in the range of 128 through 255 into a | |
153 | multibyte character. The value should be a vector, or @code{nil}. | |
969fe9b5 | 154 | If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}. |
cc6d0d2c RS |
155 | @end defvar |
156 | ||
cc6d0d2c | 157 | @defun string-make-unibyte string |
a9f0a989 | 158 | @tindex string-make-unibyte |
cc6d0d2c | 159 | This function converts the text of @var{string} to unibyte |
1911e6e5 | 160 | representation, if it isn't already, and returns the result. If |
969fe9b5 | 161 | @var{string} is a unibyte string, it is returned unchanged. |
cc6d0d2c RS |
162 | @end defun |
163 | ||
cc6d0d2c | 164 | @defun string-make-multibyte string |
a9f0a989 | 165 | @tindex string-make-multibyte |
cc6d0d2c | 166 | This function converts the text of @var{string} to multibyte |
1911e6e5 | 167 | representation, if it isn't already, and returns the result. If |
969fe9b5 | 168 | @var{string} is a multibyte string, it is returned unchanged. |
cc6d0d2c RS |
169 | @end defun |
170 | ||
171 | @node Selecting a Representation | |
172 | @section Selecting a Representation | |
173 | ||
174 | Sometimes it is useful to examine an existing buffer or string as | |
175 | multibyte when it was unibyte, or vice versa. | |
176 | ||
cc6d0d2c | 177 | @defun set-buffer-multibyte multibyte |
a9f0a989 | 178 | @tindex set-buffer-multibyte |
cc6d0d2c RS |
179 | Set the representation type of the current buffer. If @var{multibyte} |
180 | is non-@code{nil}, the buffer becomes multibyte. If @var{multibyte} | |
181 | is @code{nil}, the buffer becomes unibyte. | |
182 | ||
183 | This function leaves the buffer contents unchanged when viewed as a | |
184 | sequence of bytes. As a consequence, it can change the contents viewed | |
185 | as characters; a sequence of two bytes which is treated as one character | |
186 | in multibyte representation will count as two characters in unibyte | |
187 | representation. | |
188 | ||
189 | This function sets @code{enable-multibyte-characters} to record which | |
190 | representation is in use. It also adjusts various data in the buffer | |
969fe9b5 RS |
191 | (including overlays, text properties and markers) so that they cover the |
192 | same text as they did before. | |
cc6d0d2c RS |
193 | @end defun |
194 | ||
cc6d0d2c | 195 | @defun string-as-unibyte string |
a9f0a989 | 196 | @tindex string-as-unibyte |
cc6d0d2c RS |
197 | This function returns a string with the same bytes as @var{string} but |
198 | treating each byte as a character. This means that the value may have | |
199 | more characters than @var{string} has. | |
200 | ||
969fe9b5 | 201 | If @var{string} is unibyte already, then the value is @var{string} |
cc6d0d2c RS |
202 | itself. |
203 | @end defun | |
204 | ||
cc6d0d2c | 205 | @defun string-as-multibyte string |
a9f0a989 | 206 | @tindex string-as-multibyte |
cc6d0d2c RS |
207 | This function returns a string with the same bytes as @var{string} but |
208 | treating each multibyte sequence as one character. This means that the | |
209 | value may have fewer characters than @var{string} has. | |
210 | ||
969fe9b5 | 211 | If @var{string} is multibyte already, then the value is @var{string} |
cc6d0d2c RS |
212 | itself. |
213 | @end defun | |
214 | ||
215 | @node Character Codes | |
216 | @section Character Codes | |
217 | @cindex character codes | |
218 | ||
219 | The unibyte and multibyte text representations use different character | |
220 | codes. The valid character codes for unibyte representation range from | |
221 | 0 to 255---the values that can fit in one byte. The valid character | |
222 | codes for multibyte representation range from 0 to 524287, but not all | |
223 | values in that range are valid. In particular, the values 128 through | |
969fe9b5 RS |
224 | 255 are not legitimate in multibyte text (though they can occur in ``raw |
225 | bytes''; @pxref{Explicit Encoding}). Only the @sc{ASCII} codes 0 | |
226 | through 127 are fully legitimate in both representations. | |
cc6d0d2c RS |
227 | |
228 | @defun char-valid-p charcode | |
229 | This returns @code{t} if @var{charcode} is valid for either one of the two | |
230 | text representations. | |
231 | ||
232 | @example | |
233 | (char-valid-p 65) | |
234 | @result{} t | |
235 | (char-valid-p 256) | |
236 | @result{} nil | |
237 | (char-valid-p 2248) | |
238 | @result{} t | |
239 | @end example | |
240 | @end defun | |
241 | ||
242 | @node Character Sets | |
243 | @section Character Sets | |
244 | @cindex character sets | |
245 | ||
246 | Emacs classifies characters into various @dfn{character sets}, each of | |
247 | which has a name which is a symbol. Each character belongs to one and | |
248 | only one character set. | |
249 | ||
250 | In general, there is one character set for each distinct script. For | |
251 | example, @code{latin-iso8859-1} is one character set, | |
252 | @code{greek-iso8859-7} is another, and @code{ascii} is another. An | |
969fe9b5 RS |
253 | Emacs character set can hold at most 9025 characters; therefore, in some |
254 | cases, characters that would logically be grouped together are split | |
a9f0a989 RS |
255 | into several character sets. For example, one set of Chinese |
256 | characters, generally known as Big 5, is divided into two Emacs | |
257 | character sets, @code{chinese-big5-1} and @code{chinese-big5-2}. | |
cc6d0d2c | 258 | |
cc6d0d2c | 259 | @defun charsetp object |
a9f0a989 | 260 | @tindex charsetp |
cc6d0d2c RS |
261 | Return @code{t} if @var{object} is a character set name symbol, |
262 | @code{nil} otherwise. | |
263 | @end defun | |
264 | ||
cc6d0d2c | 265 | @defun charset-list |
a9f0a989 | 266 | @tindex charset-list |
cc6d0d2c RS |
267 | This function returns a list of all defined character set names. |
268 | @end defun | |
269 | ||
cc6d0d2c | 270 | @defun char-charset character |
a9f0a989 RS |
271 | @tindex char-charset |
272 | This function returns the name of the character | |
cc6d0d2c RS |
273 | set that @var{character} belongs to. |
274 | @end defun | |
275 | ||
cc6d0d2c RS |
276 | @node Chars and Bytes |
277 | @section Characters and Bytes | |
278 | @cindex bytes and characters | |
279 | ||
a9f0a989 RS |
280 | @cindex introduction sequence |
281 | @cindex dimension (of character set) | |
cc6d0d2c | 282 | In multibyte representation, each character occupies one or more |
a9f0a989 | 283 | bytes. Each character set has an @dfn{introduction sequence}, which is |
1911e6e5 RS |
284 | normally one or two bytes long. (Exception: the @sc{ASCII} character |
285 | set has a zero-length introduction sequence.) The introduction sequence | |
286 | is the beginning of the byte sequence for any character in the character | |
287 | set. The rest of the character's bytes distinguish it from the other | |
288 | characters in the same character set. Depending on the character set, | |
289 | there are either one or two distinguishing bytes; the number of such | |
290 | bytes is called the @dfn{dimension} of the character set. | |
a9f0a989 RS |
291 | |
292 | @defun charset-dimension charset | |
293 | @tindex charset-dimension | |
294 | This function returns the dimension of @var{charset}; | |
1911e6e5 | 295 | at present, the dimension is always 1 or 2. |
a9f0a989 RS |
296 | @end defun |
297 | ||
298 | This is the simplest way to determine the byte length of a character | |
299 | set's introduction sequence: | |
300 | ||
301 | @example | |
302 | (- (char-bytes (make-char @var{charset})) | |
303 | (charset-dimension @var{charset})) | |
304 | @end example | |
305 | ||
306 | @node Splitting Characters | |
307 | @section Splitting Characters | |
308 | ||
309 | The functions in this section convert between characters and the byte | |
310 | values used to represent them. For most purposes, there is no need to | |
311 | be concerned with the sequence of bytes used to represent a character, | |
969fe9b5 | 312 | because Emacs translates automatically when necessary. |
cc6d0d2c | 313 | |
cc6d0d2c | 314 | @defun char-bytes character |
a9f0a989 | 315 | @tindex char-bytes |
cc6d0d2c | 316 | This function returns the number of bytes used to represent the |
a9f0a989 RS |
317 | character @var{character}. This depends only on the character set that |
318 | @var{character} belongs to; it equals the dimension of that character | |
319 | set (@pxref{Character Sets}), plus the length of its introduction | |
320 | sequence. | |
cc6d0d2c RS |
321 | |
322 | @example | |
323 | (char-bytes 2248) | |
324 | @result{} 2 | |
325 | (char-bytes 65) | |
326 | @result{} 1 | |
cc6d0d2c RS |
327 | (char-bytes 192) |
328 | @result{} 1 | |
329 | @end example | |
a9f0a989 RS |
330 | |
331 | The reason this function can give correct results for both multibyte and | |
332 | unibyte representations is that the non-@sc{ASCII} character codes used | |
333 | in those two representations do not overlap. | |
cc6d0d2c RS |
334 | @end defun |
335 | ||
cc6d0d2c | 336 | @defun split-char character |
a9f0a989 | 337 | @tindex split-char |
cc6d0d2c | 338 | Return a list containing the name of the character set of |
a9f0a989 RS |
339 | @var{character}, followed by one or two byte values (integers) which |
340 | identify @var{character} within that character set. The number of byte | |
341 | values is the character set's dimension. | |
cc6d0d2c RS |
342 | |
343 | @example | |
344 | (split-char 2248) | |
345 | @result{} (latin-iso8859-1 72) | |
346 | (split-char 65) | |
347 | @result{} (ascii 65) | |
348 | @end example | |
349 | ||
350 | Unibyte non-@sc{ASCII} characters are considered as part of | |
351 | the @code{ascii} character set: | |
352 | ||
353 | @example | |
354 | (split-char 192) | |
355 | @result{} (ascii 192) | |
356 | @end example | |
357 | @end defun | |
358 | ||
cc6d0d2c | 359 | @defun make-char charset &rest byte-values |
a9f0a989 RS |
360 | @tindex make-char |
361 | This function returns the character in character set @var{charset} | |
362 | identified by @var{byte-values}. This is roughly the inverse of | |
363 | @code{split-char}. Normally, you should specify either one or two | |
364 | @var{byte-values}, according to the dimension of @var{charset}. For | |
365 | example, | |
cc6d0d2c RS |
366 | |
367 | @example | |
368 | (make-char 'latin-iso8859-1 72) | |
369 | @result{} 2248 | |
370 | @end example | |
371 | @end defun | |
372 | ||
a9f0a989 RS |
373 | @cindex generic characters |
374 | If you call @code{make-char} with no @var{byte-values}, the result is | |
375 | a @dfn{generic character} which stands for @var{charset}. A generic | |
376 | character is an integer, but it is @emph{not} valid for insertion in the | |
377 | buffer as a character. It can be used in @code{char-table-range} to | |
378 | refer to the whole character set (@pxref{Char-Tables}). | |
379 | @code{char-valid-p} returns @code{nil} for generic characters. | |
380 | For example: | |
381 | ||
382 | @example | |
383 | (make-char 'latin-iso8859-1) | |
384 | @result{} 2176 | |
385 | (char-valid-p 2176) | |
386 | @result{} nil | |
387 | (split-char 2176) | |
388 | @result{} (latin-iso8859-1 0) | |
389 | @end example | |
390 | ||
391 | @node Scanning Charsets | |
392 | @section Scanning for Character Sets | |
393 | ||
394 | Sometimes it is useful to find out which character sets appear in a | |
395 | part of a buffer or a string. One use for this is in determining which | |
396 | coding systems (@pxref{Coding Systems}) are capable of representing all | |
397 | of the text in question. | |
398 | ||
399 | @defun find-charset-region beg end &optional translation | |
400 | @tindex find-charset-region | |
401 | This function returns a list of the character sets that appear in the | |
402 | current buffer between positions @var{beg} and @var{end}. | |
403 | ||
404 | The optional argument @var{translation} specifies a translation table to | |
405 | be used in scanning the text (@pxref{Translation of Characters}). If it | |
406 | is non-@code{nil}, then each character in the region is translated | |
407 | through this table, and the value returned describes the translated | |
408 | characters instead of the characters actually in the buffer. | |
409 | @end defun | |
410 | ||
411 | @defun find-charset-string string &optional translation | |
412 | @tindex find-charset-string | |
413 | This function returns a list of the character sets | |
414 | that appear in the string @var{string}. | |
415 | ||
416 | The optional argument @var{translation} specifies a | |
417 | translation table; see @code{find-charset-region}, above. | |
418 | @end defun | |
419 | ||
420 | @node Translation of Characters | |
421 | @section Translation of Characters | |
422 | @cindex character translation tables | |
423 | @cindex translation tables | |
424 | ||
425 | A @dfn{translation table} specifies a mapping of characters | |
426 | into characters. These tables are used in encoding and decoding, and | |
427 | for other purposes. Some coding systems specify their own particular | |
428 | translation tables; there are also default translation tables which | |
429 | apply to all other coding systems. | |
430 | ||
431 | @defun make-translation-table translations | |
432 | This function returns a translation table based on the arguments | |
433 | @var{translations}. Each argument---each element of | |
434 | @var{translations}---should be a list of the form @code{(@var{from} | |
435 | . @var{to})}; this says to translate the character @var{from} into | |
436 | @var{to}. | |
437 | ||
438 | You can also map one whole character set into another character set with | |
439 | the same dimension. To do this, you specify a generic character (which | |
440 | designates a character set) for @var{from} (@pxref{Splitting Characters}). | |
441 | In this case, @var{to} should also be a generic character, for another | |
442 | character set of the same dimension. Then the translation table | |
443 | translates each character of @var{from}'s character set into the | |
444 | corresponding character of @var{to}'s character set. | |
445 | @end defun | |
446 | ||
447 | In decoding, the translation table's translations are applied to the | |
448 | characters that result from ordinary decoding. If a coding system has | |
449 | property @code{character-translation-table-for-decode}, that specifies | |
450 | the translation table to use. Otherwise, if | |
451 | @code{standard-character-translation-table-for-decode} is | |
452 | non-@code{nil}, decoding uses that table. | |
453 | ||
454 | In encoding, the translation table's translations are applied to the | |
455 | characters in the buffer, and the result of translation is actually | |
456 | encoded. If a coding system has property | |
457 | @code{character-translation-table-for-encode}, that specifies the | |
458 | translation table to use. Otherwise the variable | |
459 | @code{standard-character-translation-table-for-encode} specifies the | |
460 | translation table. | |
461 | ||
462 | @defvar standard-character-translation-table-for-decode | |
463 | This is the default translation table for decoding, for | |
464 | coding systems that don't specify any other translation table. | |
465 | @end defvar | |
466 | ||
467 | @defvar standard-character-translation-table-for-encode | |
468 | This is the default translation table for encoding, for | |
469 | coding systems that don't specify any other translation table. | |
470 | @end defvar | |
471 | ||
cc6d0d2c RS |
472 | @node Coding Systems |
473 | @section Coding Systems | |
474 | ||
475 | @cindex coding system | |
476 | When Emacs reads or writes a file, and when Emacs sends text to a | |
477 | subprocess or receives text from a subprocess, it normally performs | |
478 | character code conversion and end-of-line conversion as specified | |
479 | by a particular @dfn{coding system}. | |
480 | ||
a9f0a989 RS |
481 | @menu |
482 | * Coding System Basics:: | |
483 | * Encoding and I/O:: | |
484 | * Lisp and Coding Systems:: | |
1911e6e5 | 485 | * User-Chosen Coding Systems:: |
a9f0a989 RS |
486 | * Default Coding Systems:: |
487 | * Specifying Coding Systems:: | |
488 | * Explicit Encoding:: | |
489 | * Terminal I/O Encoding:: | |
490 | * MS-DOS File Types:: | |
491 | @end menu | |
492 | ||
493 | @node Coding System Basics | |
494 | @subsection Basic Concepts of Coding Systems | |
495 | ||
cc6d0d2c RS |
496 | @cindex character code conversion |
497 | @dfn{Character code conversion} involves conversion between the encoding | |
498 | used inside Emacs and some other encoding. Emacs supports many | |
499 | different encodings, in that it can convert to and from them. For | |
500 | example, it can convert text to or from encodings such as Latin 1, Latin | |
501 | 2, Latin 3, Latin 4, Latin 5, and several variants of ISO 2022. In some | |
502 | cases, Emacs supports several alternative encodings for the same | |
503 | characters; for example, there are three coding systems for the Cyrillic | |
504 | (Russian) alphabet: ISO, Alternativnyj, and KOI8. | |
505 | ||
cc6d0d2c RS |
506 | Most coding systems specify a particular character code for |
507 | conversion, but some of them leave this unspecified---to be chosen | |
508 | heuristically based on the data. | |
509 | ||
969fe9b5 RS |
510 | @cindex end of line conversion |
511 | @dfn{End of line conversion} handles three different conventions used | |
512 | on various systems for representing end of line in files. The Unix | |
513 | convention is to use the linefeed character (also called newline). The | |
514 | DOS convention is to use the two character sequence, carriage-return | |
515 | linefeed, at the end of a line. The Mac convention is to use just | |
516 | carriage-return. | |
517 | ||
cc6d0d2c RS |
518 | @cindex base coding system |
519 | @cindex variant coding system | |
520 | @dfn{Base coding systems} such as @code{latin-1} leave the end-of-line | |
521 | conversion unspecified, to be chosen based on the data. @dfn{Variant | |
522 | coding systems} such as @code{latin-1-unix}, @code{latin-1-dos} and | |
523 | @code{latin-1-mac} specify the end-of-line conversion explicitly as | |
a9f0a989 | 524 | well. Most base coding systems have three corresponding variants whose |
cc6d0d2c RS |
525 | names are formed by adding @samp{-unix}, @samp{-dos} and @samp{-mac}. |
526 | ||
a9f0a989 RS |
527 | The coding system @code{raw-text} is special in that it prevents |
528 | character code conversion, and causes the buffer visited with that | |
529 | coding system to be a unibyte buffer. It does not specify the | |
530 | end-of-line conversion, allowing that to be determined as usual by the | |
531 | data, and has the usual three variants which specify the end-of-line | |
532 | conversion. @code{no-conversion} is equivalent to @code{raw-text-unix}: | |
533 | it specifies no conversion of either character codes or end-of-line. | |
534 | ||
535 | The coding system @code{emacs-mule} specifies that the data is | |
536 | represented in the internal Emacs encoding. This is like | |
537 | @code{raw-text} in that no code conversion happens, but different in | |
538 | that the result is multibyte data. | |
539 | ||
540 | @defun coding-system-get coding-system property | |
541 | @tindex coding-system-get | |
542 | This function returns the specified property of the coding system | |
543 | @var{coding-system}. Most coding system properties exist for internal | |
544 | purposes, but one that you might find useful is @code{mime-charset}. | |
545 | That property's value is the name used in MIME for the character coding | |
546 | which this coding system can read and write. Examples: | |
547 | ||
548 | @example | |
549 | (coding-system-get 'iso-latin-1 'mime-charset) | |
550 | @result{} iso-8859-1 | |
551 | (coding-system-get 'iso-2022-cn 'mime-charset) | |
552 | @result{} iso-2022-cn | |
553 | (coding-system-get 'cyrillic-koi8 'mime-charset) | |
554 | @result{} koi8-r | |
555 | @end example | |
556 | ||
557 | The value of the @code{mime-charset} property is also defined | |
558 | as an alias for the coding system. | |
559 | @end defun | |
560 | ||
561 | @node Encoding and I/O | |
562 | @subsection Encoding and I/O | |
563 | ||
1911e6e5 | 564 | The principal purpose of coding systems is for use in reading and |
a9f0a989 RS |
565 | writing files. The function @code{insert-file-contents} uses |
566 | a coding system for decoding the file data, and @code{write-region} | |
567 | uses one to encode the buffer contents. | |
568 | ||
569 | You can specify the coding system to use either explicitly | |
570 | (@pxref{Specifying Coding Systems}), or implicitly using the defaulting | |
571 | mechanism (@pxref{Default Coding Systems}). But these methods may not | |
572 | completely specify what to do. For example, they may choose a coding | |
573 | system such as @code{undefined} which leaves the character code | |
574 | conversion to be determined from the data. In these cases, the I/O | |
575 | operation finishes the job of choosing a coding system. Very often | |
576 | you will want to find out afterwards which coding system was chosen. | |
577 | ||
578 | @defvar buffer-file-coding-system | |
579 | @tindex buffer-file-coding-system | |
580 | This variable records the coding system that was used for visiting the | |
581 | current buffer. It is used for saving the buffer, and for writing part | |
582 | of the buffer with @code{write-region}. When those operations ask the | |
583 | user to specify a different coding system, | |
584 | @code{buffer-file-coding-system} is updated to the coding system | |
585 | specified. | |
586 | @end defvar | |
587 | ||
588 | @defvar save-buffer-coding-system | |
589 | @tindex save-buffer-coding-system | |
590 | This variable specifies the coding system for saving the buffer---but it | |
591 | is not used for @code{write-region}. When saving the buffer asks the | |
592 | user to specify a different coding system, and | |
593 | @code{save-buffer-coding-system} was used, then it is updated to the | |
594 | coding system that was specified. | |
595 | @end defvar | |
596 | ||
597 | @defvar last-coding-system-used | |
598 | @tindex last-coding-system-used | |
599 | I/O operations for files and subprocesses set this variable to the | |
600 | coding system name that was used. The explicit encoding and decoding | |
601 | functions (@pxref{Explicit Encoding}) set it too. | |
602 | ||
603 | @strong{Warning:} Since receiving subprocess output sets this variable, | |
604 | it can change whenever Emacs waits; therefore, you should use copy the | |
605 | value shortly after the function call which stores the value you are | |
606 | interested in. | |
607 | @end defvar | |
608 | ||
2eb4136f RS |
609 | The variable @code{selection-coding-system} specifies how to encode |
610 | selections for the window system. @xref{Window System Selections}. | |
611 | ||
969fe9b5 RS |
612 | @node Lisp and Coding Systems |
613 | @subsection Coding Systems in Lisp | |
614 | ||
cc6d0d2c RS |
615 | Here are Lisp facilities for working with coding systems; |
616 | ||
cc6d0d2c | 617 | @defun coding-system-list &optional base-only |
a9f0a989 | 618 | @tindex coding-system-list |
cc6d0d2c RS |
619 | This function returns a list of all coding system names (symbols). If |
620 | @var{base-only} is non-@code{nil}, the value includes only the | |
621 | base coding systems. Otherwise, it includes variant coding systems as well. | |
622 | @end defun | |
623 | ||
cc6d0d2c | 624 | @defun coding-system-p object |
a9f0a989 | 625 | @tindex coding-system-p |
cc6d0d2c RS |
626 | This function returns @code{t} if @var{object} is a coding system |
627 | name. | |
628 | @end defun | |
629 | ||
cc6d0d2c | 630 | @defun check-coding-system coding-system |
a9f0a989 | 631 | @tindex check-coding-system |
cc6d0d2c RS |
632 | This function checks the validity of @var{coding-system}. |
633 | If that is valid, it returns @var{coding-system}. | |
634 | Otherwise it signals an error with condition @code{coding-system-error}. | |
635 | @end defun | |
636 | ||
a9f0a989 RS |
637 | @defun coding-system-change-eol-conversion coding-system eol-type |
638 | @tindex coding-system-change-eol-conversion | |
639 | This function returns a coding system which is like @var{coding-system} | |
1911e6e5 | 640 | except for its eol conversion, which is specified by @code{eol-type}. |
a9f0a989 RS |
641 | @var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or |
642 | @code{nil}. If it is @code{nil}, the returned coding system determines | |
643 | the end-of-line conversion from the data. | |
644 | @end defun | |
969fe9b5 | 645 | |
a9f0a989 RS |
646 | @defun coding-system-change-text-conversion eol-coding text-coding |
647 | @tindex coding-system-change-text-conversion | |
648 | This function returns a coding system which uses the end-of-line | |
649 | conversion of @var{eol-coding}, and the text conversion of | |
650 | @var{text-coding}. If @var{text-coding} is @code{nil}, it returns | |
651 | @code{undecided}, or one of its variants according to @var{eol-coding}. | |
969fe9b5 RS |
652 | @end defun |
653 | ||
a9f0a989 RS |
654 | @defun find-coding-systems-region from to |
655 | @tindex find-coding-systems-region | |
656 | This function returns a list of coding systems that could be used to | |
657 | encode a text between @var{from} and @var{to}. All coding systems in | |
658 | the list can safely encode any multibyte characters in that portion of | |
659 | the text. | |
660 | ||
661 | If the text contains no multibyte characters, the function returns the | |
662 | list @code{(undecided)}. | |
663 | @end defun | |
664 | ||
665 | @defun find-coding-systems-string string | |
666 | @tindex find-coding-systems-string | |
667 | This function returns a list of coding systems that could be used to | |
668 | encode the text of @var{string}. All coding systems in the list can | |
669 | safely encode any multibyte characters in @var{string}. If the text | |
670 | contains no multibyte characters, this returns the list | |
671 | @code{(undecided)}. | |
672 | @end defun | |
673 | ||
674 | @defun find-coding-systems-for-charsets charsets | |
675 | @tindex find-coding-systems-for-charsets | |
676 | This function returns a list of coding systems that could be used to | |
677 | encode all the character sets in the list @var{charsets}. | |
678 | @end defun | |
679 | ||
680 | @defun detect-coding-region start end &optional highest | |
cc6d0d2c | 681 | @tindex detect-coding-region |
cc6d0d2c RS |
682 | This function chooses a plausible coding system for decoding the text |
683 | from @var{start} to @var{end}. This text should be ``raw bytes'' | |
969fe9b5 | 684 | (@pxref{Explicit Encoding}). |
cc6d0d2c | 685 | |
a9f0a989 | 686 | Normally this function returns a list of coding systems that could |
cc6d0d2c | 687 | handle decoding the text that was scanned. They are listed in order of |
a9f0a989 RS |
688 | decreasing priority. But if @var{highest} is non-@code{nil}, then the |
689 | return value is just one coding system, the one that is highest in | |
690 | priority. | |
691 | ||
692 | If the region contains only @sc{ASCII} characters, the value | |
693 | is @code{undecided} or @code{(undecided)}. | |
cc6d0d2c RS |
694 | @end defun |
695 | ||
a9f0a989 RS |
696 | @defun detect-coding-string string highest |
697 | @tindex detect-coding-string | |
cc6d0d2c RS |
698 | This function is like @code{detect-coding-region} except that it |
699 | operates on the contents of @var{string} instead of bytes in the buffer. | |
1911e6e5 RS |
700 | @end defun |
701 | ||
702 | @xref{Process Information}, for how to examine or set the coding | |
703 | systems used for I/O to a subprocess. | |
704 | ||
705 | @node User-Chosen Coding Systems | |
706 | @subsection User-Chosen Coding Systems | |
707 | ||
708 | @tindex select-safe-coding-system | |
709 | @defun select-safe-coding-system from to &optional preferred-coding-system | |
ebc6903b | 710 | This function selects a coding system for encoding the text between |
1911e6e5 RS |
711 | @var{from} and @var{to}, asking the user to choose if necessary. |
712 | ||
713 | The optional argument @var{preferred-coding-system} specifies a coding | |
ebc6903b RS |
714 | system to try first. If that one can handle the text in the specified |
715 | region, then it is used. If this argument is omitted, the current | |
716 | buffer's value of @code{buffer-file-coding-system} is tried first. | |
1911e6e5 RS |
717 | |
718 | If the region contains some multibyte characters that the preferred | |
719 | coding system cannot encode, this function asks the user to choose from | |
720 | a list of coding systems which can encode the text, and returns the | |
721 | user's choice. | |
722 | ||
723 | One other kludgy feature: if @var{from} is a string, the string is the | |
724 | target text, and @var{to} is ignored. | |
969fe9b5 RS |
725 | @end defun |
726 | ||
727 | Here are two functions you can use to let the user specify a coding | |
728 | system, with completion. @xref{Completion}. | |
729 | ||
a9f0a989 | 730 | @defun read-coding-system prompt &optional default |
969fe9b5 | 731 | @tindex read-coding-system |
969fe9b5 RS |
732 | This function reads a coding system using the minibuffer, prompting with |
733 | string @var{prompt}, and returns the coding system name as a symbol. If | |
734 | the user enters null input, @var{default} specifies which coding system | |
735 | to return. It should be a symbol or a string. | |
736 | @end defun | |
737 | ||
969fe9b5 | 738 | @defun read-non-nil-coding-system prompt |
a9f0a989 | 739 | @tindex read-non-nil-coding-system |
969fe9b5 | 740 | This function reads a coding system using the minibuffer, prompting with |
a9f0a989 | 741 | string @var{prompt}, and returns the coding system name as a symbol. If |
969fe9b5 RS |
742 | the user tries to enter null input, it asks the user to try again. |
743 | @xref{Coding Systems}. | |
cc6d0d2c RS |
744 | @end defun |
745 | ||
746 | @node Default Coding Systems | |
a9f0a989 | 747 | @subsection Default Coding Systems |
cc6d0d2c | 748 | |
a9f0a989 RS |
749 | This section describes variables that specify the default coding |
750 | system for certain files or when running certain subprograms, and the | |
1911e6e5 | 751 | function that I/O operations use to access them. |
a9f0a989 RS |
752 | |
753 | The idea of these variables is that you set them once and for all to the | |
754 | defaults you want, and then do not change them again. To specify a | |
755 | particular coding system for a particular operation in a Lisp program, | |
756 | don't change these variables; instead, override them using | |
757 | @code{coding-system-for-read} and @code{coding-system-for-write} | |
758 | (@pxref{Specifying Coding Systems}). | |
cc6d0d2c | 759 | |
cc6d0d2c | 760 | @defvar file-coding-system-alist |
a9f0a989 | 761 | @tindex file-coding-system-alist |
cc6d0d2c RS |
762 | This variable is an alist that specifies the coding systems to use for |
763 | reading and writing particular files. Each element has the form | |
764 | @code{(@var{pattern} . @var{coding})}, where @var{pattern} is a regular | |
765 | expression that matches certain file names. The element applies to file | |
766 | names that match @var{pattern}. | |
767 | ||
1911e6e5 | 768 | The @sc{cdr} of the element, @var{coding}, should be either a coding |
cc6d0d2c RS |
769 | system, a cons cell containing two coding systems, or a function symbol. |
770 | If @var{val} is a coding system, that coding system is used for both | |
771 | reading the file and writing it. If @var{val} is a cons cell containing | |
772 | two coding systems, its @sc{car} specifies the coding system for | |
773 | decoding, and its @sc{cdr} specifies the coding system for encoding. | |
774 | ||
775 | If @var{val} is a function symbol, the function must return a coding | |
776 | system or a cons cell containing two coding systems. This value is used | |
777 | as described above. | |
778 | @end defvar | |
779 | ||
cc6d0d2c | 780 | @defvar process-coding-system-alist |
a9f0a989 | 781 | @tindex process-coding-system-alist |
cc6d0d2c RS |
782 | This variable is an alist specifying which coding systems to use for a |
783 | subprocess, depending on which program is running in the subprocess. It | |
784 | works like @code{file-coding-system-alist}, except that @var{pattern} is | |
785 | matched against the program name used to start the subprocess. The coding | |
786 | system or systems specified in this alist are used to initialize the | |
787 | coding systems used for I/O to the subprocess, but you can specify | |
788 | other coding systems later using @code{set-process-coding-system}. | |
789 | @end defvar | |
790 | ||
a9f0a989 RS |
791 | @strong{Warning:} Coding systems such as @code{undecided} which |
792 | determine the coding system from the data do not work entirely reliably | |
1911e6e5 | 793 | with asynchronous subprocess output. This is because Emacs handles |
a9f0a989 RS |
794 | asynchronous subprocess output in batches, as it arrives. If the coding |
795 | system leaves the character code conversion unspecified, or leaves the | |
796 | end-of-line conversion unspecified, Emacs must try to detect the proper | |
797 | conversion from one batch at a time, and this does not always work. | |
798 | ||
799 | Therefore, with an asynchronous subprocess, if at all possible, use a | |
800 | coding system which determines both the character code conversion and | |
801 | the end of line conversion---that is, one like @code{latin-1-unix}, | |
802 | rather than @code{undecided} or @code{latin-1}. | |
803 | ||
cc6d0d2c | 804 | @defvar network-coding-system-alist |
a9f0a989 | 805 | @tindex network-coding-system-alist |
cc6d0d2c RS |
806 | This variable is an alist that specifies the coding system to use for |
807 | network streams. It works much like @code{file-coding-system-alist}, | |
969fe9b5 | 808 | with the difference that the @var{pattern} in an element may be either a |
cc6d0d2c RS |
809 | port number or a regular expression. If it is a regular expression, it |
810 | is matched against the network service name used to open the network | |
811 | stream. | |
812 | @end defvar | |
813 | ||
cc6d0d2c | 814 | @defvar default-process-coding-system |
a9f0a989 | 815 | @tindex default-process-coding-system |
cc6d0d2c RS |
816 | This variable specifies the coding systems to use for subprocess (and |
817 | network stream) input and output, when nothing else specifies what to | |
818 | do. | |
819 | ||
a9f0a989 RS |
820 | The value should be a cons cell of the form @code{(@var{input-coding} |
821 | . @var{output-coding})}. Here @var{input-coding} applies to input from | |
822 | the subprocess, and @var{output-coding} applies to output to it. | |
cc6d0d2c RS |
823 | @end defvar |
824 | ||
a9f0a989 RS |
825 | @defun find-operation-coding-system operation &rest arguments |
826 | @tindex find-operation-coding-system | |
827 | This function returns the coding system to use (by default) for | |
828 | performing @var{operation} with @var{arguments}. The value has this | |
829 | form: | |
830 | ||
831 | @example | |
832 | (@var{decoding-system} @var{encoding-system}) | |
833 | @end example | |
834 | ||
835 | The first element, @var{decoding-system}, is the coding system to use | |
836 | for decoding (in case @var{operation} does decoding), and | |
837 | @var{encoding-system} is the coding system for encoding (in case | |
838 | @var{operation} does encoding). | |
839 | ||
840 | The argument @var{operation} should be an Emacs I/O primitive: | |
841 | @code{insert-file-contents}, @code{write-region}, @code{call-process}, | |
842 | @code{call-process-region}, @code{start-process}, or | |
843 | @code{open-network-stream}. | |
844 | ||
845 | The remaining arguments should be the same arguments that might be given | |
846 | to that I/O primitive. Depending on which primitive, one of those | |
847 | arguments is selected as the @dfn{target}. For example, if | |
848 | @var{operation} does file I/O, whichever argument specifies the file | |
849 | name is the target. For subprocess primitives, the process name is the | |
850 | target. For @code{open-network-stream}, the target is the service name | |
851 | or port number. | |
852 | ||
853 | This function looks up the target in @code{file-coding-system-alist}, | |
854 | @code{process-coding-system-alist}, or | |
855 | @code{network-coding-system-alist}, depending on @var{operation}. | |
856 | @xref{Default Coding Systems}. | |
857 | @end defun | |
858 | ||
cc6d0d2c | 859 | @node Specifying Coding Systems |
a9f0a989 | 860 | @subsection Specifying a Coding System for One Operation |
cc6d0d2c RS |
861 | |
862 | You can specify the coding system for a specific operation by binding | |
863 | the variables @code{coding-system-for-read} and/or | |
864 | @code{coding-system-for-write}. | |
865 | ||
cc6d0d2c | 866 | @defvar coding-system-for-read |
a9f0a989 | 867 | @tindex coding-system-for-read |
cc6d0d2c RS |
868 | If this variable is non-@code{nil}, it specifies the coding system to |
869 | use for reading a file, or for input from a synchronous subprocess. | |
870 | ||
871 | It also applies to any asynchronous subprocess or network stream, but in | |
872 | a different way: the value of @code{coding-system-for-read} when you | |
873 | start the subprocess or open the network stream specifies the input | |
874 | decoding method for that subprocess or network stream. It remains in | |
875 | use for that subprocess or network stream unless and until overridden. | |
876 | ||
877 | The right way to use this variable is to bind it with @code{let} for a | |
878 | specific I/O operation. Its global value is normally @code{nil}, and | |
879 | you should not globally set it to any other value. Here is an example | |
880 | of the right way to use the variable: | |
881 | ||
882 | @example | |
883 | ;; @r{Read the file with no character code conversion.} | |
969fe9b5 | 884 | ;; @r{Assume @sc{crlf} represents end-of-line.} |
cc6d0d2c RS |
885 | (let ((coding-system-for-write 'emacs-mule-dos)) |
886 | (insert-file-contents filename)) | |
887 | @end example | |
888 | ||
889 | When its value is non-@code{nil}, @code{coding-system-for-read} takes | |
a9f0a989 | 890 | precedence over all other methods of specifying a coding system to use for |
cc6d0d2c RS |
891 | input, including @code{file-coding-system-alist}, |
892 | @code{process-coding-system-alist} and | |
893 | @code{network-coding-system-alist}. | |
894 | @end defvar | |
895 | ||
cc6d0d2c | 896 | @defvar coding-system-for-write |
a9f0a989 | 897 | @tindex coding-system-for-write |
cc6d0d2c RS |
898 | This works much like @code{coding-system-for-read}, except that it |
899 | applies to output rather than input. It affects writing to files, | |
900 | subprocesses, and net connections. | |
901 | ||
902 | When a single operation does both input and output, as do | |
903 | @code{call-process-region} and @code{start-process}, both | |
904 | @code{coding-system-for-read} and @code{coding-system-for-write} | |
905 | affect it. | |
906 | @end defvar | |
907 | ||
cc6d0d2c | 908 | @defvar inhibit-eol-conversion |
a9f0a989 | 909 | @tindex inhibit-eol-conversion |
cc6d0d2c RS |
910 | When this variable is non-@code{nil}, no end-of-line conversion is done, |
911 | no matter which coding system is specified. This applies to all the | |
912 | Emacs I/O and subprocess primitives, and to the explicit encoding and | |
913 | decoding functions (@pxref{Explicit Encoding}). | |
914 | @end defvar | |
915 | ||
cc6d0d2c | 916 | @node Explicit Encoding |
a9f0a989 | 917 | @subsection Explicit Encoding and Decoding |
cc6d0d2c RS |
918 | @cindex encoding text |
919 | @cindex decoding text | |
920 | ||
921 | All the operations that transfer text in and out of Emacs have the | |
922 | ability to use a coding system to encode or decode the text. | |
923 | You can also explicitly encode and decode text using the functions | |
924 | in this section. | |
925 | ||
926 | @cindex raw bytes | |
927 | The result of encoding, and the input to decoding, are not ordinary | |
928 | text. They are ``raw bytes''---bytes that represent text in the same | |
929 | way that an external file would. When a buffer contains raw bytes, it | |
930 | is most natural to mark that buffer as using unibyte representation, | |
931 | using @code{set-buffer-multibyte} (@pxref{Selecting a Representation}), | |
969fe9b5 RS |
932 | but this is not required. If the buffer's contents are only temporarily |
933 | raw, leave the buffer multibyte, which will be correct after you decode | |
934 | them. | |
cc6d0d2c RS |
935 | |
936 | The usual way to get raw bytes in a buffer, for explicit decoding, is | |
969fe9b5 | 937 | to read them from a file with @code{insert-file-contents-literally} |
cc6d0d2c | 938 | (@pxref{Reading from Files}) or specify a non-@code{nil} @var{rawfile} |
969fe9b5 | 939 | argument when visiting a file with @code{find-file-noselect}. |
cc6d0d2c RS |
940 | |
941 | The usual way to use the raw bytes that result from explicitly | |
942 | encoding text is to copy them to a file or process---for example, to | |
969fe9b5 | 943 | write them with @code{write-region} (@pxref{Writing to Files}), and |
cc6d0d2c RS |
944 | suppress encoding for that @code{write-region} call by binding |
945 | @code{coding-system-for-write} to @code{no-conversion}. | |
946 | ||
1911e6e5 RS |
947 | Raw bytes sometimes contain overlong byte-sequences that look like a |
948 | proper multibyte character plus extra bytes containing trailing codes. | |
949 | For most purposes, Emacs treats such a sequence in a buffer or string as | |
950 | a single character, and if you look at its character code, you get the | |
951 | value that corresponds to the multibyte character sequence---the extra | |
952 | bytes are disregarded. This behavior is not quite clean, but raw bytes | |
953 | are used only in limited places in Emacs, so as a practical matter | |
954 | problems can be avoided. | |
955 | ||
cc6d0d2c | 956 | @defun encode-coding-region start end coding-system |
a9f0a989 | 957 | @tindex encode-coding-region |
cc6d0d2c | 958 | This function encodes the text from @var{start} to @var{end} according |
969fe9b5 RS |
959 | to coding system @var{coding-system}. The encoded text replaces the |
960 | original text in the buffer. The result of encoding is ``raw bytes,'' | |
961 | but the buffer remains multibyte if it was multibyte before. | |
cc6d0d2c RS |
962 | @end defun |
963 | ||
cc6d0d2c | 964 | @defun encode-coding-string string coding-system |
a9f0a989 | 965 | @tindex encode-coding-string |
cc6d0d2c RS |
966 | This function encodes the text in @var{string} according to coding |
967 | system @var{coding-system}. It returns a new string containing the | |
969fe9b5 | 968 | encoded text. The result of encoding is a unibyte string of ``raw bytes.'' |
cc6d0d2c RS |
969 | @end defun |
970 | ||
cc6d0d2c | 971 | @defun decode-coding-region start end coding-system |
a9f0a989 | 972 | @tindex decode-coding-region |
cc6d0d2c RS |
973 | This function decodes the text from @var{start} to @var{end} according |
974 | to coding system @var{coding-system}. The decoded text replaces the | |
975 | original text in the buffer. To make explicit decoding useful, the text | |
976 | before decoding ought to be ``raw bytes.'' | |
977 | @end defun | |
978 | ||
cc6d0d2c | 979 | @defun decode-coding-string string coding-system |
a9f0a989 | 980 | @tindex decode-coding-string |
cc6d0d2c RS |
981 | This function decodes the text in @var{string} according to coding |
982 | system @var{coding-system}. It returns a new string containing the | |
983 | decoded text. To make explicit decoding useful, the contents of | |
984 | @var{string} ought to be ``raw bytes.'' | |
985 | @end defun | |
969fe9b5 | 986 | |
a9f0a989 RS |
987 | @node Terminal I/O Encoding |
988 | @subsection Terminal I/O Encoding | |
989 | ||
990 | Emacs can decode keyboard input using a coding system, and encode | |
2eb4136f RS |
991 | terminal output. This is useful for terminals that transmit or display |
992 | text using a particular encoding such as Latin-1. Emacs does not set | |
993 | @code{last-coding-system-used} for encoding or decoding for the | |
994 | terminal. | |
a9f0a989 RS |
995 | |
996 | @defun keyboard-coding-system | |
997 | @tindex keyboard-coding-system | |
998 | This function returns the coding system that is in use for decoding | |
999 | keyboard input---or @code{nil} if no coding system is to be used. | |
1000 | @end defun | |
1001 | ||
1002 | @defun set-keyboard-coding-system coding-system | |
1003 | @tindex set-keyboard-coding-system | |
1004 | This function specifies @var{coding-system} as the coding system to | |
1005 | use for decoding keyboard input. If @var{coding-system} is @code{nil}, | |
1006 | that means do not decode keyboard input. | |
1007 | @end defun | |
1008 | ||
1009 | @defun terminal-coding-system | |
1010 | @tindex terminal-coding-system | |
1011 | This function returns the coding system that is in use for encoding | |
1012 | terminal output---or @code{nil} for no encoding. | |
1013 | @end defun | |
1014 | ||
1015 | @defun set-terminal-coding-system coding-system | |
1016 | @tindex set-terminal-coding-system | |
1017 | This function specifies @var{coding-system} as the coding system to use | |
1018 | for encoding terminal output. If @var{coding-system} is @code{nil}, | |
1019 | that means do not encode terminal output. | |
1020 | @end defun | |
1021 | ||
969fe9b5 | 1022 | @node MS-DOS File Types |
a9f0a989 | 1023 | @subsection MS-DOS File Types |
969fe9b5 RS |
1024 | @cindex DOS file types |
1025 | @cindex MS-DOS file types | |
1026 | @cindex Windows file types | |
1027 | @cindex file types on MS-DOS and Windows | |
1028 | @cindex text files and binary files | |
1029 | @cindex binary files and text files | |
1030 | ||
1031 | Emacs on MS-DOS and on MS-Windows recognizes certain file names as | |
a9f0a989 RS |
1032 | text files or binary files. By ``binary file'' we mean a file of |
1033 | literal byte values that are not necessary meant to be characters. | |
1034 | Emacs does no end-of-line conversion and no character code conversion | |
1035 | for a binary file. Meanwhile, when you create a new file which is | |
1036 | marked by its name as a ``text file'', Emacs uses DOS end-of-line | |
1037 | conversion. | |
969fe9b5 RS |
1038 | |
1039 | @defvar buffer-file-type | |
1040 | This variable, automatically buffer-local in each buffer, records the | |
a9f0a989 RS |
1041 | file type of the buffer's visited file. When a buffer does not specify |
1042 | a coding system with @code{buffer-file-coding-system}, this variable is | |
1043 | used to determine which coding system to use when writing the contents | |
1044 | of the buffer. It should be @code{nil} for text, @code{t} for binary. | |
1045 | If it is @code{t}, the coding system is @code{no-conversion}. | |
1046 | Otherwise, @code{undecided-dos} is used. | |
1047 | ||
1048 | Normally this variable is set by visiting a file; it is set to | |
1049 | @code{nil} if the file was visited without any actual conversion. | |
969fe9b5 RS |
1050 | @end defvar |
1051 | ||
1052 | @defopt file-name-buffer-file-type-alist | |
1053 | This variable holds an alist for recognizing text and binary files. | |
1054 | Each element has the form (@var{regexp} . @var{type}), where | |
1055 | @var{regexp} is matched against the file name, and @var{type} may be | |
1056 | @code{nil} for text, @code{t} for binary, or a function to call to | |
1057 | compute which. If it is a function, then it is called with a single | |
1058 | argument (the file name) and should return @code{t} or @code{nil}. | |
1059 | ||
1060 | Emacs when running on MS-DOS or MS-Windows checks this alist to decide | |
1061 | which coding system to use when reading a file. For a text file, | |
1062 | @code{undecided-dos} is used. For a binary file, @code{no-conversion} | |
1063 | is used. | |
1064 | ||
1065 | If no element in this alist matches a given file name, then | |
1066 | @code{default-buffer-file-type} says how to treat the file. | |
1067 | @end defopt | |
1068 | ||
1069 | @defopt default-buffer-file-type | |
1070 | This variable says how to handle files for which | |
1071 | @code{file-name-buffer-file-type-alist} says nothing about the type. | |
1072 | ||
1073 | If this variable is non-@code{nil}, then these files are treated as | |
a9f0a989 RS |
1074 | binary: the coding system @code{no-conversion} is used. Otherwise, |
1075 | nothing special is done for them---the coding system is deduced solely | |
1076 | from the file contents, in the usual Emacs fashion. | |
969fe9b5 RS |
1077 | @end defopt |
1078 | ||
a9f0a989 RS |
1079 | @node Input Methods |
1080 | @section Input Methods | |
1081 | @cindex input methods | |
1082 | ||
1083 | @dfn{Input methods} provide convenient ways of entering non-@sc{ASCII} | |
1084 | characters from the keyboard. Unlike coding systems, which translate | |
1085 | non-@sc{ASCII} characters to and from encodings meant to be read by | |
1086 | programs, input methods provide human-friendly commands. (@xref{Input | |
1087 | Methods,,, emacs, The GNU Emacs Manual}, for information on how users | |
1088 | use input methods to enter text.) How to define input methods is not | |
1089 | yet documented in this manual, but here we describe how to use them. | |
1090 | ||
1091 | Each input method has a name, which is currently a string; | |
1092 | in the future, symbols may also be usable as input method names. | |
1093 | ||
1094 | @tindex current-input-method | |
1095 | @defvar current-input-method | |
1096 | This variable holds the name of the input method now active in the | |
1097 | current buffer. (It automatically becomes local in each buffer when set | |
1098 | in any fashion.) It is @code{nil} if no input method is active in the | |
1099 | buffer now. | |
969fe9b5 RS |
1100 | @end defvar |
1101 | ||
a9f0a989 RS |
1102 | @tindex default-input-method |
1103 | @defvar default-input-method | |
1104 | This variable holds the default input method for commands that choose an | |
1105 | input method. Unlike @code{current-input-method}, this variable is | |
1106 | normally global. | |
969fe9b5 | 1107 | @end defvar |
a9f0a989 RS |
1108 | |
1109 | @tindex set-input-method | |
1110 | @defun set-input-method input-method | |
1111 | This function activates input method @var{input-method} for the current | |
1112 | buffer. It also sets @code{default-input-method} to @var{input-method}. | |
1113 | If @var{input-method} is @code{nil}, this function deactivates any input | |
1114 | method for the current buffer. | |
1115 | @end defun | |
1116 | ||
1117 | @tindex read-input-method-name | |
1118 | @defun read-input-method-name prompt &optional default inhibit-null | |
1119 | This function reads an input method name with the minibuffer, prompting | |
1120 | with @var{prompt}. If @var{default} is non-@code{nil}, that is returned | |
1121 | by default, if the user enters empty input. However, if | |
1122 | @var{inhibit-null} is non-@code{nil}, empty input signals an error. | |
1123 | ||
1124 | The returned value is a string. | |
1125 | @end defun | |
1126 | ||
1127 | @tindex input-method-alist | |
1128 | @defvar input-method-alist | |
1129 | This variable defines all the supported input methods. | |
1130 | Each element defines one input method, and should have the form: | |
1131 | ||
1132 | @example | |
1911e6e5 RS |
1133 | (@var{input-method} @var{language-env} @var{activate-func} |
1134 | @var{title} @var{description} @var{args}...) | |
a9f0a989 RS |
1135 | @end example |
1136 | ||
1911e6e5 RS |
1137 | Here @var{input-method} is the input method name, a string; |
1138 | @var{language-env} is another string, the name of the language | |
1139 | environment this input method is recommended for. (That serves only for | |
1140 | documentation purposes.) | |
a9f0a989 RS |
1141 | |
1142 | @var{title} is a string to display in the mode line while this method is | |
1143 | active. @var{description} is a string describing this method and what | |
1144 | it is good for. | |
1145 | ||
1146 | @var{activate-func} is a function to call to activate this method. The | |
1147 | @var{args}, if any, are passed as arguments to @var{activate-func}. All | |
1148 | told, the arguments to @var{activate-func} are @var{input-method} and | |
1149 | the @var{args}. | |
1911e6e5 | 1150 | @end defvar |
a9f0a989 | 1151 | |
2eb4136f RS |
1152 | The fundamental interface to input methods is through the |
1153 | variable @code{input-method-function}. @xref{Reading One Event}. |