@c -*-texinfo-*-
@c This is part of the GNU Emacs Lisp Reference Manual.
-@c Copyright (C) 1998 Free Software Foundation, Inc.
+@c Copyright (C) 1998, 1999 Free Software Foundation, Inc.
@c See the file elisp.texi for copying conditions.
@setfilename ../info/characters
@node Non-ASCII Characters, Searching and Matching, Text, Top
-@chapter Non-ASCII Characters
+@chapter Non-@sc{ascii} Characters
@cindex multibyte characters
-@cindex non-ASCII characters
+@cindex non-@sc{ascii} characters
- This chapter covers the special issues relating to non-@sc{ASCII}
+ This chapter covers the special issues relating to non-@sc{ascii}
characters and how they are stored in strings and buffers.
@menu
* Translation of Characters::
* Coding Systems::
* Input Methods::
+* Locales:: Interacting with the POSIX locale.
@end menu
@node Text Representations
@cindex unibyte text
In unibyte representation, each character occupies one byte and
therefore the possible character codes range from 0 to 255. Codes 0
-through 127 are @sc{ASCII} characters; the codes from 128 through 255
-are used for one non-@sc{ASCII} character set (you can choose which
+through 127 are @sc{ascii} characters; the codes from 128 through 255
+are used for one non-@sc{ascii} character set (you can choose which
character set by setting the variable @code{nonascii-insert-offset}).
@cindex leading code
character are always in the range 160 through 255 (octal 0240 through
0377); these values are @dfn{trailing codes}.
+ Some sequences of bytes do not form meaningful multibyte characters:
+for example, a single isolated byte in the range 128 through 255 is
+never meaningful. Such byte sequences are not entirely valid, and never
+appear in proper multibyte text (since that consists of a sequence of
+@emph{characters}); but they can appear as part of ``raw bytes''
+(@pxref{Explicit Encoding}).
+
In a buffer, the buffer-local value of the variable
@code{enable-multibyte-characters} specifies the representation used.
-The representation for a string is determined based on the string
-contents when the string is constructed.
+The representation for a string is determined and recorded in the string
+when the string is constructed.
@defvar enable-multibyte-characters
-@tindex enable-multibyte-characters
This variable specifies the current buffer's text representation.
If it is non-@code{nil}, the buffer contains multibyte text; otherwise,
it contains unibyte text.
@end defvar
@defvar default-enable-multibyte-characters
-@tindex default-enable-multibyte-characters
This variable's value is entirely equivalent to @code{(default-value
'enable-multibyte-characters)}, and setting this variable changes that
default value. Setting the local binding of
default value to @code{nil} early in startup.
@end defvar
+@defun position-bytes position
+@tindex position-bytes
+Return the byte-position corresponding to buffer position @var{position}
+in the current buffer.
+@end defun
+
+@defun byte-to-position byte-position
+@tindex byte-to-position
+Return the buffer position corresponding to byte-position
+@var{byte-position} in the current buffer.
+@end defun
+
@defun multibyte-string-p string
-@tindex multibyte-string-p
-Return @code{t} if @var{string} contains multibyte characters.
+Return @code{t} if @var{string} is a multibyte string.
@end defun
@node Converting Representations
acceptable because the buffer's representation is a choice made by the
user that cannot be overridden automatically.
- Converting unibyte text to multibyte text leaves @sc{ASCII} characters
-unchanged, and likewise 128 through 159. It converts the non-@sc{ASCII}
+ Converting unibyte text to multibyte text leaves @sc{ascii} characters
+unchanged, and likewise 128 through 159. It converts the non-@sc{ascii}
codes 160 through 255 by adding the value @code{nonascii-insert-offset}
to each character code. By setting this variable, you specify which
character set the unibyte characters correspond to (@pxref{Character
Sets}). For example, if @code{nonascii-insert-offset} is 2048, which is
@code{(- (make-char 'latin-iso8859-1) 128)}, then the unibyte
-non-@sc{ASCII} characters correspond to Latin 1. If it is 2688, which
+non-@sc{ascii} characters correspond to Latin 1. If it is 2688, which
is @code{(- (make-char 'greek-iso8859-7) 128)}, then they correspond to
Greek letters.
- Converting multibyte text to unibyte is simpler: it performs
-logical-and of each character code with 255. If
-@code{nonascii-insert-offset} has a reasonable value, corresponding to
-the beginning of some character set, this conversion is the inverse of
-the other: converting unibyte text to multibyte and back to unibyte
-reproduces the original unibyte text.
+ Converting multibyte text to unibyte is simpler: it discards all but
+the low 8 bits of each character code. If @code{nonascii-insert-offset}
+has a reasonable value, corresponding to the beginning of some character
+set, this conversion is the inverse of the other: converting unibyte
+text to multibyte and back to unibyte reproduces the original unibyte
+text.
@defvar nonascii-insert-offset
-@tindex nonascii-insert-offset
-This variable specifies the amount to add to a non-@sc{ASCII} character
+This variable specifies the amount to add to a non-@sc{ascii} character
when converting unibyte text to multibyte. It also applies when
@code{self-insert-command} inserts a character in the unibyte
-non-@sc{ASCII} range, 128 through 255. However, the function
+non-@sc{ascii} range, 128 through 255. However, the function
@code{insert-char} does not perform this conversion.
The right value to use to select character set @var{cs} is @code{(-
@end defvar
@defvar nonascii-translation-table
-@tindex nonascii-translation-table
This variable provides a more general alternative to
@code{nonascii-insert-offset}. You can use it to specify independently
how to translate each code in the range of 128 through 255 into a
@end defvar
@defun string-make-unibyte string
-@tindex string-make-unibyte
This function converts the text of @var{string} to unibyte
representation, if it isn't already, and returns the result. If
@var{string} is a unibyte string, it is returned unchanged.
@end defun
@defun string-make-multibyte string
-@tindex string-make-multibyte
This function converts the text of @var{string} to multibyte
representation, if it isn't already, and returns the result. If
@var{string} is a multibyte string, it is returned unchanged.
multibyte when it was unibyte, or vice versa.
@defun set-buffer-multibyte multibyte
-@tindex set-buffer-multibyte
Set the representation type of the current buffer. If @var{multibyte}
is non-@code{nil}, the buffer becomes multibyte. If @var{multibyte}
is @code{nil}, the buffer becomes unibyte.
representation is in use. It also adjusts various data in the buffer
(including overlays, text properties and markers) so that they cover the
same text as they did before.
+
+You cannot use @code{set-buffer-multibyte} on an indirect buffer,
+because indirect buffers always inherit the representation of the
+base buffer.
@end defun
@defun string-as-unibyte string
-@tindex string-as-unibyte
This function returns a string with the same bytes as @var{string} but
treating each byte as a character. This means that the value may have
more characters than @var{string} has.
-If @var{string} is unibyte already, then the value is @var{string}
-itself.
+If @var{string} is already a unibyte string, then the value is
+@var{string} itself.
@end defun
@defun string-as-multibyte string
-@tindex string-as-multibyte
This function returns a string with the same bytes as @var{string} but
treating each multibyte sequence as one character. This means that the
value may have fewer characters than @var{string} has.
-If @var{string} is multibyte already, then the value is @var{string}
-itself.
+If @var{string} is already a multibyte string, then the value is
+@var{string} itself.
@end defun
@node Character Codes
codes for multibyte representation range from 0 to 524287, but not all
values in that range are valid. In particular, the values 128 through
255 are not legitimate in multibyte text (though they can occur in ``raw
-bytes''; @pxref{Explicit Encoding}). Only the @sc{ASCII} codes 0
+bytes''; @pxref{Explicit Encoding}). Only the @sc{ascii} codes 0
through 127 are fully legitimate in both representations.
@defun char-valid-p charcode
character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
@defun charsetp object
-@tindex charsetp
-Return @code{t} if @var{object} is a character set name symbol,
+Returns @code{t} if @var{object} is a symbol that names a character set,
@code{nil} otherwise.
@end defun
@defun charset-list
-@tindex charset-list
This function returns a list of all defined character set names.
@end defun
@defun char-charset character
-@tindex char-charset
-This function returns the name of the character
-set that @var{character} belongs to.
+This function returns the name of the character set that @var{character}
+belongs to.
+@end defun
+
+@defun charset-plist charset
+@tindex charset-plist
+This function returns the charset property list of the character set
+@var{charset}. Although @var{charset} is a symbol, this is not the same
+as the property list of that symbol. Charset properties are used for
+special purposes within Emacs; for example, @code{x-charset-registry}
+helps determine which fonts to use (@pxref{Font Selection}).
@end defun
@node Chars and Bytes
@cindex dimension (of character set)
In multibyte representation, each character occupies one or more
bytes. Each character set has an @dfn{introduction sequence}, which is
-normally one or two bytes long. (Exception: the @sc{ASCII} character
+normally one or two bytes long. (Exception: the @sc{ascii} character
set has a zero-length introduction sequence.) The introduction sequence
is the beginning of the byte sequence for any character in the character
set. The rest of the character's bytes distinguish it from the other
bytes is called the @dfn{dimension} of the character set.
@defun charset-dimension charset
-@tindex charset-dimension
-This function returns the dimension of @var{charset};
-at present, the dimension is always 1 or 2.
+This function returns the dimension of @var{charset}; at present, the
+dimension is always 1 or 2.
+@end defun
+
+@defun charset-bytes charset
+@tindex charset-bytes
+This function returns the number of bytes used to represent a character
+in character set @var{charset}.
@end defun
This is the simplest way to determine the byte length of a character
set's introduction sequence:
@example
-(- (char-bytes (make-char @var{charset}))
+(- (charset-bytes @var{charset})
(charset-dimension @var{charset}))
@end example
be concerned with the sequence of bytes used to represent a character,
because Emacs translates automatically when necessary.
-@defun char-bytes character
-@tindex char-bytes
-This function returns the number of bytes used to represent the
-character @var{character}. This depends only on the character set that
-@var{character} belongs to; it equals the dimension of that character
-set (@pxref{Character Sets}), plus the length of its introduction
-sequence.
-
-@example
-(char-bytes 2248)
- @result{} 2
-(char-bytes 65)
- @result{} 1
-(char-bytes 192)
- @result{} 1
-@end example
-
-The reason this function can give correct results for both multibyte and
-unibyte representations is that the non-@sc{ASCII} character codes used
-in those two representations do not overlap.
-@end defun
-
@defun split-char character
-@tindex split-char
Return a list containing the name of the character set of
@var{character}, followed by one or two byte values (integers) which
identify @var{character} within that character set. The number of byte
@result{} (ascii 65)
@end example
-Unibyte non-@sc{ASCII} characters are considered as part of
+Unibyte non-@sc{ascii} characters are considered as part of
the @code{ascii} character set:
@example
@end defun
@defun make-char charset &rest byte-values
-@tindex make-char
This function returns the character in character set @var{charset}
identified by @var{byte-values}. This is roughly the inverse of
@code{split-char}. Normally, you should specify either one or two
of the text in question.
@defun find-charset-region beg end &optional translation
-@tindex find-charset-region
This function returns a list of the character sets that appear in the
current buffer between positions @var{beg} and @var{end}.
is non-@code{nil}, then each character in the region is translated
through this table, and the value returned describes the translated
characters instead of the characters actually in the buffer.
+
+In two peculiar cases, the value includes the symbol @code{unknown}:
+
+@itemize @bullet
+@item
+When a unibyte buffer contains non-@sc{ascii} characters.
+
+@item
+When a multibyte buffer contains invalid byte-sequences (raw bytes).
+@xref{Explicit Encoding}.
+@end itemize
@end defun
@defun find-charset-string string &optional translation
-@tindex find-charset-string
-This function returns a list of the character sets
-that appear in the string @var{string}.
-
-The optional argument @var{translation} specifies a
-translation table; see @code{find-charset-region}, above.
+This function returns a list of the character sets that appear in the
+string @var{string}. It is just like @code{find-charset-region}, except
+that it applies to the contents of @var{string} instead of part of the
+current buffer.
@end defun
@node Translation of Characters
translation tables; there are also default translation tables which
apply to all other coding systems.
-@defun make-translation-table translations
-This function returns a translation table based on the arguments
-@var{translations}. Each argument---each element of
-@var{translations}---should be a list of the form @code{(@var{from}
+@defun make-translation-table &rest translations
+This function returns a translation table based on the argument
+@var{translations}. Each element of
+@var{translations} should be a list of the form @code{(@var{from}
. @var{to})}; this says to translate the character @var{from} into
@var{to}.
characters that result from ordinary decoding. If a coding system has
property @code{character-translation-table-for-decode}, that specifies
the translation table to use. Otherwise, if
-@code{standard-character-translation-table-for-decode} is
-non-@code{nil}, decoding uses that table.
+@code{standard-translation-table-for-decode} is non-@code{nil}, decoding
+uses that table.
In encoding, the translation table's translations are applied to the
characters in the buffer, and the result of translation is actually
encoded. If a coding system has property
@code{character-translation-table-for-encode}, that specifies the
translation table to use. Otherwise the variable
-@code{standard-character-translation-table-for-encode} specifies the
-translation table.
+@code{standard-translation-table-for-encode} specifies the translation
+table.
-@defvar standard-character-translation-table-for-decode
+@defvar standard-translation-table-for-decode
This is the default translation table for decoding, for
coding systems that don't specify any other translation table.
@end defvar
-@defvar standard-character-translation-table-for-encode
+@defvar standard-translation-table-for-encode
This is the default translation table for encoding, for
coding systems that don't specify any other translation table.
@end defvar
character code conversion and end-of-line conversion as specified
by a particular @dfn{coding system}.
+ How to define a coding system is an arcane matter, and is not
+documented here.
+
@menu
* Coding System Basics::
* Encoding and I/O::
(Russian) alphabet: ISO, Alternativnyj, and KOI8.
Most coding systems specify a particular character code for
-conversion, but some of them leave this unspecified---to be chosen
-heuristically based on the data.
+conversion, but some of them leave the choice unspecified---to be chosen
+heuristically for each file, based on the data.
@cindex end of line conversion
@dfn{End of line conversion} handles three different conventions used
on various systems for representing end of line in files. The Unix
convention is to use the linefeed character (also called newline). The
-DOS convention is to use the two character sequence, carriage-return
-linefeed, at the end of a line. The Mac convention is to use just
-carriage-return.
+DOS convention is to use a carriage-return and a linefeed at the end of
+a line. The Mac convention is to use just carriage-return.
@cindex base coding system
@cindex variant coding system
that the result is multibyte data.
@defun coding-system-get coding-system property
-@tindex coding-system-get
This function returns the specified property of the coding system
@var{coding-system}. Most coding system properties exist for internal
purposes, but one that you might find useful is @code{mime-charset}.
you will want to find out afterwards which coding system was chosen.
@defvar buffer-file-coding-system
-@tindex buffer-file-coding-system
This variable records the coding system that was used for visiting the
current buffer. It is used for saving the buffer, and for writing part
of the buffer with @code{write-region}. When those operations ask the
user to specify a different coding system,
@code{buffer-file-coding-system} is updated to the coding system
specified.
+
+However, @code{buffer-file-coding-system} does not affect sending text
+to a subprocess.
@end defvar
@defvar save-buffer-coding-system
-@tindex save-buffer-coding-system
This variable specifies the coding system for saving the buffer---but it
-is not used for @code{write-region}. When saving the buffer asks the
-user to specify a different coding system, and
-@code{save-buffer-coding-system} was used, then it is updated to the
-coding system that was specified.
+is not used for @code{write-region}.
+
+When a command to save the buffer starts out to use
+@code{save-buffer-coding-system}, and that coding system cannot handle
+the actual text in the buffer, the command asks the user to choose
+another coding system. After that happens, the command also updates
+@code{save-buffer-coding-system} to represent the coding system that the
+user specified.
@end defvar
@defvar last-coding-system-used
-@tindex last-coding-system-used
I/O operations for files and subprocesses set this variable to the
coding system name that was used. The explicit encoding and decoding
functions (@pxref{Explicit Encoding}) set it too.
@strong{Warning:} Since receiving subprocess output sets this variable,
-it can change whenever Emacs waits; therefore, you should use copy the
-value shortly after the function call which stores the value you are
+it can change whenever Emacs waits; therefore, you should copy the
+value shortly after the function call that stores the value you are
interested in.
@end defvar
+ The variable @code{selection-coding-system} specifies how to encode
+selections for the window system. @xref{Window System Selections}.
+
@node Lisp and Coding Systems
@subsection Coding Systems in Lisp
- Here are Lisp facilities for working with coding systems;
+ Here are the Lisp facilities for working with coding systems:
@defun coding-system-list &optional base-only
-@tindex coding-system-list
This function returns a list of all coding system names (symbols). If
@var{base-only} is non-@code{nil}, the value includes only the
base coding systems. Otherwise, it includes variant coding systems as well.
@end defun
@defun coding-system-p object
-@tindex coding-system-p
This function returns @code{t} if @var{object} is a coding system
name.
@end defun
@defun check-coding-system coding-system
-@tindex check-coding-system
This function checks the validity of @var{coding-system}.
If that is valid, it returns @var{coding-system}.
Otherwise it signals an error with condition @code{coding-system-error}.
@end defun
@defun coding-system-change-eol-conversion coding-system eol-type
-@tindex coding-system-change-eol-conversion
This function returns a coding system which is like @var{coding-system}
except for its eol conversion, which is specified by @code{eol-type}.
@var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or
@end defun
@defun coding-system-change-text-conversion eol-coding text-coding
-@tindex coding-system-change-text-conversion
This function returns a coding system which uses the end-of-line
conversion of @var{eol-coding}, and the text conversion of
@var{text-coding}. If @var{text-coding} is @code{nil}, it returns
@end defun
@defun find-coding-systems-region from to
-@tindex find-coding-systems-region
This function returns a list of coding systems that could be used to
encode a text between @var{from} and @var{to}. All coding systems in
the list can safely encode any multibyte characters in that portion of
@end defun
@defun find-coding-systems-string string
-@tindex find-coding-systems-string
This function returns a list of coding systems that could be used to
encode the text of @var{string}. All coding systems in the list can
safely encode any multibyte characters in @var{string}. If the text
@end defun
@defun find-coding-systems-for-charsets charsets
-@tindex find-coding-systems-for-charsets
This function returns a list of coding systems that could be used to
encode all the character sets in the list @var{charsets}.
@end defun
@defun detect-coding-region start end &optional highest
-@tindex detect-coding-region
This function chooses a plausible coding system for decoding the text
from @var{start} to @var{end}. This text should be ``raw bytes''
(@pxref{Explicit Encoding}).
return value is just one coding system, the one that is highest in
priority.
-If the region contains only @sc{ASCII} characters, the value
+If the region contains only @sc{ascii} characters, the value
is @code{undecided} or @code{(undecided)}.
@end defun
@defun detect-coding-string string highest
-@tindex detect-coding-string
This function is like @code{detect-coding-region} except that it
operates on the contents of @var{string} instead of bytes in the buffer.
@end defun
@node User-Chosen Coding Systems
@subsection User-Chosen Coding Systems
-@tindex select-safe-coding-system
@defun select-safe-coding-system from to &optional preferred-coding-system
-This function selects a coding system for encoding the between
+This function selects a coding system for encoding the text between
@var{from} and @var{to}, asking the user to choose if necessary.
The optional argument @var{preferred-coding-system} specifies a coding
-system try first. If it can handle the text in the specified region,
-then it is used. If this argument is omitted, the current buffer's
-value of @code{buffer-file-coding-system} is tried first.
+system to try first. If that one can handle the text in the specified
+region, then it is used. If this argument is omitted, the current
+buffer's value of @code{buffer-file-coding-system} is tried first.
If the region contains some multibyte characters that the preferred
coding system cannot encode, this function asks the user to choose from
system, with completion. @xref{Completion}.
@defun read-coding-system prompt &optional default
-@tindex read-coding-system
This function reads a coding system using the minibuffer, prompting with
string @var{prompt}, and returns the coding system name as a symbol. If
the user enters null input, @var{default} specifies which coding system
@end defun
@defun read-non-nil-coding-system prompt
-@tindex read-non-nil-coding-system
This function reads a coding system using the minibuffer, prompting with
string @var{prompt}, and returns the coding system name as a symbol. If
the user tries to enter null input, it asks the user to try again.
(@pxref{Specifying Coding Systems}).
@defvar file-coding-system-alist
-@tindex file-coding-system-alist
This variable is an alist that specifies the coding systems to use for
reading and writing particular files. Each element has the form
@code{(@var{pattern} . @var{coding})}, where @var{pattern} is a regular
names that match @var{pattern}.
The @sc{cdr} of the element, @var{coding}, should be either a coding
-system, a cons cell containing two coding systems, or a function symbol.
-If @var{val} is a coding system, that coding system is used for both
-reading the file and writing it. If @var{val} is a cons cell containing
-two coding systems, its @sc{car} specifies the coding system for
-decoding, and its @sc{cdr} specifies the coding system for encoding.
-
-If @var{val} is a function symbol, the function must return a coding
+system, a cons cell containing two coding systems, or a function name (a
+symbol with a function definition). If @var{coding} is a coding system,
+that coding system is used for both reading the file and writing it. If
+@var{coding} is a cons cell containing two coding systems, its @sc{car}
+specifies the coding system for decoding, and its @sc{cdr} specifies the
+coding system for encoding.
+
+If @var{coding} is a function name, the function must return a coding
system or a cons cell containing two coding systems. This value is used
as described above.
@end defvar
@defvar process-coding-system-alist
-@tindex process-coding-system-alist
This variable is an alist specifying which coding systems to use for a
subprocess, depending on which program is running in the subprocess. It
works like @code{file-coding-system-alist}, except that @var{pattern} is
other coding systems later using @code{set-process-coding-system}.
@end defvar
- @strong{Warning:} Coding systems such as @code{undecided} which
-determine the coding system from the data do not work entirely reliably
+ @strong{Warning:} Coding systems such as @code{undecided}, which
+determine the coding system from the data, do not work entirely reliably
with asynchronous subprocess output. This is because Emacs handles
asynchronous subprocess output in batches, as it arrives. If the coding
system leaves the character code conversion unspecified, or leaves the
rather than @code{undecided} or @code{latin-1}.
@defvar network-coding-system-alist
-@tindex network-coding-system-alist
This variable is an alist that specifies the coding system to use for
network streams. It works much like @code{file-coding-system-alist},
with the difference that the @var{pattern} in an element may be either a
@end defvar
@defvar default-process-coding-system
-@tindex default-process-coding-system
This variable specifies the coding systems to use for subprocess (and
network stream) input and output, when nothing else specifies what to
do.
@end defvar
@defun find-operation-coding-system operation &rest arguments
-@tindex find-operation-coding-system
This function returns the coding system to use (by default) for
performing @var{operation} with @var{arguments}. The value has this
form:
@var{encoding-system} is the coding system for encoding (in case
@var{operation} does encoding).
-The argument @var{operation} should be an Emacs I/O primitive:
+The argument @var{operation} should be a symbol, one of
@code{insert-file-contents}, @code{write-region}, @code{call-process},
@code{call-process-region}, @code{start-process}, or
-@code{open-network-stream}.
+@code{open-network-stream}. These are the names of the Emacs I/O primitives
+that can do coding system conversion.
The remaining arguments should be the same arguments that might be given
-to that I/O primitive. Depending on which primitive, one of those
+to that I/O primitive. Depending on the primitive, one of those
arguments is selected as the @dfn{target}. For example, if
@var{operation} does file I/O, whichever argument specifies the file
name is the target. For subprocess primitives, the process name is the
@code{coding-system-for-write}.
@defvar coding-system-for-read
-@tindex coding-system-for-read
If this variable is non-@code{nil}, it specifies the coding system to
use for reading a file, or for input from a synchronous subprocess.
@end defvar
@defvar coding-system-for-write
-@tindex coding-system-for-write
This works much like @code{coding-system-for-read}, except that it
applies to output rather than input. It affects writing to files,
-subprocesses, and net connections.
+as well as sending output to subprocesses and net connections.
When a single operation does both input and output, as do
@code{call-process-region} and @code{start-process}, both
@end defvar
@defvar inhibit-eol-conversion
-@tindex inhibit-eol-conversion
When this variable is non-@code{nil}, no end-of-line conversion is done,
no matter which coding system is specified. This applies to all the
Emacs I/O and subprocess primitives, and to the explicit encoding and
suppress encoding for that @code{write-region} call by binding
@code{coding-system-for-write} to @code{no-conversion}.
+ Raw bytes typically contain stray individual bytes with values in the
+range 128 through 255, that are legitimate only as part of multibyte
+sequences. Even if the buffer is multibyte, Emacs treats each such
+individual byte as a character and uses the byte value as its character
+code. In this way, character codes 128 through 255 can be found in a
+multibyte buffer, even though they are not legitimate multibyte
+character codes.
+
Raw bytes sometimes contain overlong byte-sequences that look like a
-proper multibyte character plus extra bytes containing trailing codes.
-For most purposes, Emacs treats such a sequence in a buffer or string as
-a single character, and if you look at its character code, you get the
-value that corresponds to the multibyte character sequence---the extra
-bytes are disregarded. This behavior is not quite clean, but raw bytes
-are used only in limited places in Emacs, so as a practical matter
-problems can be avoided.
+proper multibyte character plus extra superfluous trailing codes. For
+most purposes, Emacs treats such a sequence in a buffer or string as a
+single character, and if you look at its character code, you get the
+value that corresponds to the multibyte character
+sequence---disregarding the extra trailing codes. This is not quite
+clean, but raw bytes are used only in limited ways, so as a practical
+matter it is not worth the trouble to treat this case differently.
+
+ When a multibyte buffer contains illegitimate byte sequences,
+sometimes insertion or deletion can cause them to coalesce into a
+legitimate multibyte character. For example, suppose the buffer
+contains the sequence 129 68 192, 68 being the character @samp{D}. If
+you delete the @samp{D}, the bytes 129 and 192 become adjacent, and thus
+become one multibyte character (Latin-1 A with grave accent). Point
+moves to one side or the other of the character, since it cannot be
+within a character. Don't be alarmed by this.
+
+ Some really peculiar situations prevent proper coalescence. For
+example, if you narrow the buffer so that the accessible portion begins
+just before the @samp{D}, then delete the @samp{D}, the two surrounding
+bytes cannot coalesce because one of them is outside the accessible
+portion of the buffer. In this case, the deletion cannot be done, so
+@code{delete-region} signals an error.
+
+ Here are the functions to perform explicit encoding or decoding. The
+decoding functions produce ``raw bytes''; the encoding functions are
+meant to operate on ``raw bytes''. All of these functions discard text
+properties.
@defun encode-coding-region start end coding-system
-@tindex encode-coding-region
This function encodes the text from @var{start} to @var{end} according
to coding system @var{coding-system}. The encoded text replaces the
original text in the buffer. The result of encoding is ``raw bytes,''
@end defun
@defun encode-coding-string string coding-system
-@tindex encode-coding-string
This function encodes the text in @var{string} according to coding
system @var{coding-system}. It returns a new string containing the
encoded text. The result of encoding is a unibyte string of ``raw bytes.''
@end defun
@defun decode-coding-region start end coding-system
-@tindex decode-coding-region
This function decodes the text from @var{start} to @var{end} according
to coding system @var{coding-system}. The decoded text replaces the
original text in the buffer. To make explicit decoding useful, the text
@end defun
@defun decode-coding-string string coding-system
-@tindex decode-coding-string
This function decodes the text in @var{string} according to coding
system @var{coding-system}. It returns a new string containing the
decoded text. To make explicit decoding useful, the contents of
@subsection Terminal I/O Encoding
Emacs can decode keyboard input using a coding system, and encode
-terminal output. This kind of decoding and encoding does not set
-@code{last-coding-system-used}.
+terminal output. This is useful for terminals that transmit or display
+text using a particular encoding such as Latin-1. Emacs does not set
+@code{last-coding-system-used} for encoding or decoding for the
+terminal.
@defun keyboard-coding-system
-@tindex keyboard-coding-system
This function returns the coding system that is in use for decoding
keyboard input---or @code{nil} if no coding system is to be used.
@end defun
@defun set-keyboard-coding-system coding-system
-@tindex set-keyboard-coding-system
This function specifies @var{coding-system} as the coding system to
use for decoding keyboard input. If @var{coding-system} is @code{nil},
that means do not decode keyboard input.
@end defun
@defun terminal-coding-system
-@tindex terminal-coding-system
This function returns the coding system that is in use for encoding
terminal output---or @code{nil} for no encoding.
@end defun
@defun set-terminal-coding-system coding-system
-@tindex set-terminal-coding-system
This function specifies @var{coding-system} as the coding system to use
for encoding terminal output. If @var{coding-system} is @code{nil},
that means do not encode terminal output.
@cindex text files and binary files
@cindex binary files and text files
- Emacs on MS-DOS and on MS-Windows recognizes certain file names as
-text files or binary files. By ``binary file'' we mean a file of
-literal byte values that are not necessary meant to be characters.
-Emacs does no end-of-line conversion and no character code conversion
-for a binary file. Meanwhile, when you create a new file which is
-marked by its name as a ``text file'', Emacs uses DOS end-of-line
-conversion.
+ On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
+end-of-line conversion for a file by looking at the file's name. This
+feature classifies fils as @dfn{text files} and @dfn{binary files}. By
+``binary file'' we mean a file of literal byte values that are not
+necessarily meant to be characters; Emacs does no end-of-line conversion
+and no character code conversion for them. On the other hand, the bytes
+in a text file are intended to represent characters; when you create a
+new file whose name implies that it is a text file, Emacs uses DOS
+end-of-line conversion.
@defvar buffer-file-type
This variable, automatically buffer-local in each buffer, records the
compute which. If it is a function, then it is called with a single
argument (the file name) and should return @code{t} or @code{nil}.
-Emacs when running on MS-DOS or MS-Windows checks this alist to decide
+When running on MS-DOS or MS-Windows, Emacs checks this alist to decide
which coding system to use when reading a file. For a text file,
@code{undecided-dos} is used. For a binary file, @code{no-conversion}
is used.
@section Input Methods
@cindex input methods
- @dfn{Input methods} provide convenient ways of entering non-@sc{ASCII}
+ @dfn{Input methods} provide convenient ways of entering non-@sc{ascii}
characters from the keyboard. Unlike coding systems, which translate
-non-@sc{ASCII} characters to and from encodings meant to be read by
+non-@sc{ascii} characters to and from encodings meant to be read by
programs, input methods provide human-friendly commands. (@xref{Input
Methods,,, emacs, The GNU Emacs Manual}, for information on how users
use input methods to enter text.) How to define input methods is not
Each input method has a name, which is currently a string;
in the future, symbols may also be usable as input method names.
-@tindex current-input-method
@defvar current-input-method
This variable holds the name of the input method now active in the
current buffer. (It automatically becomes local in each buffer when set
buffer now.
@end defvar
-@tindex default-input-method
@defvar default-input-method
This variable holds the default input method for commands that choose an
input method. Unlike @code{current-input-method}, this variable is
normally global.
@end defvar
-@tindex set-input-method
@defun set-input-method input-method
This function activates input method @var{input-method} for the current
buffer. It also sets @code{default-input-method} to @var{input-method}.
method for the current buffer.
@end defun
-@tindex read-input-method-name
@defun read-input-method-name prompt &optional default inhibit-null
This function reads an input method name with the minibuffer, prompting
with @var{prompt}. If @var{default} is non-@code{nil}, that is returned
The returned value is a string.
@end defun
-@tindex input-method-alist
@defvar input-method-alist
This variable defines all the supported input methods.
Each element defines one input method, and should have the form:
the @var{args}.
@end defvar
+ The fundamental interface to input methods is through the
+variable @code{input-method-function}. @xref{Reading One Event}.
+
+@node Locales
+@section Locales
+@cindex locale
+
+ POSIX defines a concept of ``locales'' which control which language
+to use in language-related features. These Emacs variables control
+how Emacs interacts with these features.
+
+@defvar locale-coding-system
+@tindex locale-coding-system
+This variable specifies the coding system to use for decoding system
+error messages, for encoding the format argument to
+@code{format-time-string}, and for decoding the return value of
+@code{format-time-string}.
+@end defvar
+
+@defvar system-messages-locale
+@tindex system-messages-locale
+This variable specifies the locale to use for generating system error
+messages. Changing the locale can cause messages to come out in a
+different language or in a different orthography. If the variable is
+@code{nil}, the locale is specified by environment variables in the
+usual POSIX fashion.
+@end defvar
+@defvar system-time-locale
+@tindex system-time-locale
+This variable specifies the locale to use for formatting time values.
+Changing the locale can cause messages to appear according to the
+conventions of a different language. If the variable is @code{nil}, the
+locale is specified by environment variables in the usual POSIX fashion.
+@end defvar
HCoop / bpt / emacs.git / blobdiff