*** empty log message ***

[bpt/emacs.git] / lispref / nonascii.texi

diff --git a/lispref/nonascii.texi b/lispref/nonascii.texi
index e5ead72..ba001ca 100644 (file)
--- a/lispref/nonascii.texi
+++ b/lispref/nonascii.texi
@@ -1,29 +1,33 @@
 @c -*-texinfo-*-
 @c This is part of the GNU Emacs Lisp Reference Manual.
 @c -*-texinfo-*-
 @c This is part of the GNU Emacs Lisp Reference Manual.

-@c Copyright (C) 1998, 1999 Free Software Foundation, Inc. 
+@c Copyright (C) 1998, 1999, 2002, 2003, 2004,
+@c   2005, 2006 Free Software Foundation, Inc.

 @c See the file elisp.texi for copying conditions.
 @setfilename ../info/characters
 @node Non-ASCII Characters, Searching and Matching, Text, Top
 @c See the file elisp.texi for copying conditions.
 @setfilename ../info/characters
 @node Non-ASCII Characters, Searching and Matching, Text, Top

-@chapter Non-ASCII Characters
+@chapter Non-@acronym{ASCII} Characters

 @cindex multibyte characters
 @cindex multibyte characters

-@cindex non-ASCII characters
+@cindex non-@acronym{ASCII} characters

 
 

-  This chapter covers the special issues relating to non-@sc{ascii}
+  This chapter covers the special issues relating to non-@acronym{ASCII}

 characters and how they are stored in strings and buffers.
 
 @menu
 characters and how they are stored in strings and buffers.
 
 @menu

-* Text Representations::
-* Converting Representations::
-* Selecting a Representation::
-* Character Codes::
-* Character Sets::
-* Chars and Bytes::
-* Splitting Characters::
-* Scanning Charsets::
-* Translation of Characters::
-* Coding Systems::
-* Input Methods::
-* Locales::             Interacting with the POSIX locale.
+* Text Representations::    Unibyte and multibyte representations
+* Converting Representations::  Converting unibyte to multibyte and vice versa.
+* Selecting a Representation::  Treating a byte sequence as unibyte or multi.
+* Character Codes::         How unibyte and multibyte relate to
+                                codes of individual characters.
+* Character Sets::          The space of possible character codes
+                                is divided into various character sets.
+* Chars and Bytes::         More information about multibyte encodings.
+* Splitting Characters::    Converting a character to its byte sequence.
+* Scanning Charsets::       Which character sets are used in a buffer?
+* Translation of Characters::   Translation tables are used for conversion.
+* Coding Systems::          Coding systems are conversions for saving files.
+* Input Methods::           Input methods allow users to enter various
+                                non-ASCII characters without special keyboards.
+* Locales::                 Interacting with the POSIX locale.

 @end menu
 
 @node Text Representations
 @end menu
 
 @node Text Representations


@@ -41,8 +45,8 @@ attention to the difference.
 @cindex unibyte text
   In unibyte representation, each character occupies one byte and
 therefore the possible character codes range from 0 to 255.  Codes 0
 @cindex unibyte text
   In unibyte representation, each character occupies one byte and
 therefore the possible character codes range from 0 to 255.  Codes 0

-through 127 are @sc{ascii} characters; the codes from 128 through 255
-are used for one non-@sc{ascii} character set (you can choose which
+through 127 are @acronym{ASCII} characters; the codes from 128 through 255
+are used for one non-@acronym{ASCII} character set (you can choose which

 character set by setting the variable @code{nonascii-insert-offset}).
 
 @cindex leading code
 character set by setting the variable @code{nonascii-insert-offset}).
 
 @cindex leading code


@@ -56,12 +60,13 @@ stored.  The first byte of a multibyte character is always in the range
 character are always in the range 160 through 255 (octal 0240 through
 0377); these values are @dfn{trailing codes}.
 
 character are always in the range 160 through 255 (octal 0240 through
 0377); these values are @dfn{trailing codes}.
 

-  Some sequences of bytes do not form meaningful multibyte characters:
-for example, a single isolated byte in the range 128 through 255 is
-never meaningful.  Such byte sequences are not entirely valid, and never
-appear in proper multibyte text (since that consists of a sequence of
-@emph{characters}); but they can appear as part of ``raw bytes''
-(@pxref{Explicit Encoding}).
+  Some sequences of bytes are not valid in multibyte text: for example,
+a single isolated byte in the range 128 through 159 is not allowed.  But
+character codes 128 through 159 can appear in multibyte text,
+represented as two-byte sequences.  All the character codes 128 through
+255 are possible (though slightly abnormal) in multibyte text; they
+appear in multibyte buffers and strings when you do explicit encoding
+and decoding (@pxref{Explicit Encoding}).

 
   In a buffer, the buffer-local value of the variable
 @code{enable-multibyte-characters} specifies the representation used.
 
   In a buffer, the buffer-local value of the variable
 @code{enable-multibyte-characters} specifies the representation used.


@@ -90,15 +95,16 @@ default value to @code{nil} early in startup.
 @end defvar
 
 @defun position-bytes position
 @end defvar
 
 @defun position-bytes position

-@tindex position-bytes
-Return the byte-position corresponding to buffer position @var{position}
-in the current buffer.
+Return the byte-position corresponding to buffer position
+@var{position} in the current buffer.  This is 1 at the start of the
+buffer, and counts upward in bytes.  If @var{position} is out of
+range, the value is @code{nil}.

 @end defun
 
 @defun byte-to-position byte-position
 @end defun
 
 @defun byte-to-position byte-position

-@tindex byte-to-position

 Return the buffer position corresponding to byte-position
 Return the buffer position corresponding to byte-position

-@var{byte-position} in the current buffer.
+@var{byte-position} in the current buffer.  If @var{byte-position} is
+out of range, the value is @code{nil}.

 @end defun
 
 @defun multibyte-string-p string
 @end defun
 
 @defun multibyte-string-p string


@@ -130,16 +136,16 @@ alternative, to convert the buffer contents to multibyte, is not
 acceptable because the buffer's representation is a choice made by the
 user that cannot be overridden automatically.
 
 acceptable because the buffer's representation is a choice made by the
 user that cannot be overridden automatically.
 

-  Converting unibyte text to multibyte text leaves @sc{ascii} characters
-unchanged, and likewise 128 through 159.  It converts the non-@sc{ascii}
-codes 160 through 255 by adding the value @code{nonascii-insert-offset}
-to each character code.  By setting this variable, you specify which
-character set the unibyte characters correspond to (@pxref{Character
-Sets}).  For example, if @code{nonascii-insert-offset} is 2048, which is
-@code{(- (make-char 'latin-iso8859-1) 128)}, then the unibyte
-non-@sc{ascii} characters correspond to Latin 1.  If it is 2688, which
-is @code{(- (make-char 'greek-iso8859-7) 128)}, then they correspond to
-Greek letters.
+  Converting unibyte text to multibyte text leaves @acronym{ASCII} characters
+unchanged, and likewise character codes 128 through 159.  It converts
+the non-@acronym{ASCII} codes 160 through 255 by adding the value
+@code{nonascii-insert-offset} to each character code.  By setting this
+variable, you specify which character set the unibyte characters
+correspond to (@pxref{Character Sets}).  For example, if
+@code{nonascii-insert-offset} is 2048, which is @code{(- (make-char
+'latin-iso8859-1) 128)}, then the unibyte non-@acronym{ASCII} characters
+correspond to Latin 1.  If it is 2688, which is @code{(- (make-char
+'greek-iso8859-7) 128)}, then they correspond to Greek letters.

 
   Converting multibyte text to unibyte is simpler: it discards all but
 the low 8 bits of each character code.  If @code{nonascii-insert-offset}
 
   Converting multibyte text to unibyte is simpler: it discards all but
 the low 8 bits of each character code.  If @code{nonascii-insert-offset}


@@ -149,11 +155,11 @@ text to multibyte and back to unibyte reproduces the original unibyte
 text.
 
 @defvar nonascii-insert-offset
 text.
 
 @defvar nonascii-insert-offset

-This variable specifies the amount to add to a non-@sc{ascii} character
+This variable specifies the amount to add to a non-@acronym{ASCII} character

 when converting unibyte text to multibyte.  It also applies when
 @code{self-insert-command} inserts a character in the unibyte
 when converting unibyte text to multibyte.  It also applies when
 @code{self-insert-command} inserts a character in the unibyte

-non-@sc{ascii} range, 128 through 255.  However, the function
-@code{insert-char} does not perform this conversion.
+non-@acronym{ASCII} range, 128 through 255.  However, the functions
+@code{insert} and @code{insert-char} do not perform this conversion.

 
 The right value to use to select character set @var{cs} is @code{(-
 (make-char @var{cs}) 128)}.  If the value of
 
 The right value to use to select character set @var{cs} is @code{(-
 (make-char @var{cs}) 128)}.  If the value of


@@ -165,20 +171,55 @@ value for the Latin 1 character set, rather than zero.
 This variable provides a more general alternative to
 @code{nonascii-insert-offset}.  You can use it to specify independently
 how to translate each code in the range of 128 through 255 into a
 This variable provides a more general alternative to
 @code{nonascii-insert-offset}.  You can use it to specify independently
 how to translate each code in the range of 128 through 255 into a

-multibyte character.  The value should be a vector, or @code{nil}.
+multibyte character.  The value should be a char-table, or @code{nil}.

 If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}.
 @end defvar
 
 If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}.
 @end defvar
 

+The next three functions either return the argument @var{string}, or a
+newly created string with no text properties.
+

 @defun string-make-unibyte string
 This function converts the text of @var{string} to unibyte
 representation, if it isn't already, and returns the result.  If
 @defun string-make-unibyte string
 This function converts the text of @var{string} to unibyte
 representation, if it isn't already, and returns the result.  If

-@var{string} is a unibyte string, it is returned unchanged.
+@var{string} is a unibyte string, it is returned unchanged.  Multibyte
+character codes are converted to unibyte according to
+@code{nonascii-translation-table} or, if that is @code{nil}, using
+@code{nonascii-insert-offset}.  If the lookup in the translation table
+fails, this function takes just the low 8 bits of each character.

 @end defun
 
 @defun string-make-multibyte string
 This function converts the text of @var{string} to multibyte
 representation, if it isn't already, and returns the result.  If
 @end defun
 
 @defun string-make-multibyte string
 This function converts the text of @var{string} to multibyte
 representation, if it isn't already, and returns the result.  If

-@var{string} is a multibyte string, it is returned unchanged.
+@var{string} is a multibyte string or consists entirely of
+@acronym{ASCII} characters, it is returned unchanged.  In particular,
+if @var{string} is unibyte and entirely @acronym{ASCII}, the returned
+string is unibyte.  (When the characters are all @acronym{ASCII},
+Emacs primitives will treat the string the same way whether it is
+unibyte or multibyte.)  If @var{string} is unibyte and contains
+non-@acronym{ASCII} characters, the function
+@code{unibyte-char-to-multibyte} is used to convert each unibyte
+character to a multibyte character.
+@end defun
+
+@defun string-to-multibyte string
+This function returns a multibyte string containing the same sequence
+of character codes as @var{string}.  Unlike
+@code{string-make-multibyte}, this function unconditionally returns a
+multibyte string.  If @var{string} is a multibyte string, it is
+returned unchanged.
+@end defun
+
+@defun multibyte-char-to-unibyte char
+This convert the multibyte character @var{char} to a unibyte
+character, based on @code{nonascii-translation-table} and
+@code{nonascii-insert-offset}.
+@end defun
+
+@defun unibyte-char-to-multibyte char
+This convert the unibyte character @var{char} to a multibyte
+character, based on @code{nonascii-translation-table} and
+@code{nonascii-insert-offset}.

 @end defun
 
 @node Selecting a Representation
 @end defun
 
 @node Selecting a Representation


@@ -196,7 +237,10 @@ This function leaves the buffer contents unchanged when viewed as a
 sequence of bytes.  As a consequence, it can change the contents viewed
 as characters; a sequence of two bytes which is treated as one character
 in multibyte representation will count as two characters in unibyte
 sequence of bytes.  As a consequence, it can change the contents viewed
 as characters; a sequence of two bytes which is treated as one character
 in multibyte representation will count as two characters in unibyte

-representation.
+representation.  Character codes 128 through 159 are an exception.  They
+are represented by one byte in a unibyte buffer, but when the buffer is
+set to multibyte, they are converted to two-byte sequences, and vice
+versa.

 
 This function sets @code{enable-multibyte-characters} to record which
 representation is in use.  It also adjusts various data in the buffer
 
 This function sets @code{enable-multibyte-characters} to record which
 representation is in use.  It also adjusts various data in the buffer


@@ -214,7 +258,10 @@ treating each byte as a character.  This means that the value may have
 more characters than @var{string} has.
 
 If @var{string} is already a unibyte string, then the value is
 more characters than @var{string} has.
 
 If @var{string} is already a unibyte string, then the value is

-@var{string} itself.
+@var{string} itself.  Otherwise it is a newly created string, with no
+text properties.  If @var{string} is multibyte, any characters it
+contains of charset @code{eight-bit-control} or @code{eight-bit-graphic}
+are converted to the corresponding single byte.

 @end defun
 
 @defun string-as-multibyte string
 @end defun
 
 @defun string-as-multibyte string


@@ -223,7 +270,11 @@ treating each multibyte sequence as one character.  This means that the
 value may have fewer characters than @var{string} has.
 
 If @var{string} is already a multibyte string, then the value is
 value may have fewer characters than @var{string} has.
 
 If @var{string} is already a multibyte string, then the value is

-@var{string} itself.
+@var{string} itself.  Otherwise it is a newly created string, with no
+text properties.  If @var{string} is unibyte and contains any individual
+8-bit bytes (i.e.@: not part of a multibyte form), they are converted to
+the corresponding multibyte character of charset @code{eight-bit-control}
+or @code{eight-bit-graphic}.

 @end defun
 
 @node Character Codes
 @end defun
 
 @node Character Codes


@@ -234,14 +285,15 @@ If @var{string} is already a multibyte string, then the value is
 codes.  The valid character codes for unibyte representation range from
 0 to 255---the values that can fit in one byte.  The valid character
 codes for multibyte representation range from 0 to 524287, but not all
 codes.  The valid character codes for unibyte representation range from
 0 to 255---the values that can fit in one byte.  The valid character
 codes for multibyte representation range from 0 to 524287, but not all

-values in that range are valid.  In particular, the values 128 through
-255 are not legitimate in multibyte text (though they can occur in ``raw
-bytes''; @pxref{Explicit Encoding}).  Only the @sc{ascii} codes 0
-through 127 are fully legitimate in both representations.
+values in that range are valid.  The values 128 through 255 are not
+entirely proper in multibyte text, but they can occur if you do explicit
+encoding and decoding (@pxref{Explicit Encoding}).  Some other character
+codes cannot occur at all in multibyte text.  Only the @acronym{ASCII} codes
+0 through 127 are completely legitimate in both representations.

 
 

-@defun char-valid-p charcode
-This returns @code{t} if @var{charcode} is valid for either one of the two
-text representations.
+@defun char-valid-p charcode &optional genericp
+This returns @code{t} if @var{charcode} is valid (either for unibyte
+text or for multibyte text).

 
 @example
 (char-valid-p 65)
 
 @example
 (char-valid-p 65)


@@ -251,6 +303,10 @@ text representations.
 (char-valid-p 2248)
      @result{} t
 @end example
 (char-valid-p 2248)
      @result{} t
 @end example

+
+If the optional argument @var{genericp} is non-@code{nil}, this
+function also returns @code{t} if @var{charcode} is a generic
+character (@pxref{Splitting Characters}).

 @end defun
 
 @node Character Sets
 @end defun
 
 @node Character Sets


@@ -270,29 +326,43 @@ into several character sets.  For example, one set of Chinese
 characters, generally known as Big 5, is divided into two Emacs
 character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
 
 characters, generally known as Big 5, is divided into two Emacs
 character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
 

+  @acronym{ASCII} characters are in character set @code{ascii}.  The
+non-@acronym{ASCII} characters 128 through 159 are in character set
+@code{eight-bit-control}, and codes 160 through 255 are in character set
+@code{eight-bit-graphic}.
+

 @defun charsetp object
 Returns @code{t} if @var{object} is a symbol that names a character set,
 @code{nil} otherwise.
 @end defun
 
 @defun charsetp object
 Returns @code{t} if @var{object} is a symbol that names a character set,
 @code{nil} otherwise.
 @end defun
 

+@defvar charset-list
+The value is a list of all defined character set names.
+@end defvar
+

 @defun charset-list
 @defun charset-list

-This function returns a list of all defined character set names.
+This function returns the value of @code{charset-list}.  It is only
+provided for backward compatibility.

 @end defun
 
 @defun char-charset character
 This function returns the name of the character set that @var{character}
 @end defun
 
 @defun char-charset character
 This function returns the name of the character set that @var{character}

-belongs to.
+belongs to, or the symbol @code{unknown} if @var{character} is not a
+valid character.

 @end defun
 
 @defun charset-plist charset
 @end defun
 
 @defun charset-plist charset

-@tindex charset-plist

 This function returns the charset property list of the character set
 @var{charset}.  Although @var{charset} is a symbol, this is not the same
 as the property list of that symbol.  Charset properties are used for
 This function returns the charset property list of the character set
 @var{charset}.  Although @var{charset} is a symbol, this is not the same
 as the property list of that symbol.  Charset properties are used for

-special purposes within Emacs; for example, @code{x-charset-registry}
-helps determine which fonts to use (@pxref{Font Selection}).
+special purposes within Emacs.

 @end defun
 
 @end defun
 

+@deffn Command list-charset-chars charset
+This command displays a list of characters in the character set
+@var{charset}.
+@end deffn
+

 @node Chars and Bytes
 @section Characters and Bytes
 @cindex bytes and characters
 @node Chars and Bytes
 @section Characters and Bytes
 @cindex bytes and characters


@@ -301,13 +371,14 @@ helps determine which fonts to use (@pxref{Font Selection}).
 @cindex dimension (of character set)
   In multibyte representation, each character occupies one or more
 bytes.  Each character set has an @dfn{introduction sequence}, which is
 @cindex dimension (of character set)
   In multibyte representation, each character occupies one or more
 bytes.  Each character set has an @dfn{introduction sequence}, which is

-normally one or two bytes long.  (Exception: the @sc{ascii} character
-set has a zero-length introduction sequence.)  The introduction sequence
-is the beginning of the byte sequence for any character in the character
-set.  The rest of the character's bytes distinguish it from the other
-characters in the same character set.  Depending on the character set,
-there are either one or two distinguishing bytes; the number of such
-bytes is called the @dfn{dimension} of the character set.
+normally one or two bytes long.  (Exception: the @code{ascii} character
+set and the @code{eight-bit-graphic} character set have a zero-length
+introduction sequence.)  The introduction sequence is the beginning of
+the byte sequence for any character in the character set.  The rest of
+the character's bytes distinguish it from the other characters in the
+same character set.  Depending on the character set, there are either
+one or two distinguishing bytes; the number of such bytes is called the
+@dfn{dimension} of the character set.

 
 @defun charset-dimension charset
 This function returns the dimension of @var{charset}; at present, the
 
 @defun charset-dimension charset
 This function returns the dimension of @var{charset}; at present, the


@@ -315,7 +386,6 @@ dimension is always 1 or 2.
 @end defun
 
 @defun charset-bytes charset
 @end defun
 
 @defun charset-bytes charset

-@tindex charset-bytes

 This function returns the number of bytes used to represent a character
 in character set @var{charset}.
 @end defun
 This function returns the number of bytes used to represent a character
 in character set @var{charset}.
 @end defun


@@ -342,33 +412,35 @@ Return a list containing the name of the character set of
 identify @var{character} within that character set.  The number of byte
 values is the character set's dimension.
 
 identify @var{character} within that character set.  The number of byte
 values is the character set's dimension.
 

+If @var{character} is invalid as a character code, @code{split-char}
+returns a list consisting of the symbol @code{unknown} and @var{character}.
+

 @example
 (split-char 2248)
      @result{} (latin-iso8859-1 72)
 (split-char 65)
      @result{} (ascii 65)
 @example
 (split-char 2248)
      @result{} (latin-iso8859-1 72)
 (split-char 65)
      @result{} (ascii 65)

-@end example
-
-Unibyte non-@sc{ascii} characters are considered as part of
-the @code{ascii} character set:
-
-@example
-(split-char 192)
-     @result{} (ascii 192)
+(split-char 128)
+     @result{} (eight-bit-control 128)

 @end example
 @end defun
 
 @end example
 @end defun
 

-@defun make-char charset &rest byte-values
-This function returns the character in character set @var{charset}
-identified by @var{byte-values}.  This is roughly the inverse of
-@code{split-char}.  Normally, you should specify either one or two
-@var{byte-values}, according to the dimension of @var{charset}.  For
-example,
+@defun make-char charset &optional code1 code2
+This function returns the character in character set @var{charset} whose
+position codes are @var{code1} and @var{code2}.  This is roughly the
+inverse of @code{split-char}.  Normally, you should specify either one
+or both of @var{code1} and @var{code2} according to the dimension of
+@var{charset}.  For example,

 
 @example
 (make-char 'latin-iso8859-1 72)
      @result{} 2248
 @end example
 
 @example
 (make-char 'latin-iso8859-1 72)
      @result{} 2248
 @end example

+
+Actually, the eighth bit of both @var{code1} and @var{code2} is zeroed
+before they are used to index @var{charset}.  Thus you may use, for
+instance, an ISO 8859 character code rather than subtracting 128, as
+is necessary to index the corresponding Emacs charset.

 @end defun
 
 @cindex generic characters
 @end defun
 
 @cindex generic characters


@@ -385,10 +457,18 @@ For example:
      @result{} 2176
 (char-valid-p 2176)
      @result{} nil
      @result{} 2176
 (char-valid-p 2176)
      @result{} nil

+(char-valid-p 2176 t)
+     @result{} t

 (split-char 2176)
      @result{} (latin-iso8859-1 0)
 @end example
 
 (split-char 2176)
      @result{} (latin-iso8859-1 0)
 @end example
 

+The character sets @code{ascii}, @code{eight-bit-control}, and
+@code{eight-bit-graphic} don't have corresponding generic characters.  If
+@var{charset} is one of them and you don't supply @var{code1},
+@code{make-char} returns the character code corresponding to the
+smallest code in @var{charset}.
+

 @node Scanning Charsets
 @section Scanning for Character Sets
 
 @node Scanning Charsets
 @section Scanning for Character Sets
 


@@ -397,6 +477,13 @@ part of a buffer or a string.  One use for this is in determining which
 coding systems (@pxref{Coding Systems}) are capable of representing all
 of the text in question.
 
 coding systems (@pxref{Coding Systems}) are capable of representing all
 of the text in question.
 

+@defun charset-after &optional pos
+This function return the charset of a character in the current buffer
+at position @var{pos}.  If @var{pos} is omitted or @code{nil}, it
+defaults to the current value of point.  If @var{pos} is out of range,
+the value is @code{nil}.
+@end defun
+

 @defun find-charset-region beg end &optional translation
 This function returns a list of the character sets that appear in the
 current buffer between positions @var{beg} and @var{end}.
 @defun find-charset-region beg end &optional translation
 This function returns a list of the character sets that appear in the
 current buffer between positions @var{beg} and @var{end}.


@@ -406,17 +493,6 @@ be used in scanning the text (@pxref{Translation of Characters}).  If it
 is non-@code{nil}, then each character in the region is translated
 through this table, and the value returned describes the translated
 characters instead of the characters actually in the buffer.
 is non-@code{nil}, then each character in the region is translated
 through this table, and the value returned describes the translated
 characters instead of the characters actually in the buffer.

-
-In two peculiar cases, the value includes the symbol @code{unknown}:
-
-@itemize @bullet
-@item
-When a unibyte buffer contains non-@sc{ascii} characters.
-
-@item
-When a multibyte buffer contains invalid byte-sequences (raw bytes).
-@xref{Explicit Encoding}.
-@end itemize

 @end defun
 
 @defun find-charset-string string &optional translation
 @end defun
 
 @defun find-charset-string string &optional translation


@@ -431,40 +507,58 @@ current buffer.
 @cindex character translation tables
 @cindex translation tables
 
 @cindex character translation tables
 @cindex translation tables
 

-  A @dfn{translation table} specifies a mapping of characters
-into characters.  These tables are used in encoding and decoding, and
-for other purposes.  Some coding systems specify their own particular
-translation tables; there are also default translation tables which
-apply to all other coding systems.
+  A @dfn{translation table} is a char-table that specifies a mapping
+of characters into characters.  These tables are used in encoding and
+decoding, and for other purposes.  Some coding systems specify their
+own particular translation tables; there are also default translation
+tables which apply to all other coding systems.
+
+  For instance, the coding-system @code{utf-8} has a translation table
+that maps characters of various charsets (e.g.,
+@code{latin-iso8859-@var{x}}) into Unicode character sets.  This way,
+it can encode Latin-2 characters into UTF-8.  Meanwhile,
+@code{unify-8859-on-decoding-mode} operates by specifying
+@code{standard-translation-table-for-decode} to translate
+Latin-@var{x} characters into corresponding Unicode characters.

 
 @defun make-translation-table &rest translations
 This function returns a translation table based on the argument
 
 @defun make-translation-table &rest translations
 This function returns a translation table based on the argument

-@var{translations}.  Each element of
-@var{translations} should be a list of the form @code{(@var{from}
-. @var{to})}; this says to translate the character @var{from} into
-@var{to}.
+@var{translations}.  Each element of @var{translations} should be a
+list of elements of the form @code{(@var{from} . @var{to})}; this says
+to translate the character @var{from} into @var{to}.
+
+The arguments and the forms in each argument are processed in order,
+and if a previous form already translates @var{to} to some other
+character, say @var{to-alt}, @var{from} is also translated to
+@var{to-alt}.

 
 You can also map one whole character set into another character set with
 the same dimension.  To do this, you specify a generic character (which
 designates a character set) for @var{from} (@pxref{Splitting Characters}).
 
 You can also map one whole character set into another character set with
 the same dimension.  To do this, you specify a generic character (which
 designates a character set) for @var{from} (@pxref{Splitting Characters}).

-In this case, @var{to} should also be a generic character, for another
-character set of the same dimension.  Then the translation table
-translates each character of @var{from}'s character set into the
-corresponding character of @var{to}'s character set.
+In this case, if @var{to} is also a generic character, its character
+set should have the same dimension as @var{from}'s.  Then the
+translation table translates each character of @var{from}'s character
+set into the corresponding character of @var{to}'s character set.  If
+@var{from} is a generic character and @var{to} is an ordinary
+character, then the translation table translates every character of
+@var{from}'s character set into @var{to}.

 @end defun
 
   In decoding, the translation table's translations are applied to the
 characters that result from ordinary decoding.  If a coding system has
 @end defun
 
   In decoding, the translation table's translations are applied to the
 characters that result from ordinary decoding.  If a coding system has

-property @code{character-translation-table-for-decode}, that specifies
-the translation table to use.  Otherwise, if
-@code{standard-translation-table-for-decode} is non-@code{nil}, decoding
-uses that table.
+property @code{translation-table-for-decode}, that specifies the
+translation table to use.  (This is a property of the coding system,
+as returned by @code{coding-system-get}, not a property of the symbol
+that is the coding system's name. @xref{Coding System Basics,, Basic
+Concepts of Coding Systems}.)  Otherwise, if
+@code{standard-translation-table-for-decode} is non-@code{nil},
+decoding uses that table.

 
   In encoding, the translation table's translations are applied to the
 characters in the buffer, and the result of translation is actually
 encoded.  If a coding system has property
 
   In encoding, the translation table's translations are applied to the
 characters in the buffer, and the result of translation is actually
 encoded.  If a coding system has property

-@code{character-translation-table-for-encode}, that specifies the
-translation table to use.  Otherwise the variable
+@code{translation-table-for-encode}, that specifies the translation
+table to use.  Otherwise the variable

 @code{standard-translation-table-for-encode} specifies the translation
 table.
 
 @code{standard-translation-table-for-encode} specifies the translation
 table.
 


@@ -478,6 +572,18 @@ This is the default translation table for encoding, for
 coding systems that don't specify any other translation table.
 @end defvar
 
 coding systems that don't specify any other translation table.
 @end defvar
 

+@defvar translation-table-for-input
+Self-inserting characters are translated through this translation
+table before they are inserted.  Search commands also translate their
+input through this table, so they can compare more reliably with
+what's in the buffer.
+
+@code{set-buffer-file-coding-system} sets this variable so that your
+keyboard input gets translated into the character sets that the buffer
+is likely to contain.  This variable automatically becomes
+buffer-local when set.
+@end defvar
+

 @node Coding Systems
 @section Coding Systems
 
 @node Coding Systems
 @section Coding Systems
 


@@ -491,15 +597,17 @@ by a particular @dfn{coding system}.
 documented here.
 
 @menu
 documented here.
 
 @menu

-* Coding System Basics::
-* Encoding and I/O::
-* Lisp and Coding Systems::
-* User-Chosen Coding Systems::
-* Default Coding Systems::
-* Specifying Coding Systems::
-* Explicit Encoding::
-* Terminal I/O Encoding::
-* MS-DOS File Types::
+* Coding System Basics::        Basic concepts.
+* Encoding and I/O::            How file I/O functions handle coding systems.
+* Lisp and Coding Systems::     Functions to operate on coding system names.
+* User-Chosen Coding Systems::  Asking the user to choose a coding system.
+* Default Coding Systems::      Controlling the default choices.
+* Specifying Coding Systems::   Requesting a particular coding system
+                                    for a single file operation.
+* Explicit Encoding::           Encoding or decoding text without doing I/O.
+* Terminal I/O Encoding::       Use of encoding for terminal I/O.
+* MS-DOS File Types::           How DOS "text" and "binary" files
+                                    relate to coding systems.

 @end menu
 
 @node Coding System Basics
 @end menu
 
 @node Coding System Basics


@@ -519,6 +627,27 @@ characters; for example, there are three coding systems for the Cyrillic
 conversion, but some of them leave the choice unspecified---to be chosen
 heuristically for each file, based on the data.
 
 conversion, but some of them leave the choice unspecified---to be chosen
 heuristically for each file, based on the data.
 

+  In general, a coding system doesn't guarantee roundtrip identity:
+decoding a byte sequence using coding system, then encoding the
+resulting text in the same coding system, can produce a different byte
+sequence.  However, the following coding systems do guarantee that the
+byte sequence will be the same as what you originally decoded:
+
+@quotation
+chinese-big5 chinese-iso-8bit cyrillic-iso-8bit emacs-mule
+greek-iso-8bit hebrew-iso-8bit iso-latin-1 iso-latin-2 iso-latin-3
+iso-latin-4 iso-latin-5 iso-latin-8 iso-latin-9 iso-safe
+japanese-iso-8bit japanese-shift-jis korean-iso-8bit raw-text
+@end quotation
+
+  Encoding buffer text and then decoding the result can also fail to
+reproduce the original text.  For instance, if you encode Latin-2
+characters with @code{utf-8} and decode the result using the same
+coding system, you'll get Unicode characters (of charset
+@code{mule-unicode-0100-24ff}).  If you encode Unicode characters with
+@code{iso-latin-2} and decode the result with the same coding system,
+you'll get Latin-2 characters.
+

 @cindex end of line conversion
   @dfn{End of line conversion} handles three different conventions used
 on various systems for representing end of line in files.  The Unix
 @cindex end of line conversion
   @dfn{End of line conversion} handles three different conventions used
 on various systems for representing end of line in files.  The Unix


@@ -577,7 +706,7 @@ a coding system for decoding the file data, and @code{write-region}
 uses one to encode the buffer contents.
 
   You can specify the coding system to use either explicitly
 uses one to encode the buffer contents.
 
   You can specify the coding system to use either explicitly

-(@pxref{Specifying Coding Systems}), or implicitly using the defaulting
+(@pxref{Specifying Coding Systems}), or implicitly using a default

 mechanism (@pxref{Default Coding Systems}).  But these methods may not
 completely specify what to do.  For example, they may choose a coding
 system such as @code{undefined} which leaves the character code
 mechanism (@pxref{Default Coding Systems}).  But these methods may not
 completely specify what to do.  For example, they may choose a coding
 system such as @code{undefined} which leaves the character code


@@ -586,27 +715,33 @@ operation finishes the job of choosing a coding system.  Very often
 you will want to find out afterwards which coding system was chosen.
 
 @defvar buffer-file-coding-system
 you will want to find out afterwards which coding system was chosen.
 
 @defvar buffer-file-coding-system

-This variable records the coding system that was used for visiting the
-current buffer.  It is used for saving the buffer, and for writing part
-of the buffer with @code{write-region}.  When those operations ask the
-user to specify a different coding system,
-@code{buffer-file-coding-system} is updated to the coding system
-specified.
-
-However, @code{buffer-file-coding-system} does not affect sending text
+This buffer-local variable records the coding system that was used to visit
+the current buffer.  It is used for saving the buffer, and for writing part
+of the buffer with @code{write-region}.  If the text to be written
+cannot be safely encoded using the coding system specified by this
+variable, these operations select an alternative encoding by calling
+the function @code{select-safe-coding-system} (@pxref{User-Chosen
+Coding Systems}).  If selecting a different encoding requires to ask
+the user to specify a coding system, @code{buffer-file-coding-system}
+is updated to the newly selected coding system.
+
+@code{buffer-file-coding-system} does @emph{not} affect sending text

 to a subprocess.
 @end defvar
 
 @defvar save-buffer-coding-system
 to a subprocess.
 @end defvar
 
 @defvar save-buffer-coding-system

-This variable specifies the coding system for saving the buffer---but it
-is not used for @code{write-region}.
+This variable specifies the coding system for saving the buffer (by
+overriding @code{buffer-file-coding-system}).  Note that it is not used
+for @code{write-region}.

 
 When a command to save the buffer starts out to use
 
 When a command to save the buffer starts out to use

-@code{save-buffer-coding-system}, and that coding system cannot handle
+@code{buffer-file-coding-system} (or @code{save-buffer-coding-system}),
+and that coding system cannot handle

 the actual text in the buffer, the command asks the user to choose
 the actual text in the buffer, the command asks the user to choose

-another coding system.  After that happens, the command also updates
-@code{save-buffer-coding-system} to represent the coding system that the
-user specified.
+another coding system (by calling @code{select-safe-coding-system}).
+After that happens, the command also updates
+@code{buffer-file-coding-system} to represent the coding system that
+the user specified.

 @end defvar
 
 @defvar last-coding-system-used
 @end defvar
 
 @defvar last-coding-system-used


@@ -623,6 +758,26 @@ interested in.
   The variable @code{selection-coding-system} specifies how to encode
 selections for the window system.  @xref{Window System Selections}.
 
   The variable @code{selection-coding-system} specifies how to encode
 selections for the window system.  @xref{Window System Selections}.
 

+@defvar file-name-coding-system
+The variable @code{file-name-coding-system} specifies the coding
+system to use for encoding file names.  Emacs encodes file names using
+that coding system for all file operations.  If
+@code{file-name-coding-system} is @code{nil}, Emacs uses a default
+coding system determined by the selected language environment.  In the
+default language environment, any non-@acronym{ASCII} characters in
+file names are not encoded specially; they appear in the file system
+using the internal Emacs representation.
+@end defvar
+
+  @strong{Warning:} if you change @code{file-name-coding-system} (or
+the language environment) in the middle of an Emacs session, problems
+can result if you have already visited files whose names were encoded
+using the earlier coding system and are handled differently under the
+new coding system.  If you try to save one of these buffers under the
+visited file name, saving may use the wrong file name, or it may get
+an error.  If such a problem happens, use @kbd{C-x C-w} to specify a
+new file name for that buffer.
+

 @node Lisp and Coding Systems
 @subsection Coding Systems in Lisp
 
 @node Lisp and Coding Systems
 @subsection Coding Systems in Lisp
 


@@ -631,12 +786,13 @@ selections for the window system.  @xref{Window System Selections}.
 @defun coding-system-list &optional base-only
 This function returns a list of all coding system names (symbols).  If
 @var{base-only} is non-@code{nil}, the value includes only the
 @defun coding-system-list &optional base-only
 This function returns a list of all coding system names (symbols).  If
 @var{base-only} is non-@code{nil}, the value includes only the

-base coding systems.  Otherwise, it includes variant coding systems as well.
+base coding systems.  Otherwise, it includes alias and variant coding
+systems as well.

 @end defun
 
 @defun coding-system-p object
 This function returns @code{t} if @var{object} is a coding system
 @end defun
 
 @defun coding-system-p object
 This function returns @code{t} if @var{object} is a coding system

-name.
+name or @code{nil}.

 @end defun
 
 @defun check-coding-system coding-system
 @end defun
 
 @defun check-coding-system coding-system


@@ -645,12 +801,44 @@ If that is valid, it returns @var{coding-system}.
 Otherwise it signals an error with condition @code{coding-system-error}.
 @end defun
 
 Otherwise it signals an error with condition @code{coding-system-error}.
 @end defun
 

+@cindex EOL conversion
+@cindex end-of-line conversion
+@cindex line end conversion
+@defun coding-system-eol-type coding-system
+This function returns the type of end-of-line (a.k.a.@: @dfn{eol})
+conversion used by @var{coding-system}.  If @var{coding-system}
+specifies a certain eol conversion, the return value is an integer 0,
+1, or 2, standing for @code{unix}, @code{dos}, and @code{mac},
+respectively.  If @var{coding-system} doesn't specify eol conversion
+explicitly, the return value is a vector of coding systems, each one
+with one of the possible eol conversion types, like this:
+
+@lisp
+(coding-system-eol-type 'latin-1)
+     @result{} [latin-1-unix latin-1-dos latin-1-mac]
+@end lisp
+
+@noindent
+If this function returns a vector, Emacs will decide, as part of the
+text encoding or decoding process, what eol conversion to use.  For
+decoding, the end-of-line format of the text is auto-detected, and the
+eol conversion is set to match it (e.g., DOS-style CRLF format will
+imply @code{dos} eol conversion).  For encoding, the eol conversion is
+taken from the appropriate default coding system (e.g.,
+@code{default-buffer-file-coding-system} for
+@code{buffer-file-coding-system}), or from the default eol conversion
+appropriate for the underlying platform.
+@end defun
+

 @defun coding-system-change-eol-conversion coding-system eol-type
 This function returns a coding system which is like @var{coding-system}
 except for its eol conversion, which is specified by @code{eol-type}.
 @var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or
 @code{nil}.  If it is @code{nil}, the returned coding system determines
 the end-of-line conversion from the data.
 @defun coding-system-change-eol-conversion coding-system eol-type
 This function returns a coding system which is like @var{coding-system}
 except for its eol conversion, which is specified by @code{eol-type}.
 @var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or
 @code{nil}.  If it is @code{nil}, the returned coding system determines
 the end-of-line conversion from the data.

+
+@var{eol-type} may also be 0, 1 or 2, standing for @code{unix},
+@code{dos} and @code{mac}, respectively.

 @end defun
 
 @defun coding-system-change-text-conversion eol-coding text-coding
 @end defun
 
 @defun coding-system-change-text-conversion eol-coding text-coding


@@ -685,7 +873,7 @@ encode all the character sets in the list @var{charsets}.
 
 @defun detect-coding-region start end &optional highest
 This function chooses a plausible coding system for decoding the text
 
 @defun detect-coding-region start end &optional highest
 This function chooses a plausible coding system for decoding the text

-from @var{start} to @var{end}.  This text should be ``raw bytes''
+from @var{start} to @var{end}.  This text should be a byte sequence

 (@pxref{Explicit Encoding}).
 
 Normally this function returns a list of coding systems that could
 (@pxref{Explicit Encoding}).
 
 Normally this function returns a list of coding systems that could


@@ -694,37 +882,81 @@ decreasing priority.  But if @var{highest} is non-@code{nil}, then the
 return value is just one coding system, the one that is highest in
 priority.
 
 return value is just one coding system, the one that is highest in
 priority.
 

-If the region contains only @sc{ascii} characters, the value
-is @code{undecided} or @code{(undecided)}.
+If the region contains only @acronym{ASCII} characters except for such
+ISO-2022 control characters ISO-2022 as @code{ESC}, the value is
+@code{undecided} or @code{(undecided)}, or a variant specifying
+end-of-line conversion, if that can be deduced from the text.

 @end defun
 
 @end defun
 

-@defun detect-coding-string string highest
+@defun detect-coding-string string &optional highest

 This function is like @code{detect-coding-region} except that it
 operates on the contents of @var{string} instead of bytes in the buffer.
 @end defun
 
 This function is like @code{detect-coding-region} except that it
 operates on the contents of @var{string} instead of bytes in the buffer.
 @end defun
 

-  @xref{Process Information}, for how to examine or set the coding
-systems used for I/O to a subprocess.
+  @xref{Coding systems for a subprocess,, Process Information}, in
+particular the description of the functions
+@code{process-coding-system} and @code{set-process-coding-system}, for
+how to examine or set the coding systems used for I/O to a subprocess.

 
 @node User-Chosen Coding Systems
 @subsection User-Chosen Coding Systems
 
 
 @node User-Chosen Coding Systems
 @subsection User-Chosen Coding Systems
 

-@defun select-safe-coding-system from to &optional preferred-coding-system
-This function selects a coding system for encoding the text between
-@var{from} and @var{to}, asking the user to choose if necessary.
-
-The optional argument @var{preferred-coding-system} specifies a coding
-system to try first.  If that one can handle the text in the specified
-region, then it is used.  If this argument is omitted, the current
-buffer's value of @code{buffer-file-coding-system} is tried first.
-
-If the region contains some multibyte characters that the preferred
-coding system cannot encode, this function asks the user to choose from
-a list of coding systems which can encode the text, and returns the
-user's choice.
-
-One other kludgy feature: if @var{from} is a string, the string is the
-target text, and @var{to} is ignored.
+@cindex select safe coding system
+@defun select-safe-coding-system from to &optional default-coding-system accept-default-p file
+This function selects a coding system for encoding specified text,
+asking the user to choose if necessary.  Normally the specified text
+is the text in the current buffer between @var{from} and @var{to}.  If
+@var{from} is a string, the string specifies the text to encode, and
+@var{to} is ignored.
+
+If @var{default-coding-system} is non-@code{nil}, that is the first
+coding system to try; if that can handle the text,
+@code{select-safe-coding-system} returns that coding system.  It can
+also be a list of coding systems; then the function tries each of them
+one by one.  After trying all of them, it next tries the current
+buffer's value of @code{buffer-file-coding-system} (if it is not
+@code{undecided}), then the value of
+@code{default-buffer-file-coding-system} and finally the user's most
+preferred coding system, which the user can set using the command
+@code{prefer-coding-system} (@pxref{Recognize Coding,, Recognizing
+Coding Systems, emacs, The GNU Emacs Manual}).
+
+If one of those coding systems can safely encode all the specified
+text, @code{select-safe-coding-system} chooses it and returns it.
+Otherwise, it asks the user to choose from a list of coding systems
+which can encode all the text, and returns the user's choice.
+
+@var{default-coding-system} can also be a list whose first element is
+t and whose other elements are coding systems.  Then, if no coding
+system in the list can handle the text, @code{select-safe-coding-system}
+queries the user immediately, without trying any of the three
+alternatives described above.
+
+The optional argument @var{accept-default-p}, if non-@code{nil},
+should be a function to determine whether a coding system selected
+without user interaction is acceptable. @code{select-safe-coding-system}
+calls this function with one argument, the base coding system of the
+selected coding system.  If @var{accept-default-p} returns @code{nil},
+@code{select-safe-coding-system} rejects the silently selected coding
+system, and asks the user to select a coding system from a list of
+possible candidates.
+
+@vindex select-safe-coding-system-accept-default-p
+If the variable @code{select-safe-coding-system-accept-default-p} is
+non-@code{nil}, its value overrides the value of
+@var{accept-default-p}.
+
+As a final step, before returning the chosen coding system,
+@code{select-safe-coding-system} checks whether that coding system is
+consistent with what would be selected if the contents of the region
+were read from a file.  (If not, this could lead to data corruption in
+a file subsequently re-visited and edited.)  Normally,
+@code{select-safe-coding-system} uses @code{buffer-file-name} as the
+file for this purpose, but if @var{file} is non-@code{nil}, it uses
+that file instead (this can be relevant for @code{write-region} and
+similar functions).  If it detects an apparent inconsistency,
+@code{select-safe-coding-system} queries the user before selecting the
+coding system.

 @end defun
 
   Here are two functions you can use to let the user specify a coding
 @end defun
 
   Here are two functions you can use to let the user specify a coding


@@ -758,6 +990,18 @@ don't change these variables; instead, override them using
 @code{coding-system-for-read} and @code{coding-system-for-write}
 (@pxref{Specifying Coding Systems}).
 
 @code{coding-system-for-read} and @code{coding-system-for-write}
 (@pxref{Specifying Coding Systems}).
 

+@defvar auto-coding-regexp-alist
+This variable is an alist of text patterns and corresponding coding
+systems. Each element has the form @code{(@var{regexp}
+. @var{coding-system})}; a file whose first few kilobytes match
+@var{regexp} is decoded with @var{coding-system} when its contents are
+read into a buffer.  The settings in this alist take priority over
+@code{coding:} tags in the files and the contents of
+@code{file-coding-system-alist} (see below).  The default value is set
+so that Emacs automatically recognizes mail files in Babyl format and
+reads them with no code conversions.
+@end defvar
+

 @defvar file-coding-system-alist
 This variable is an alist that specifies the coding systems to use for
 reading and writing particular files.  Each element has the form
 @defvar file-coding-system-alist
 This variable is an alist that specifies the coding systems to use for
 reading and writing particular files.  Each element has the form


@@ -773,9 +1017,11 @@ that coding system is used for both reading the file and writing it.  If
 specifies the coding system for decoding, and its @sc{cdr} specifies the
 coding system for encoding.
 
 specifies the coding system for decoding, and its @sc{cdr} specifies the
 coding system for encoding.
 

-If @var{coding} is a function name, the function must return a coding
-system or a cons cell containing two coding systems.  This value is used
-as described above.
+If @var{coding} is a function name, the function should take one
+argument, a list of all arguments passed to
+@code{find-operation-coding-system}.  It must return a coding system
+or a cons cell containing two coding systems.  This value has the same
+meaning as described above.

 @end defvar
 
 @defvar process-coding-system-alist
 @end defvar
 
 @defvar process-coding-system-alist


@@ -820,13 +1066,29 @@ The value should be a cons cell of the form @code{(@var{input-coding}
 the subprocess, and @var{output-coding} applies to output to it.
 @end defvar
 
 the subprocess, and @var{output-coding} applies to output to it.
 @end defvar
 

+@defvar auto-coding-functions
+This variable holds a list of functions that try to determine a
+coding system for a file based on its undecoded contents.
+
+Each function in this list should be written to look at text in the
+current buffer, but should not modify it in any way.  The buffer will
+contain undecoded text of parts of the file.  Each function should
+take one argument, @var{size}, which tells it how many characters to
+look at, starting from point.  If the function succeeds in determining
+a coding system for the file, it should return that coding system.
+Otherwise, it should return @code{nil}.
+
+If a file has a @samp{coding:} tag, that takes precedence, so these
+functions won't be called.
+@end defvar
+

 @defun find-operation-coding-system operation &rest arguments
 This function returns the coding system to use (by default) for
 performing @var{operation} with @var{arguments}.  The value has this
 form:
 
 @example
 @defun find-operation-coding-system operation &rest arguments
 This function returns the coding system to use (by default) for
 performing @var{operation} with @var{arguments}.  The value has this
 form:
 
 @example

-(@var{decoding-system} @var{encoding-system})
+(@var{decoding-system} . @var{encoding-system})

 @end example
 
 The first element, @var{decoding-system}, is the coding system to use
 @end example
 
 The first element, @var{decoding-system}, is the coding system to use


@@ -834,24 +1096,35 @@ for decoding (in case @var{operation} does decoding), and
 @var{encoding-system} is the coding system for encoding (in case
 @var{operation} does encoding).
 
 @var{encoding-system} is the coding system for encoding (in case
 @var{operation} does encoding).
 

-The argument @var{operation} should be a symbol, one of
-@code{insert-file-contents}, @code{write-region}, @code{call-process},
-@code{call-process-region}, @code{start-process}, or
-@code{open-network-stream}.  These are the names of the Emacs I/O primitives
-that can do coding system conversion.
+The argument @var{operation} should be a symbol, any one of
+@code{insert-file-contents}, @code{write-region},
+@code{start-process}, @code{call-process}, @code{call-process-region},
+or @code{open-network-stream}.  These are the names of the Emacs I/O
+primitives that can do character code and eol conversion.

 
 The remaining arguments should be the same arguments that might be given
 
 The remaining arguments should be the same arguments that might be given

-to that I/O primitive.  Depending on the primitive, one of those
-arguments is selected as the @dfn{target}.  For example, if
+to the corresponding I/O primitive.  Depending on the primitive, one
+of those arguments is selected as the @dfn{target}.  For example, if

 @var{operation} does file I/O, whichever argument specifies the file
 name is the target.  For subprocess primitives, the process name is the
 target.  For @code{open-network-stream}, the target is the service name
 or port number.
 
 @var{operation} does file I/O, whichever argument specifies the file
 name is the target.  For subprocess primitives, the process name is the
 target.  For @code{open-network-stream}, the target is the service name
 or port number.
 

-This function looks up the target in @code{file-coding-system-alist},
-@code{process-coding-system-alist}, or
-@code{network-coding-system-alist}, depending on @var{operation}.
-@xref{Default Coding Systems}.
+Depending on @var{operation}, this function looks up the target in
+@code{file-coding-system-alist}, @code{process-coding-system-alist},
+or @code{network-coding-system-alist}.  If the target is found in the
+alist, @code{find-operation-coding-system} returns its association in
+the alist; otherwise it returns @code{nil}.
+
+If @var{operation} is @code{insert-file-contents}, the argument
+corresponding to the target may be a cons cell of the form
+@code{(@var{filename} . @var{buffer})}).  In that case, @var{filename}
+is a file name to look up in @code{file-coding-system-alist}, and
+@var{buffer} is a buffer that contains the file's contents (not yet
+decoded).  If @code{file-coding-system-alist} specifies a function to
+call for this file, and that function needs to examine the file's
+contents (as it usually does), it should examine the contents of
+@var{buffer} instead of reading the file.

 @end defun
 
 @node Specifying Coding Systems
 @end defun
 
 @node Specifying Coding Systems


@@ -878,8 +1151,8 @@ of the right way to use the variable:
 
 @example
 ;; @r{Read the file with no character code conversion.}
 
 @example
 ;; @r{Read the file with no character code conversion.}

-;; @r{Assume @sc{crlf} represents end-of-line.}
-(let ((coding-system-for-write 'emacs-mule-dos))
+;; @r{Assume @acronym{crlf} represents end-of-line.}
+(let ((coding-system-for-read 'emacs-mule-dos))

   (insert-file-contents filename))
 @end example
 
   (insert-file-contents filename))
 @end example
 


@@ -918,90 +1191,78 @@ ability to use a coding system to encode or decode the text.
 You can also explicitly encode and decode text using the functions
 in this section.
 
 You can also explicitly encode and decode text using the functions
 in this section.
 

-@cindex raw bytes

   The result of encoding, and the input to decoding, are not ordinary
   The result of encoding, and the input to decoding, are not ordinary

-text.  They are ``raw bytes''---bytes that represent text in the same
-way that an external file would.  When a buffer contains raw bytes, it
-is most natural to mark that buffer as using unibyte representation,
-using @code{set-buffer-multibyte} (@pxref{Selecting a Representation}),
-but this is not required.  If the buffer's contents are only temporarily
-raw, leave the buffer multibyte, which will be correct after you decode
-them.
-
-  The usual way to get raw bytes in a buffer, for explicit decoding, is
-to read them from a file with @code{insert-file-contents-literally}
-(@pxref{Reading from Files}) or specify a non-@code{nil} @var{rawfile}
-argument when visiting a file with @code{find-file-noselect}.
-
-  The usual way to use the raw bytes that result from explicitly
-encoding text is to copy them to a file or process---for example, to
-write them with @code{write-region} (@pxref{Writing to Files}), and
-suppress encoding for that @code{write-region} call by binding
-@code{coding-system-for-write} to @code{no-conversion}.
-
-  Raw bytes typically contain stray individual bytes with values in the
-range 128 through 255, that are legitimate only as part of multibyte
-sequences.  Even if the buffer is multibyte, Emacs treats each such
-individual byte as a character and uses the byte value as its character
-code.  In this way, character codes 128 through 255 can be found in a
-multibyte buffer, even though they are not legitimate multibyte
-character codes.
-
-  Raw bytes sometimes contain overlong byte-sequences that look like a
-proper multibyte character plus extra superfluous trailing codes.  For
-most purposes, Emacs treats such a sequence in a buffer or string as a
-single character, and if you look at its character code, you get the
-value that corresponds to the multibyte character
-sequence---disregarding the extra trailing codes.  This is not quite
-clean, but raw bytes are used only in limited ways, so as a practical
-matter it is not worth the trouble to treat this case differently.
-
-  When a multibyte buffer contains illegitimate byte sequences,
-sometimes insertion or deletion can cause them to coalesce into a
-legitimate multibyte character.  For example, suppose the buffer
-contains the sequence 129 68 192, 68 being the character @samp{D}.  If
-you delete the @samp{D}, the bytes 129 and 192 become adjacent, and thus
-become one multibyte character (Latin-1 A with grave accent).  Point
-moves to one side or the other of the character, since it cannot be
-within a character.  Don't be alarmed by this.
-
-  Some really peculiar situations prevent proper coalescence.  For
-example, if you narrow the buffer so that the accessible portion begins
-just before the @samp{D}, then delete the @samp{D}, the two surrounding
-bytes cannot coalesce because one of them is outside the accessible
-portion of the buffer.  In this case, the deletion cannot be done, so
-@code{delete-region} signals an error.
+text.  They logically consist of a series of byte values; that is, a
+series of characters whose codes are in the range 0 through 255.  In a
+multibyte buffer or string, character codes 128 through 159 are
+represented by multibyte sequences, but this is invisible to Lisp
+programs.
+
+  The usual way to read a file into a buffer as a sequence of bytes, so
+you can decode the contents explicitly, is with
+@code{insert-file-contents-literally} (@pxref{Reading from Files});
+alternatively, specify a non-@code{nil} @var{rawfile} argument when
+visiting a file with @code{find-file-noselect}.  These methods result in
+a unibyte buffer.
+
+  The usual way to use the byte sequence that results from explicitly
+encoding text is to copy it to a file or process---for example, to write
+it with @code{write-region} (@pxref{Writing to Files}), and suppress
+encoding by binding @code{coding-system-for-write} to
+@code{no-conversion}.

 
   Here are the functions to perform explicit encoding or decoding.  The
 
   Here are the functions to perform explicit encoding or decoding.  The

-decoding functions produce ``raw bytes''; the encoding functions are
-meant to operate on ``raw bytes''.  All of these functions discard text
-properties.
+encoding functions produce sequences of bytes; the decoding functions
+are meant to operate on sequences of bytes.  All of these functions
+discard text properties.

 
 

-@defun encode-coding-region start end coding-system
-This function encodes the text from @var{start} to @var{end} according
+@deffn Command encode-coding-region start end coding-system
+This command encodes the text from @var{start} to @var{end} according

 to coding system @var{coding-system}.  The encoded text replaces the
 to coding system @var{coding-system}.  The encoded text replaces the

-original text in the buffer.  The result of encoding is ``raw bytes,''
-but the buffer remains multibyte if it was multibyte before.
-@end defun
+original text in the buffer.  The result of encoding is logically a
+sequence of bytes, but the buffer remains multibyte if it was multibyte
+before.
+
+This command returns the length of the encoded text.
+@end deffn

 
 

-@defun encode-coding-string string coding-system
+@defun encode-coding-string string coding-system &optional nocopy

 This function encodes the text in @var{string} according to coding
 system @var{coding-system}.  It returns a new string containing the
 This function encodes the text in @var{string} according to coding
 system @var{coding-system}.  It returns a new string containing the

-encoded text.  The result of encoding is a unibyte string of ``raw bytes.''
+encoded text, except when @var{nocopy} is non-@code{nil}, in which
+case the function may return @var{string} itself if the encoding
+operation is trivial.  The result of encoding is a unibyte string.

 @end defun
 
 @end defun
 

-@defun decode-coding-region start end coding-system
-This function decodes the text from @var{start} to @var{end} according
+@deffn Command decode-coding-region start end coding-system
+This command decodes the text from @var{start} to @var{end} according

 to coding system @var{coding-system}.  The decoded text replaces the
 original text in the buffer.  To make explicit decoding useful, the text
 to coding system @var{coding-system}.  The decoded text replaces the
 original text in the buffer.  To make explicit decoding useful, the text

-before decoding ought to be ``raw bytes.''
-@end defun
+before decoding ought to be a sequence of byte values, but both
+multibyte and unibyte buffers are acceptable.
+
+This command returns the length of the decoded text.
+@end deffn

 
 

-@defun decode-coding-string string coding-system
+@defun decode-coding-string string coding-system &optional nocopy

 This function decodes the text in @var{string} according to coding
 system @var{coding-system}.  It returns a new string containing the
 This function decodes the text in @var{string} according to coding
 system @var{coding-system}.  It returns a new string containing the

-decoded text.  To make explicit decoding useful, the contents of
-@var{string} ought to be ``raw bytes.''
+decoded text, except when @var{nocopy} is non-@code{nil}, in which
+case the function may return @var{string} itself if the decoding
+operation is trivial.  To make explicit decoding useful, the contents
+of @var{string} ought to be a sequence of byte values, but a multibyte
+string is acceptable.
+@end defun
+
+@defun decode-coding-inserted-region from to filename &optional visit beg end replace
+This function decodes the text from @var{from} to @var{to} as if
+it were being read from file @var{filename} using @code{insert-file-contents}
+using the rest of the arguments provided.
+
+The normal way to use this function is after reading text from a file
+without decoding, if you decide you would rather have decoded it.
+Instead of deleting the text and reading it again, this time with
+decoding, you can call this function.

 @end defun
 
 @node Terminal I/O Encoding
 @end defun
 
 @node Terminal I/O Encoding


@@ -1018,22 +1279,22 @@ This function returns the coding system that is in use for decoding
 keyboard input---or @code{nil} if no coding system is to be used.
 @end defun
 
 keyboard input---or @code{nil} if no coding system is to be used.
 @end defun
 

-@defun set-keyboard-coding-system coding-system
-This function specifies @var{coding-system} as the coding system to
+@deffn Command set-keyboard-coding-system coding-system
+This command specifies @var{coding-system} as the coding system to

 use for decoding keyboard input.  If @var{coding-system} is @code{nil},
 that means do not decode keyboard input.
 use for decoding keyboard input.  If @var{coding-system} is @code{nil},
 that means do not decode keyboard input.

-@end defun
+@end deffn

 
 @defun terminal-coding-system
 This function returns the coding system that is in use for encoding
 terminal output---or @code{nil} for no encoding.
 @end defun
 
 
 @defun terminal-coding-system
 This function returns the coding system that is in use for encoding
 terminal output---or @code{nil} for no encoding.
 @end defun
 

-@defun set-terminal-coding-system coding-system
-This function specifies @var{coding-system} as the coding system to use
+@deffn Command set-terminal-coding-system coding-system
+This command specifies @var{coding-system} as the coding system to use

 for encoding terminal output.  If @var{coding-system} is @code{nil},
 that means do not encode terminal output.
 for encoding terminal output.  If @var{coding-system} is @code{nil},
 that means do not encode terminal output.

-@end defun
+@end deffn

 
 @node MS-DOS File Types
 @subsection MS-DOS File Types
 
 @node MS-DOS File Types
 @subsection MS-DOS File Types


@@ -1046,7 +1307,7 @@ that means do not encode terminal output.
 
   On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
 end-of-line conversion for a file by looking at the file's name.  This
 
   On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
 end-of-line conversion for a file by looking at the file's name.  This

-feature classifies fils as @dfn{text files} and @dfn{binary files}.  By
+feature classifies files as @dfn{text files} and @dfn{binary files}.  By

 ``binary file'' we mean a file of literal byte values that are not
 necessarily meant to be characters; Emacs does no end-of-line conversion
 and no character code conversion for them.  On the other hand, the bytes
 ``binary file'' we mean a file of literal byte values that are not
 necessarily meant to be characters; Emacs does no end-of-line conversion
 and no character code conversion for them.  On the other hand, the bytes


@@ -1098,9 +1359,9 @@ from the file contents, in the usual Emacs fashion.
 @section Input Methods
 @cindex input methods
 
 @section Input Methods
 @cindex input methods
 

-  @dfn{Input methods} provide convenient ways of entering non-@sc{ascii}
+  @dfn{Input methods} provide convenient ways of entering non-@acronym{ASCII}

 characters from the keyboard.  Unlike coding systems, which translate
 characters from the keyboard.  Unlike coding systems, which translate

-non-@sc{ascii} characters to and from encodings meant to be read by
+non-@acronym{ASCII} characters to and from encodings meant to be read by

 programs, input methods provide human-friendly commands.  (@xref{Input
 Methods,,, emacs, The GNU Emacs Manual}, for information on how users
 use input methods to enter text.)  How to define input methods is not
 programs, input methods provide human-friendly commands.  (@xref{Input
 Methods,,, emacs, The GNU Emacs Manual}, for information on how users
 use input methods to enter text.)  How to define input methods is not


@@ -1116,18 +1377,18 @@ in any fashion.)  It is @code{nil} if no input method is active in the
 buffer now.
 @end defvar
 
 buffer now.
 @end defvar
 

-@defvar default-input-method
+@defopt default-input-method

 This variable holds the default input method for commands that choose an
 input method.  Unlike @code{current-input-method}, this variable is
 normally global.
 This variable holds the default input method for commands that choose an
 input method.  Unlike @code{current-input-method}, this variable is
 normally global.

-@end defvar
+@end defopt

 
 

-@defun set-input-method input-method
-This function activates input method @var{input-method} for the current
+@deffn Command set-input-method input-method
+This command activates input method @var{input-method} for the current

 buffer.  It also sets @code{default-input-method} to @var{input-method}.
 buffer.  It also sets @code{default-input-method} to @var{input-method}.

-If @var{input-method} is @code{nil}, this function deactivates any input
+If @var{input-method} is @code{nil}, this command deactivates any input

 method for the current buffer.
 method for the current buffer.

-@end defun
+@end deffn

 
 @defun read-input-method-name prompt &optional default inhibit-null
 This function reads an input method name with the minibuffer, prompting
 
 @defun read-input-method-name prompt &optional default inhibit-null
 This function reads an input method name with the minibuffer, prompting


@@ -1152,18 +1413,19 @@ Here @var{input-method} is the input method name, a string;
 environment this input method is recommended for.  (That serves only for
 documentation purposes.)
 
 environment this input method is recommended for.  (That serves only for
 documentation purposes.)
 

-@var{title} is a string to display in the mode line while this method is
-active.  @var{description} is a string describing this method and what
-it is good for.
-

 @var{activate-func} is a function to call to activate this method.  The
 @var{args}, if any, are passed as arguments to @var{activate-func}.  All
 told, the arguments to @var{activate-func} are @var{input-method} and
 the @var{args}.
 @var{activate-func} is a function to call to activate this method.  The
 @var{args}, if any, are passed as arguments to @var{activate-func}.  All
 told, the arguments to @var{activate-func} are @var{input-method} and
 the @var{args}.

+
+@var{title} is a string to display in the mode line while this method is
+active.  @var{description} is a string describing this method and what
+it is good for.

 @end defvar
 
   The fundamental interface to input methods is through the
 @end defvar
 
   The fundamental interface to input methods is through the

-variable @code{input-method-function}.  @xref{Reading One Event}.
+variable @code{input-method-function}.  @xref{Reading One Event},
+and @ref{Invoking the Input Method}.

 
 @node Locales
 @section Locales
 
 @node Locales
 @section Locales


@@ -1174,26 +1436,57 @@ to use in language-related features.  These Emacs variables control
 how Emacs interacts with these features.
 
 @defvar locale-coding-system
 how Emacs interacts with these features.
 
 @defvar locale-coding-system

-@tindex locale-coding-system
+@cindex keyboard input decoding on X

 This variable specifies the coding system to use for decoding system
 This variable specifies the coding system to use for decoding system

-error messages, for encoding the format argument to
-@code{format-time-string}, and for decoding the return value of
-@code{format-time-string}.
+error messages and---on X Window system only---keyboard input, for
+encoding the format argument to @code{format-time-string}, and for
+decoding the return value of @code{format-time-string}.

 @end defvar
 
 @defvar system-messages-locale
 @end defvar
 
 @defvar system-messages-locale

-@tindex system-messages-locale

 This variable specifies the locale to use for generating system error
 messages.  Changing the locale can cause messages to come out in a
 This variable specifies the locale to use for generating system error
 messages.  Changing the locale can cause messages to come out in a

-different language or in a different orthorgraphy.  If the variable is
+different language or in a different orthography.  If the variable is

 @code{nil}, the locale is specified by environment variables in the
 usual POSIX fashion.
 @end defvar
 
 @defvar system-time-locale
 @code{nil}, the locale is specified by environment variables in the
 usual POSIX fashion.
 @end defvar
 
 @defvar system-time-locale

-@tindex system-time-locale

 This variable specifies the locale to use for formatting time values.
 Changing the locale can cause messages to appear according to the
 conventions of a different language.  If the variable is @code{nil}, the
 locale is specified by environment variables in the usual POSIX fashion.
 @end defvar
 This variable specifies the locale to use for formatting time values.
 Changing the locale can cause messages to appear according to the
 conventions of a different language.  If the variable is @code{nil}, the
 locale is specified by environment variables in the usual POSIX fashion.
 @end defvar

+
+@defun locale-info item
+This function returns locale data @var{item} for the current POSIX
+locale, if available.  @var{item} should be one of these symbols:
+
+@table @code
+@item codeset
+Return the character set as a string (locale item @code{CODESET}).
+
+@item days
+Return a 7-element vector of day names (locale items
+@code{DAY_1} through @code{DAY_7});
+
+@item months
+Return a 12-element vector of month names (locale items @code{MON_1}
+through @code{MON_12}).
+
+@item paper
+Return a list @code{(@var{width} @var{height})} for the default paper
+size measured in millimeters (locale items @code{PAPER_WIDTH} and
+@code{PAPER_HEIGHT}).
+@end table
+
+If the system can't provide the requested information, or if
+@var{item} is not one of those symbols, the value is @code{nil}.  All
+strings in the return value are decoded using
+@code{locale-coding-system}.  @xref{Locales,,, libc, The GNU Libc Manual},
+for more information about locales and locale items.
+@end defun
+
+@ignore
+   arch-tag: be705bf8-941b-4c35-84fc-ad7d20ddb7cb
+@end ignore