@c This is part of the Emacs manual.
-@c Copyright (C) 1997, 1999-2012 Free Software Foundation, Inc.
+@c Copyright (C) 1997, 1999-2014 Free Software Foundation, Inc.
@c See file emacs.texi for copying conditions.
-@node International, Modes, Frames, Top
+@node International
@chapter International Character Set Support
@c This node is referenced in the tutorial. When renaming or deleting
@c it, the tutorial needs to be adjusted. (TUTORIAL.de)
-@cindex MULE
@cindex international scripts
@cindex multibyte characters
@cindex encoding of characters
@menu
* International Chars:: Basic concepts of multibyte characters.
-* Disabling Multibyte:: Controlling whether to use multibyte characters.
* Language Environments:: Setting things up for the language you use.
* Input Methods:: Entering text characters not on your keyboard.
* Select Input Method:: Specifying your choice of input methods.
used, generally don't have keys for all the characters in them. You
can insert characters that your keyboard does not support, using
@kbd{C-q} (@code{quoted-insert}) or @kbd{C-x 8 @key{RET}}
-(@code{ucs-insert}). @xref{Inserting Text}. Emacs also supports
+(@code{insert-char}). @xref{Inserting Text}. Emacs also supports
various @dfn{input methods}, typically one for each script or
language, which make it easier to type characters in the script.
@xref{Input Methods}.
decomposition: (65 768) ('A' '`')
@end smallexample
-@c FIXME? Does this section even belong in the user manual?
-@c Seems more appropriate to the lispref?
-@node Disabling Multibyte
-@section Disabling Multibyte Characters
-
- By default, Emacs starts in multibyte mode: it stores the contents
-of buffers and strings using an internal encoding that represents
-non-@acronym{ASCII} characters using multi-byte sequences. Multibyte
-mode allows you to use all the supported languages and scripts without
-limitations.
-
-@cindex turn multibyte support on or off
- Under very special circumstances, you may want to disable multibyte
-character support, for a specific buffer.
-When multibyte characters are disabled in a buffer, we call
-that @dfn{unibyte mode}. In unibyte mode, each character in the
-buffer has a character code ranging from 0 through 255 (0377 octal); 0
-through 127 (0177 octal) represent @acronym{ASCII} characters, and 128
-(0200 octal) through 255 (0377 octal) represent non-@acronym{ASCII}
-characters.
-
- To edit a particular file in unibyte representation, visit it using
-@code{find-file-literally}. @xref{Visiting}. You can convert a
-multibyte buffer to unibyte by saving it to a file, killing the
-buffer, and visiting the file again with @code{find-file-literally}.
-Alternatively, you can use @kbd{C-x @key{RET} c}
-(@code{universal-coding-system-argument}) and specify @samp{raw-text}
-as the coding system with which to visit or save a file. @xref{Text
-Coding}. Unlike @code{find-file-literally}, finding a file as
-@samp{raw-text} doesn't disable format conversion, uncompression, or
-auto mode selection.
-
-@c Not a single file in Emacs uses this feature. Is it really worth
-@c mentioning in the _user_ manual? Also, this duplicates somewhat
-@c "Loading Non-ASCII" from the lispref.
-@cindex Lisp files, and multibyte operation
-@cindex multibyte operation, and Lisp files
-@cindex unibyte operation, and Lisp files
-@cindex init file, and non-@acronym{ASCII} characters
- Emacs normally loads Lisp files as multibyte.
-This includes the Emacs initialization
-file, @file{.emacs}, and the initialization files of packages
-such as Gnus. However, you can specify unibyte loading for a
-particular Lisp file, by adding an entry @samp{unibyte: t} in a file
-local variables section (@pxref{File Variables}). Then that file is
-always loaded as unibyte text. Note that this does not represent a
-real @code{unibyte} variable, rather it just acts as an indicator
-to Emacs in the same way as @code{coding} does (@pxref{Specify Coding}).
-@ignore
-@c I don't see the point of this statement:
-The motivation for these conventions is that it is more reliable to
-always load any particular Lisp file in the same way.
-@end ignore
-Note also that this feature only applies to @emph{loading} Lisp files
-for evaluation, not to visiting them for editing. You can also load a
-Lisp file as unibyte, on any one occasion, by typing @kbd{C-x
-@key{RET} c raw-text @key{RET}} immediately before loading it.
-
-@c See http://debbugs.gnu.org/11226 for lack of unibyte tooltip.
-@vindex enable-multibyte-characters
-The buffer-local variable @code{enable-multibyte-characters} is
-non-@code{nil} in multibyte buffers, and @code{nil} in unibyte ones.
-The mode line also indicates whether a buffer is multibyte or not.
-@xref{Mode Line}. With a graphical display, in a multibyte buffer,
-the portion of the mode line that indicates the character set has a
-tooltip that (amongst other things) says that the buffer is multibyte.
-In a unibyte buffer, the character set indicator is absent. Thus, in
-a unibyte buffer (when using a graphical display) there is normally
-nothing before the indication of the visited file's end-of-line
-convention (colon, backslash, etc.), unless you are using an input
-method.
-
-@findex toggle-enable-multibyte-characters
-You can turn off multibyte support in a specific buffer by invoking the
-command @code{toggle-enable-multibyte-characters} in that buffer.
-
@node Language Environments
@section Language Environments
@cindex language environments
@cindex Euro sign
@cindex UTF-8
@quotation
-ASCII, Belarusian, Bengali, Brazilian Portuguese, Bulgarian,
+ASCII, Belarusian, Bengali, Brazilian Portuguese, Bulgarian, Cham,
Chinese-BIG5, Chinese-CNS, Chinese-EUC-TW, Chinese-GB, Chinese-GBK,
Chinese-GB18030, Croatian, Cyrillic-ALT, Cyrillic-ISO, Cyrillic-KOI8,
Czech, Devanagari, Dutch, English, Esperanto, Ethiopic, French,
which prefers Cyrillic characters and files encoded in Windows-1255).
@end quotation
-@cindex fonts for various scripts
-@cindex Intlfonts package, installation
To display the script(s) used by your language environment on a
-graphical display, you need to have a suitable font. If some of the
-characters appear as empty boxes or hex codes, you should install
-extra fonts. Your operating system may have optional fonts that
-you can install; or you can install the
-GNU Intlfonts package, which includes fonts for most supported
-scripts.@footnote{If you run Emacs on X, you may need to inform the X
-server about the location of the newly installed fonts with
-commands such as:
-@c FIXME? I feel like this may be out of date.
-@c Eg the intlfonts tarfile is ~ 10 years old.
-
-@example
- xset fp+ /usr/local/share/emacs/fonts
- xset fp rehash
-@end example
-}
+graphical display, you need to have suitable fonts.
@xref{Fontsets}, for more details about setting up your fonts.
@findex set-locale-environment
not when you are in the minibuffer).
Another facility for typing characters not on your keyboard is by
-using @kbd{C-x 8 @key{RET}} (@code{ucs-insert}) to insert a single
+using @kbd{C-x 8 @key{RET}} (@code{insert-char}) to insert a single
character based on its Unicode name or code-point; see @ref{Inserting
Text}.
Unlike the previous two, this variable does not override any
@samp{-*-coding:-*-} tag.
-@c FIXME? This seems somewhat out of place. Move to the Rmail section?
-@vindex rmail-decode-mime-charset
-@vindex rmail-file-coding-system
- When you get new mail in Rmail, each message is translated
-automatically from the coding system it is written in, as if it were a
-separate file. This uses the priority list of coding systems that you
-have specified. If a MIME message specifies a character set, Rmail
-obeys that specification. For reading and saving Rmail files
-themselves, Emacs uses the coding system specified by the variable
-@code{rmail-file-coding-system}. The default value is @code{nil},
-which means that Rmail files are not translated (they are read and
-written in the Emacs internal character code).
-
@node Specify Coding
@section Specifying a File's Coding System
behaves a bit differently. It additionally checks whether the
@c What determines this?
most-preferred coding system is recommended for use in MIME messages;
-if not, Emacs tells you that the most-preferred coding system is not
-recommended and prompts you for another coding system. This is so you
-won't inadvertently send a message encoded in a way that your
-recipient's mail software will have difficulty decoding. (You can
-still use an unsuitable coding system if you type its name in response
-to the question.)
+if not, it informs you of this fact and prompts you for another coding
+system. This is so you won't inadvertently send a message encoded in
+a way that your recipient's mail software will have difficulty
+decoding. (You can still use an unsuitable coding system if you enter
+its name at the prompt.)
@c It seems that select-message-coding-system does this.
-@c Both sendmail.el and smptmail.el call it; i.e. smtpmail.el still
+@c Both sendmail.el and smptmail.el call it; i.e., smtpmail.el still
@c obeys sendmail-coding-system.
@vindex sendmail-coding-system
When you send a mail message (@pxref{Sending Mail}),
@findex set-buffer-file-coding-system
The command @kbd{C-x @key{RET} f}
(@code{set-buffer-file-coding-system}) sets the file coding system for
-the current buffer---in other words, it says which coding system to
-use when saving or reverting the visited file. You specify which
-coding system using the minibuffer. If you specify a coding system
-that cannot handle all of the characters in the buffer, Emacs warns
-you about the troublesome characters when you actually save the
-buffer.
+the current buffer (i.e., the coding system to use when saving or
+reverting the file). You specify which coding system using the
+minibuffer. You can also invoke this command by clicking with
+@kbd{Mouse-3} on the coding system indicator in the mode line
+(@pxref{Mode Line}).
+
+ If you specify a coding system that cannot handle all the characters
+in the buffer, Emacs will warn you about the troublesome characters,
+and ask you to choose another coding system, when you try to save the
+buffer (@pxref{Output Coding}).
@cindex specify end-of-line conversion
You can also use this command to specify the end-of-line conversion
@table @kbd
@item C-x @key{RET} x @var{coding} @key{RET}
Use coding system @var{coding} for transferring selections to and from
-other window-based applications (@code{set-selection-coding-system}).
+other graphical applications (@code{set-selection-coding-system}).
@item C-x @key{RET} X @var{coding} @key{RET}
Use coding system @var{coding} for transferring @emph{one}
-selection---the next one---to or from another window-based application
+selection---the next one---to or from another graphical application
(@code{set-next-selection-coding-system}).
@item C-x @key{RET} p @var{input-coding} @key{RET} @var{output-coding} @key{RET}
file names are not encoded specially; they appear in the file system
using the internal Emacs representation.
+@cindex file-name encoding, MS-Windows
+@vindex w32-unicode-filenames
+ When Emacs runs on MS-Windows versions that are descendants of the
+NT family (Windows 2000, XP, Vista, Windows 7, and Windows 8), the
+value of @code{file-name-coding-system} is largely ignored, as Emacs
+by default uses APIs that allow to pass Unicode file names directly.
+By contrast, on Windows 9X, file names are encoded using
+@code{file-name-coding-system}, which should be set to the codepage
+(@pxref{Coding Systems, codepage}) pertinent for the current system
+locale. The value of the variable @code{w32-unicode-filenames}
+controls whether Emacs uses the Unicode APIs when it calls OS
+functions that accept file names. This variable is set by the startup
+code to @code{nil} on Windows 9X, and to @code{t} on newer versions of
+MS-Windows.
+
@strong{Warning:} if you change @code{file-name-coding-system} (or the
language environment) in the middle of an Emacs session, problems can
result if you have already visited files whose names were encoded using
A font typically defines shapes for a single alphabet or script.
Therefore, displaying the entire range of scripts that Emacs supports
requires a collection of many fonts. In Emacs, such a collection is
-called a @dfn{fontset}. A fontset is defined by a list of font specs,
+called a @dfn{fontset}. A fontset is defined by a list of font specifications,
each assigned to handle a range of character codes, and may fall back
-on another fontset for characters which are not covered by the fonts
+on another fontset for characters that are not covered by the fonts
it specifies.
+@cindex fonts for various scripts
+@cindex Intlfonts package, installation
Each fontset has a name, like a font. However, while fonts are
stored in the system and the available font names are defined by the
system, fontsets are defined within Emacs itself. Once you have
defined a fontset, you can use it within Emacs by specifying its name,
anywhere that you could use a single font. Of course, Emacs fontsets
-can use only the fonts that the system supports; if certain characters
-appear on the screen as hollow boxes, this means that the fontset in
-use for them has no font for those characters.@footnote{The Emacs
-installation instructions have information on additional font
-support.}
+can use only the fonts that the system supports. If some characters
+appear on the screen as empty boxes or hex codes, this means that the
+fontset in use for them has no font for those characters. In this
+case, or if the characters are shown, but not as well as you would
+like, you may need to install extra fonts. Your operating system may
+have optional fonts that you can install; or you can install the GNU
+Intlfonts package, which includes fonts for most supported
+scripts.@footnote{If you run Emacs on X, you may need to inform the X
+server about the location of the newly installed fonts with commands
+such as:
+@c FIXME? I feel like this may be out of date.
+@c E.g., the intlfonts tarfile is ~ 10 years old.
+
+@example
+ xset fp+ /usr/local/share/emacs/fonts
+ xset fp rehash
+@end example
+}
Emacs creates three fontsets automatically: the @dfn{standard
fontset}, the @dfn{startup fontset} and the @dfn{default fontset}.
+@c FIXME? The doc of *standard*-fontset-spec says:
+@c "You have the biggest chance to display international characters
+@c with correct glyphs by using the *standard* fontset." (my emphasis)
+@c See http://lists.gnu.org/archive/html/emacs-devel/2012-04/msg00430.html
The default fontset is most likely to have fonts for a wide variety of
-non-@acronym{ASCII} characters and is the default fallback for the
+non-@acronym{ASCII} characters, and is the default fallback for the
other two fontsets, and if you set a default font rather than fontset.
-However it does not specify font family names, so results can be
+However, it does not specify font family names, so results can be
somewhat random if you use it directly. You can specify use of a
-specific fontset with the @samp{-fn} option. For example,
+particular fontset by starting Emacs with the @samp{-fn} option.
+For example,
@example
emacs -fn fontset-standard
@noindent
or just @samp{fontset-standard} for short.
- On GNUstep and Mac, fontset-standard is created using the value of
-@code{ns-standard-fontset-spec}, and on Windows it is
+ On GNUstep and Mac OS X, the standard fontset is created using the value of
+@code{ns-standard-fontset-spec}, and on MS Windows it is
created using the value of @code{w32-standard-fontset-spec}.
+@c FIXME? How does one access these, or do anything with them?
+@c Does it matter?
Bold, italic, and bold-italic variants of the standard fontset are
created automatically. Their names have @samp{bold} instead of
@samp{medium}, or @samp{i} instead of @samp{r}, or both.
@var{charset_encoding} field with @samp{startup}, then using the
resulting string to specify a fontset.
- For instance, if you start Emacs this way,
+ For instance, if you start Emacs with a font of this form,
+@c FIXME? I think this is a little misleading, because you cannot (?)
+@c actually specify a font with wildcards, it has to be a complete spec.
+@c Also, an X font specification of this form hasn't (?) been
+@c mentioned before now, and is somewhat obsolete these days.
+@c People are more likely to use a form like
+@c emacs -fn "DejaVu Sans Mono-12"
+@c How does any of this apply in that case?
@example
emacs -fn "*courier-medium-r-normal--14-140-*-iso8859-1"
@end example
-*-courier-medium-r-normal-*-14-140-*-*-*-*-fontset-startup
@end example
- The startup fontset will use the font that you specify or a variant
-with a different registry and encoding for all the characters which
+ The startup fontset will use the font that you specify, or a variant
+with a different registry and encoding, for all the characters that
are supported by that font, and fallback on @samp{fontset-default} for
other characters.
just like an actual font name. But be careful not to specify a fontset
name in a wildcard resource like @samp{Emacs*Font}---that wildcard
specification matches various other resources, such as for menus, and
-menus cannot handle fontsets.
+@c FIXME is this still true?
+menus cannot handle fontsets. @xref{X Resources}.
You can specify additional fontsets using X resources named
@samp{Fontset-@var{n}}, where @var{n} is an integer starting from 0.
@end smallexample
@noindent
-@var{fontpattern} should have the form of a standard X font name, except
+@var{fontpattern} should have the form of a standard X font name (see
+the previous fontset-startup example), except
for the last two fields. They should have the form
@samp{fontset-@var{alias}}.
In addition, when several consecutive fields are wildcards, Emacs
collapses them into a single wildcard. This is to prevent use of
auto-scaled fonts. Fonts made by scaling larger fonts are not usable
-for editing, and scaling a smaller font is not useful because it is
+for editing, and scaling a smaller font is not also useful, because it is
better to use the smaller font in its own size, which is what Emacs
does.
You may not have any Chinese font matching the above font
specification. Most X distributions include only Chinese fonts that
-have @samp{song ti} or @samp{fangsong ti} in @var{family} field. In
-such a case, @samp{Fontset-@var{n}} can be specified as below:
+have @samp{song ti} or @samp{fangsong ti} in the @var{family} field. In
+such a case, @samp{Fontset-@var{n}} can be specified as:
@smallexample
Emacs.Fontset-0: -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24,\
Fontsets can be modified using the function @code{set-fontset-font},
specifying a character, a charset, a script, or a range of characters
-to modify the font for, and a font-spec for the font to be used. Some
-examples are:
+to modify the font for, and a font specification for the font to be
+used. Some examples are:
@example
;; Use Liberation Mono for latin-3 charset.
@node Undisplayable Characters
@section Undisplayable Characters
- There may be a some non-@acronym{ASCII} characters that your
+ There may be some non-@acronym{ASCII} characters that your
terminal cannot display. Most text terminals support just a single
character set (use the variable @code{default-terminal-coding-system}
-(@pxref{Terminal Coding}) to tell Emacs which one); characters which
+to tell Emacs which one, @ref{Terminal Coding}); characters that
can't be encoded in that coding system are displayed as @samp{?} by
default.
If you use Latin-1 characters but your terminal can't display
Latin-1, you can arrange to display mnemonic @acronym{ASCII} sequences
-instead, e.g.@: @samp{"o} for o-umlaut. Load the library
+instead, e.g., @samp{"o} for o-umlaut. Load the library
@file{iso-ascii} to do this.
@vindex latin1-display
accented letters and punctuation needed by various European languages
(and some non-European ones). Note that Emacs considers bytes with
codes in this range as raw bytes, not as characters, even in a unibyte
-session, i.e.@: if you disable multibyte characters. However, Emacs
-can still handle these character codes as if they belonged to
-@emph{one} of the single-byte character sets at a time. To specify
-@emph{which} of these codes to use, invoke @kbd{M-x
-set-language-environment} and specify a suitable language environment
-such as @samp{Latin-@var{n}}.
-
- For more information about unibyte operation, see @ref{Disabling
-Multibyte}. Note particularly that you probably want to ensure that
-your initialization files are read as unibyte if they contain
-non-@acronym{ASCII} characters.
+buffer, i.e., if you disable multibyte characters. However, Emacs can
+still handle these character codes as if they belonged to @emph{one}
+of the single-byte character sets at a time. To specify @emph{which}
+of these codes to use, invoke @kbd{M-x set-language-environment} and
+specify a suitable language environment such as @samp{Latin-@var{n}}.
+@xref{Disabling Multibyte, , Disabling Multibyte Characters, elisp,
+GNU Emacs Lisp Reference Manual}.
@vindex unibyte-display-via-language-environment
Emacs can also display bytes in the range 160 to 255 as readable
set, Emacs can display these characters as @acronym{ASCII} sequences which at
least give you a clear idea of what the characters are. To do this,
load the library @code{iso-ascii}. Similar libraries for other
-Latin-@var{n} character sets could be implemented, but we don't have
-them yet.
+Latin-@var{n} character sets could be implemented, but have not been
+so far.
@findex standard-display-8bit
@cindex 8-bit display
On a graphical display, you should not need to do anything special to
use these keys; they should simply work. On a text terminal, you
-should use the command @code{M-x set-keyboard-coding-system} or the
+should use the command @code{M-x set-keyboard-coding-system} or customize the
variable @code{keyboard-coding-system} to specify which coding system
your keyboard uses (@pxref{Terminal Coding}). Enabling this feature
will probably require you to use @kbd{ESC} to type Meta characters;
@findex list-character-sets
@kbd{M-x list-character-sets} displays a list of all supported
charsets. The list gives the names of charsets and additional
-information to identity each charset (see
-@url{http://www.itscj.ipsj.or.jp/ISO-IR/} for details). In this list,
+information to identity each charset; see the
+@url{http://www.itscj.ipsj.or.jp/ISO-IR/, International Register of
+Coded Character Sets} for more details. In this list,
charsets are divided into two categories: @dfn{normal charsets} are
listed first, followed by @dfn{supplementary charsets}. A
supplementary charset is one that is used to define another charset
Hebrew, whose natural ordering of horizontal text for display is from
right to left. However, digits and Latin text embedded in these
scripts are still displayed left to right. It is also not uncommon to
-have small portions of text in Arabic or Hebrew embedded in otherwise
-Latin document, e.g., as comments and strings in a program source
+have small portions of text in Arabic or Hebrew embedded in an otherwise
+Latin document; e.g., as comments and strings in a program source
file. For these reasons, text that uses these scripts is actually
@dfn{bidirectional}: a mixture of runs of left-to-right and
right-to-left characters.
directionality when they are displayed. The default value is
@code{t}.
+@cindex base direction of paragraphs
+@cindex paragraph, base direction
Each paragraph of bidirectional text can have its own @dfn{base
direction}, either right-to-left or left-to-right. (Paragraph
-boundaries are empty lines, i.e.@: lines consisting entirely of
-whitespace characters.) Text in left-to-right paragraphs begins at
-the left margin of the window and is truncated or continued when it
-reaches the right margin. By contrast, text in right-to-left
-paragraphs begins at the right margin and is continued or truncated at
-the left margin.
+@c paragraph-separate etc have no influence on this?
+boundaries are empty lines, i.e., lines consisting entirely of
+whitespace characters.) Text in left-to-right paragraphs begins on
+the screen at the left margin of the window and is truncated or
+continued when it reaches the right margin. By contrast, text in
+right-to-left paragraphs is displayed starting at the right margin and
+is continued or truncated at the left margin.
@vindex bidi-paragraph-direction
Emacs determines the base direction of each paragraph dynamically,
jump when point traverses reordered bidirectional text. Similarly, a
highlighted region covering a contiguous range of character positions
may look discontinuous if the region spans reordered text. This is
-normal and similar to behavior of other programs that support
-bidirectional text.
+normal and similar to the behavior of other programs that support
+bidirectional text. If you set @code{visual-order-cursor-movement} to
+a non-@code{nil} value, cursor motion by the arrow keys follows the
+visual order on screen (@pxref{Moving Point, visual-order movement}).
HCoop / bpt / emacs.git / blobdiff