HCoop / bpt / emacs.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
(syms_of_coding) <translation-table-for-input>: Modify doc string to
[bpt/emacs.git] / src / coding.c
CommitLineData
9542cb1f 1/* Coding system handler (conversion, detection, etc).
aaef169d 2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
76b6f707 3 2006, 2007, 2008, 2009 Free Software Foundation, Inc.
7976eda0 4 Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
76b6f707 5 2005, 2006, 2007, 2008, 2009
ce03bf76
KH		 6 National Institute of Advanced Industrial Science and Technology (AIST)
7 Registration Number H14PRO021
8f924df7 8 Copyright (C) 2003
df7492f9
KH		 9 National Institute of Advanced Industrial Science and Technology (AIST)
10 Registration Number H13PRO009
4ed46869 11
369314dc
KH		 12This file is part of GNU Emacs.
13
9ec0b715 14GNU Emacs is free software: you can redistribute it and/or modify
369314dc 15it under the terms of the GNU General Public License as published by
9ec0b715
GM		 16the Free Software Foundation, either version 3 of the License, or
17(at your option) any later version.
4ed46869 18
369314dc
KH		 19GNU Emacs is distributed in the hope that it will be useful,
20but WITHOUT ANY WARRANTY; without even the implied warranty of
21MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
22GNU General Public License for more details.
4ed46869 23
369314dc 24You should have received a copy of the GNU General Public License
9ec0b715 25along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. */
4ed46869
KH		 26
27/*** TABLE OF CONTENTS ***
28
b73bfc1c 29 0. General comments
4ed46869 30 1. Preamble
df7492f9
KH		 31 2. Emacs' internal format (emacs-utf-8) handlers
32 3. UTF-8 handlers
33 4. UTF-16 handlers
34 5. Charset-base coding systems handlers
35 6. emacs-mule (old Emacs' internal format) handlers
36 7. ISO2022 handlers
37 8. Shift-JIS and BIG5 handlers
38 9. CCL handlers
39 10. C library functions
40 11. Emacs Lisp library functions
41 12. Postamble
4ed46869
KH		 42
43*/
44
df7492f9 45/*** 0. General comments ***
b73bfc1c
KH		 46
47
df7492f9 48CODING SYSTEM
4ed46869 49
5bad0796
DL		 50 A coding system is an object for an encoding mechanism that contains
51 information about how to convert byte sequences to character
e19c3639
KH		 52 sequences and vice versa. When we say "decode", it means converting
53 a byte sequence of a specific coding system into a character
54 sequence that is represented by Emacs' internal coding system
55 `emacs-utf-8', and when we say "encode", it means converting a
56 character sequence of emacs-utf-8 to a byte sequence of a specific
0ef69138 57 coding system.
4ed46869 58
e19c3639
KH		 59 In Emacs Lisp, a coding system is represented by a Lisp symbol. In
60 C level, a coding system is represented by a vector of attributes
5bad0796 61 stored in the hash table Vcharset_hash_table. The conversion from
e19c3639
KH		 62 coding system symbol to attributes vector is done by looking up
63 Vcharset_hash_table by the symbol.
4ed46869 64
e19c3639 65 Coding systems are classified into the following types depending on
5bad0796 66 the encoding mechanism. Here's a brief description of the types.
4ed46869 67
df7492f9
KH		 68 o UTF-8
69
70 o UTF-16
71
72 o Charset-base coding system
73
74 A coding system defined by one or more (coded) character sets.
5bad0796 75 Decoding and encoding are done by a code converter defined for each
df7492f9
KH		 76 character set.
77
5bad0796 78 o Old Emacs internal format (emacs-mule)
df7492f9 79
5bad0796 80 The coding system adopted by old versions of Emacs (20 and 21).
4ed46869 81
df7492f9 82 o ISO2022-base coding system
4ed46869
KH		 83
84 The most famous coding system for multiple character sets. X's
df7492f9
KH		 85 Compound Text, various EUCs (Extended Unix Code), and coding systems
86 used in the Internet communication such as ISO-2022-JP are all
87 variants of ISO2022.
4ed46869 88
df7492f9 89 o SJIS (or Shift-JIS or MS-Kanji-Code)
93dec019 90
4ed46869
KH		 91 A coding system to encode character sets: ASCII, JISX0201, and
92 JISX0208. Widely used for PC's in Japan. Details are described in
df7492f9 93 section 8.
4ed46869 94
df7492f9 95 o BIG5
4ed46869 96
df7492f9 97 A coding system to encode character sets: ASCII and Big5. Widely
cfb43547 98 used for Chinese (mainly in Taiwan and Hong Kong). Details are
df7492f9
KH		 99 described in section 8. In this file, when we write "big5" (all
100 lowercase), we mean the coding system, and when we write "Big5"
101 (capitalized), we mean the character set.
4ed46869 102
df7492f9 103 o CCL
27901516 104
5bad0796 105 If a user wants to decode/encode text encoded in a coding system
df7492f9
KH		 106 not listed above, he can supply a decoder and an encoder for it in
107 CCL (Code Conversion Language) programs. Emacs executes the CCL
108 program while decoding/encoding.
27901516 109
df7492f9 110 o Raw-text
4ed46869 111
5a936b46 112 A coding system for text containing raw eight-bit data. Emacs
5bad0796 113 treats each byte of source text as a character (except for
df7492f9 114 end-of-line conversion).
4ed46869 115
df7492f9
KH		 116 o No-conversion
117
118 Like raw text, but don't do end-of-line conversion.
4ed46869 119
4ed46869 120
df7492f9 121END-OF-LINE FORMAT
4ed46869 122
5bad0796 123 How text end-of-line is encoded depends on operating system. For
df7492f9 124 instance, Unix's format is just one byte of LF (line-feed) code,
f4dee582 125 whereas DOS's format is two-byte sequence of `carriage-return' and
d46c5b12
KH		 126 `line-feed' codes. MacOS's format is usually one byte of
127 `carriage-return'.
4ed46869 128
cfb43547 129 Since text character encoding and end-of-line encoding are
df7492f9
KH		 130 independent, any coding system described above can take any format
131 of end-of-line (except for no-conversion).
4ed46869 132
e19c3639
KH		 133STRUCT CODING_SYSTEM
134
135 Before using a coding system for code conversion (i.e. decoding and
136 encoding), we setup a structure of type `struct coding_system'.
137 This structure keeps various information about a specific code
5bad0796 138 conversion (e.g. the location of source and destination data).
4ed46869
KH		 139
140*/
141
df7492f9
KH		 142/* COMMON MACROS */
143
144
4ed46869
KH		 145/*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
146
df7492f9 147 These functions check if a byte sequence specified as a source in
ff0dacd7
KH		 148 CODING conforms to the format of XXX, and update the members of
149 DETECT_INFO.
df7492f9 150
ff0dacd7 151 Return 1 if the byte sequence conforms to XXX, otherwise return 0.
df7492f9
KH		 152
153 Below is the template of these functions. */
154
4ed46869 155#if 0
df7492f9 156static int
ff0dacd7 157detect_coding_XXX (coding, detect_info)
df7492f9 158 struct coding_system *coding;
ff0dacd7 159 struct coding_detection_info *detect_info;
4ed46869 160{
f1d34bca
MB		 161 const unsigned char *src = coding->source;
162 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 163 int multibytep = coding->src_multibyte;
ff0dacd7 164 int consumed_chars = 0;
df7492f9
KH		 165 int found = 0;
166 ...;
167
168 while (1)
169 {
170 /* Get one byte from the source. If the souce is exausted, jump
171 to no_more_source:. */
172 ONE_MORE_BYTE (c);
ff0dacd7
KH		 173
174 if (! __C_conforms_to_XXX___ (c))
175 break;
176 if (! __C_strongly_suggests_XXX__ (c))
177 found = CATEGORY_MASK_XXX;
df7492f9 178 }
ff0dacd7
KH		 179 /* The byte sequence is invalid for XXX. */
180 detect_info->rejected |= CATEGORY_MASK_XXX;
df7492f9 181 return 0;
ff0dacd7 182
df7492f9 183 no_more_source:
ff0dacd7
KH		 184 /* The source exausted successfully. */
185 detect_info->found |= found;
df7492f9 186 return 1;
4ed46869
KH		 187}
188#endif
189
190/*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
191
df7492f9
KH		 192 These functions decode a byte sequence specified as a source by
193 CODING. The resulting multibyte text goes to a place pointed to by
194 CODING->charbuf, the length of which should not exceed
195 CODING->charbuf_size;
d46c5b12 196
df7492f9
KH		 197 These functions set the information of original and decoded texts in
198 CODING->consumed, CODING->consumed_char, and CODING->charbuf_used.
199 They also set CODING->result to one of CODING_RESULT_XXX indicating
200 how the decoding is finished.
d46c5b12 201
df7492f9 202 Below is the template of these functions. */
d46c5b12 203
4ed46869 204#if 0
b73bfc1c 205static void
df7492f9 206decode_coding_XXXX (coding)
4ed46869 207 struct coding_system *coding;
4ed46869 208{
f1d34bca
MB		 209 const unsigned char *src = coding->source + coding->consumed;
210 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 211 /* SRC_BASE remembers the start position in source in each loop.
212 The loop will be exited when there's not enough source code, or
213 when there's no room in CHARBUF for a decoded character. */
f1d34bca 214 const unsigned char *src_base;
df7492f9 215 /* A buffer to produce decoded characters. */
69a80ea3
KH		 216 int *charbuf = coding->charbuf + coding->charbuf_used;
217 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 218 int multibytep = coding->src_multibyte;
219
220 while (1)
221 {
222 src_base = src;
223 if (charbuf < charbuf_end)
224 /* No more room to produce a decoded character. */
225 break;
226 ONE_MORE_BYTE (c);
227 /* Decode it. */
228 }
229
230 no_more_source:
231 if (src_base < src_end
232 && coding->mode & CODING_MODE_LAST_BLOCK)
233 /* If the source ends by partial bytes to construct a character,
234 treat them as eight-bit raw data. */
235 while (src_base < src_end && charbuf < charbuf_end)
236 *charbuf++ = *src_base++;
237 /* Remember how many bytes and characters we consumed. If the
238 source is multibyte, the bytes and chars are not identical. */
239 coding->consumed = coding->consumed_char = src_base - coding->source;
240 /* Remember how many characters we produced. */
241 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 242}
243#endif
244
245/*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
246
df7492f9
KH		 247 These functions encode SRC_BYTES length text at SOURCE of Emacs'
248 internal multibyte format by CODING. The resulting byte sequence
b73bfc1c
KH		 249 goes to a place pointed to by DESTINATION, the length of which
250 should not exceed DST_BYTES.
d46c5b12 251
df7492f9
KH		 252 These functions set the information of original and encoded texts in
253 the members produced, produced_char, consumed, and consumed_char of
254 the structure *CODING. They also set the member result to one of
255 CODING_RESULT_XXX indicating how the encoding finished.
d46c5b12 256
df7492f9
KH		 257 DST_BYTES zero means that source area and destination area are
258 overlapped, which means that we can produce a encoded text until it
259 reaches at the head of not-yet-encoded source text.
d46c5b12 260
df7492f9 261 Below is a template of these functions. */
4ed46869 262#if 0
b73bfc1c 263static void
df7492f9 264encode_coding_XXX (coding)
4ed46869 265 struct coding_system *coding;
4ed46869 266{
df7492f9
KH		 267 int multibytep = coding->dst_multibyte;
268 int *charbuf = coding->charbuf;
269 int *charbuf_end = charbuf->charbuf + coding->charbuf_used;
270 unsigned char *dst = coding->destination + coding->produced;
271 unsigned char *dst_end = coding->destination + coding->dst_bytes;
272 unsigned char *adjusted_dst_end = dst_end - _MAX_BYTES_PRODUCED_IN_LOOP_;
273 int produced_chars = 0;
274
275 for (; charbuf < charbuf_end && dst < adjusted_dst_end; charbuf++)
276 {
277 int c = *charbuf;
278 /* Encode C into DST, and increment DST. */
279 }
280 label_no_more_destination:
281 /* How many chars and bytes we produced. */
282 coding->produced_char += produced_chars;
283 coding->produced = dst - coding->destination;
4ed46869
KH		 284}
285#endif
286
4ed46869
KH		 287\f
288/*** 1. Preamble ***/
289
68c45bf0 290#include <config.h>
4ed46869
KH		 291#include <stdio.h>
292
4ed46869
KH		 293#include "lisp.h"
294#include "buffer.h"
df7492f9 295#include "character.h"
4ed46869
KH		 296#include "charset.h"
297#include "ccl.h"
df7492f9 298#include "composite.h"
4ed46869
KH		 299#include "coding.h"
300#include "window.h"
b8299c66
KL		 301#include "frame.h"
302#include "termhooks.h"
4ed46869 303
df7492f9 304Lisp_Object Vcoding_system_hash_table;
4ed46869 305
df7492f9 306Lisp_Object Qcoding_system, Qcoding_aliases, Qeol_type;
1965cb73
DL		 307Lisp_Object Qunix, Qdos;
308extern Lisp_Object Qmac; /* frame.c */
4ed46869
KH		 309Lisp_Object Qbuffer_file_coding_system;
310Lisp_Object Qpost_read_conversion, Qpre_write_conversion;
df7492f9 311Lisp_Object Qdefault_char;
27901516 312Lisp_Object Qno_conversion, Qundecided;
df7492f9 313Lisp_Object Qcharset, Qiso_2022, Qutf_8, Qutf_16, Qshift_jis, Qbig5;
b49a1807 314Lisp_Object Qbig, Qlittle;
bb0115a2 315Lisp_Object Qcoding_system_history;
1397dc18 316Lisp_Object Qvalid_codes;
2133e2d1 317Lisp_Object QCcategory, QCmnemonic, QCdefault_char;
a6f87d34
KH		 318Lisp_Object QCdecode_translation_table, QCencode_translation_table;
319Lisp_Object QCpost_read_conversion, QCpre_write_conversion;
35befdaa 320Lisp_Object QCascii_compatible_p;
4ed46869
KH		 321
322extern Lisp_Object Qinsert_file_contents, Qwrite_region;
387f6ba5 323Lisp_Object Qcall_process, Qcall_process_region;
4ed46869
KH		 324Lisp_Object Qstart_process, Qopen_network_stream;
325Lisp_Object Qtarget_idx;
326
065e3595
KH		 327Lisp_Object Qinsufficient_source, Qinconsistent_eol, Qinvalid_source;
328Lisp_Object Qinterrupted, Qinsufficient_memory;
329
c7183fb8
GM		 330extern Lisp_Object Qcompletion_ignore_case;
331
44e8490d
KH		 332/* If a symbol has this property, evaluate the value to define the
333 symbol as a coding system. */
334static Lisp_Object Qcoding_system_define_form;
335
5d5bf4d8
KH		 336int coding_system_require_warning;
337
d46c5b12
KH		 338Lisp_Object Vselect_safe_coding_system_function;
339
7722baf9
EZ		 340/* Mnemonic string for each format of end-of-line. */
341Lisp_Object eol_mnemonic_unix, eol_mnemonic_dos, eol_mnemonic_mac;
342/* Mnemonic string to indicate format of end-of-line is not yet
4ed46869 343 decided. */
7722baf9 344Lisp_Object eol_mnemonic_undecided;
4ed46869 345
fcbcfb64
KH		 346/* Format of end-of-line decided by system. This is Qunix on
347 Unix and Mac, Qdos on DOS/Windows.
348 This has an effect only for external encoding (i.e. for output to
349 file and process), not for in-buffer or Lisp string encoding. */
350static Lisp_Object system_eol_type;
351
4ed46869
KH		 352#ifdef emacs
353
4608c386
KH		 354Lisp_Object Vcoding_system_list, Vcoding_system_alist;
355
356Lisp_Object Qcoding_system_p, Qcoding_system_error;
4ed46869 357
d46c5b12
KH		 358/* Coding system emacs-mule and raw-text are for converting only
359 end-of-line format. */
360Lisp_Object Qemacs_mule, Qraw_text;
8f924df7 361Lisp_Object Qutf_8_emacs;
ecf488bc 362
4ed46869
KH		 363/* Coding-systems are handed between Emacs Lisp programs and C internal
364 routines by the following three variables. */
365/* Coding-system for reading files and receiving data from process. */
366Lisp_Object Vcoding_system_for_read;
367/* Coding-system for writing files and sending data to process. */
368Lisp_Object Vcoding_system_for_write;
369/* Coding-system actually used in the latest I/O. */
370Lisp_Object Vlast_coding_system_used;
065e3595
KH		 371/* Set to non-nil when an error is detected while code conversion. */
372Lisp_Object Vlast_code_conversion_error;
c4825358 373/* A vector of length 256 which contains information about special
94487c4e 374 Latin codes (especially for dealing with Microsoft codes). */
3f003981 375Lisp_Object Vlatin_extra_code_table;
c4825358 376
9ce27fde
KH		 377/* Flag to inhibit code conversion of end-of-line format. */
378int inhibit_eol_conversion;
379
74383408
KH		 380/* Flag to inhibit ISO2022 escape sequence detection. */
381int inhibit_iso_escape_detection;
382
97b1b294
EZ		 383/* Flag to inhibit detection of binary files through null bytes. */
384int inhibit_null_byte_detection;
385
ed29121d
EZ		 386/* Flag to make buffer-file-coding-system inherit from process-coding. */
387int inherit_process_coding_system;
388
c4825358
KH		 389/* Coding system to be used to encode text for terminal display when
390 terminal coding system is nil. */
391struct coding_system safe_terminal_coding;
392
02ba4723
KH		 393Lisp_Object Vfile_coding_system_alist;
394Lisp_Object Vprocess_coding_system_alist;
395Lisp_Object Vnetwork_coding_system_alist;
4ed46869 396
68c45bf0
PE		 397Lisp_Object Vlocale_coding_system;
398
4ed46869
KH		 399#endif /* emacs */
400
f967223b
KH		 401/* Flag to tell if we look up translation table on character code
402 conversion. */
84fbb8a0 403Lisp_Object Venable_character_translation;
f967223b
KH		 404/* Standard translation table to look up on decoding (reading). */
405Lisp_Object Vstandard_translation_table_for_decode;
406/* Standard translation table to look up on encoding (writing). */
407Lisp_Object Vstandard_translation_table_for_encode;
84fbb8a0 408
f967223b
KH		 409Lisp_Object Qtranslation_table;
410Lisp_Object Qtranslation_table_id;
411Lisp_Object Qtranslation_table_for_decode;
412Lisp_Object Qtranslation_table_for_encode;
4ed46869
KH		 413
414/* Alist of charsets vs revision number. */
df7492f9 415static Lisp_Object Vcharset_revision_table;
4ed46869 416
02ba4723
KH		 417/* Default coding systems used for process I/O. */
418Lisp_Object Vdefault_process_coding_system;
419
002fdb44
DL		 420/* Char table for translating Quail and self-inserting input. */
421Lisp_Object Vtranslation_table_for_input;
422
df7492f9
KH		 423/* Two special coding systems. */
424Lisp_Object Vsjis_coding_system;
425Lisp_Object Vbig5_coding_system;
426
df7492f9
KH		 427/* ISO2022 section */
428
429#define CODING_ISO_INITIAL(coding, reg) \
430 (XINT (AREF (AREF (CODING_ID_ATTRS ((coding)->id), \
431 coding_attr_iso_initial), \
432 reg)))
433
434
435#define CODING_ISO_REQUEST(coding, charset_id) \
436 ((charset_id <= (coding)->max_charset_id \
437 ? (coding)->safe_charsets[charset_id] \
438 : -1))
439
440
441#define CODING_ISO_FLAGS(coding) \
442 ((coding)->spec.iso_2022.flags)
443#define CODING_ISO_DESIGNATION(coding, reg) \
444 ((coding)->spec.iso_2022.current_designation[reg])
445#define CODING_ISO_INVOCATION(coding, plane) \
446 ((coding)->spec.iso_2022.current_invocation[plane])
447#define CODING_ISO_SINGLE_SHIFTING(coding) \
448 ((coding)->spec.iso_2022.single_shifting)
449#define CODING_ISO_BOL(coding) \
450 ((coding)->spec.iso_2022.bol)
451#define CODING_ISO_INVOKED_CHARSET(coding, plane) \
452 CODING_ISO_DESIGNATION ((coding), CODING_ISO_INVOCATION ((coding), (plane)))
453
454/* Control characters of ISO2022. */
455 /* code */ /* function */
456#define ISO_CODE_LF 0x0A /* line-feed */
457#define ISO_CODE_CR 0x0D /* carriage-return */
458#define ISO_CODE_SO 0x0E /* shift-out */
459#define ISO_CODE_SI 0x0F /* shift-in */
460#define ISO_CODE_SS2_7 0x19 /* single-shift-2 for 7-bit code */
461#define ISO_CODE_ESC 0x1B /* escape */
462#define ISO_CODE_SS2 0x8E /* single-shift-2 */
463#define ISO_CODE_SS3 0x8F /* single-shift-3 */
464#define ISO_CODE_CSI 0x9B /* control-sequence-introducer */
465
466/* All code (1-byte) of ISO2022 is classified into one of the
467 followings. */
468enum iso_code_class_type
469 {
470 ISO_control_0, /* Control codes in the range
471 0x00..0x1F and 0x7F, except for the
472 following 5 codes. */
df7492f9
KH		 473 ISO_shift_out, /* ISO_CODE_SO (0x0E) */
474 ISO_shift_in, /* ISO_CODE_SI (0x0F) */
475 ISO_single_shift_2_7, /* ISO_CODE_SS2_7 (0x19) */
476 ISO_escape, /* ISO_CODE_SO (0x1B) */
477 ISO_control_1, /* Control codes in the range
478 0x80..0x9F, except for the
479 following 3 codes. */
480 ISO_single_shift_2, /* ISO_CODE_SS2 (0x8E) */
481 ISO_single_shift_3, /* ISO_CODE_SS3 (0x8F) */
482 ISO_control_sequence_introducer, /* ISO_CODE_CSI (0x9B) */
483 ISO_0x20_or_0x7F, /* Codes of the values 0x20 or 0x7F. */
484 ISO_graphic_plane_0, /* Graphic codes in the range 0x21..0x7E. */
485 ISO_0xA0_or_0xFF, /* Codes of the values 0xA0 or 0xFF. */
486 ISO_graphic_plane_1 /* Graphic codes in the range 0xA1..0xFE. */
487 };
05e6f5dc 488
df7492f9
KH		 489/** The macros CODING_ISO_FLAG_XXX defines a flag bit of the
490 `iso-flags' attribute of an iso2022 coding system. */
05e6f5dc 491
df7492f9
KH		 492/* If set, produce long-form designation sequence (e.g. ESC $ ( A)
493 instead of the correct short-form sequence (e.g. ESC $ A). */
494#define CODING_ISO_FLAG_LONG_FORM 0x0001
93dec019 495
df7492f9
KH		 496/* If set, reset graphic planes and registers at end-of-line to the
497 initial state. */
498#define CODING_ISO_FLAG_RESET_AT_EOL 0x0002
05e6f5dc 499
df7492f9
KH		 500/* If set, reset graphic planes and registers before any control
501 characters to the initial state. */
502#define CODING_ISO_FLAG_RESET_AT_CNTL 0x0004
05e6f5dc 503
df7492f9
KH		 504/* If set, encode by 7-bit environment. */
505#define CODING_ISO_FLAG_SEVEN_BITS 0x0008
4ed46869 506
df7492f9
KH		 507/* If set, use locking-shift function. */
508#define CODING_ISO_FLAG_LOCKING_SHIFT 0x0010
b73bfc1c 509
df7492f9
KH		 510/* If set, use single-shift function. Overwrite
511 CODING_ISO_FLAG_LOCKING_SHIFT. */
512#define CODING_ISO_FLAG_SINGLE_SHIFT 0x0020
b73bfc1c 513
df7492f9
KH		 514/* If set, use designation escape sequence. */
515#define CODING_ISO_FLAG_DESIGNATION 0x0040
b73bfc1c 516
df7492f9
KH		 517/* If set, produce revision number sequence. */
518#define CODING_ISO_FLAG_REVISION 0x0080
b73bfc1c 519
df7492f9
KH		 520/* If set, produce ISO6429's direction specifying sequence. */
521#define CODING_ISO_FLAG_DIRECTION 0x0100
f4dee582 522
df7492f9
KH		 523/* If set, assume designation states are reset at beginning of line on
524 output. */
525#define CODING_ISO_FLAG_INIT_AT_BOL 0x0200
4ed46869 526
df7492f9
KH		 527/* If set, designation sequence should be placed at beginning of line
528 on output. */
529#define CODING_ISO_FLAG_DESIGNATE_AT_BOL 0x0400
aa72b389 530
df7492f9
KH		 531/* If set, do not encode unsafe charactes on output. */
532#define CODING_ISO_FLAG_SAFE 0x0800
aa72b389 533
df7492f9
KH		 534/* If set, extra latin codes (128..159) are accepted as a valid code
535 on input. */
536#define CODING_ISO_FLAG_LATIN_EXTRA 0x1000
aa72b389 537
df7492f9 538#define CODING_ISO_FLAG_COMPOSITION 0x2000
aa72b389 539
df7492f9 540#define CODING_ISO_FLAG_EUC_TW_SHIFT 0x4000
aa72b389 541
bf16eb23 542#define CODING_ISO_FLAG_USE_ROMAN 0x8000
aa72b389 543
bf16eb23 544#define CODING_ISO_FLAG_USE_OLDJIS 0x10000
aa72b389 545
bf16eb23 546#define CODING_ISO_FLAG_FULL_SUPPORT 0x100000
aa72b389 547
df7492f9
KH		 548/* A character to be produced on output if encoding of the original
549 character is prohibited by CODING_ISO_FLAG_SAFE. */
550#define CODING_INHIBIT_CHARACTER_SUBSTITUTION '?'
aa72b389 551
a470d443
KH		 552/* UTF-8 section */
553#define CODING_UTF_8_BOM(coding) \
554 ((coding)->spec.utf_8_bom)
4ed46869 555
df7492f9
KH		 556/* UTF-16 section */
557#define CODING_UTF_16_BOM(coding) \
558 ((coding)->spec.utf_16.bom)
4ed46869 559
df7492f9
KH		 560#define CODING_UTF_16_ENDIAN(coding) \
561 ((coding)->spec.utf_16.endian)
4ed46869 562
df7492f9
KH		 563#define CODING_UTF_16_SURROGATE(coding) \
564 ((coding)->spec.utf_16.surrogate)
4ed46869 565
4ed46869 566
df7492f9
KH		 567/* CCL section */
568#define CODING_CCL_DECODER(coding) \
569 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_decoder)
570#define CODING_CCL_ENCODER(coding) \
571 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_encoder)
572#define CODING_CCL_VALIDS(coding) \
8f924df7 573 (SDATA (AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_valids)))
4ed46869 574
5a936b46 575/* Index for each coding category in `coding_categories' */
4ed46869 576
df7492f9
KH		 577enum coding_category
578 {
579 coding_category_iso_7,
580 coding_category_iso_7_tight,
581 coding_category_iso_8_1,
582 coding_category_iso_8_2,
583 coding_category_iso_7_else,
584 coding_category_iso_8_else,
a470d443
KH		 585 coding_category_utf_8_auto,
586 coding_category_utf_8_nosig,
587 coding_category_utf_8_sig,
df7492f9
KH		 588 coding_category_utf_16_auto,
589 coding_category_utf_16_be,
590 coding_category_utf_16_le,
591 coding_category_utf_16_be_nosig,
592 coding_category_utf_16_le_nosig,
593 coding_category_charset,
594 coding_category_sjis,
595 coding_category_big5,
596 coding_category_ccl,
597 coding_category_emacs_mule,
598 /* All above are targets of code detection. */
599 coding_category_raw_text,
600 coding_category_undecided,
601 coding_category_max
602 };
603
604/* Definitions of flag bits used in detect_coding_XXXX. */
605#define CATEGORY_MASK_ISO_7 (1 << coding_category_iso_7)
606#define CATEGORY_MASK_ISO_7_TIGHT (1 << coding_category_iso_7_tight)
607#define CATEGORY_MASK_ISO_8_1 (1 << coding_category_iso_8_1)
608#define CATEGORY_MASK_ISO_8_2 (1 << coding_category_iso_8_2)
609#define CATEGORY_MASK_ISO_7_ELSE (1 << coding_category_iso_7_else)
610#define CATEGORY_MASK_ISO_8_ELSE (1 << coding_category_iso_8_else)
a470d443
KH		 611#define CATEGORY_MASK_UTF_8_AUTO (1 << coding_category_utf_8_auto)
612#define CATEGORY_MASK_UTF_8_NOSIG (1 << coding_category_utf_8_nosig)
613#define CATEGORY_MASK_UTF_8_SIG (1 << coding_category_utf_8_sig)
b49a1807 614#define CATEGORY_MASK_UTF_16_AUTO (1 << coding_category_utf_16_auto)
df7492f9
KH		 615#define CATEGORY_MASK_UTF_16_BE (1 << coding_category_utf_16_be)
616#define CATEGORY_MASK_UTF_16_LE (1 << coding_category_utf_16_le)
617#define CATEGORY_MASK_UTF_16_BE_NOSIG (1 << coding_category_utf_16_be_nosig)
618#define CATEGORY_MASK_UTF_16_LE_NOSIG (1 << coding_category_utf_16_le_nosig)
619#define CATEGORY_MASK_CHARSET (1 << coding_category_charset)
620#define CATEGORY_MASK_SJIS (1 << coding_category_sjis)
621#define CATEGORY_MASK_BIG5 (1 << coding_category_big5)
622#define CATEGORY_MASK_CCL (1 << coding_category_ccl)
623#define CATEGORY_MASK_EMACS_MULE (1 << coding_category_emacs_mule)
ff0dacd7 624#define CATEGORY_MASK_RAW_TEXT (1 << coding_category_raw_text)
df7492f9
KH		 625
626/* This value is returned if detect_coding_mask () find nothing other
627 than ASCII characters. */
628#define CATEGORY_MASK_ANY \
629 (CATEGORY_MASK_ISO_7 \
630 | CATEGORY_MASK_ISO_7_TIGHT \
631 | CATEGORY_MASK_ISO_8_1 \
632 | CATEGORY_MASK_ISO_8_2 \
633 | CATEGORY_MASK_ISO_7_ELSE \
634 | CATEGORY_MASK_ISO_8_ELSE \
a470d443
KH		 635 | CATEGORY_MASK_UTF_8_AUTO \
636 | CATEGORY_MASK_UTF_8_NOSIG \
637 | CATEGORY_MASK_UTF_8_SIG \
2f3cbb32 638 | CATEGORY_MASK_UTF_16_AUTO \
df7492f9
KH		 639 | CATEGORY_MASK_UTF_16_BE \
640 | CATEGORY_MASK_UTF_16_LE \
641 | CATEGORY_MASK_UTF_16_BE_NOSIG \
642 | CATEGORY_MASK_UTF_16_LE_NOSIG \
643 | CATEGORY_MASK_CHARSET \
644 | CATEGORY_MASK_SJIS \
645 | CATEGORY_MASK_BIG5 \
646 | CATEGORY_MASK_CCL \
647 | CATEGORY_MASK_EMACS_MULE)
648
649
650#define CATEGORY_MASK_ISO_7BIT \
651 (CATEGORY_MASK_ISO_7 | CATEGORY_MASK_ISO_7_TIGHT)
652
653#define CATEGORY_MASK_ISO_8BIT \
654 (CATEGORY_MASK_ISO_8_1 | CATEGORY_MASK_ISO_8_2)
655
656#define CATEGORY_MASK_ISO_ELSE \
657 (CATEGORY_MASK_ISO_7_ELSE | CATEGORY_MASK_ISO_8_ELSE)
658
659#define CATEGORY_MASK_ISO_ESCAPE \
660 (CATEGORY_MASK_ISO_7 \
661 | CATEGORY_MASK_ISO_7_TIGHT \
662 | CATEGORY_MASK_ISO_7_ELSE \
663 | CATEGORY_MASK_ISO_8_ELSE)
664
665#define CATEGORY_MASK_ISO \
666 ( CATEGORY_MASK_ISO_7BIT \
667 | CATEGORY_MASK_ISO_8BIT \
668 | CATEGORY_MASK_ISO_ELSE)
669
670#define CATEGORY_MASK_UTF_16 \
2f3cbb32
KH		 671 (CATEGORY_MASK_UTF_16_AUTO \
672 | CATEGORY_MASK_UTF_16_BE \
df7492f9
KH		 673 | CATEGORY_MASK_UTF_16_LE \
674 | CATEGORY_MASK_UTF_16_BE_NOSIG \
675 | CATEGORY_MASK_UTF_16_LE_NOSIG)
676
a470d443
KH		 677#define CATEGORY_MASK_UTF_8 \
678 (CATEGORY_MASK_UTF_8_AUTO \
679 | CATEGORY_MASK_UTF_8_NOSIG \
680 | CATEGORY_MASK_UTF_8_SIG)
df7492f9
KH		 681
682/* List of symbols `coding-category-xxx' ordered by priority. This
683 variable is exposed to Emacs Lisp. */
684static Lisp_Object Vcoding_category_list;
685
686/* Table of coding categories (Lisp symbols). This variable is for
687 internal use oly. */
688static Lisp_Object Vcoding_category_table;
689
690/* Table of coding-categories ordered by priority. */
691static enum coding_category coding_priorities[coding_category_max];
692
693/* Nth element is a coding context for the coding system bound to the
694 Nth coding category. */
695static struct coding_system coding_categories[coding_category_max];
696
df7492f9
KH		 697/*** Commonly used macros and functions ***/
698
699#ifndef min
700#define min(a, b) ((a) < (b) ? (a) : (b))
701#endif
702#ifndef max
703#define max(a, b) ((a) > (b) ? (a) : (b))
704#endif
4ed46869 705
24a73b0a
KH		 706#define CODING_GET_INFO(coding, attrs, charset_list) \
707 do { \
708 (attrs) = CODING_ID_ATTRS ((coding)->id); \
709 (charset_list) = CODING_ATTR_CHARSET_LIST (attrs); \
df7492f9 710 } while (0)
4ed46869 711
4ed46869 712
df7492f9
KH		 713/* Safely get one byte from the source text pointed by SRC which ends
714 at SRC_END, and set C to that byte. If there are not enough bytes
065e3595
KH		 715 in the source, it jumps to `no_more_source'. If multibytep is
716 nonzero, and a multibyte character is found at SRC, set C to the
717 negative value of the character code. The caller should declare
718 and set these variables appropriately in advance:
719 src, src_end, multibytep */
aa72b389 720
065e3595
KH		 721#define ONE_MORE_BYTE(c) \
722 do { \
723 if (src == src_end) \
724 { \
725 if (src_base < src) \
726 record_conversion_result \
727 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
728 goto no_more_source; \
729 } \
730 c = *src++; \
731 if (multibytep && (c & 0x80)) \
732 { \
733 if ((c & 0xFE) == 0xC0) \
734 c = ((c & 1) << 6) | *src++; \
735 else \
736 { \
35befdaa
KH		 737 src--; \
738 c = - string_char (src, &src, NULL); \
065e3595
KH		 739 record_conversion_result \
740 (coding, CODING_RESULT_INVALID_SRC); \
741 } \
742 } \
743 consumed_chars++; \
aa72b389
KH		 744 } while (0)
745
f56a4450 746/* Safely get two bytes from the source text pointed by SRC which ends
220eeac9
KH		 747 at SRC_END, and set C1 and C2 to those bytes while skipping the
748 heading multibyte characters. If there are not enough bytes in the
749 source, it jumps to `no_more_source'. If multibytep is nonzero and
750 a multibyte character is found for C2, set C2 to the negative value
751 of the character code. The caller should declare and set these
752 variables appropriately in advance:
f56a4450
KH		 753 src, src_end, multibytep
754 It is intended that this macro is used in detect_coding_utf_16. */
755
220eeac9
KH		 756#define TWO_MORE_BYTES(c1, c2) \
757 do { \
758 do { \
759 if (src == src_end) \
760 goto no_more_source; \
761 c1 = *src++; \
762 if (multibytep && (c1 & 0x80)) \
763 { \
764 if ((c1 & 0xFE) == 0xC0) \
765 c1 = ((c1 & 1) << 6) | *src++; \
766 else \
767 { \
768 src += BYTES_BY_CHAR_HEAD (c1) - 1; \
769 c1 = -1; \
770 } \
771 } \
772 } while (c1 < 0); \
773 if (src == src_end) \
774 goto no_more_source; \
775 c2 = *src++; \
776 if (multibytep && (c2 & 0x80)) \
777 { \
778 if ((c2 & 0xFE) == 0xC0) \
779 c2 = ((c2 & 1) << 6) | *src++; \
780 else \
781 c2 = -1; \
782 } \
f56a4450
KH		 783 } while (0)
784
aa72b389 785
065e3595
KH		 786#define ONE_MORE_BYTE_NO_CHECK(c) \
787 do { \
788 c = *src++; \
789 if (multibytep && (c & 0x80)) \
790 { \
791 if ((c & 0xFE) == 0xC0) \
792 c = ((c & 1) << 6) | *src++; \
793 else \
794 { \
35befdaa
KH		 795 src--; \
796 c = - string_char (src, &src, NULL); \
065e3595
KH		 797 record_conversion_result \
798 (coding, CODING_RESULT_INVALID_SRC); \
799 } \
800 } \
801 consumed_chars++; \
aa72b389
KH		 802 } while (0)
803
aa72b389 804
df7492f9
KH		 805/* Store a byte C in the place pointed by DST and increment DST to the
806 next free point, and increment PRODUCED_CHARS. The caller should
807 assure that C is 0..127, and declare and set the variable `dst'
808 appropriately in advance.
809*/
aa72b389
KH		 810
811
df7492f9
KH		 812#define EMIT_ONE_ASCII_BYTE(c) \
813 do { \
814 produced_chars++; \
815 *dst++ = (c); \
b6871cc7 816 } while (0)
aa72b389
KH		 817
818
df7492f9 819/* Like EMIT_ONE_ASCII_BYTE byt store two bytes; C1 and C2. */
aa72b389 820
df7492f9
KH		 821#define EMIT_TWO_ASCII_BYTES(c1, c2) \
822 do { \
823 produced_chars += 2; \
824 *dst++ = (c1), *dst++ = (c2); \
825 } while (0)
aa72b389
KH		 826
827
df7492f9
KH		 828/* Store a byte C in the place pointed by DST and increment DST to the
829 next free point, and increment PRODUCED_CHARS. If MULTIBYTEP is
830 nonzero, store in an appropriate multibyte from. The caller should
831 declare and set the variables `dst' and `multibytep' appropriately
832 in advance. */
833
834#define EMIT_ONE_BYTE(c) \
835 do { \
836 produced_chars++; \
837 if (multibytep) \
838 { \
839 int ch = (c); \
840 if (ch >= 0x80) \
841 ch = BYTE8_TO_CHAR (ch); \
842 CHAR_STRING_ADVANCE (ch, dst); \
843 } \
844 else \
845 *dst++ = (c); \
aa72b389 846 } while (0)
aa72b389 847
aa72b389 848
df7492f9 849/* Like EMIT_ONE_BYTE, but emit two bytes; C1 and C2. */
aa72b389 850
e19c3639
KH		 851#define EMIT_TWO_BYTES(c1, c2) \
852 do { \
853 produced_chars += 2; \
854 if (multibytep) \
855 { \
856 int ch; \
857 \
858 ch = (c1); \
859 if (ch >= 0x80) \
860 ch = BYTE8_TO_CHAR (ch); \
861 CHAR_STRING_ADVANCE (ch, dst); \
862 ch = (c2); \
863 if (ch >= 0x80) \
864 ch = BYTE8_TO_CHAR (ch); \
865 CHAR_STRING_ADVANCE (ch, dst); \
866 } \
867 else \
868 { \
869 *dst++ = (c1); \
870 *dst++ = (c2); \
871 } \
aa72b389
KH		 872 } while (0)
873
874
df7492f9
KH		 875#define EMIT_THREE_BYTES(c1, c2, c3) \
876 do { \
877 EMIT_ONE_BYTE (c1); \
878 EMIT_TWO_BYTES (c2, c3); \
879 } while (0)
aa72b389 880
aa72b389 881
df7492f9
KH		 882#define EMIT_FOUR_BYTES(c1, c2, c3, c4) \
883 do { \
884 EMIT_TWO_BYTES (c1, c2); \
885 EMIT_TWO_BYTES (c3, c4); \
886 } while (0)
aa72b389 887
aa72b389 888
f6cbaf43
KH		 889/* Prototypes for static functions. */
890static void record_conversion_result P_ ((struct coding_system *coding,
891 enum coding_result_code result));
892static int detect_coding_utf_8 P_ ((struct coding_system *,
893 struct coding_detection_info *info));
894static void decode_coding_utf_8 P_ ((struct coding_system *));
895static int encode_coding_utf_8 P_ ((struct coding_system *));
896
897static int detect_coding_utf_16 P_ ((struct coding_system *,
898 struct coding_detection_info *info));
899static void decode_coding_utf_16 P_ ((struct coding_system *));
900static int encode_coding_utf_16 P_ ((struct coding_system *));
901
902static int detect_coding_iso_2022 P_ ((struct coding_system *,
903 struct coding_detection_info *info));
904static void decode_coding_iso_2022 P_ ((struct coding_system *));
905static int encode_coding_iso_2022 P_ ((struct coding_system *));
906
907static int detect_coding_emacs_mule P_ ((struct coding_system *,
908 struct coding_detection_info *info));
909static void decode_coding_emacs_mule P_ ((struct coding_system *));
910static int encode_coding_emacs_mule P_ ((struct coding_system *));
911
912static int detect_coding_sjis P_ ((struct coding_system *,
913 struct coding_detection_info *info));
914static void decode_coding_sjis P_ ((struct coding_system *));
915static int encode_coding_sjis P_ ((struct coding_system *));
916
917static int detect_coding_big5 P_ ((struct coding_system *,
918 struct coding_detection_info *info));
919static void decode_coding_big5 P_ ((struct coding_system *));
920static int encode_coding_big5 P_ ((struct coding_system *));
921
922static int detect_coding_ccl P_ ((struct coding_system *,
923 struct coding_detection_info *info));
924static void decode_coding_ccl P_ ((struct coding_system *));
925static int encode_coding_ccl P_ ((struct coding_system *));
926
927static void decode_coding_raw_text P_ ((struct coding_system *));
928static int encode_coding_raw_text P_ ((struct coding_system *));
929
930static void coding_set_source P_ ((struct coding_system *));
931static void coding_set_destination P_ ((struct coding_system *));
932static void coding_alloc_by_realloc P_ ((struct coding_system *, EMACS_INT));
933static void coding_alloc_by_making_gap P_ ((struct coding_system *,
287c57d7 934 EMACS_INT, EMACS_INT));
f6cbaf43
KH		 935static unsigned char *alloc_destination P_ ((struct coding_system *,
936 EMACS_INT, unsigned char *));
937static void setup_iso_safe_charsets P_ ((Lisp_Object));
938static unsigned char *encode_designation_at_bol P_ ((struct coding_system *,
939 int *, int *,
940 unsigned char *));
941static int detect_eol P_ ((const unsigned char *,
942 EMACS_INT, enum coding_category));
943static Lisp_Object adjust_coding_eol_type P_ ((struct coding_system *, int));
944static void decode_eol P_ ((struct coding_system *));
945static Lisp_Object get_translation_table P_ ((Lisp_Object, int, int *));
946static Lisp_Object get_translation P_ ((Lisp_Object, int *, int *,
947 int, int *, int *));
948static int produce_chars P_ ((struct coding_system *, Lisp_Object, int));
949static INLINE void produce_composition P_ ((struct coding_system *, int *,
950 EMACS_INT));
951static INLINE void produce_charset P_ ((struct coding_system *, int *,
952 EMACS_INT));
953static void produce_annotation P_ ((struct coding_system *, EMACS_INT));
954static int decode_coding P_ ((struct coding_system *));
955static INLINE int *handle_composition_annotation P_ ((EMACS_INT, EMACS_INT,
3ed051d4 956 struct coding_system *,
f6cbaf43
KH		 957 int *, EMACS_INT *));
958static INLINE int *handle_charset_annotation P_ ((EMACS_INT, EMACS_INT,
959 struct coding_system *,
960 int *, EMACS_INT *));
961static void consume_chars P_ ((struct coding_system *, Lisp_Object, int));
962static int encode_coding P_ ((struct coding_system *));
963static Lisp_Object make_conversion_work_buffer P_ ((int));
964static Lisp_Object code_conversion_restore P_ ((Lisp_Object));
965static INLINE int char_encodable_p P_ ((int, Lisp_Object));
966static Lisp_Object make_subsidiaries P_ ((Lisp_Object));
967
065e3595
KH		 968static void
969record_conversion_result (struct coding_system *coding,
970 enum coding_result_code result)
971{
972 coding->result = result;
973 switch (result)
974 {
975 case CODING_RESULT_INSUFFICIENT_SRC:
976 Vlast_code_conversion_error = Qinsufficient_source;
977 break;
978 case CODING_RESULT_INCONSISTENT_EOL:
979 Vlast_code_conversion_error = Qinconsistent_eol;
980 break;
981 case CODING_RESULT_INVALID_SRC:
982 Vlast_code_conversion_error = Qinvalid_source;
983 break;
984 case CODING_RESULT_INTERRUPT:
985 Vlast_code_conversion_error = Qinterrupted;
986 break;
987 case CODING_RESULT_INSUFFICIENT_MEM:
988 Vlast_code_conversion_error = Qinsufficient_memory;
989 break;
35befdaa
KH		 990 default:
991 Vlast_code_conversion_error = intern ("Unknown error");
065e3595
KH		 992 }
993}
994
df7492f9
KH		 995#define CODING_DECODE_CHAR(coding, src, src_base, src_end, charset, code, c) \
996 do { \
997 charset_map_loaded = 0; \
998 c = DECODE_CHAR (charset, code); \
999 if (charset_map_loaded) \
1000 { \
8f924df7 1001 const unsigned char *orig = coding->source; \
df7492f9
KH		 1002 EMACS_INT offset; \
1003 \
1004 coding_set_source (coding); \
1005 offset = coding->source - orig; \
1006 src += offset; \
1007 src_base += offset; \
1008 src_end += offset; \
1009 } \
aa72b389
KH		 1010 } while (0)
1011
1012
119852e7
KH		 1013/* If there are at least BYTES length of room at dst, allocate memory
1014 for coding->destination and update dst and dst_end. We don't have
1015 to take care of coding->source which will be relocated. It is
1016 handled by calling coding_set_source in encode_coding. */
1017
df7492f9
KH		 1018#define ASSURE_DESTINATION(bytes) \
1019 do { \
1020 if (dst + (bytes) >= dst_end) \
1021 { \
1022 int more_bytes = charbuf_end - charbuf + (bytes); \
1023 \
1024 dst = alloc_destination (coding, more_bytes, dst); \
1025 dst_end = coding->destination + coding->dst_bytes; \
1026 } \
1027 } while (0)
aa72b389 1028
aa72b389 1029
db274c7a
KH		 1030/* Store multibyte form of the character C in P, and advance P to the
1031 end of the multibyte form. This is like CHAR_STRING_ADVANCE but it
1032 never calls MAYBE_UNIFY_CHAR. */
1033
1034#define CHAR_STRING_ADVANCE_NO_UNIFY(c, p) \
1035 do { \
1036 if ((c) <= MAX_1_BYTE_CHAR) \
1037 *(p)++ = (c); \
1038 else if ((c) <= MAX_2_BYTE_CHAR) \
1039 *(p)++ = (0xC0 | ((c) >> 6)), \
1040 *(p)++ = (0x80 | ((c) & 0x3F)); \
1041 else if ((c) <= MAX_3_BYTE_CHAR) \
1042 *(p)++ = (0xE0 | ((c) >> 12)), \
1043 *(p)++ = (0x80 | (((c) >> 6) & 0x3F)), \
1044 *(p)++ = (0x80 | ((c) & 0x3F)); \
1045 else if ((c) <= MAX_4_BYTE_CHAR) \
1046 *(p)++ = (0xF0 | (c >> 18)), \
1047 *(p)++ = (0x80 | ((c >> 12) & 0x3F)), \
1048 *(p)++ = (0x80 | ((c >> 6) & 0x3F)), \
1049 *(p)++ = (0x80 | (c & 0x3F)); \
1050 else if ((c) <= MAX_5_BYTE_CHAR) \
1051 *(p)++ = 0xF8, \
1052 *(p)++ = (0x80 | ((c >> 18) & 0x0F)), \
1053 *(p)++ = (0x80 | ((c >> 12) & 0x3F)), \
1054 *(p)++ = (0x80 | ((c >> 6) & 0x3F)), \
1055 *(p)++ = (0x80 | (c & 0x3F)); \
1056 else \
1057 (p) += BYTE8_STRING ((c) - 0x3FFF80, p); \
1058 } while (0)
1059
1060
1061/* Return the character code of character whose multibyte form is at
1062 P, and advance P to the end of the multibyte form. This is like
1063 STRING_CHAR_ADVANCE, but it never calls MAYBE_UNIFY_CHAR. */
1064
1065#define STRING_CHAR_ADVANCE_NO_UNIFY(p) \
1066 (!((p)[0] & 0x80) \
1067 ? *(p)++ \
1068 : ! ((p)[0] & 0x20) \
1069 ? ((p) += 2, \
1070 ((((p)[-2] & 0x1F) << 6) \
1071 | ((p)[-1] & 0x3F) \
1072 | ((unsigned char) ((p)[-2]) < 0xC2 ? 0x3FFF80 : 0))) \
1073 : ! ((p)[0] & 0x10) \
1074 ? ((p) += 3, \
1075 ((((p)[-3] & 0x0F) << 12) \
1076 | (((p)[-2] & 0x3F) << 6) \
1077 | ((p)[-1] & 0x3F))) \
1078 : ! ((p)[0] & 0x08) \
1079 ? ((p) += 4, \
1080 ((((p)[-4] & 0xF) << 18) \
1081 | (((p)[-3] & 0x3F) << 12) \
1082 | (((p)[-2] & 0x3F) << 6) \
1083 | ((p)[-1] & 0x3F))) \
1084 : ((p) += 5, \
1085 ((((p)[-4] & 0x3F) << 18) \
1086 | (((p)[-3] & 0x3F) << 12) \
1087 | (((p)[-2] & 0x3F) << 6) \
1088 | ((p)[-1] & 0x3F))))
1089
aa72b389 1090
df7492f9
KH		 1091static void
1092coding_set_source (coding)
aa72b389 1093 struct coding_system *coding;
aa72b389 1094{
df7492f9
KH		 1095 if (BUFFERP (coding->src_object))
1096 {
2cb26057 1097 struct buffer *buf = XBUFFER (coding->src_object);
aa72b389 1098
df7492f9 1099 if (coding->src_pos < 0)
2cb26057 1100 coding->source = BUF_GAP_END_ADDR (buf) + coding->src_pos_byte;
df7492f9 1101 else
2cb26057 1102 coding->source = BUF_BYTE_ADDRESS (buf, coding->src_pos_byte);
aa72b389 1103 }
df7492f9 1104 else if (STRINGP (coding->src_object))
aa72b389 1105 {
8f924df7 1106 coding->source = SDATA (coding->src_object) + coding->src_pos_byte;
aa72b389 1107 }
df7492f9
KH		 1108 else
1109 /* Otherwise, the source is C string and is never relocated
1110 automatically. Thus we don't have to update anything. */
1111 ;
1112}
aa72b389 1113
df7492f9
KH		 1114static void
1115coding_set_destination (coding)
1116 struct coding_system *coding;
1117{
1118 if (BUFFERP (coding->dst_object))
aa72b389 1119 {
df7492f9 1120 if (coding->src_pos < 0)
aa72b389 1121 {
13818c30 1122 coding->destination = BEG_ADDR + coding->dst_pos_byte - BEG_BYTE;
28f67a95
KH		 1123 coding->dst_bytes = (GAP_END_ADDR
1124 - (coding->src_bytes - coding->consumed)
1125 - coding->destination);
aa72b389 1126 }
df7492f9 1127 else
28f67a95
KH		 1128 {
1129 /* We are sure that coding->dst_pos_byte is before the gap
1130 of the buffer. */
1131 coding->destination = (BUF_BEG_ADDR (XBUFFER (coding->dst_object))
13818c30 1132 + coding->dst_pos_byte - BEG_BYTE);
28f67a95
KH		 1133 coding->dst_bytes = (BUF_GAP_END_ADDR (XBUFFER (coding->dst_object))
1134 - coding->destination);
1135 }
df7492f9
KH		 1136 }
1137 else
1138 /* Otherwise, the destination is C string and is never relocated
1139 automatically. Thus we don't have to update anything. */
1140 ;
1141}
1142
1143
1144static void
1145coding_alloc_by_realloc (coding, bytes)
1146 struct coding_system *coding;
1147 EMACS_INT bytes;
1148{
1149 coding->destination = (unsigned char *) xrealloc (coding->destination,
1150 coding->dst_bytes + bytes);
1151 coding->dst_bytes += bytes;
1152}
1153
1154static void
db274c7a 1155coding_alloc_by_making_gap (coding, gap_head_used, bytes)
df7492f9 1156 struct coding_system *coding;
db274c7a 1157 EMACS_INT gap_head_used, bytes;
df7492f9 1158{
db274c7a 1159 if (EQ (coding->src_object, coding->dst_object))
df7492f9 1160 {
db274c7a
KH		 1161 /* The gap may contain the produced data at the head and not-yet
1162 consumed data at the tail. To preserve those data, we at
1163 first make the gap size to zero, then increase the gap
1164 size. */
1165 EMACS_INT add = GAP_SIZE;
1166
1167 GPT += gap_head_used, GPT_BYTE += gap_head_used;
1168 GAP_SIZE = 0; ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
df7492f9
KH		 1169 make_gap (bytes);
1170 GAP_SIZE += add; ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
db274c7a 1171 GPT -= gap_head_used, GPT_BYTE -= gap_head_used;
df7492f9 1172 }
730fff51 1173 else
df7492f9 1174 {
2c78b7e1
KH		 1175 Lisp_Object this_buffer;
1176
1177 this_buffer = Fcurrent_buffer ();
df7492f9
KH		 1178 set_buffer_internal (XBUFFER (coding->dst_object));
1179 make_gap (bytes);
1180 set_buffer_internal (XBUFFER (this_buffer));
aa72b389 1181 }
df7492f9 1182}
8f924df7 1183
df7492f9
KH		 1184
1185static unsigned char *
1186alloc_destination (coding, nbytes, dst)
1187 struct coding_system *coding;
3e139625 1188 EMACS_INT nbytes;
df7492f9
KH		 1189 unsigned char *dst;
1190{
1191 EMACS_INT offset = dst - coding->destination;
1192
1193 if (BUFFERP (coding->dst_object))
db274c7a
KH		 1194 {
1195 struct buffer *buf = XBUFFER (coding->dst_object);
1196
1197 coding_alloc_by_making_gap (coding, dst - BUF_GPT_ADDR (buf), nbytes);
1198 }
aa72b389 1199 else
df7492f9 1200 coding_alloc_by_realloc (coding, nbytes);
065e3595 1201 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1202 coding_set_destination (coding);
1203 dst = coding->destination + offset;
1204 return dst;
1205}
aa72b389 1206
ff0dacd7
KH		 1207/** Macros for annotations. */
1208
1209/* Maximum length of annotation data (sum of annotations for
1210 composition and charset). */
69a80ea3 1211#define MAX_ANNOTATION_LENGTH (4 + (MAX_COMPOSITION_COMPONENTS * 2) - 1 + 4)
ff0dacd7
KH		 1212
1213/* An annotation data is stored in the array coding->charbuf in this
1214 format:
69a80ea3 1215 [ -LENGTH ANNOTATION_MASK NCHARS ... ]
ff0dacd7
KH		 1216 LENGTH is the number of elements in the annotation.
1217 ANNOTATION_MASK is one of CODING_ANNOTATE_XXX_MASK.
69a80ea3 1218 NCHARS is the number of characters in the text annotated.
ff0dacd7
KH		 1219
1220 The format of the following elements depend on ANNOTATION_MASK.
1221
1222 In the case of CODING_ANNOTATE_COMPOSITION_MASK, these elements
1223 follows:
1224 ... METHOD [ COMPOSITION-COMPONENTS ... ]
1225 METHOD is one of enum composition_method.
1226 Optionnal COMPOSITION-COMPONENTS are characters and composition
1227 rules.
1228
1229 In the case of CODING_ANNOTATE_CHARSET_MASK, one element CHARSET-ID
1230 follows. */
1231
69a80ea3 1232#define ADD_ANNOTATION_DATA(buf, len, mask, nchars) \
ff0dacd7
KH		 1233 do { \
1234 *(buf)++ = -(len); \
1235 *(buf)++ = (mask); \
69a80ea3 1236 *(buf)++ = (nchars); \
ff0dacd7
KH		 1237 coding->annotated = 1; \
1238 } while (0);
1239
69a80ea3
KH		 1240#define ADD_COMPOSITION_DATA(buf, nchars, method) \
1241 do { \
1242 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_COMPOSITION_MASK, nchars); \
1243 *buf++ = method; \
ff0dacd7
KH		 1244 } while (0)
1245
1246
69a80ea3
KH		 1247#define ADD_CHARSET_DATA(buf, nchars, id) \
1248 do { \
1249 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_CHARSET_MASK, nchars); \
1250 *buf++ = id; \
ff0dacd7
KH		 1251 } while (0)
1252
df7492f9
KH		 1253\f
1254/*** 2. Emacs' internal format (emacs-utf-8) ***/
1255
1256
1257
1258\f
1259/*** 3. UTF-8 ***/
1260
1261/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1262 Check if a text is encoded in UTF-8. If it is, return 1, else
1263 return 0. */
df7492f9
KH		 1264
1265#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
1266#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
1267#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
1268#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
1269#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
1270#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
1271
a470d443
KH		 1272#define UTF_BOM 0xFEFF
1273#define UTF_8_BOM_1 0xEF
1274#define UTF_8_BOM_2 0xBB
1275#define UTF_8_BOM_3 0xBF
1276
df7492f9 1277static int
ff0dacd7 1278detect_coding_utf_8 (coding, detect_info)
df7492f9 1279 struct coding_system *coding;
ff0dacd7 1280 struct coding_detection_info *detect_info;
df7492f9 1281{
065e3595 1282 const unsigned char *src = coding->source, *src_base;
8f924df7 1283 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 1284 int multibytep = coding->src_multibyte;
1285 int consumed_chars = 0;
a470d443 1286 int bom_found = 0;
df7492f9
KH		 1287 int found = 0;
1288
ff0dacd7 1289 detect_info->checked |= CATEGORY_MASK_UTF_8;
df7492f9
KH		 1290 /* A coding system of this category is always ASCII compatible. */
1291 src += coding->head_ascii;
1292
1293 while (1)
aa72b389 1294 {
df7492f9 1295 int c, c1, c2, c3, c4;
aa72b389 1296
065e3595 1297 src_base = src;
df7492f9 1298 ONE_MORE_BYTE (c);
065e3595 1299 if (c < 0 || UTF_8_1_OCTET_P (c))
df7492f9
KH		 1300 continue;
1301 ONE_MORE_BYTE (c1);
065e3595 1302 if (c1 < 0 || ! UTF_8_EXTRA_OCTET_P (c1))
df7492f9
KH		 1303 break;
1304 if (UTF_8_2_OCTET_LEADING_P (c))
aa72b389 1305 {
a470d443 1306 found = 1;
df7492f9 1307 continue;
aa72b389 1308 }
df7492f9 1309 ONE_MORE_BYTE (c2);
065e3595 1310 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1311 break;
1312 if (UTF_8_3_OCTET_LEADING_P (c))
aa72b389 1313 {
a470d443
KH		 1314 found = 1;
1315 if (src_base == coding->source
1316 && c == UTF_8_BOM_1 && c1 == UTF_8_BOM_2 && c2 == UTF_8_BOM_3)
1317 bom_found = 1;
df7492f9 1318 continue;
aa72b389 1319 }
df7492f9 1320 ONE_MORE_BYTE (c3);
065e3595 1321 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1322 break;
1323 if (UTF_8_4_OCTET_LEADING_P (c))
aa72b389 1324 {
a470d443 1325 found = 1;
df7492f9
KH		 1326 continue;
1327 }
1328 ONE_MORE_BYTE (c4);
065e3595 1329 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1330 break;
1331 if (UTF_8_5_OCTET_LEADING_P (c))
1332 {
a470d443 1333 found = 1;
df7492f9
KH		 1334 continue;
1335 }
1336 break;
aa72b389 1337 }
ff0dacd7 1338 detect_info->rejected |= CATEGORY_MASK_UTF_8;
df7492f9 1339 return 0;
aa72b389 1340
df7492f9 1341 no_more_source:
065e3595 1342 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
aa72b389 1343 {
ff0dacd7 1344 detect_info->rejected |= CATEGORY_MASK_UTF_8;
89528eb3 1345 return 0;
aa72b389 1346 }
a470d443
KH		 1347 if (bom_found)
1348 {
1349 /* The first character 0xFFFE doesn't necessarily mean a BOM. */
1350 detect_info->found |= CATEGORY_MASK_UTF_8_SIG | CATEGORY_MASK_UTF_8_NOSIG;
1351 }
1352 else
1353 {
1354 detect_info->rejected |= CATEGORY_MASK_UTF_8_SIG;
0e17387a
KH		 1355 if (found)
1356 detect_info->found |= CATEGORY_MASK_UTF_8_NOSIG;
a470d443 1357 }
ff0dacd7 1358 return 1;
aa72b389
KH		 1359}
1360
4ed46869 1361
b73bfc1c 1362static void
df7492f9 1363decode_coding_utf_8 (coding)
b73bfc1c 1364 struct coding_system *coding;
b73bfc1c 1365{
8f924df7
KH		 1366 const unsigned char *src = coding->source + coding->consumed;
1367 const unsigned char *src_end = coding->source + coding->src_bytes;
1368 const unsigned char *src_base;
69a80ea3
KH		 1369 int *charbuf = coding->charbuf + coding->charbuf_used;
1370 int *charbuf_end = coding->charbuf + coding->charbuf_size;
453b38f0 1371 int consumed_chars = 0, consumed_chars_base = 0;
df7492f9 1372 int multibytep = coding->src_multibyte;
a470d443 1373 enum utf_bom_type bom = CODING_UTF_8_BOM (coding);
24a73b0a 1374 Lisp_Object attr, charset_list;
119852e7
KH		 1375 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
1376 int byte_after_cr = -1;
4ed46869 1377
24a73b0a 1378 CODING_GET_INFO (coding, attr, charset_list);
df7492f9 1379
a470d443
KH		 1380 if (bom != utf_without_bom)
1381 {
1382 int c1, c2, c3;
1383
1384 src_base = src;
1385 ONE_MORE_BYTE (c1);
1386 if (! UTF_8_3_OCTET_LEADING_P (c1))
1387 src = src_base;
1388 else
1389 {
159bd5a2 1390 ONE_MORE_BYTE (c2);
a470d443
KH		 1391 if (! UTF_8_EXTRA_OCTET_P (c2))
1392 src = src_base;
1393 else
1394 {
159bd5a2 1395 ONE_MORE_BYTE (c3);
a470d443
KH		 1396 if (! UTF_8_EXTRA_OCTET_P (c3))
1397 src = src_base;
1398 else
1399 {
1400 if ((c1 != UTF_8_BOM_1)
1401 || (c2 != UTF_8_BOM_2) || (c3 != UTF_8_BOM_3))
1402 src = src_base;
1403 else
1404 CODING_UTF_8_BOM (coding) = utf_without_bom;
1405 }
1406 }
1407 }
1408 }
1409 CODING_UTF_8_BOM (coding) = utf_without_bom;
1410
1411
1412
df7492f9 1413 while (1)
b73bfc1c 1414 {
df7492f9 1415 int c, c1, c2, c3, c4, c5;
ec6d2bb8 1416
df7492f9
KH		 1417 src_base = src;
1418 consumed_chars_base = consumed_chars;
4af310db 1419
df7492f9 1420 if (charbuf >= charbuf_end)
b71f6f73
KH		 1421 {
1422 if (byte_after_cr >= 0)
1423 src_base--;
1424 break;
1425 }
df7492f9 1426
119852e7
KH		 1427 if (byte_after_cr >= 0)
1428 c1 = byte_after_cr, byte_after_cr = -1;
1429 else
1430 ONE_MORE_BYTE (c1);
065e3595
KH		 1431 if (c1 < 0)
1432 {
1433 c = - c1;
1434 }
1435 else if (UTF_8_1_OCTET_P(c1))
df7492f9 1436 {
119852e7
KH		 1437 if (eol_crlf && c1 == '\r')
1438 ONE_MORE_BYTE (byte_after_cr);
df7492f9 1439 c = c1;
4af310db 1440 }
df7492f9 1441 else
4af310db 1442 {
df7492f9 1443 ONE_MORE_BYTE (c2);
065e3595 1444 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1445 goto invalid_code;
1446 if (UTF_8_2_OCTET_LEADING_P (c1))
4af310db 1447 {
b0edb2c5
DL		 1448 c = ((c1 & 0x1F) << 6) | (c2 & 0x3F);
1449 /* Reject overlong sequences here and below. Encoders
1450 producing them are incorrect, they can be misleading,
1451 and they mess up read/write invariance. */
1452 if (c < 128)
1453 goto invalid_code;
4af310db 1454 }
df7492f9 1455 else
aa72b389 1456 {
df7492f9 1457 ONE_MORE_BYTE (c3);
065e3595 1458 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1459 goto invalid_code;
1460 if (UTF_8_3_OCTET_LEADING_P (c1))
b0edb2c5
DL		 1461 {
1462 c = (((c1 & 0xF) << 12)
1463 | ((c2 & 0x3F) << 6) | (c3 & 0x3F));
72fe1301
DL		 1464 if (c < 0x800
1465 || (c >= 0xd800 && c < 0xe000)) /* surrogates (invalid) */
b0edb2c5
DL		 1466 goto invalid_code;
1467 }
df7492f9
KH		 1468 else
1469 {
1470 ONE_MORE_BYTE (c4);
065e3595 1471 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1472 goto invalid_code;
1473 if (UTF_8_4_OCTET_LEADING_P (c1))
b0edb2c5 1474 {
df7492f9
KH		 1475 c = (((c1 & 0x7) << 18) | ((c2 & 0x3F) << 12)
1476 | ((c3 & 0x3F) << 6) | (c4 & 0x3F));
b0edb2c5
DL		 1477 if (c < 0x10000)
1478 goto invalid_code;
1479 }
df7492f9
KH		 1480 else
1481 {
1482 ONE_MORE_BYTE (c5);
065e3595 1483 if (c5 < 0 || ! UTF_8_EXTRA_OCTET_P (c5))
df7492f9
KH		 1484 goto invalid_code;
1485 if (UTF_8_5_OCTET_LEADING_P (c1))
1486 {
1487 c = (((c1 & 0x3) << 24) | ((c2 & 0x3F) << 18)
1488 | ((c3 & 0x3F) << 12) | ((c4 & 0x3F) << 6)
1489 | (c5 & 0x3F));
b0edb2c5 1490 if ((c > MAX_CHAR) || (c < 0x200000))
df7492f9
KH		 1491 goto invalid_code;
1492 }
1493 else
1494 goto invalid_code;
1495 }
1496 }
aa72b389 1497 }
b73bfc1c 1498 }
df7492f9
KH		 1499
1500 *charbuf++ = c;
1501 continue;
1502
1503 invalid_code:
1504 src = src_base;
1505 consumed_chars = consumed_chars_base;
1506 ONE_MORE_BYTE (c);
1507 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
1508 coding->errors++;
aa72b389
KH		 1509 }
1510
df7492f9
KH		 1511 no_more_source:
1512 coding->consumed_char += consumed_chars_base;
1513 coding->consumed = src_base - coding->source;
1514 coding->charbuf_used = charbuf - coding->charbuf;
1515}
1516
1517
1518static int
1519encode_coding_utf_8 (coding)
1520 struct coding_system *coding;
1521{
1522 int multibytep = coding->dst_multibyte;
1523 int *charbuf = coding->charbuf;
1524 int *charbuf_end = charbuf + coding->charbuf_used;
1525 unsigned char *dst = coding->destination + coding->produced;
1526 unsigned char *dst_end = coding->destination + coding->dst_bytes;
e19c3639 1527 int produced_chars = 0;
df7492f9
KH		 1528 int c;
1529
a470d443
KH		 1530 if (CODING_UTF_8_BOM (coding) == utf_with_bom)
1531 {
1532 ASSURE_DESTINATION (3);
1533 EMIT_THREE_BYTES (UTF_8_BOM_1, UTF_8_BOM_2, UTF_8_BOM_3);
1534 CODING_UTF_8_BOM (coding) = utf_without_bom;
1535 }
1536
df7492f9 1537 if (multibytep)
aa72b389 1538 {
df7492f9
KH		 1539 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
1540
1541 while (charbuf < charbuf_end)
b73bfc1c 1542 {
df7492f9 1543 unsigned char str[MAX_MULTIBYTE_LENGTH], *p, *pend = str;
8f924df7 1544
df7492f9
KH		 1545 ASSURE_DESTINATION (safe_room);
1546 c = *charbuf++;
28f67a95
KH		 1547 if (CHAR_BYTE8_P (c))
1548 {
1549 c = CHAR_TO_BYTE8 (c);
1550 EMIT_ONE_BYTE (c);
1551 }
1552 else
1553 {
db274c7a 1554 CHAR_STRING_ADVANCE_NO_UNIFY (c, pend);
28f67a95
KH		 1555 for (p = str; p < pend; p++)
1556 EMIT_ONE_BYTE (*p);
1557 }
b73bfc1c 1558 }
aa72b389 1559 }
df7492f9
KH		 1560 else
1561 {
1562 int safe_room = MAX_MULTIBYTE_LENGTH;
1563
1564 while (charbuf < charbuf_end)
b73bfc1c 1565 {
df7492f9
KH		 1566 ASSURE_DESTINATION (safe_room);
1567 c = *charbuf++;
f03caae0
KH		 1568 if (CHAR_BYTE8_P (c))
1569 *dst++ = CHAR_TO_BYTE8 (c);
1570 else
db274c7a 1571 CHAR_STRING_ADVANCE_NO_UNIFY (c, dst);
df7492f9 1572 produced_chars++;
4ed46869
KH		 1573 }
1574 }
065e3595 1575 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1576 coding->produced_char += produced_chars;
1577 coding->produced = dst - coding->destination;
1578 return 0;
4ed46869
KH		 1579}
1580
b73bfc1c 1581
df7492f9 1582/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1583 Check if a text is encoded in one of UTF-16 based coding systems.
1584 If it is, return 1, else return 0. */
aa72b389 1585
df7492f9
KH		 1586#define UTF_16_HIGH_SURROGATE_P(val) \
1587 (((val) & 0xFC00) == 0xD800)
1588
1589#define UTF_16_LOW_SURROGATE_P(val) \
1590 (((val) & 0xFC00) == 0xDC00)
93dec019 1591
df7492f9
KH		 1592#define UTF_16_INVALID_P(val) \
1593 (((val) == 0xFFFE) \
1594 || ((val) == 0xFFFF) \
1595 || UTF_16_LOW_SURROGATE_P (val))
aa72b389 1596
aa72b389 1597
df7492f9 1598static int
ff0dacd7 1599detect_coding_utf_16 (coding, detect_info)
aa72b389 1600 struct coding_system *coding;
ff0dacd7 1601 struct coding_detection_info *detect_info;
aa72b389 1602{
8f924df7
KH		 1603 const unsigned char *src = coding->source, *src_base = src;
1604 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 1605 int multibytep = coding->src_multibyte;
1606 int consumed_chars = 0;
1607 int c1, c2;
aa72b389 1608
ff0dacd7 1609 detect_info->checked |= CATEGORY_MASK_UTF_16;
ff0dacd7 1610 if (coding->mode & CODING_MODE_LAST_BLOCK
24a73b0a 1611 && (coding->src_chars & 1))
ff0dacd7
KH		 1612 {
1613 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1614 return 0;
1615 }
24a73b0a 1616
f56a4450 1617 TWO_MORE_BYTES (c1, c2);
df7492f9 1618 if ((c1 == 0xFF) && (c2 == 0xFE))
aa72b389 1619 {
b49a1807
KH		 1620 detect_info->found |= (CATEGORY_MASK_UTF_16_LE
1621 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1622 detect_info->rejected |= (CATEGORY_MASK_UTF_16_BE
1623 | CATEGORY_MASK_UTF_16_BE_NOSIG
1624 | CATEGORY_MASK_UTF_16_LE_NOSIG);
aa72b389 1625 }
df7492f9 1626 else if ((c1 == 0xFE) && (c2 == 0xFF))
ff0dacd7 1627 {
b49a1807
KH		 1628 detect_info->found |= (CATEGORY_MASK_UTF_16_BE
1629 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1630 detect_info->rejected |= (CATEGORY_MASK_UTF_16_LE
1631 | CATEGORY_MASK_UTF_16_BE_NOSIG
1632 | CATEGORY_MASK_UTF_16_LE_NOSIG);
1633 }
220eeac9 1634 else if (c2 < 0)
f56a4450
KH		 1635 {
1636 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1637 return 0;
1638 }
2f3cbb32 1639 else
24a73b0a 1640 {
2f3cbb32
KH		 1641 /* We check the dispersion of Eth and Oth bytes where E is even and
1642 O is odd. If both are high, we assume binary data.*/
1643 unsigned char e[256], o[256];
1644 unsigned e_num = 1, o_num = 1;
1645
1646 memset (e, 0, 256);
1647 memset (o, 0, 256);
1648 e[c1] = 1;
1649 o[c2] = 1;
1650
24a73b0a
KH		 1651 detect_info->rejected
1652 |= (CATEGORY_MASK_UTF_16_BE | CATEGORY_MASK_UTF_16_LE);
2f3cbb32
KH		 1653
1654 while (1)
1655 {
f56a4450 1656 TWO_MORE_BYTES (c1, c2);
220eeac9 1657 if (c2 < 0)
f56a4450 1658 break;
2f3cbb32
KH		 1659 if (! e[c1])
1660 {
1661 e[c1] = 1;
1662 e_num++;
1663 if (e_num >= 128)
1664 break;
1665 }
1666 if (! o[c2])
1667 {
1668 o[c1] = 1;
1669 o_num++;
1670 if (o_num >= 128)
1671 break;
1672 }
1673 }
1674 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1675 return 0;
ff0dacd7 1676 }
2f3cbb32 1677
df7492f9 1678 no_more_source:
ff0dacd7 1679 return 1;
df7492f9 1680}
aa72b389 1681
df7492f9
KH		 1682static void
1683decode_coding_utf_16 (coding)
1684 struct coding_system *coding;
1685{
8f924df7
KH		 1686 const unsigned char *src = coding->source + coding->consumed;
1687 const unsigned char *src_end = coding->source + coding->src_bytes;
1688 const unsigned char *src_base;
69a80ea3
KH		 1689 int *charbuf = coding->charbuf + coding->charbuf_used;
1690 int *charbuf_end = coding->charbuf + coding->charbuf_size;
3a8406e1 1691 int consumed_chars = 0, consumed_chars_base = 0;
df7492f9 1692 int multibytep = coding->src_multibyte;
a470d443 1693 enum utf_bom_type bom = CODING_UTF_16_BOM (coding);
df7492f9
KH		 1694 enum utf_16_endian_type endian = CODING_UTF_16_ENDIAN (coding);
1695 int surrogate = CODING_UTF_16_SURROGATE (coding);
24a73b0a 1696 Lisp_Object attr, charset_list;
119852e7
KH		 1697 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
1698 int byte_after_cr1 = -1, byte_after_cr2 = -1;
df7492f9 1699
24a73b0a 1700 CODING_GET_INFO (coding, attr, charset_list);
df7492f9 1701
a470d443 1702 if (bom == utf_with_bom)
aa72b389 1703 {
df7492f9 1704 int c, c1, c2;
4af310db 1705
aa72b389 1706 src_base = src;
df7492f9
KH		 1707 ONE_MORE_BYTE (c1);
1708 ONE_MORE_BYTE (c2);
e19c3639 1709 c = (c1 << 8) | c2;
aa72b389 1710
b49a1807
KH		 1711 if (endian == utf_16_big_endian
1712 ? c != 0xFEFF : c != 0xFFFE)
aa72b389 1713 {
b49a1807
KH		 1714 /* The first two bytes are not BOM. Treat them as bytes
1715 for a normal character. */
1716 src = src_base;
1717 coding->errors++;
aa72b389 1718 }
a470d443 1719 CODING_UTF_16_BOM (coding) = utf_without_bom;
b49a1807 1720 }
a470d443 1721 else if (bom == utf_detect_bom)
b49a1807
KH		 1722 {
1723 /* We have already tried to detect BOM and failed in
1724 detect_coding. */
a470d443 1725 CODING_UTF_16_BOM (coding) = utf_without_bom;
df7492f9 1726 }
aa72b389 1727
df7492f9
KH		 1728 while (1)
1729 {
1730 int c, c1, c2;
1731
1732 src_base = src;
1733 consumed_chars_base = consumed_chars;
1734
1735 if (charbuf + 2 >= charbuf_end)
b71f6f73
KH		 1736 {
1737 if (byte_after_cr1 >= 0)
1738 src_base -= 2;
1739 break;
1740 }
df7492f9 1741
119852e7
KH		 1742 if (byte_after_cr1 >= 0)
1743 c1 = byte_after_cr1, byte_after_cr1 = -1;
1744 else
1745 ONE_MORE_BYTE (c1);
065e3595
KH		 1746 if (c1 < 0)
1747 {
1748 *charbuf++ = -c1;
1749 continue;
1750 }
119852e7
KH		 1751 if (byte_after_cr2 >= 0)
1752 c2 = byte_after_cr2, byte_after_cr2 = -1;
1753 else
1754 ONE_MORE_BYTE (c2);
065e3595
KH		 1755 if (c2 < 0)
1756 {
1757 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
1758 *charbuf++ = -c2;
1759 continue;
1760 }
df7492f9 1761 c = (endian == utf_16_big_endian
e19c3639 1762 ? ((c1 << 8) | c2) : ((c2 << 8) | c1));
119852e7 1763
df7492f9 1764 if (surrogate)
fd3ae0b9 1765 {
df7492f9 1766 if (! UTF_16_LOW_SURROGATE_P (c))
fd3ae0b9 1767 {
df7492f9
KH		 1768 if (endian == utf_16_big_endian)
1769 c1 = surrogate >> 8, c2 = surrogate & 0xFF;
1770 else
1771 c1 = surrogate & 0xFF, c2 = surrogate >> 8;
1772 *charbuf++ = c1;
1773 *charbuf++ = c2;
1774 coding->errors++;
1775 if (UTF_16_HIGH_SURROGATE_P (c))
1776 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
fd3ae0b9 1777 else
df7492f9 1778 *charbuf++ = c;
fd3ae0b9
KH		 1779 }
1780 else
df7492f9
KH		 1781 {
1782 c = ((surrogate - 0xD800) << 10) | (c - 0xDC00);
1783 CODING_UTF_16_SURROGATE (coding) = surrogate = 0;
29f7ffd0 1784 *charbuf++ = 0x10000 + c;
df7492f9 1785 }
fd3ae0b9 1786 }
aa72b389 1787 else
df7492f9
KH		 1788 {
1789 if (UTF_16_HIGH_SURROGATE_P (c))
1790 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1791 else
119852e7
KH		 1792 {
1793 if (eol_crlf && c == '\r')
1794 {
1795 ONE_MORE_BYTE (byte_after_cr1);
1796 ONE_MORE_BYTE (byte_after_cr2);
1797 }
1798 *charbuf++ = c;
1799 }
8f924df7 1800 }
aa72b389 1801 }
df7492f9
KH		 1802
1803 no_more_source:
1804 coding->consumed_char += consumed_chars_base;
1805 coding->consumed = src_base - coding->source;
1806 coding->charbuf_used = charbuf - coding->charbuf;
aa72b389 1807}
b73bfc1c 1808
df7492f9
KH		 1809static int
1810encode_coding_utf_16 (coding)
1811 struct coding_system *coding;
1812{
1813 int multibytep = coding->dst_multibyte;
1814 int *charbuf = coding->charbuf;
1815 int *charbuf_end = charbuf + coding->charbuf_used;
1816 unsigned char *dst = coding->destination + coding->produced;
1817 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1818 int safe_room = 8;
a470d443 1819 enum utf_bom_type bom = CODING_UTF_16_BOM (coding);
df7492f9
KH		 1820 int big_endian = CODING_UTF_16_ENDIAN (coding) == utf_16_big_endian;
1821 int produced_chars = 0;
24a73b0a 1822 Lisp_Object attrs, charset_list;
df7492f9 1823 int c;
4ed46869 1824
24a73b0a 1825 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 1826
a470d443 1827 if (bom != utf_without_bom)
df7492f9
KH		 1828 {
1829 ASSURE_DESTINATION (safe_room);
1830 if (big_endian)
df7492f9 1831 EMIT_TWO_BYTES (0xFE, 0xFF);
880cf180
KH		 1832 else
1833 EMIT_TWO_BYTES (0xFF, 0xFE);
a470d443 1834 CODING_UTF_16_BOM (coding) = utf_without_bom;
df7492f9
KH		 1835 }
1836
1837 while (charbuf < charbuf_end)
1838 {
1839 ASSURE_DESTINATION (safe_room);
1840 c = *charbuf++;
e19c3639
KH		 1841 if (c >= MAX_UNICODE_CHAR)
1842 c = coding->default_char;
df7492f9
KH		 1843
1844 if (c < 0x10000)
1845 {
1846 if (big_endian)
1847 EMIT_TWO_BYTES (c >> 8, c & 0xFF);
1848 else
1849 EMIT_TWO_BYTES (c & 0xFF, c >> 8);
1850 }
1851 else
1852 {
1853 int c1, c2;
1854
1855 c -= 0x10000;
1856 c1 = (c >> 10) + 0xD800;
1857 c2 = (c & 0x3FF) + 0xDC00;
1858 if (big_endian)
1859 EMIT_FOUR_BYTES (c1 >> 8, c1 & 0xFF, c2 >> 8, c2 & 0xFF);
1860 else
1861 EMIT_FOUR_BYTES (c1 & 0xFF, c1 >> 8, c2 & 0xFF, c2 >> 8);
1862 }
1863 }
065e3595 1864 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1865 coding->produced = dst - coding->destination;
1866 coding->produced_char += produced_chars;
1867 return 0;
1868}
1869
1870\f
1871/*** 6. Old Emacs' internal format (emacs-mule) ***/
1872
1873/* Emacs' internal format for representation of multiple character
1874 sets is a kind of multi-byte encoding, i.e. characters are
1875 represented by variable-length sequences of one-byte codes.
1876
1877 ASCII characters and control characters (e.g. `tab', `newline') are
1878 represented by one-byte sequences which are their ASCII codes, in
1879 the range 0x00 through 0x7F.
1880
1881 8-bit characters of the range 0x80..0x9F are represented by
1882 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
1883 code + 0x20).
1884
1885 8-bit characters of the range 0xA0..0xFF are represented by
1886 one-byte sequences which are their 8-bit code.
1887
1888 The other characters are represented by a sequence of `base
1889 leading-code', optional `extended leading-code', and one or two
1890 `position-code's. The length of the sequence is determined by the
1891 base leading-code. Leading-code takes the range 0x81 through 0x9D,
1892 whereas extended leading-code and position-code take the range 0xA0
1893 through 0xFF. See `charset.h' for more details about leading-code
1894 and position-code.
1895
1896 --- CODE RANGE of Emacs' internal format ---
1897 character set range
1898 ------------- -----
1899 ascii 0x00..0x7F
1900 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
1901 eight-bit-graphic 0xA0..0xBF
1902 ELSE 0x81..0x9D + [0xA0..0xFF]+
1903 ---------------------------------------------
1904
1905 As this is the internal character representation, the format is
1906 usually not used externally (i.e. in a file or in a data sent to a
1907 process). But, it is possible to have a text externally in this
1908 format (i.e. by encoding by the coding system `emacs-mule').
1909
1910 In that case, a sequence of one-byte codes has a slightly different
1911 form.
1912
1913 At first, all characters in eight-bit-control are represented by
1914 one-byte sequences which are their 8-bit code.
1915
1916 Next, character composition data are represented by the byte
1917 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
1918 where,
1919 METHOD is 0xF0 plus one of composition method (enum
1920 composition_method),
1921
1922 BYTES is 0xA0 plus a byte length of this composition data,
1923
1924 CHARS is 0x20 plus a number of characters composed by this
1925 data,
1926
1927 COMPONENTs are characters of multibye form or composition
1928 rules encoded by two-byte of ASCII codes.
1929
1930 In addition, for backward compatibility, the following formats are
1931 also recognized as composition data on decoding.
1932
1933 0x80 MSEQ ...
1934 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
1935
1936 Here,
1937 MSEQ is a multibyte form but in these special format:
1938 ASCII: 0xA0 ASCII_CODE+0x80,
1939 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
1940 RULE is a one byte code of the range 0xA0..0xF0 that
1941 represents a composition rule.
1942 */
1943
1944char emacs_mule_bytes[256];
1945
df7492f9 1946int
ff0dacd7 1947emacs_mule_char (coding, src, nbytes, nchars, id)
df7492f9 1948 struct coding_system *coding;
065e3595 1949 const unsigned char *src;
ff0dacd7 1950 int *nbytes, *nchars, *id;
df7492f9 1951{
8f924df7
KH		 1952 const unsigned char *src_end = coding->source + coding->src_bytes;
1953 const unsigned char *src_base = src;
df7492f9 1954 int multibytep = coding->src_multibyte;
df7492f9
KH		 1955 struct charset *charset;
1956 unsigned code;
1957 int c;
1958 int consumed_chars = 0;
1959
1960 ONE_MORE_BYTE (c);
065e3595 1961 if (c < 0)
df7492f9 1962 {
065e3595
KH		 1963 c = -c;
1964 charset = emacs_mule_charset[0];
1965 }
1966 else
1967 {
4d41e8b7
KH		 1968 if (c >= 0xA0)
1969 {
b3af4b28 1970 /* Old style component character of a composition. */
4d41e8b7
KH		 1971 if (c == 0xA0)
1972 {
1973 ONE_MORE_BYTE (c);
1974 c -= 0x80;
1975 }
1976 else
1977 c -= 0x20;
1978 }
1979
065e3595 1980 switch (emacs_mule_bytes[c])
b73bfc1c 1981 {
065e3595 1982 case 2:
df7492f9
KH		 1983 if (! (charset = emacs_mule_charset[c]))
1984 goto invalid_code;
1985 ONE_MORE_BYTE (c);
9ffd559c 1986 if (c < 0xA0)
065e3595 1987 goto invalid_code;
df7492f9 1988 code = c & 0x7F;
065e3595
KH		 1989 break;
1990
1991 case 3:
1992 if (c == EMACS_MULE_LEADING_CODE_PRIVATE_11
1993 || c == EMACS_MULE_LEADING_CODE_PRIVATE_12)
1994 {
1995 ONE_MORE_BYTE (c);
9ffd559c 1996 if (c < 0xA0 || ! (charset = emacs_mule_charset[c]))
065e3595
KH		 1997 goto invalid_code;
1998 ONE_MORE_BYTE (c);
9ffd559c 1999 if (c < 0xA0)
065e3595
KH		 2000 goto invalid_code;
2001 code = c & 0x7F;
2002 }
2003 else
2004 {
2005 if (! (charset = emacs_mule_charset[c]))
2006 goto invalid_code;
2007 ONE_MORE_BYTE (c);
9ffd559c 2008 if (c < 0xA0)
065e3595
KH		 2009 goto invalid_code;
2010 code = (c & 0x7F) << 8;
2011 ONE_MORE_BYTE (c);
9ffd559c 2012 if (c < 0xA0)
065e3595
KH		 2013 goto invalid_code;
2014 code |= c & 0x7F;
2015 }
2016 break;
2017
2018 case 4:
2019 ONE_MORE_BYTE (c);
2020 if (c < 0 || ! (charset = emacs_mule_charset[c]))
df7492f9
KH		 2021 goto invalid_code;
2022 ONE_MORE_BYTE (c);
9ffd559c 2023 if (c < 0xA0)
065e3595 2024 goto invalid_code;
781d7a48 2025 code = (c & 0x7F) << 8;
df7492f9 2026 ONE_MORE_BYTE (c);
9ffd559c 2027 if (c < 0xA0)
065e3595 2028 goto invalid_code;
df7492f9 2029 code |= c & 0x7F;
065e3595 2030 break;
df7492f9 2031
065e3595
KH		 2032 case 1:
2033 code = c;
2034 charset = CHARSET_FROM_ID (ASCII_BYTE_P (code)
2035 ? charset_ascii : charset_eight_bit);
2036 break;
df7492f9 2037
065e3595
KH		 2038 default:
2039 abort ();
2040 }
2041 c = DECODE_CHAR (charset, code);
2042 if (c < 0)
2043 goto invalid_code;
df7492f9 2044 }
df7492f9
KH		 2045 *nbytes = src - src_base;
2046 *nchars = consumed_chars;
ff0dacd7
KH		 2047 if (id)
2048 *id = charset->id;
df7492f9
KH		 2049 return c;
2050
2051 no_more_source:
2052 return -2;
2053
2054 invalid_code:
2055 return -1;
2056}
2057
2058
2059/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 2060 Check if a text is encoded in `emacs-mule'. If it is, return 1,
2061 else return 0. */
df7492f9
KH		 2062
2063static int
ff0dacd7 2064detect_coding_emacs_mule (coding, detect_info)
df7492f9 2065 struct coding_system *coding;
ff0dacd7 2066 struct coding_detection_info *detect_info;
df7492f9 2067{
065e3595 2068 const unsigned char *src = coding->source, *src_base;
8f924df7 2069 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 2070 int multibytep = coding->src_multibyte;
2071 int consumed_chars = 0;
2072 int c;
2073 int found = 0;
2074
ff0dacd7 2075 detect_info->checked |= CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 2076 /* A coding system of this category is always ASCII compatible. */
2077 src += coding->head_ascii;
2078
2079 while (1)
2080 {
065e3595 2081 src_base = src;
df7492f9 2082 ONE_MORE_BYTE (c);
065e3595
KH		 2083 if (c < 0)
2084 continue;
df7492f9
KH		 2085 if (c == 0x80)
2086 {
2087 /* Perhaps the start of composite character. We simple skip
2088 it because analyzing it is too heavy for detecting. But,
2089 at least, we check that the composite character
3ed051d4 2090 constitutes of more than 4 bytes. */
8f924df7 2091 const unsigned char *src_base;
df7492f9
KH		 2092
2093 repeat:
2094 src_base = src;
2095 do
2096 {
2097 ONE_MORE_BYTE (c);
2098 }
2099 while (c >= 0xA0);
2100
2101 if (src - src_base <= 4)
2102 break;
ff0dacd7 2103 found = CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 2104 if (c == 0x80)
2105 goto repeat;
b73bfc1c 2106 }
df7492f9
KH		 2107
2108 if (c < 0x80)
b73bfc1c 2109 {
df7492f9
KH		 2110 if (c < 0x20
2111 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO))
2112 break;
2113 }
2114 else
2115 {
0e219d54 2116 int more_bytes = emacs_mule_bytes[*src_base] - 1;
df7492f9 2117
0e219d54 2118 while (more_bytes > 0)
df7492f9
KH		 2119 {
2120 ONE_MORE_BYTE (c);
0e219d54
KH		 2121 if (c < 0xA0)
2122 {
2123 src--; /* Unread the last byte. */
2124 break;
2125 }
2126 more_bytes--;
df7492f9 2127 }
0e219d54 2128 if (more_bytes != 0)
df7492f9 2129 break;
ff0dacd7 2130 found = CATEGORY_MASK_EMACS_MULE;
4ed46869
KH		 2131 }
2132 }
ff0dacd7 2133 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 2134 return 0;
2135
2136 no_more_source:
065e3595 2137 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 2138 {
ff0dacd7 2139 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
89528eb3
KH		 2140 return 0;
2141 }
ff0dacd7
KH		 2142 detect_info->found |= found;
2143 return 1;
4ed46869
KH		 2144}
2145
b73bfc1c 2146
df7492f9
KH		 2147/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
2148
2149/* Decode a character represented as a component of composition
2150 sequence of Emacs 20/21 style at SRC. Set C to that character and
2151 update SRC to the head of next character (or an encoded composition
2152 rule). If SRC doesn't points a composition component, set C to -1.
2153 If SRC points an invalid byte sequence, global exit by a return
2154 value 0. */
2155
2156#define DECODE_EMACS_MULE_COMPOSITION_CHAR(buf) \
f937a7db 2157 do \
df7492f9
KH		 2158 { \
2159 int c; \
2160 int nbytes, nchars; \
2161 \
2162 if (src == src_end) \
2163 break; \
ff0dacd7 2164 c = emacs_mule_char (coding, src, &nbytes, &nchars, NULL);\
df7492f9
KH		 2165 if (c < 0) \
2166 { \
2167 if (c == -2) \
2168 break; \
2169 goto invalid_code; \
2170 } \
2171 *buf++ = c; \
2172 src += nbytes; \
2173 consumed_chars += nchars; \
2174 } \
f937a7db 2175 while (0)
df7492f9
KH		 2176
2177
2178/* Decode a composition rule represented as a component of composition
781d7a48
KH		 2179 sequence of Emacs 20 style at SRC. Store the decoded rule in *BUF,
2180 and increment BUF. If SRC points an invalid byte sequence, set C
2181 to -1. */
df7492f9 2182
781d7a48 2183#define DECODE_EMACS_MULE_COMPOSITION_RULE_20(buf) \
df7492f9
KH		 2184 do { \
2185 int c, gref, nref; \
2186 \
781d7a48 2187 if (src >= src_end) \
df7492f9
KH		 2188 goto invalid_code; \
2189 ONE_MORE_BYTE_NO_CHECK (c); \
4d41e8b7 2190 c -= 0xA0; \
df7492f9
KH		 2191 if (c < 0 || c >= 81) \
2192 goto invalid_code; \
2193 \
2194 gref = c / 9, nref = c % 9; \
2195 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
2196 } while (0)
2197
2198
781d7a48
KH		 2199/* Decode a composition rule represented as a component of composition
2200 sequence of Emacs 21 style at SRC. Store the decoded rule in *BUF,
2201 and increment BUF. If SRC points an invalid byte sequence, set C
2202 to -1. */
2203
2204#define DECODE_EMACS_MULE_COMPOSITION_RULE_21(buf) \
2205 do { \
2206 int gref, nref; \
2207 \
2208 if (src + 1>= src_end) \
2209 goto invalid_code; \
2210 ONE_MORE_BYTE_NO_CHECK (gref); \
2211 gref -= 0x20; \
2212 ONE_MORE_BYTE_NO_CHECK (nref); \
2213 nref -= 0x20; \
2214 if (gref < 0 || gref >= 81 \
2215 || nref < 0 || nref >= 81) \
2216 goto invalid_code; \
2217 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
2218 } while (0)
2219
2220
df7492f9 2221#define DECODE_EMACS_MULE_21_COMPOSITION(c) \
aa72b389 2222 do { \
df7492f9 2223 /* Emacs 21 style format. The first three bytes at SRC are \
781d7a48 2224 (METHOD - 0xF2), (BYTES - 0xA0), (CHARS - 0xA0), where BYTES is \
df7492f9
KH		 2225 the byte length of this composition information, CHARS is the \
2226 number of characters composed by this composition. */ \
781d7a48
KH		 2227 enum composition_method method = c - 0xF2; \
2228 int *charbuf_base = charbuf; \
df7492f9
KH		 2229 int consumed_chars_limit; \
2230 int nbytes, nchars; \
2231 \
2232 ONE_MORE_BYTE (c); \
065e3595
KH		 2233 if (c < 0) \
2234 goto invalid_code; \
df7492f9
KH		 2235 nbytes = c - 0xA0; \
2236 if (nbytes < 3) \
2237 goto invalid_code; \
2238 ONE_MORE_BYTE (c); \
065e3595
KH		 2239 if (c < 0) \
2240 goto invalid_code; \
df7492f9 2241 nchars = c - 0xA0; \
69a80ea3 2242 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
df7492f9
KH		 2243 consumed_chars_limit = consumed_chars_base + nbytes; \
2244 if (method != COMPOSITION_RELATIVE) \
aa72b389 2245 { \
df7492f9
KH		 2246 int i = 0; \
2247 while (consumed_chars < consumed_chars_limit) \
aa72b389 2248 { \
df7492f9 2249 if (i % 2 && method != COMPOSITION_WITH_ALTCHARS) \
781d7a48 2250 DECODE_EMACS_MULE_COMPOSITION_RULE_21 (charbuf); \
df7492f9
KH		 2251 else \
2252 DECODE_EMACS_MULE_COMPOSITION_CHAR (charbuf); \
781d7a48 2253 i++; \
aa72b389 2254 } \
df7492f9
KH		 2255 if (consumed_chars < consumed_chars_limit) \
2256 goto invalid_code; \
781d7a48 2257 charbuf_base[0] -= i; \
aa72b389
KH		 2258 } \
2259 } while (0)
93dec019 2260
aa72b389 2261
d959f512
KH		 2262#define DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION(c) \
2263 do { \
2264 /* Emacs 20 style format for relative composition. */ \
2265 /* Store multibyte form of characters to be composed. */ \
2266 enum composition_method method = COMPOSITION_RELATIVE; \
2267 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
2268 int *buf = components; \
2269 int i, j; \
2270 \
2271 src = src_base; \
2272 ONE_MORE_BYTE (c); /* skip 0x80 */ \
2273 for (i = 0; *src >= 0xA0 && i < MAX_COMPOSITION_COMPONENTS; i++) \
2274 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2275 if (i < 2) \
2276 goto invalid_code; \
2277 ADD_COMPOSITION_DATA (charbuf, i, method); \
2278 for (j = 0; j < i; j++) \
2279 *charbuf++ = components[j]; \
df7492f9
KH		 2280 } while (0)
2281
2282
2283#define DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION(c) \
2284 do { \
2285 /* Emacs 20 style format for rule-base composition. */ \
2286 /* Store multibyte form of characters to be composed. */ \
ff0dacd7 2287 enum composition_method method = COMPOSITION_WITH_RULE; \
4d41e8b7 2288 int *charbuf_base = charbuf; \
df7492f9
KH		 2289 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
2290 int *buf = components; \
2291 int i, j; \
4d41e8b7 2292 \
df7492f9 2293 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
4d41e8b7 2294 for (i = 1; i < MAX_COMPOSITION_COMPONENTS; i++) \
df7492f9 2295 { \
4d41e8b7
KH		 2296 if (*src < 0xA0) \
2297 break; \
781d7a48 2298 DECODE_EMACS_MULE_COMPOSITION_RULE_20 (buf); \
df7492f9
KH		 2299 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2300 } \
4d41e8b7 2301 if (i <= 1 || (buf - components) % 2 == 0) \
df7492f9 2302 goto invalid_code; \
4d41e8b7 2303 if (charbuf + i + (i / 2) + 1 >= charbuf_end) \
df7492f9 2304 goto no_more_source; \
4d41e8b7
KH		 2305 ADD_COMPOSITION_DATA (charbuf, i, method); \
2306 i = i * 2 - 1; \
df7492f9
KH		 2307 for (j = 0; j < i; j++) \
2308 *charbuf++ = components[j]; \
4d41e8b7 2309 charbuf_base[0] -= i; \
df7492f9
KH		 2310 for (j = 0; j < i; j += 2) \
2311 *charbuf++ = components[j]; \
2312 } while (0)
2313
aa72b389
KH		 2314
2315static void
df7492f9 2316decode_coding_emacs_mule (coding)
aa72b389 2317 struct coding_system *coding;
aa72b389 2318{
8f924df7
KH		 2319 const unsigned char *src = coding->source + coding->consumed;
2320 const unsigned char *src_end = coding->source + coding->src_bytes;
2321 const unsigned char *src_base;
69a80ea3
KH		 2322 int *charbuf = coding->charbuf + coding->charbuf_used;
2323 int *charbuf_end
2324 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9 2325 int consumed_chars = 0, consumed_chars_base;
df7492f9 2326 int multibytep = coding->src_multibyte;
24a73b0a 2327 Lisp_Object attrs, charset_list;
ff0dacd7
KH		 2328 int char_offset = coding->produced_char;
2329 int last_offset = char_offset;
2330 int last_id = charset_ascii;
119852e7
KH		 2331 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
2332 int byte_after_cr = -1;
aa72b389 2333
24a73b0a 2334 CODING_GET_INFO (coding, attrs, charset_list);
aa72b389 2335
aa72b389
KH		 2336 while (1)
2337 {
df7492f9
KH		 2338 int c;
2339
aa72b389 2340 src_base = src;
df7492f9
KH		 2341 consumed_chars_base = consumed_chars;
2342
2343 if (charbuf >= charbuf_end)
b71f6f73
KH		 2344 {
2345 if (byte_after_cr >= 0)
2346 src_base--;
2347 break;
2348 }
aa72b389 2349
119852e7
KH		 2350 if (byte_after_cr >= 0)
2351 c = byte_after_cr, byte_after_cr = -1;
2352 else
2353 ONE_MORE_BYTE (c);
065e3595
KH		 2354 if (c < 0)
2355 {
2356 *charbuf++ = -c;
2357 char_offset++;
2358 }
2359 else if (c < 0x80)
aa72b389 2360 {
119852e7
KH		 2361 if (eol_crlf && c == '\r')
2362 ONE_MORE_BYTE (byte_after_cr);
df7492f9
KH		 2363 *charbuf++ = c;
2364 char_offset++;
aa72b389 2365 }
df7492f9
KH		 2366 else if (c == 0x80)
2367 {
df7492f9 2368 ONE_MORE_BYTE (c);
065e3595
KH		 2369 if (c < 0)
2370 goto invalid_code;
781d7a48
KH		 2371 if (c - 0xF2 >= COMPOSITION_RELATIVE
2372 && c - 0xF2 <= COMPOSITION_WITH_RULE_ALTCHARS)
df7492f9
KH		 2373 DECODE_EMACS_MULE_21_COMPOSITION (c);
2374 else if (c < 0xC0)
2375 DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION (c);
2376 else if (c == 0xFF)
2377 DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION (c);
2378 else
2379 goto invalid_code;
2380 }
2381 else if (c < 0xA0 && emacs_mule_bytes[c] > 1)
2382 {
2383 int nbytes, nchars;
ff0dacd7
KH		 2384 int id;
2385
781d7a48
KH		 2386 src = src_base;
2387 consumed_chars = consumed_chars_base;
ff0dacd7 2388 c = emacs_mule_char (coding, src, &nbytes, &nchars, &id);
df7492f9
KH		 2389 if (c < 0)
2390 {
2391 if (c == -2)
2392 break;
2393 goto invalid_code;
2394 }
ff0dacd7
KH		 2395 if (last_id != id)
2396 {
2397 if (last_id != charset_ascii)
69a80ea3 2398 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
ff0dacd7
KH		 2399 last_id = id;
2400 last_offset = char_offset;
2401 }
df7492f9 2402 *charbuf++ = c;
781d7a48
KH		 2403 src += nbytes;
2404 consumed_chars += nchars;
df7492f9
KH		 2405 char_offset++;
2406 }
4d41e8b7
KH		 2407 else
2408 goto invalid_code;
df7492f9
KH		 2409 continue;
2410
2411 invalid_code:
2412 src = src_base;
2413 consumed_chars = consumed_chars_base;
2414 ONE_MORE_BYTE (c);
2415 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 2416 char_offset++;
df7492f9
KH		 2417 coding->errors++;
2418 }
2419
2420 no_more_source:
ff0dacd7 2421 if (last_id != charset_ascii)
69a80ea3 2422 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 2423 coding->consumed_char += consumed_chars_base;
2424 coding->consumed = src_base - coding->source;
2425 coding->charbuf_used = charbuf - coding->charbuf;
2426}
2427
2428
2429#define EMACS_MULE_LEADING_CODES(id, codes) \
2430 do { \
2431 if (id < 0xA0) \
2432 codes[0] = id, codes[1] = 0; \
2433 else if (id < 0xE0) \
2434 codes[0] = 0x9A, codes[1] = id; \
2435 else if (id < 0xF0) \
2436 codes[0] = 0x9B, codes[1] = id; \
2437 else if (id < 0xF5) \
2438 codes[0] = 0x9C, codes[1] = id; \
2439 else \
2440 codes[0] = 0x9D, codes[1] = id; \
2441 } while (0);
2442
aa72b389 2443
df7492f9
KH		 2444static int
2445encode_coding_emacs_mule (coding)
2446 struct coding_system *coding;
2447{
2448 int multibytep = coding->dst_multibyte;
2449 int *charbuf = coding->charbuf;
2450 int *charbuf_end = charbuf + coding->charbuf_used;
2451 unsigned char *dst = coding->destination + coding->produced;
2452 unsigned char *dst_end = coding->destination + coding->dst_bytes;
2453 int safe_room = 8;
df7492f9 2454 int produced_chars = 0;
24a73b0a 2455 Lisp_Object attrs, charset_list;
df7492f9 2456 int c;
ff0dacd7 2457 int preferred_charset_id = -1;
df7492f9 2458
24a73b0a 2459 CODING_GET_INFO (coding, attrs, charset_list);
eccb6815
KH		 2460 if (! EQ (charset_list, Vemacs_mule_charset_list))
2461 {
2462 CODING_ATTR_CHARSET_LIST (attrs)
2463 = charset_list = Vemacs_mule_charset_list;
2464 }
df7492f9
KH		 2465
2466 while (charbuf < charbuf_end)
2467 {
2468 ASSURE_DESTINATION (safe_room);
2469 c = *charbuf++;
ff0dacd7
KH		 2470
2471 if (c < 0)
2472 {
2473 /* Handle an annotation. */
2474 switch (*charbuf)
2475 {
2476 case CODING_ANNOTATE_COMPOSITION_MASK:
2477 /* Not yet implemented. */
2478 break;
2479 case CODING_ANNOTATE_CHARSET_MASK:
2480 preferred_charset_id = charbuf[3];
2481 if (preferred_charset_id >= 0
2482 && NILP (Fmemq (make_number (preferred_charset_id),
2483 charset_list)))
2484 preferred_charset_id = -1;
2485 break;
2486 default:
2487 abort ();
2488 }
2489 charbuf += -c - 1;
2490 continue;
2491 }
2492
df7492f9
KH		 2493 if (ASCII_CHAR_P (c))
2494 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 2495 else if (CHAR_BYTE8_P (c))
2496 {
2497 c = CHAR_TO_BYTE8 (c);
2498 EMIT_ONE_BYTE (c);
2499 }
df7492f9 2500 else
aa72b389 2501 {
df7492f9
KH		 2502 struct charset *charset;
2503 unsigned code;
2504 int dimension;
2505 int emacs_mule_id;
2506 unsigned char leading_codes[2];
2507
ff0dacd7
KH		 2508 if (preferred_charset_id >= 0)
2509 {
2510 charset = CHARSET_FROM_ID (preferred_charset_id);
905ca9d2
KH		 2511 if (CHAR_CHARSET_P (c, charset))
2512 code = ENCODE_CHAR (charset, c);
2513 else
2514 charset = char_charset (c, charset_list, &code);
ff0dacd7
KH		 2515 }
2516 else
2517 charset = char_charset (c, charset_list, &code);
df7492f9
KH		 2518 if (! charset)
2519 {
2520 c = coding->default_char;
2521 if (ASCII_CHAR_P (c))
2522 {
2523 EMIT_ONE_ASCII_BYTE (c);
2524 continue;
2525 }
2526 charset = char_charset (c, charset_list, &code);
2527 }
2528 dimension = CHARSET_DIMENSION (charset);
2529 emacs_mule_id = CHARSET_EMACS_MULE_ID (charset);
2530 EMACS_MULE_LEADING_CODES (emacs_mule_id, leading_codes);
2531 EMIT_ONE_BYTE (leading_codes[0]);
2532 if (leading_codes[1])
2533 EMIT_ONE_BYTE (leading_codes[1]);
2534 if (dimension == 1)
1fa663f9 2535 EMIT_ONE_BYTE (code | 0x80);
aa72b389 2536 else
df7492f9 2537 {
1fa663f9 2538 code |= 0x8080;
df7492f9
KH		 2539 EMIT_ONE_BYTE (code >> 8);
2540 EMIT_ONE_BYTE (code & 0xFF);
2541 }
aa72b389 2542 }
aa72b389 2543 }
065e3595 2544 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 2545 coding->produced_char += produced_chars;
2546 coding->produced = dst - coding->destination;
2547 return 0;
aa72b389 2548}
b73bfc1c 2549
4ed46869 2550\f
df7492f9 2551/*** 7. ISO2022 handlers ***/
4ed46869
KH		 2552
2553/* The following note describes the coding system ISO2022 briefly.
39787efd 2554 Since the intention of this note is to help understand the
5a936b46 2555 functions in this file, some parts are NOT ACCURATE or are OVERLY
39787efd 2556 SIMPLIFIED. For thorough understanding, please refer to the
5a936b46 2557 original document of ISO2022. This is equivalent to the standard
cfb43547 2558 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
4ed46869
KH		 2559
2560 ISO2022 provides many mechanisms to encode several character sets
cfb43547 2561 in 7-bit and 8-bit environments. For 7-bit environments, all text
39787efd
KH		 2562 is encoded using bytes less than 128. This may make the encoded
2563 text a little bit longer, but the text passes more easily through
cfb43547 2564 several types of gateway, some of which strip off the MSB (Most
8ca3766a 2565 Significant Bit).
b73bfc1c 2566
cfb43547
DL		 2567 There are two kinds of character sets: control character sets and
2568 graphic character sets. The former contain control characters such
4ed46869 2569 as `newline' and `escape' to provide control functions (control
39787efd 2570 functions are also provided by escape sequences). The latter
cfb43547 2571 contain graphic characters such as 'A' and '-'. Emacs recognizes
4ed46869
KH		 2572 two control character sets and many graphic character sets.
2573
2574 Graphic character sets are classified into one of the following
39787efd
KH		 2575 four classes, according to the number of bytes (DIMENSION) and
2576 number of characters in one dimension (CHARS) of the set:
2577 - DIMENSION1_CHARS94
2578 - DIMENSION1_CHARS96
2579 - DIMENSION2_CHARS94
2580 - DIMENSION2_CHARS96
2581
2582 In addition, each character set is assigned an identification tag,
cfb43547 2583 unique for each set, called the "final character" (denoted as <F>
39787efd
KH		 2584 hereafter). The <F> of each character set is decided by ECMA(*)
2585 when it is registered in ISO. The code range of <F> is 0x30..0x7F
2586 (0x30..0x3F are for private use only).
4ed46869
KH		 2587
2588 Note (*): ECMA = European Computer Manufacturers Association
2589
cfb43547 2590 Here are examples of graphic character sets [NAME(<F>)]:
4ed46869
KH		 2591 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
2592 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
2593 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
2594 o DIMENSION2_CHARS96 -- none for the moment
2595
39787efd 2596 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
4ed46869
KH		 2597 C0 [0x00..0x1F] -- control character plane 0
2598 GL [0x20..0x7F] -- graphic character plane 0
2599 C1 [0x80..0x9F] -- control character plane 1
2600 GR [0xA0..0xFF] -- graphic character plane 1
2601
2602 A control character set is directly designated and invoked to C0 or
39787efd
KH		 2603 C1 by an escape sequence. The most common case is that:
2604 - ISO646's control character set is designated/invoked to C0, and
2605 - ISO6429's control character set is designated/invoked to C1,
2606 and usually these designations/invocations are omitted in encoded
2607 text. In a 7-bit environment, only C0 can be used, and a control
2608 character for C1 is encoded by an appropriate escape sequence to
2609 fit into the environment. All control characters for C1 are
2610 defined to have corresponding escape sequences.
4ed46869
KH		 2611
2612 A graphic character set is at first designated to one of four
2613 graphic registers (G0 through G3), then these graphic registers are
2614 invoked to GL or GR. These designations and invocations can be
2615 done independently. The most common case is that G0 is invoked to
39787efd
KH		 2616 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
2617 these invocations and designations are omitted in encoded text.
2618 In a 7-bit environment, only GL can be used.
4ed46869 2619
39787efd
KH		 2620 When a graphic character set of CHARS94 is invoked to GL, codes
2621 0x20 and 0x7F of the GL area work as control characters SPACE and
2622 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
2623 be used.
4ed46869
KH		 2624
2625 There are two ways of invocation: locking-shift and single-shift.
2626 With locking-shift, the invocation lasts until the next different
39787efd
KH		 2627 invocation, whereas with single-shift, the invocation affects the
2628 following character only and doesn't affect the locking-shift
2629 state. Invocations are done by the following control characters or
2630 escape sequences:
4ed46869
KH		 2631
2632 ----------------------------------------------------------------------
39787efd 2633 abbrev function cntrl escape seq description
4ed46869 2634 ----------------------------------------------------------------------
39787efd
KH		 2635 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
2636 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
2637 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
2638 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
2639 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
2640 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
2641 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
2642 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
2643 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
4ed46869 2644 ----------------------------------------------------------------------
39787efd
KH		 2645 (*) These are not used by any known coding system.
2646
2647 Control characters for these functions are defined by macros
2648 ISO_CODE_XXX in `coding.h'.
4ed46869 2649
39787efd 2650 Designations are done by the following escape sequences:
4ed46869
KH		 2651 ----------------------------------------------------------------------
2652 escape sequence description
2653 ----------------------------------------------------------------------
2654 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
2655 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
2656 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
2657 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
2658 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
2659 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
2660 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
2661 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
2662 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
2663 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
2664 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
2665 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
2666 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
2667 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
2668 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
2669 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
2670 ----------------------------------------------------------------------
2671
2672 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
39787efd 2673 of dimension 1, chars 94, and final character <F>, etc...
4ed46869
KH		 2674
2675 Note (*): Although these designations are not allowed in ISO2022,
2676 Emacs accepts them on decoding, and produces them on encoding
39787efd 2677 CHARS96 character sets in a coding system which is characterized as
4ed46869
KH		 2678 7-bit environment, non-locking-shift, and non-single-shift.
2679
2680 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
df7492f9 2681 '(' must be omitted. We refer to this as "short-form" hereafter.
4ed46869 2682
cfb43547 2683 Now you may notice that there are a lot of ways of encoding the
39787efd
KH		 2684 same multilingual text in ISO2022. Actually, there exist many
2685 coding systems such as Compound Text (used in X11's inter client
8ca3766a
DL		 2686 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
2687 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
4ed46869
KH		 2688 localized platforms), and all of these are variants of ISO2022.
2689
2690 In addition to the above, Emacs handles two more kinds of escape
2691 sequences: ISO6429's direction specification and Emacs' private
2692 sequence for specifying character composition.
2693
39787efd 2694 ISO6429's direction specification takes the following form:
4ed46869
KH		 2695 o CSI ']' -- end of the current direction
2696 o CSI '0' ']' -- end of the current direction
2697 o CSI '1' ']' -- start of left-to-right text
2698 o CSI '2' ']' -- start of right-to-left text
2699 The control character CSI (0x9B: control sequence introducer) is
39787efd
KH		 2700 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
2701
2702 Character composition specification takes the following form:
ec6d2bb8
KH		 2703 o ESC '0' -- start relative composition
2704 o ESC '1' -- end composition
2705 o ESC '2' -- start rule-base composition (*)
2706 o ESC '3' -- start relative composition with alternate chars (**)
2707 o ESC '4' -- start rule-base composition with alternate chars (**)
b73bfc1c 2708 Since these are not standard escape sequences of any ISO standard,
cfb43547 2709 the use of them with these meanings is restricted to Emacs only.
ec6d2bb8 2710
5a936b46
DL		 2711 (*) This form is used only in Emacs 20.7 and older versions,
2712 but newer versions can safely decode it.
cfb43547 2713 (**) This form is used only in Emacs 21.1 and newer versions,
5a936b46 2714 and older versions can't decode it.
ec6d2bb8 2715
cfb43547 2716 Here's a list of example usages of these composition escape
b73bfc1c 2717 sequences (categorized by `enum composition_method').
ec6d2bb8 2718
b73bfc1c 2719 COMPOSITION_RELATIVE:
ec6d2bb8 2720 ESC 0 CHAR [ CHAR ] ESC 1
8ca3766a 2721 COMPOSITION_WITH_RULE:
ec6d2bb8 2722 ESC 2 CHAR [ RULE CHAR ] ESC 1
b73bfc1c 2723 COMPOSITION_WITH_ALTCHARS:
ec6d2bb8 2724 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
b73bfc1c 2725 COMPOSITION_WITH_RULE_ALTCHARS:
ec6d2bb8 2726 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
4ed46869
KH		 2727
2728enum iso_code_class_type iso_code_class[256];
2729
df7492f9
KH		 2730#define SAFE_CHARSET_P(coding, id) \
2731 ((id) <= (coding)->max_charset_id \
2732 && (coding)->safe_charsets[id] >= 0)
2733
2734
2735#define SHIFT_OUT_OK(category) \
2736 (CODING_ISO_INITIAL (&coding_categories[category], 1) >= 0)
2737
2738static void
f0064e1f
DL		 2739setup_iso_safe_charsets (attrs)
2740 Lisp_Object attrs;
df7492f9
KH		 2741{
2742 Lisp_Object charset_list, safe_charsets;
2743 Lisp_Object request;
2744 Lisp_Object reg_usage;
2745 Lisp_Object tail;
2746 int reg94, reg96;
2747 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
2748 int max_charset_id;
2749
2750 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2751 if ((flags & CODING_ISO_FLAG_FULL_SUPPORT)
2752 && ! EQ (charset_list, Viso_2022_charset_list))
2753 {
2754 CODING_ATTR_CHARSET_LIST (attrs)
2755 = charset_list = Viso_2022_charset_list;
2756 ASET (attrs, coding_attr_safe_charsets, Qnil);
2757 }
2758
2759 if (STRINGP (AREF (attrs, coding_attr_safe_charsets)))
2760 return;
2761
2762 max_charset_id = 0;
2763 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2764 {
2765 int id = XINT (XCAR (tail));
2766 if (max_charset_id < id)
2767 max_charset_id = id;
2768 }
d46c5b12 2769
df7492f9
KH		 2770 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
2771 make_number (255));
2772 request = AREF (attrs, coding_attr_iso_request);
2773 reg_usage = AREF (attrs, coding_attr_iso_usage);
2774 reg94 = XINT (XCAR (reg_usage));
2775 reg96 = XINT (XCDR (reg_usage));
2776
2777 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2778 {
2779 Lisp_Object id;
2780 Lisp_Object reg;
2781 struct charset *charset;
2782
2783 id = XCAR (tail);
2784 charset = CHARSET_FROM_ID (XINT (id));
bf16eb23 2785 reg = Fcdr (Fassq (id, request));
df7492f9 2786 if (! NILP (reg))
8f924df7 2787 SSET (safe_charsets, XINT (id), XINT (reg));
df7492f9
KH		 2788 else if (charset->iso_chars_96)
2789 {
2790 if (reg96 < 4)
8f924df7 2791 SSET (safe_charsets, XINT (id), reg96);
df7492f9
KH		 2792 }
2793 else
2794 {
2795 if (reg94 < 4)
8f924df7 2796 SSET (safe_charsets, XINT (id), reg94);
df7492f9
KH		 2797 }
2798 }
2799 ASET (attrs, coding_attr_safe_charsets, safe_charsets);
2800}
d46c5b12 2801
b6871cc7 2802
4ed46869 2803/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 2804 Check if a text is encoded in one of ISO-2022 based codig systems.
2805 If it is, return 1, else return 0. */
4ed46869 2806
0a28aafb 2807static int
ff0dacd7 2808detect_coding_iso_2022 (coding, detect_info)
df7492f9 2809 struct coding_system *coding;
ff0dacd7 2810 struct coding_detection_info *detect_info;
4ed46869 2811{
8f924df7
KH		 2812 const unsigned char *src = coding->source, *src_base = src;
2813 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 2814 int multibytep = coding->src_multibyte;
ff0dacd7 2815 int single_shifting = 0;
df7492f9
KH		 2816 int id;
2817 int c, c1;
2818 int consumed_chars = 0;
2819 int i;
ff0dacd7
KH		 2820 int rejected = 0;
2821 int found = 0;
cee53ed4 2822 int composition_count = -1;
ff0dacd7
KH		 2823
2824 detect_info->checked |= CATEGORY_MASK_ISO;
df7492f9
KH		 2825
2826 for (i = coding_category_iso_7; i <= coding_category_iso_8_else; i++)
2827 {
2828 struct coding_system *this = &(coding_categories[i]);
2829 Lisp_Object attrs, val;
2830
c6b278e7
KH		 2831 if (this->id < 0)
2832 continue;
df7492f9
KH		 2833 attrs = CODING_ID_ATTRS (this->id);
2834 if (CODING_ISO_FLAGS (this) & CODING_ISO_FLAG_FULL_SUPPORT
2835 && ! EQ (CODING_ATTR_SAFE_CHARSETS (attrs), Viso_2022_charset_list))
2836 setup_iso_safe_charsets (attrs);
2837 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 2838 this->max_charset_id = SCHARS (val) - 1;
2839 this->safe_charsets = (char *) SDATA (val);
df7492f9
KH		 2840 }
2841
2842 /* A coding system of this category is always ASCII compatible. */
2843 src += coding->head_ascii;
3f003981 2844
ff0dacd7 2845 while (rejected != CATEGORY_MASK_ISO)
4ed46869 2846 {
065e3595 2847 src_base = src;
df7492f9 2848 ONE_MORE_BYTE (c);
4ed46869
KH		 2849 switch (c)
2850 {
2851 case ISO_CODE_ESC:
74383408
KH		 2852 if (inhibit_iso_escape_detection)
2853 break;
f46869e4 2854 single_shifting = 0;
df7492f9 2855 ONE_MORE_BYTE (c);
d46c5b12 2856 if (c >= '(' && c <= '/')
4ed46869 2857 {
bf9cdd4e 2858 /* Designation sequence for a charset of dimension 1. */
df7492f9 2859 ONE_MORE_BYTE (c1);
d46c5b12 2860 if (c1 < ' ' || c1 >= 0x80
df7492f9 2861 || (id = iso_charset_table[0][c >= ','][c1]) < 0)
d46c5b12
KH		 2862 /* Invalid designation sequence. Just ignore. */
2863 break;
bf9cdd4e
KH		 2864 }
2865 else if (c == '$')
2866 {
2867 /* Designation sequence for a charset of dimension 2. */
df7492f9 2868 ONE_MORE_BYTE (c);
bf9cdd4e
KH		 2869 if (c >= '@' && c <= 'B')
2870 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
ff0dacd7 2871 id = iso_charset_table[1][0][c];
bf9cdd4e 2872 else if (c >= '(' && c <= '/')
bcf26d6a 2873 {
df7492f9 2874 ONE_MORE_BYTE (c1);
d46c5b12 2875 if (c1 < ' ' || c1 >= 0x80
df7492f9 2876 || (id = iso_charset_table[1][c >= ','][c1]) < 0)
d46c5b12
KH		 2877 /* Invalid designation sequence. Just ignore. */
2878 break;
bcf26d6a 2879 }
bf9cdd4e 2880 else
ff0dacd7 2881 /* Invalid designation sequence. Just ignore it. */
d46c5b12
KH		 2882 break;
2883 }
ae9ff118 2884 else if (c == 'N' || c == 'O')
d46c5b12 2885 {
ae9ff118 2886 /* ESC <Fe> for SS2 or SS3. */
ff0dacd7
KH		 2887 single_shifting = 1;
2888 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
d46c5b12 2889 break;
4ed46869 2890 }
cee53ed4
KH		 2891 else if (c == '1')
2892 {
2893 /* End of composition. */
2894 if (composition_count < 0
2895 || composition_count > MAX_COMPOSITION_COMPONENTS)
2896 /* Invalid */
2897 break;
2898 composition_count = -1;
2899 found |= CATEGORY_MASK_ISO;
2900 }
ec6d2bb8
KH		 2901 else if (c >= '0' && c <= '4')
2902 {
2903 /* ESC <Fp> for start/end composition. */
cee53ed4 2904 composition_count = 0;
ec6d2bb8
KH		 2905 break;
2906 }
bf9cdd4e 2907 else
df7492f9 2908 {
ff0dacd7 2909 /* Invalid escape sequence. Just ignore it. */
df7492f9
KH		 2910 break;
2911 }
d46c5b12
KH		 2912
2913 /* We found a valid designation sequence for CHARSET. */
ff0dacd7 2914 rejected |= CATEGORY_MASK_ISO_8BIT;
df7492f9
KH		 2915 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7],
2916 id))
ff0dacd7 2917 found |= CATEGORY_MASK_ISO_7;
d46c5b12 2918 else
ff0dacd7 2919 rejected |= CATEGORY_MASK_ISO_7;
df7492f9
KH		 2920 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_tight],
2921 id))
ff0dacd7 2922 found |= CATEGORY_MASK_ISO_7_TIGHT;
d46c5b12 2923 else
ff0dacd7 2924 rejected |= CATEGORY_MASK_ISO_7_TIGHT;
df7492f9
KH		 2925 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_else],
2926 id))
ff0dacd7 2927 found |= CATEGORY_MASK_ISO_7_ELSE;
ae9ff118 2928 else
ff0dacd7 2929 rejected |= CATEGORY_MASK_ISO_7_ELSE;
df7492f9
KH		 2930 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_8_else],
2931 id))
ff0dacd7 2932 found |= CATEGORY_MASK_ISO_8_ELSE;
ae9ff118 2933 else
ff0dacd7 2934 rejected |= CATEGORY_MASK_ISO_8_ELSE;
4ed46869
KH		 2935 break;
2936
4ed46869 2937 case ISO_CODE_SO:
d46c5b12 2938 case ISO_CODE_SI:
ff0dacd7 2939 /* Locking shift out/in. */
74383408
KH		 2940 if (inhibit_iso_escape_detection)
2941 break;
f46869e4 2942 single_shifting = 0;
ff0dacd7 2943 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
d46c5b12
KH		 2944 break;
2945
4ed46869 2946 case ISO_CODE_CSI:
ff0dacd7 2947 /* Control sequence introducer. */
f46869e4 2948 single_shifting = 0;
ff0dacd7
KH		 2949 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2950 found |= CATEGORY_MASK_ISO_8_ELSE;
2951 goto check_extra_latin;
2952
4ed46869
KH		 2953 case ISO_CODE_SS2:
2954 case ISO_CODE_SS3:
ff0dacd7
KH		 2955 /* Single shift. */
2956 if (inhibit_iso_escape_detection)
2957 break;
75e2a253 2958 single_shifting = 0;
ff0dacd7
KH		 2959 rejected |= CATEGORY_MASK_ISO_7BIT;
2960 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
2961 & CODING_ISO_FLAG_SINGLE_SHIFT)
75e2a253 2962 found |= CATEGORY_MASK_ISO_8_1, single_shifting = 1;
ff0dacd7
KH		 2963 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
2964 & CODING_ISO_FLAG_SINGLE_SHIFT)
75e2a253
KH		 2965 found |= CATEGORY_MASK_ISO_8_2, single_shifting = 1;
2966 if (single_shifting)
2967 break;
ff0dacd7 2968 goto check_extra_latin;
4ed46869
KH		 2969
2970 default:
065e3595
KH		 2971 if (c < 0)
2972 continue;
4ed46869 2973 if (c < 0x80)
f46869e4 2974 {
cee53ed4
KH		 2975 if (composition_count >= 0)
2976 composition_count++;
f46869e4
KH		 2977 single_shifting = 0;
2978 break;
2979 }
ff0dacd7 2980 if (c >= 0xA0)
c4825358 2981 {
ff0dacd7
KH		 2982 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2983 found |= CATEGORY_MASK_ISO_8_1;
f46869e4 2984 /* Check the length of succeeding codes of the range
ff0dacd7
KH		 2985 0xA0..0FF. If the byte length is even, we include
2986 CATEGORY_MASK_ISO_8_2 in `found'. We can check this
2987 only when we are not single shifting. */
2988 if (! single_shifting
2989 && ! (rejected & CATEGORY_MASK_ISO_8_2))
f46869e4 2990 {
e17de821 2991 int i = 1;
b73bfc1c
KH		 2992 while (src < src_end)
2993 {
df7492f9 2994 ONE_MORE_BYTE (c);
b73bfc1c
KH		 2995 if (c < 0xA0)
2996 break;
2997 i++;
2998 }
2999
3000 if (i & 1 && src < src_end)
cee53ed4
KH		 3001 {
3002 rejected |= CATEGORY_MASK_ISO_8_2;
3003 if (composition_count >= 0)
3004 composition_count += i;
3005 }
f46869e4 3006 else
cee53ed4
KH		 3007 {
3008 found |= CATEGORY_MASK_ISO_8_2;
3009 if (composition_count >= 0)
3010 composition_count += i / 2;
3011 }
f46869e4 3012 }
ff0dacd7 3013 break;
4ed46869 3014 }
ff0dacd7
KH		 3015 check_extra_latin:
3016 single_shifting = 0;
3017 if (! VECTORP (Vlatin_extra_code_table)
3018 || NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
3019 {
3020 rejected = CATEGORY_MASK_ISO;
3021 break;
3022 }
3023 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
3024 & CODING_ISO_FLAG_LATIN_EXTRA)
3025 found |= CATEGORY_MASK_ISO_8_1;
3026 else
3027 rejected |= CATEGORY_MASK_ISO_8_1;
75e2a253 3028 rejected |= CATEGORY_MASK_ISO_8_2;
4ed46869
KH		 3029 }
3030 }
ff0dacd7
KH		 3031 detect_info->rejected |= CATEGORY_MASK_ISO;
3032 return 0;
4ed46869 3033
df7492f9 3034 no_more_source:
ff0dacd7
KH		 3035 detect_info->rejected |= rejected;
3036 detect_info->found |= (found & ~rejected);
df7492f9 3037 return 1;
4ed46869 3038}
ec6d2bb8 3039
4ed46869 3040
134b9549
KH		 3041/* Set designation state into CODING. Set CHARS_96 to -1 if the
3042 escape sequence should be kept. */
df7492f9
KH		 3043#define DECODE_DESIGNATION(reg, dim, chars_96, final) \
3044 do { \
3045 int id, prev; \
3046 \
3047 if (final < '0' || final >= 128 \
3048 || ((id = ISO_CHARSET_TABLE (dim, chars_96, final)) < 0) \
3049 || !SAFE_CHARSET_P (coding, id)) \
3050 { \
3051 CODING_ISO_DESIGNATION (coding, reg) = -2; \
134b9549
KH		 3052 chars_96 = -1; \
3053 break; \
df7492f9
KH		 3054 } \
3055 prev = CODING_ISO_DESIGNATION (coding, reg); \
bf16eb23
KH		 3056 if (id == charset_jisx0201_roman) \
3057 { \
3058 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
3059 id = charset_ascii; \
3060 } \
3061 else if (id == charset_jisx0208_1978) \
3062 { \
3063 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
3064 id = charset_jisx0208; \
3065 } \
df7492f9
KH		 3066 CODING_ISO_DESIGNATION (coding, reg) = id; \
3067 /* If there was an invalid designation to REG previously, and this \
3068 designation is ASCII to REG, we should keep this designation \
3069 sequence. */ \
3070 if (prev == -2 && id == charset_ascii) \
134b9549 3071 chars_96 = -1; \
4ed46869
KH		 3072 } while (0)
3073
d46c5b12 3074
df7492f9
KH		 3075#define MAYBE_FINISH_COMPOSITION() \
3076 do { \
3077 int i; \
3078 if (composition_state == COMPOSING_NO) \
3079 break; \
3080 /* It is assured that we have enough room for producing \
3081 characters stored in the table `components'. */ \
3082 if (charbuf + component_idx > charbuf_end) \
3083 goto no_more_source; \
3084 composition_state = COMPOSING_NO; \
3085 if (method == COMPOSITION_RELATIVE \
3086 || method == COMPOSITION_WITH_ALTCHARS) \
3087 { \
3088 for (i = 0; i < component_idx; i++) \
3089 *charbuf++ = components[i]; \
3090 char_offset += component_idx; \
3091 } \
3092 else \
3093 { \
3094 for (i = 0; i < component_idx; i += 2) \
3095 *charbuf++ = components[i]; \
3096 char_offset += (component_idx / 2) + 1; \
3097 } \
3098 } while (0)
3099
d46c5b12 3100
aa72b389
KH		 3101/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
3102 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
3103 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
df7492f9
KH		 3104 ESC 3 : altchar composition : ESC 3 CHAR ... ESC 0 CHAR ... ESC 1
3105 ESC 4 : alt&rule composition : ESC 4 CHAR RULE ... CHAR ESC 0 CHAR ... ESC 1
aa72b389 3106 */
ec6d2bb8 3107
df7492f9
KH		 3108#define DECODE_COMPOSITION_START(c1) \
3109 do { \
3110 if (c1 == '0' \
781d7a48 3111 && composition_state == COMPOSING_COMPONENT_RULE) \
df7492f9
KH		 3112 { \
3113 component_len = component_idx; \
3114 composition_state = COMPOSING_CHAR; \
3115 } \
3116 else \
3117 { \
8f924df7 3118 const unsigned char *p; \
df7492f9
KH		 3119 \
3120 MAYBE_FINISH_COMPOSITION (); \
3121 if (charbuf + MAX_COMPOSITION_COMPONENTS > charbuf_end) \
3122 goto no_more_source; \
3123 for (p = src; p < src_end - 1; p++) \
3124 if (*p == ISO_CODE_ESC && p[1] == '1') \
3125 break; \
3126 if (p == src_end - 1) \
3127 { \
cee53ed4
KH		 3128 if (coding->mode & CODING_MODE_LAST_BLOCK) \
3129 goto invalid_code; \
9286b333
KH		 3130 /* The current composition doesn't end in the current \
3131 source. */ \
3132 record_conversion_result \
3133 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
df7492f9
KH		 3134 goto no_more_source; \
3135 } \
3136 \
3137 /* This is surely the start of a composition. */ \
3138 method = (c1 == '0' ? COMPOSITION_RELATIVE \
3139 : c1 == '2' ? COMPOSITION_WITH_RULE \
3140 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
3141 : COMPOSITION_WITH_RULE_ALTCHARS); \
3142 composition_state = (c1 <= '2' ? COMPOSING_CHAR \
3143 : COMPOSING_COMPONENT_CHAR); \
3144 component_idx = component_len = 0; \
3145 } \
ec6d2bb8
KH		 3146 } while (0)
3147
ec6d2bb8 3148
df7492f9
KH		 3149/* Handle compositoin end sequence ESC 1. */
3150
3151#define DECODE_COMPOSITION_END() \
ec6d2bb8 3152 do { \
df7492f9
KH		 3153 int nchars = (component_len > 0 ? component_idx - component_len \
3154 : method == COMPOSITION_RELATIVE ? component_idx \
3155 : (component_idx + 1) / 2); \
3156 int i; \
3157 int *saved_charbuf = charbuf; \
3158 \
69a80ea3 3159 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
df7492f9 3160 if (method != COMPOSITION_RELATIVE) \
ec6d2bb8 3161 { \
df7492f9
KH		 3162 if (component_len == 0) \
3163 for (i = 0; i < component_idx; i++) \
3164 *charbuf++ = components[i]; \
3165 else \
3166 for (i = 0; i < component_len; i++) \
3167 *charbuf++ = components[i]; \
3168 *saved_charbuf = saved_charbuf - charbuf; \
ec6d2bb8 3169 } \
df7492f9
KH		 3170 if (method == COMPOSITION_WITH_RULE) \
3171 for (i = 0; i < component_idx; i += 2, char_offset++) \
3172 *charbuf++ = components[i]; \
ec6d2bb8 3173 else \
df7492f9
KH		 3174 for (i = component_len; i < component_idx; i++, char_offset++) \
3175 *charbuf++ = components[i]; \
3176 coding->annotated = 1; \
3177 composition_state = COMPOSING_NO; \
ec6d2bb8
KH		 3178 } while (0)
3179
df7492f9 3180
ec6d2bb8
KH		 3181/* Decode a composition rule from the byte C1 (and maybe one more byte
3182 from SRC) and store one encoded composition rule in
3183 coding->cmp_data. */
3184
3185#define DECODE_COMPOSITION_RULE(c1) \
3186 do { \
ec6d2bb8
KH		 3187 (c1) -= 32; \
3188 if (c1 < 81) /* old format (before ver.21) */ \
3189 { \
3190 int gref = (c1) / 9; \
3191 int nref = (c1) % 9; \
3192 if (gref == 4) gref = 10; \
3193 if (nref == 4) nref = 10; \
df7492f9 3194 c1 = COMPOSITION_ENCODE_RULE (gref, nref); \
ec6d2bb8 3195 } \
b73bfc1c 3196 else if (c1 < 93) /* new format (after ver.21) */ \
ec6d2bb8
KH		 3197 { \
3198 ONE_MORE_BYTE (c2); \
df7492f9 3199 c1 = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
ec6d2bb8 3200 } \
df7492f9
KH		 3201 else \
3202 c1 = 0; \
ec6d2bb8 3203 } while (0)
88993dfd 3204
d46c5b12 3205
4ed46869
KH		 3206/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3207
b73bfc1c 3208static void
df7492f9 3209decode_coding_iso_2022 (coding)
4ed46869 3210 struct coding_system *coding;
4ed46869 3211{
8f924df7
KH		 3212 const unsigned char *src = coding->source + coding->consumed;
3213 const unsigned char *src_end = coding->source + coding->src_bytes;
3214 const unsigned char *src_base;
69a80ea3 3215 int *charbuf = coding->charbuf + coding->charbuf_used;
ff0dacd7 3216 int *charbuf_end
69a80ea3 3217 = coding->charbuf + coding->charbuf_size - 4 - MAX_ANNOTATION_LENGTH;
df7492f9 3218 int consumed_chars = 0, consumed_chars_base;
df7492f9 3219 int multibytep = coding->src_multibyte;
4ed46869 3220 /* Charsets invoked to graphic plane 0 and 1 respectively. */
df7492f9
KH		 3221 int charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3222 int charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
134b9549 3223 int charset_id_2, charset_id_3;
df7492f9
KH		 3224 struct charset *charset;
3225 int c;
3226 /* For handling composition sequence. */
3227#define COMPOSING_NO 0
3228#define COMPOSING_CHAR 1
3229#define COMPOSING_RULE 2
3230#define COMPOSING_COMPONENT_CHAR 3
3231#define COMPOSING_COMPONENT_RULE 4
3232
3233 int composition_state = COMPOSING_NO;
3234 enum composition_method method;
3235 int components[MAX_COMPOSITION_COMPONENTS * 2 + 1];
3236 int component_idx;
3237 int component_len;
24a73b0a 3238 Lisp_Object attrs, charset_list;
ff0dacd7
KH		 3239 int char_offset = coding->produced_char;
3240 int last_offset = char_offset;
3241 int last_id = charset_ascii;
119852e7
KH		 3242 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
3243 int byte_after_cr = -1;
df7492f9 3244
24a73b0a 3245 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 3246 setup_iso_safe_charsets (attrs);
287c57d7
KH		 3247 /* Charset list may have been changed. */
3248 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
3249 coding->safe_charsets = (char *) SDATA (CODING_ATTR_SAFE_CHARSETS(attrs));
b73bfc1c
KH		 3250
3251 while (1)
4ed46869 3252 {
463f5630 3253 int c1, c2;
b73bfc1c
KH		 3254
3255 src_base = src;
df7492f9
KH		 3256 consumed_chars_base = consumed_chars;
3257
3258 if (charbuf >= charbuf_end)
b71f6f73
KH		 3259 {
3260 if (byte_after_cr >= 0)
3261 src_base--;
3262 break;
3263 }
df7492f9 3264
119852e7
KH		 3265 if (byte_after_cr >= 0)
3266 c1 = byte_after_cr, byte_after_cr = -1;
3267 else
3268 ONE_MORE_BYTE (c1);
065e3595
KH		 3269 if (c1 < 0)
3270 goto invalid_code;
4ed46869 3271
98725083 3272 /* We produce at most one character. */
4ed46869
KH		 3273 switch (iso_code_class [c1])
3274 {
3275 case ISO_0x20_or_0x7F:
df7492f9 3276 if (composition_state != COMPOSING_NO)
ec6d2bb8 3277 {
df7492f9
KH		 3278 if (composition_state == COMPOSING_RULE
3279 || composition_state == COMPOSING_COMPONENT_RULE)
3280 {
cee53ed4
KH		 3281 if (component_idx < MAX_COMPOSITION_COMPONENTS * 2 + 1)
3282 {
3283 DECODE_COMPOSITION_RULE (c1);
3284 components[component_idx++] = c1;
3285 composition_state--;
3286 continue;
3287 }
3288 /* Too long composition. */
3289 MAYBE_FINISH_COMPOSITION ();
df7492f9 3290 }
4ed46869 3291 }
df7492f9
KH		 3292 if (charset_id_0 < 0
3293 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_0)))
781d7a48
KH		 3294 /* This is SPACE or DEL. */
3295 charset = CHARSET_FROM_ID (charset_ascii);
3296 else
3297 charset = CHARSET_FROM_ID (charset_id_0);
3298 break;
4ed46869
KH		 3299
3300 case ISO_graphic_plane_0:
781d7a48 3301 if (composition_state != COMPOSING_NO)
b73bfc1c 3302 {
781d7a48
KH		 3303 if (composition_state == COMPOSING_RULE
3304 || composition_state == COMPOSING_COMPONENT_RULE)
3305 {
cee53ed4
KH		 3306 if (component_idx < MAX_COMPOSITION_COMPONENTS * 2 + 1)
3307 {
3308 DECODE_COMPOSITION_RULE (c1);
3309 components[component_idx++] = c1;
3310 composition_state--;
3311 continue;
3312 }
3313 MAYBE_FINISH_COMPOSITION ();
781d7a48 3314 }
b73bfc1c 3315 }
134b9549
KH		 3316 if (charset_id_0 < 0)
3317 charset = CHARSET_FROM_ID (charset_ascii);
3318 else
3319 charset = CHARSET_FROM_ID (charset_id_0);
4ed46869
KH		 3320 break;
3321
3322 case ISO_0xA0_or_0xFF:
df7492f9
KH		 3323 if (charset_id_1 < 0
3324 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_1))
3325 || CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS)
3326 goto invalid_code;
4ed46869
KH		 3327 /* This is a graphic character, we fall down ... */
3328
3329 case ISO_graphic_plane_1:
df7492f9
KH		 3330 if (charset_id_1 < 0)
3331 goto invalid_code;
3332 charset = CHARSET_FROM_ID (charset_id_1);
4ed46869
KH		 3333 break;
3334
df7492f9 3335 case ISO_control_0:
119852e7
KH		 3336 if (eol_crlf && c1 == '\r')
3337 ONE_MORE_BYTE (byte_after_cr);
df7492f9
KH		 3338 MAYBE_FINISH_COMPOSITION ();
3339 charset = CHARSET_FROM_ID (charset_ascii);
4ed46869
KH		 3340 break;
3341
df7492f9
KH		 3342 case ISO_control_1:
3343 MAYBE_FINISH_COMPOSITION ();
3344 goto invalid_code;
3345
4ed46869 3346 case ISO_shift_out:
df7492f9
KH		 3347 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3348 || CODING_ISO_DESIGNATION (coding, 1) < 0)
3349 goto invalid_code;
3350 CODING_ISO_INVOCATION (coding, 0) = 1;
3351 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3352 continue;
4ed46869
KH		 3353
3354 case ISO_shift_in:
df7492f9
KH		 3355 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT))
3356 goto invalid_code;
3357 CODING_ISO_INVOCATION (coding, 0) = 0;
3358 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3359 continue;
4ed46869
KH		 3360
3361 case ISO_single_shift_2_7:
3362 case ISO_single_shift_2:
df7492f9
KH		 3363 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3364 goto invalid_code;
4ed46869
KH		 3365 /* SS2 is handled as an escape sequence of ESC 'N' */
3366 c1 = 'N';
3367 goto label_escape_sequence;
3368
3369 case ISO_single_shift_3:
df7492f9
KH		 3370 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3371 goto invalid_code;
4ed46869
KH		 3372 /* SS2 is handled as an escape sequence of ESC 'O' */
3373 c1 = 'O';
3374 goto label_escape_sequence;
3375
3376 case ISO_control_sequence_introducer:
3377 /* CSI is handled as an escape sequence of ESC '[' ... */
3378 c1 = '[';
3379 goto label_escape_sequence;
3380
3381 case ISO_escape:
3382 ONE_MORE_BYTE (c1);
3383 label_escape_sequence:
df7492f9 3384 /* Escape sequences handled here are invocation,
4ed46869
KH		 3385 designation, direction specification, and character
3386 composition specification. */
3387 switch (c1)
3388 {
3389 case '&': /* revision of following character set */
3390 ONE_MORE_BYTE (c1);
3391 if (!(c1 >= '@' && c1 <= '~'))
df7492f9 3392 goto invalid_code;
4ed46869
KH		 3393 ONE_MORE_BYTE (c1);
3394 if (c1 != ISO_CODE_ESC)
df7492f9 3395 goto invalid_code;
4ed46869
KH		 3396 ONE_MORE_BYTE (c1);
3397 goto label_escape_sequence;
3398
3399 case '$': /* designation of 2-byte character set */
df7492f9
KH		 3400 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3401 goto invalid_code;
134b9549
KH		 3402 {
3403 int reg, chars96;
3404
3405 ONE_MORE_BYTE (c1);
3406 if (c1 >= '@' && c1 <= 'B')
3407 { /* designation of JISX0208.1978, GB2312.1980,
88993dfd 3408 or JISX0208.1980 */
134b9549
KH		 3409 reg = 0, chars96 = 0;
3410 }
3411 else if (c1 >= 0x28 && c1 <= 0x2B)
3412 { /* designation of DIMENSION2_CHARS94 character set */
3413 reg = c1 - 0x28, chars96 = 0;
3414 ONE_MORE_BYTE (c1);
3415 }
3416 else if (c1 >= 0x2C && c1 <= 0x2F)
3417 { /* designation of DIMENSION2_CHARS96 character set */
3418 reg = c1 - 0x2C, chars96 = 1;
3419 ONE_MORE_BYTE (c1);
3420 }
3421 else
3422 goto invalid_code;
3423 DECODE_DESIGNATION (reg, 2, chars96, c1);
3424 /* We must update these variables now. */
3425 if (reg == 0)
3426 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3427 else if (reg == 1)
3428 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3429 if (chars96 < 0)
3430 goto invalid_code;
3431 }
b73bfc1c 3432 continue;
4ed46869
KH		 3433
3434 case 'n': /* invocation of locking-shift-2 */
df7492f9
KH		 3435 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3436 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3437 goto invalid_code;
3438 CODING_ISO_INVOCATION (coding, 0) = 2;
3439 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3440 continue;
4ed46869
KH		 3441
3442 case 'o': /* invocation of locking-shift-3 */
df7492f9
KH		 3443 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3444 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3445 goto invalid_code;
3446 CODING_ISO_INVOCATION (coding, 0) = 3;
3447 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3448 continue;
4ed46869
KH		 3449
3450 case 'N': /* invocation of single-shift-2 */
df7492f9
KH		 3451 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3452 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3453 goto invalid_code;
134b9549
KH		 3454 charset_id_2 = CODING_ISO_DESIGNATION (coding, 2);
3455 if (charset_id_2 < 0)
3456 charset = CHARSET_FROM_ID (charset_ascii);
3457 else
3458 charset = CHARSET_FROM_ID (charset_id_2);
b73bfc1c 3459 ONE_MORE_BYTE (c1);
e7046a18 3460 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3461 goto invalid_code;
4ed46869
KH		 3462 break;
3463
3464 case 'O': /* invocation of single-shift-3 */
df7492f9
KH		 3465 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3466 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3467 goto invalid_code;
134b9549
KH		 3468 charset_id_3 = CODING_ISO_DESIGNATION (coding, 3);
3469 if (charset_id_3 < 0)
3470 charset = CHARSET_FROM_ID (charset_ascii);
3471 else
3472 charset = CHARSET_FROM_ID (charset_id_3);
b73bfc1c 3473 ONE_MORE_BYTE (c1);
e7046a18 3474 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3475 goto invalid_code;
4ed46869
KH		 3476 break;
3477
ec6d2bb8 3478 case '0': case '2': case '3': case '4': /* start composition */
df7492f9
KH		 3479 if (! (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK))
3480 goto invalid_code;
ec6d2bb8 3481 DECODE_COMPOSITION_START (c1);
b73bfc1c 3482 continue;
4ed46869 3483
ec6d2bb8 3484 case '1': /* end composition */
df7492f9
KH		 3485 if (composition_state == COMPOSING_NO)
3486 goto invalid_code;
3487 DECODE_COMPOSITION_END ();
b73bfc1c 3488 continue;
4ed46869
KH		 3489
3490 case '[': /* specification of direction */
df7492f9
KH		 3491 if (! CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DIRECTION)
3492 goto invalid_code;
4ed46869 3493 /* For the moment, nested direction is not supported.
d46c5b12 3494 So, `coding->mode & CODING_MODE_DIRECTION' zero means
df7492f9 3495 left-to-right, and nozero means right-to-left. */
4ed46869
KH		 3496 ONE_MORE_BYTE (c1);
3497 switch (c1)
3498 {
3499 case ']': /* end of the current direction */
d46c5b12 3500 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869
KH		 3501
3502 case '0': /* end of the current direction */
3503 case '1': /* start of left-to-right direction */
3504 ONE_MORE_BYTE (c1);
3505 if (c1 == ']')
d46c5b12 3506 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869 3507 else
df7492f9 3508 goto invalid_code;
4ed46869
KH		 3509 break;
3510
3511 case '2': /* start of right-to-left direction */
3512 ONE_MORE_BYTE (c1);
3513 if (c1 == ']')
d46c5b12 3514 coding->mode |= CODING_MODE_DIRECTION;
4ed46869 3515 else
df7492f9 3516 goto invalid_code;
4ed46869
KH		 3517 break;
3518
3519 default:
df7492f9 3520 goto invalid_code;
4ed46869 3521 }
b73bfc1c 3522 continue;
4ed46869 3523
103e0180 3524 case '%':
103e0180
KH		 3525 ONE_MORE_BYTE (c1);
3526 if (c1 == '/')
3527 {
3528 /* CTEXT extended segment:
3529 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
3530 We keep these bytes as is for the moment.
3531 They may be decoded by post-read-conversion. */
3532 int dim, M, L;
4776e638 3533 int size;
8f924df7 3534
103e0180
KH		 3535 ONE_MORE_BYTE (dim);
3536 ONE_MORE_BYTE (M);
3537 ONE_MORE_BYTE (L);
3538 size = ((M - 128) * 128) + (L - 128);
4776e638
KH		 3539 if (charbuf + 8 + size > charbuf_end)
3540 goto break_loop;
3541 *charbuf++ = ISO_CODE_ESC;
3542 *charbuf++ = '%';
3543 *charbuf++ = '/';
3544 *charbuf++ = dim;
3545 *charbuf++ = BYTE8_TO_CHAR (M);
3546 *charbuf++ = BYTE8_TO_CHAR (L);
103e0180
KH		 3547 while (size-- > 0)
3548 {
3549 ONE_MORE_BYTE (c1);
4776e638 3550 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
103e0180 3551 }
103e0180
KH		 3552 }
3553 else if (c1 == 'G')
3554 {
103e0180
KH		 3555 /* XFree86 extension for embedding UTF-8 in CTEXT:
3556 ESC % G --UTF-8-BYTES-- ESC % @
3557 We keep these bytes as is for the moment.
3558 They may be decoded by post-read-conversion. */
4776e638
KH		 3559 int *p = charbuf;
3560
3561 if (p + 6 > charbuf_end)
3562 goto break_loop;
3563 *p++ = ISO_CODE_ESC;
3564 *p++ = '%';
3565 *p++ = 'G';
3566 while (p < charbuf_end)
103e0180
KH		 3567 {
3568 ONE_MORE_BYTE (c1);
3569 if (c1 == ISO_CODE_ESC
3570 && src + 1 < src_end
3571 && src[0] == '%'
3572 && src[1] == '@')
9ffd559c
KH		 3573 {
3574 src += 2;
3575 break;
3576 }
4776e638 3577 *p++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
103e0180 3578 }
4776e638
KH		 3579 if (p + 3 > charbuf_end)
3580 goto break_loop;
3581 *p++ = ISO_CODE_ESC;
3582 *p++ = '%';
3583 *p++ = '@';
3584 charbuf = p;
103e0180
KH		 3585 }
3586 else
4776e638 3587 goto invalid_code;
103e0180 3588 continue;
4776e638 3589 break;
103e0180 3590
4ed46869 3591 default:
df7492f9
KH		 3592 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3593 goto invalid_code;
134b9549
KH		 3594 {
3595 int reg, chars96;
3596
3597 if (c1 >= 0x28 && c1 <= 0x2B)
3598 { /* designation of DIMENSION1_CHARS94 character set */
3599 reg = c1 - 0x28, chars96 = 0;
3600 ONE_MORE_BYTE (c1);
3601 }
3602 else if (c1 >= 0x2C && c1 <= 0x2F)
3603 { /* designation of DIMENSION1_CHARS96 character set */
3604 reg = c1 - 0x2C, chars96 = 1;
3605 ONE_MORE_BYTE (c1);
3606 }
3607 else
3608 goto invalid_code;
3609 DECODE_DESIGNATION (reg, 1, chars96, c1);
3610 /* We must update these variables now. */
3611 if (reg == 0)
3612 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3613 else if (reg == 1)
3614 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3615 if (chars96 < 0)
3616 goto invalid_code;
3617 }
b73bfc1c 3618 continue;
4ed46869 3619 }
b73bfc1c 3620 }
4ed46869 3621
ff0dacd7
KH		 3622 if (charset->id != charset_ascii
3623 && last_id != charset->id)
3624 {
3625 if (last_id != charset_ascii)
69a80ea3 3626 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
ff0dacd7
KH		 3627 last_id = charset->id;
3628 last_offset = char_offset;
3629 }
3630
b73bfc1c 3631 /* Now we know CHARSET and 1st position code C1 of a character.
df7492f9
KH		 3632 Produce a decoded character while getting 2nd position code
3633 C2 if necessary. */
3634 c1 &= 0x7F;
3635 if (CHARSET_DIMENSION (charset) > 1)
b73bfc1c
KH		 3636 {
3637 ONE_MORE_BYTE (c2);
df7492f9 3638 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
b73bfc1c 3639 /* C2 is not in a valid range. */
df7492f9
KH		 3640 goto invalid_code;
3641 c1 = (c1 << 8) | (c2 & 0x7F);
3642 if (CHARSET_DIMENSION (charset) > 2)
3643 {
3644 ONE_MORE_BYTE (c2);
3645 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
3646 /* C2 is not in a valid range. */
3647 goto invalid_code;
3648 c1 = (c1 << 8) | (c2 & 0x7F);
3649 }
3650 }
3651
3652 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c1, c);
3653 if (c < 0)
3654 {
3655 MAYBE_FINISH_COMPOSITION ();
3656 for (; src_base < src; src_base++, char_offset++)
3657 {
3658 if (ASCII_BYTE_P (*src_base))
3659 *charbuf++ = *src_base;
3660 else
3661 *charbuf++ = BYTE8_TO_CHAR (*src_base);
3662 }
3663 }
3664 else if (composition_state == COMPOSING_NO)
3665 {
3666 *charbuf++ = c;
3667 char_offset++;
4ed46869 3668 }
df7492f9 3669 else
781d7a48 3670 {
cee53ed4
KH		 3671 if (component_idx < MAX_COMPOSITION_COMPONENTS * 2 + 1)
3672 {
3673 components[component_idx++] = c;
3674 if (method == COMPOSITION_WITH_RULE
3675 || (method == COMPOSITION_WITH_RULE_ALTCHARS
3676 && composition_state == COMPOSING_COMPONENT_CHAR))
3677 composition_state++;
3678 }
3679 else
3680 {
3681 MAYBE_FINISH_COMPOSITION ();
3682 *charbuf++ = c;
3683 char_offset++;
3684 }
4ed46869
KH		 3685 }
3686 continue;
3687
df7492f9
KH		 3688 invalid_code:
3689 MAYBE_FINISH_COMPOSITION ();
4ed46869 3690 src = src_base;
df7492f9
KH		 3691 consumed_chars = consumed_chars_base;
3692 ONE_MORE_BYTE (c);
065e3595 3693 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 3694 char_offset++;
df7492f9 3695 coding->errors++;
4776e638
KH		 3696 continue;
3697
3698 break_loop:
3699 break;
4ed46869 3700 }
fb88bf2d 3701
df7492f9 3702 no_more_source:
ff0dacd7 3703 if (last_id != charset_ascii)
69a80ea3 3704 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 3705 coding->consumed_char += consumed_chars_base;
3706 coding->consumed = src_base - coding->source;
3707 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 3708}
3709
b73bfc1c 3710
f4dee582 3711/* ISO2022 encoding stuff. */
4ed46869
KH		 3712
3713/*
f4dee582 3714 It is not enough to say just "ISO2022" on encoding, we have to
df7492f9 3715 specify more details. In Emacs, each coding system of ISO2022
4ed46869 3716 variant has the following specifications:
df7492f9 3717 1. Initial designation to G0 thru G3.
4ed46869
KH		 3718 2. Allows short-form designation?
3719 3. ASCII should be designated to G0 before control characters?
3720 4. ASCII should be designated to G0 at end of line?
3721 5. 7-bit environment or 8-bit environment?
3722 6. Use locking-shift?
3723 7. Use Single-shift?
3724 And the following two are only for Japanese:
3725 8. Use ASCII in place of JIS0201-1976-Roman?
3726 9. Use JISX0208-1983 in place of JISX0208-1978?
df7492f9
KH		 3727 These specifications are encoded in CODING_ISO_FLAGS (coding) as flag bits
3728 defined by macros CODING_ISO_FLAG_XXX. See `coding.h' for more
f4dee582 3729 details.
4ed46869
KH		 3730*/
3731
3732/* Produce codes (escape sequence) for designating CHARSET to graphic
b73bfc1c
KH		 3733 register REG at DST, and increment DST. If <final-char> of CHARSET is
3734 '@', 'A', or 'B' and the coding system CODING allows, produce
3735 designation sequence of short-form. */
4ed46869
KH		 3736
3737#define ENCODE_DESIGNATION(charset, reg, coding) \
3738 do { \
df7492f9 3739 unsigned char final_char = CHARSET_ISO_FINAL (charset); \
4ed46869
KH		 3740 char *intermediate_char_94 = "()*+"; \
3741 char *intermediate_char_96 = ",-./"; \
df7492f9
KH		 3742 int revision = -1; \
3743 int c; \
3744 \
3745 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_REVISION) \
c197f191 3746 revision = CHARSET_ISO_REVISION (charset); \
df7492f9
KH		 3747 \
3748 if (revision >= 0) \
70c22245 3749 { \
df7492f9
KH		 3750 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '&'); \
3751 EMIT_ONE_BYTE ('@' + revision); \
4ed46869 3752 } \
df7492f9 3753 EMIT_ONE_ASCII_BYTE (ISO_CODE_ESC); \
4ed46869
KH		 3754 if (CHARSET_DIMENSION (charset) == 1) \
3755 { \
df7492f9
KH		 3756 if (! CHARSET_ISO_CHARS_96 (charset)) \
3757 c = intermediate_char_94[reg]; \
4ed46869 3758 else \
df7492f9
KH		 3759 c = intermediate_char_96[reg]; \
3760 EMIT_ONE_ASCII_BYTE (c); \
4ed46869
KH		 3761 } \
3762 else \
3763 { \
df7492f9
KH		 3764 EMIT_ONE_ASCII_BYTE ('$'); \
3765 if (! CHARSET_ISO_CHARS_96 (charset)) \
4ed46869 3766 { \
df7492f9 3767 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LONG_FORM \
b73bfc1c
KH		 3768 || reg != 0 \
3769 || final_char < '@' || final_char > 'B') \
df7492f9 3770 EMIT_ONE_ASCII_BYTE (intermediate_char_94[reg]); \
4ed46869
KH		 3771 } \
3772 else \
df7492f9 3773 EMIT_ONE_ASCII_BYTE (intermediate_char_96[reg]); \
4ed46869 3774 } \
df7492f9
KH		 3775 EMIT_ONE_ASCII_BYTE (final_char); \
3776 \
3777 CODING_ISO_DESIGNATION (coding, reg) = CHARSET_ID (charset); \
4ed46869
KH		 3778 } while (0)
3779
df7492f9 3780
4ed46869
KH		 3781/* The following two macros produce codes (control character or escape
3782 sequence) for ISO2022 single-shift functions (single-shift-2 and
3783 single-shift-3). */
3784
df7492f9
KH		 3785#define ENCODE_SINGLE_SHIFT_2 \
3786 do { \
3787 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3788 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'N'); \
3789 else \
3790 EMIT_ONE_BYTE (ISO_CODE_SS2); \
3791 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 3792 } while (0)
3793
df7492f9
KH		 3794
3795#define ENCODE_SINGLE_SHIFT_3 \
3796 do { \
3797 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3798 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'O'); \
3799 else \
3800 EMIT_ONE_BYTE (ISO_CODE_SS3); \
3801 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 3802 } while (0)
3803
df7492f9 3804
4ed46869
KH		 3805/* The following four macros produce codes (control character or
3806 escape sequence) for ISO2022 locking-shift functions (shift-in,
3807 shift-out, locking-shift-2, and locking-shift-3). */
3808
df7492f9
KH		 3809#define ENCODE_SHIFT_IN \
3810 do { \
3811 EMIT_ONE_ASCII_BYTE (ISO_CODE_SI); \
3812 CODING_ISO_INVOCATION (coding, 0) = 0; \
4ed46869
KH		 3813 } while (0)
3814
df7492f9
KH		 3815
3816#define ENCODE_SHIFT_OUT \
3817 do { \
3818 EMIT_ONE_ASCII_BYTE (ISO_CODE_SO); \
3819 CODING_ISO_INVOCATION (coding, 0) = 1; \
4ed46869
KH		 3820 } while (0)
3821
df7492f9
KH		 3822
3823#define ENCODE_LOCKING_SHIFT_2 \
3824 do { \
3825 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3826 CODING_ISO_INVOCATION (coding, 0) = 2; \
4ed46869
KH		 3827 } while (0)
3828
df7492f9
KH		 3829
3830#define ENCODE_LOCKING_SHIFT_3 \
3831 do { \
3832 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3833 CODING_ISO_INVOCATION (coding, 0) = 3; \
4ed46869
KH		 3834 } while (0)
3835
df7492f9 3836
f4dee582
RS		 3837/* Produce codes for a DIMENSION1 character whose character set is
3838 CHARSET and whose position-code is C1. Designation and invocation
4ed46869
KH		 3839 sequences are also produced in advance if necessary. */
3840
6e85d753
KH		 3841#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
3842 do { \
df7492f9 3843 int id = CHARSET_ID (charset); \
bf16eb23
KH		 3844 \
3845 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
3846 && id == charset_ascii) \
3847 { \
3848 id = charset_jisx0201_roman; \
3849 charset = CHARSET_FROM_ID (id); \
3850 } \
3851 \
df7492f9 3852 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 3853 { \
df7492f9
KH		 3854 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3855 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753 3856 else \
df7492f9
KH		 3857 EMIT_ONE_BYTE (c1 | 0x80); \
3858 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 3859 break; \
3860 } \
df7492f9 3861 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 3862 { \
df7492f9 3863 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753
KH		 3864 break; \
3865 } \
df7492f9 3866 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 3867 { \
df7492f9 3868 EMIT_ONE_BYTE (c1 | 0x80); \
6e85d753
KH		 3869 break; \
3870 } \
6e85d753
KH		 3871 else \
3872 /* Since CHARSET is not yet invoked to any graphic planes, we \
3873 must invoke it, or, at first, designate it to some graphic \
3874 register. Then repeat the loop to actually produce the \
3875 character. */ \
df7492f9
KH		 3876 dst = encode_invocation_designation (charset, coding, dst, \
3877 &produced_chars); \
4ed46869
KH		 3878 } while (1)
3879
df7492f9 3880
f4dee582
RS		 3881/* Produce codes for a DIMENSION2 character whose character set is
3882 CHARSET and whose position-codes are C1 and C2. Designation and
4ed46869
KH		 3883 invocation codes are also produced in advance if necessary. */
3884
6e85d753
KH		 3885#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
3886 do { \
df7492f9 3887 int id = CHARSET_ID (charset); \
bf16eb23
KH		 3888 \
3889 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
3890 && id == charset_jisx0208) \
3891 { \
3892 id = charset_jisx0208_1978; \
3893 charset = CHARSET_FROM_ID (id); \
3894 } \
3895 \
df7492f9 3896 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 3897 { \
df7492f9
KH		 3898 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3899 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753 3900 else \
df7492f9
KH		 3901 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
3902 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 3903 break; \
3904 } \
df7492f9 3905 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 3906 { \
df7492f9 3907 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753
KH		 3908 break; \
3909 } \
df7492f9 3910 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 3911 { \
df7492f9 3912 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
6e85d753
KH		 3913 break; \
3914 } \
6e85d753
KH		 3915 else \
3916 /* Since CHARSET is not yet invoked to any graphic planes, we \
3917 must invoke it, or, at first, designate it to some graphic \
3918 register. Then repeat the loop to actually produce the \
3919 character. */ \
df7492f9
KH		 3920 dst = encode_invocation_designation (charset, coding, dst, \
3921 &produced_chars); \
4ed46869
KH		 3922 } while (1)
3923
05e6f5dc 3924
df7492f9
KH		 3925#define ENCODE_ISO_CHARACTER(charset, c) \
3926 do { \
3927 int code = ENCODE_CHAR ((charset),(c)); \
3928 \
3929 if (CHARSET_DIMENSION (charset) == 1) \
3930 ENCODE_ISO_CHARACTER_DIMENSION1 ((charset), code); \
3931 else \
3932 ENCODE_ISO_CHARACTER_DIMENSION2 ((charset), code >> 8, code & 0xFF); \
84fbb8a0 3933 } while (0)
bdd9fb48 3934
05e6f5dc 3935
4ed46869 3936/* Produce designation and invocation codes at a place pointed by DST
df7492f9 3937 to use CHARSET. The element `spec.iso_2022' of *CODING is updated.
4ed46869
KH		 3938 Return new DST. */
3939
3940unsigned char *
df7492f9
KH		 3941encode_invocation_designation (charset, coding, dst, p_nchars)
3942 struct charset *charset;
4ed46869
KH		 3943 struct coding_system *coding;
3944 unsigned char *dst;
df7492f9 3945 int *p_nchars;
4ed46869 3946{
df7492f9
KH		 3947 int multibytep = coding->dst_multibyte;
3948 int produced_chars = *p_nchars;
4ed46869 3949 int reg; /* graphic register number */
df7492f9 3950 int id = CHARSET_ID (charset);
4ed46869
KH		 3951
3952 /* At first, check designations. */
3953 for (reg = 0; reg < 4; reg++)
df7492f9 3954 if (id == CODING_ISO_DESIGNATION (coding, reg))
4ed46869
KH		 3955 break;
3956
3957 if (reg >= 4)
3958 {
3959 /* CHARSET is not yet designated to any graphic registers. */
3960 /* At first check the requested designation. */
df7492f9
KH		 3961 reg = CODING_ISO_REQUEST (coding, id);
3962 if (reg < 0)
1ba9e4ab
KH		 3963 /* Since CHARSET requests no special designation, designate it
3964 to graphic register 0. */
4ed46869
KH		 3965 reg = 0;
3966
3967 ENCODE_DESIGNATION (charset, reg, coding);
3968 }
3969
df7492f9
KH		 3970 if (CODING_ISO_INVOCATION (coding, 0) != reg
3971 && CODING_ISO_INVOCATION (coding, 1) != reg)
4ed46869
KH		 3972 {
3973 /* Since the graphic register REG is not invoked to any graphic
3974 planes, invoke it to graphic plane 0. */
3975 switch (reg)
3976 {
3977 case 0: /* graphic register 0 */
3978 ENCODE_SHIFT_IN;
3979 break;
3980
3981 case 1: /* graphic register 1 */
3982 ENCODE_SHIFT_OUT;
3983 break;
3984
3985 case 2: /* graphic register 2 */
df7492f9 3986 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 3987 ENCODE_SINGLE_SHIFT_2;
3988 else
3989 ENCODE_LOCKING_SHIFT_2;
3990 break;
3991
3992 case 3: /* graphic register 3 */
df7492f9 3993 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 3994 ENCODE_SINGLE_SHIFT_3;
3995 else
3996 ENCODE_LOCKING_SHIFT_3;
3997 break;
3998 }
3999 }
b73bfc1c 4000
df7492f9 4001 *p_nchars = produced_chars;
4ed46869
KH		 4002 return dst;
4003}
4004
df7492f9
KH		 4005/* The following three macros produce codes for indicating direction
4006 of text. */
4007#define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
ec6d2bb8 4008 do { \
df7492f9
KH		 4009 if (CODING_ISO_FLAGS (coding) == CODING_ISO_FLAG_SEVEN_BITS) \
4010 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '['); \
ec6d2bb8 4011 else \
df7492f9 4012 EMIT_ONE_BYTE (ISO_CODE_CSI); \
ec6d2bb8
KH		 4013 } while (0)
4014
ec6d2bb8 4015
df7492f9
KH		 4016#define ENCODE_DIRECTION_R2L() \
4017 do { \
4018 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
4019 EMIT_TWO_ASCII_BYTES ('2', ']'); \
ec6d2bb8
KH		 4020 } while (0)
4021
ec6d2bb8 4022
df7492f9 4023#define ENCODE_DIRECTION_L2R() \
ec6d2bb8 4024 do { \
df7492f9
KH		 4025 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
4026 EMIT_TWO_ASCII_BYTES ('0', ']'); \
ec6d2bb8 4027 } while (0)
4ed46869 4028
4ed46869
KH		 4029
4030/* Produce codes for designation and invocation to reset the graphic
4031 planes and registers to initial state. */
df7492f9
KH		 4032#define ENCODE_RESET_PLANE_AND_REGISTER() \
4033 do { \
4034 int reg; \
4035 struct charset *charset; \
4036 \
4037 if (CODING_ISO_INVOCATION (coding, 0) != 0) \
4038 ENCODE_SHIFT_IN; \
4039 for (reg = 0; reg < 4; reg++) \
4040 if (CODING_ISO_INITIAL (coding, reg) >= 0 \
4041 && (CODING_ISO_DESIGNATION (coding, reg) \
4042 != CODING_ISO_INITIAL (coding, reg))) \
4043 { \
4044 charset = CHARSET_FROM_ID (CODING_ISO_INITIAL (coding, reg)); \
4045 ENCODE_DESIGNATION (charset, reg, coding); \
4046 } \
4ed46869
KH		 4047 } while (0)
4048
df7492f9 4049
bdd9fb48 4050/* Produce designation sequences of charsets in the line started from
b73bfc1c 4051 SRC to a place pointed by DST, and return updated DST.
bdd9fb48
KH		 4052
4053 If the current block ends before any end-of-line, we may fail to
d46c5b12
KH		 4054 find all the necessary designations. */
4055
b73bfc1c 4056static unsigned char *
df7492f9 4057encode_designation_at_bol (coding, charbuf, charbuf_end, dst)
e0e989f6 4058 struct coding_system *coding;
df7492f9
KH		 4059 int *charbuf, *charbuf_end;
4060 unsigned char *dst;
e0e989f6 4061{
df7492f9 4062 struct charset *charset;
bdd9fb48
KH		 4063 /* Table of charsets to be designated to each graphic register. */
4064 int r[4];
df7492f9
KH		 4065 int c, found = 0, reg;
4066 int produced_chars = 0;
4067 int multibytep = coding->dst_multibyte;
4068 Lisp_Object attrs;
4069 Lisp_Object charset_list;
4070
4071 attrs = CODING_ID_ATTRS (coding->id);
4072 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
4073 if (EQ (charset_list, Qiso_2022))
4074 charset_list = Viso_2022_charset_list;
bdd9fb48
KH		 4075
4076 for (reg = 0; reg < 4; reg++)
4077 r[reg] = -1;
4078
b73bfc1c 4079 while (found < 4)
e0e989f6 4080 {
df7492f9
KH		 4081 int id;
4082
4083 c = *charbuf++;
b73bfc1c
KH		 4084 if (c == '\n')
4085 break;
df7492f9
KH		 4086 charset = char_charset (c, charset_list, NULL);
4087 id = CHARSET_ID (charset);
4088 reg = CODING_ISO_REQUEST (coding, id);
4089 if (reg >= 0 && r[reg] < 0)
bdd9fb48
KH		 4090 {
4091 found++;
df7492f9 4092 r[reg] = id;
bdd9fb48 4093 }
bdd9fb48
KH		 4094 }
4095
4096 if (found)
4097 {
4098 for (reg = 0; reg < 4; reg++)
4099 if (r[reg] >= 0
df7492f9
KH		 4100 && CODING_ISO_DESIGNATION (coding, reg) != r[reg])
4101 ENCODE_DESIGNATION (CHARSET_FROM_ID (r[reg]), reg, coding);
e0e989f6 4102 }
b73bfc1c
KH		 4103
4104 return dst;
e0e989f6
KH		 4105}
4106
4ed46869
KH		 4107/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
4108
df7492f9
KH		 4109static int
4110encode_coding_iso_2022 (coding)
4ed46869 4111 struct coding_system *coding;
4ed46869 4112{
df7492f9
KH		 4113 int multibytep = coding->dst_multibyte;
4114 int *charbuf = coding->charbuf;
4115 int *charbuf_end = charbuf + coding->charbuf_used;
4116 unsigned char *dst = coding->destination + coding->produced;
4117 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4118 int safe_room = 16;
4119 int bol_designation
4120 = (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL
4121 && CODING_ISO_BOL (coding));
4122 int produced_chars = 0;
4123 Lisp_Object attrs, eol_type, charset_list;
4124 int ascii_compatible;
b73bfc1c 4125 int c;
ff0dacd7 4126 int preferred_charset_id = -1;
05e6f5dc 4127
24a73b0a
KH		 4128 CODING_GET_INFO (coding, attrs, charset_list);
4129 eol_type = CODING_ID_EOL_TYPE (coding->id);
4130 if (VECTORP (eol_type))
4131 eol_type = Qunix;
4132
004068e4 4133 setup_iso_safe_charsets (attrs);
ff0dacd7 4134 /* Charset list may have been changed. */
287c57d7 4135 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8f924df7 4136 coding->safe_charsets = (char *) SDATA (CODING_ATTR_SAFE_CHARSETS(attrs));
0eecad43 4137
df7492f9 4138 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
bdd9fb48 4139
df7492f9 4140 while (charbuf < charbuf_end)
4ed46869 4141 {
df7492f9 4142 ASSURE_DESTINATION (safe_room);
b73bfc1c 4143
df7492f9 4144 if (bol_designation)
b73bfc1c 4145 {
df7492f9 4146 unsigned char *dst_prev = dst;
4ed46869 4147
bdd9fb48 4148 /* We have to produce designation sequences if any now. */
df7492f9
KH		 4149 dst = encode_designation_at_bol (coding, charbuf, charbuf_end, dst);
4150 bol_designation = 0;
4151 /* We are sure that designation sequences are all ASCII bytes. */
4152 produced_chars += dst - dst_prev;
e0e989f6
KH		 4153 }
4154
df7492f9 4155 c = *charbuf++;
ec6d2bb8 4156
ff0dacd7
KH		 4157 if (c < 0)
4158 {
4159 /* Handle an annotation. */
4160 switch (*charbuf)
ec6d2bb8 4161 {
ff0dacd7
KH		 4162 case CODING_ANNOTATE_COMPOSITION_MASK:
4163 /* Not yet implemented. */
4164 break;
4165 case CODING_ANNOTATE_CHARSET_MASK:
cf7dfdf5 4166 preferred_charset_id = charbuf[2];
ff0dacd7
KH		 4167 if (preferred_charset_id >= 0
4168 && NILP (Fmemq (make_number (preferred_charset_id),
4169 charset_list)))
4170 preferred_charset_id = -1;
4171 break;
4172 default:
4173 abort ();
4ed46869 4174 }
ff0dacd7
KH		 4175 charbuf += -c - 1;
4176 continue;
4ed46869 4177 }
ec6d2bb8 4178
b73bfc1c
KH		 4179 /* Now encode the character C. */
4180 if (c < 0x20 || c == 0x7F)
4181 {
df7492f9
KH		 4182 if (c == '\n'
4183 || (c == '\r' && EQ (eol_type, Qmac)))
19a8d9e0 4184 {
df7492f9
KH		 4185 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
4186 ENCODE_RESET_PLANE_AND_REGISTER ();
4187 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_INIT_AT_BOL)
b73bfc1c 4188 {
df7492f9
KH		 4189 int i;
4190
4191 for (i = 0; i < 4; i++)
4192 CODING_ISO_DESIGNATION (coding, i)
4193 = CODING_ISO_INITIAL (coding, i);
b73bfc1c 4194 }
df7492f9
KH		 4195 bol_designation
4196 = CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL;
19a8d9e0 4197 }
df7492f9
KH		 4198 else if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_CNTL)
4199 ENCODE_RESET_PLANE_AND_REGISTER ();
4200 EMIT_ONE_ASCII_BYTE (c);
4ed46869 4201 }
df7492f9 4202 else if (ASCII_CHAR_P (c))
88993dfd 4203 {
df7492f9
KH		 4204 if (ascii_compatible)
4205 EMIT_ONE_ASCII_BYTE (c);
93dec019 4206 else
19a8d9e0 4207 {
bf16eb23
KH		 4208 struct charset *charset = CHARSET_FROM_ID (charset_ascii);
4209 ENCODE_ISO_CHARACTER (charset, c);
19a8d9e0 4210 }
4ed46869 4211 }
16eafb5d 4212 else if (CHAR_BYTE8_P (c))
88993dfd 4213 {
16eafb5d
KH		 4214 c = CHAR_TO_BYTE8 (c);
4215 EMIT_ONE_BYTE (c);
88993dfd 4216 }
b73bfc1c 4217 else
df7492f9 4218 {
ff0dacd7 4219 struct charset *charset;
b73bfc1c 4220
ff0dacd7
KH		 4221 if (preferred_charset_id >= 0)
4222 {
4223 charset = CHARSET_FROM_ID (preferred_charset_id);
4224 if (! CHAR_CHARSET_P (c, charset))
4225 charset = char_charset (c, charset_list, NULL);
4226 }
4227 else
4228 charset = char_charset (c, charset_list, NULL);
df7492f9
KH		 4229 if (!charset)
4230 {
41cbe562
KH		 4231 if (coding->mode & CODING_MODE_SAFE_ENCODING)
4232 {
4233 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4234 charset = CHARSET_FROM_ID (charset_ascii);
4235 }
4236 else
4237 {
4238 c = coding->default_char;
4239 charset = char_charset (c, charset_list, NULL);
4240 }
df7492f9
KH		 4241 }
4242 ENCODE_ISO_CHARACTER (charset, c);
4243 }
84fbb8a0 4244 }
b73bfc1c 4245
df7492f9
KH		 4246 if (coding->mode & CODING_MODE_LAST_BLOCK
4247 && CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
4248 {
4249 ASSURE_DESTINATION (safe_room);
4250 ENCODE_RESET_PLANE_AND_REGISTER ();
4251 }
065e3595 4252 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4253 CODING_ISO_BOL (coding) = bol_designation;
4254 coding->produced_char += produced_chars;
4255 coding->produced = dst - coding->destination;
4256 return 0;
4ed46869
KH		 4257}
4258
4259\f
df7492f9 4260/*** 8,9. SJIS and BIG5 handlers ***/
4ed46869 4261
df7492f9 4262/* Although SJIS and BIG5 are not ISO's coding system, they are used
4ed46869
KH		 4263 quite widely. So, for the moment, Emacs supports them in the bare
4264 C code. But, in the future, they may be supported only by CCL. */
4265
4266/* SJIS is a coding system encoding three character sets: ASCII, right
4267 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
4268 as is. A character of charset katakana-jisx0201 is encoded by
4269 "position-code + 0x80". A character of charset japanese-jisx0208
4270 is encoded in 2-byte but two position-codes are divided and shifted
df7492f9 4271 so that it fit in the range below.
4ed46869
KH		 4272
4273 --- CODE RANGE of SJIS ---
4274 (character set) (range)
4275 ASCII 0x00 .. 0x7F
df7492f9 4276 KATAKANA-JISX0201 0xA0 .. 0xDF
c28a9453 4277 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
d14d03ac 4278 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
4ed46869
KH		 4279 -------------------------------
4280
4281*/
4282
4283/* BIG5 is a coding system encoding two character sets: ASCII and
4284 Big5. An ASCII character is encoded as is. Big5 is a two-byte
df7492f9 4285 character set and is encoded in two-byte.
4ed46869
KH		 4286
4287 --- CODE RANGE of BIG5 ---
4288 (character set) (range)
4289 ASCII 0x00 .. 0x7F
4290 Big5 (1st byte) 0xA1 .. 0xFE
4291 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
4292 --------------------------
4293
df7492f9 4294 */
4ed46869
KH		 4295
4296/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4297 Check if a text is encoded in SJIS. If it is, return
df7492f9 4298 CATEGORY_MASK_SJIS, else return 0. */
4ed46869 4299
0a28aafb 4300static int
ff0dacd7 4301detect_coding_sjis (coding, detect_info)
df7492f9 4302 struct coding_system *coding;
ff0dacd7 4303 struct coding_detection_info *detect_info;
4ed46869 4304{
065e3595 4305 const unsigned char *src = coding->source, *src_base;
8f924df7 4306 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4307 int multibytep = coding->src_multibyte;
4308 int consumed_chars = 0;
4309 int found = 0;
b73bfc1c 4310 int c;
df7492f9 4311
ff0dacd7 4312 detect_info->checked |= CATEGORY_MASK_SJIS;
df7492f9
KH		 4313 /* A coding system of this category is always ASCII compatible. */
4314 src += coding->head_ascii;
4ed46869 4315
b73bfc1c 4316 while (1)
4ed46869 4317 {
065e3595 4318 src_base = src;
df7492f9 4319 ONE_MORE_BYTE (c);
682169fe
KH		 4320 if (c < 0x80)
4321 continue;
df7492f9 4322 if ((c >= 0x81 && c <= 0x9F) || (c >= 0xE0 && c <= 0xEF))
4ed46869 4323 {
df7492f9 4324 ONE_MORE_BYTE (c);
682169fe 4325 if (c < 0x40 || c == 0x7F || c > 0xFC)
df7492f9 4326 break;
ff0dacd7 4327 found = CATEGORY_MASK_SJIS;
4ed46869 4328 }
df7492f9 4329 else if (c >= 0xA0 && c < 0xE0)
ff0dacd7 4330 found = CATEGORY_MASK_SJIS;
df7492f9
KH		 4331 else
4332 break;
4ed46869 4333 }
ff0dacd7 4334 detect_info->rejected |= CATEGORY_MASK_SJIS;
df7492f9
KH		 4335 return 0;
4336
4337 no_more_source:
065e3595 4338 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 4339 {
ff0dacd7 4340 detect_info->rejected |= CATEGORY_MASK_SJIS;
89528eb3 4341 return 0;
4ed46869 4342 }
ff0dacd7
KH		 4343 detect_info->found |= found;
4344 return 1;
4ed46869
KH		 4345}
4346
4347/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4348 Check if a text is encoded in BIG5. If it is, return
df7492f9 4349 CATEGORY_MASK_BIG5, else return 0. */
4ed46869 4350
0a28aafb 4351static int
ff0dacd7 4352detect_coding_big5 (coding, detect_info)
df7492f9 4353 struct coding_system *coding;
ff0dacd7 4354 struct coding_detection_info *detect_info;
4ed46869 4355{
065e3595 4356 const unsigned char *src = coding->source, *src_base;
8f924df7 4357 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4358 int multibytep = coding->src_multibyte;
4359 int consumed_chars = 0;
4360 int found = 0;
b73bfc1c 4361 int c;
fa42c37f 4362
ff0dacd7 4363 detect_info->checked |= CATEGORY_MASK_BIG5;
df7492f9
KH		 4364 /* A coding system of this category is always ASCII compatible. */
4365 src += coding->head_ascii;
fa42c37f 4366
b73bfc1c 4367 while (1)
fa42c37f 4368 {
065e3595 4369 src_base = src;
df7492f9
KH		 4370 ONE_MORE_BYTE (c);
4371 if (c < 0x80)
fa42c37f 4372 continue;
df7492f9 4373 if (c >= 0xA1)
fa42c37f 4374 {
df7492f9
KH		 4375 ONE_MORE_BYTE (c);
4376 if (c < 0x40 || (c >= 0x7F && c <= 0xA0))
fa42c37f 4377 return 0;
ff0dacd7 4378 found = CATEGORY_MASK_BIG5;
fa42c37f 4379 }
df7492f9
KH		 4380 else
4381 break;
fa42c37f 4382 }
ff0dacd7 4383 detect_info->rejected |= CATEGORY_MASK_BIG5;
fa42c37f 4384 return 0;
fa42c37f 4385
df7492f9 4386 no_more_source:
065e3595 4387 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 4388 {
ff0dacd7 4389 detect_info->rejected |= CATEGORY_MASK_BIG5;
89528eb3
KH		 4390 return 0;
4391 }
ff0dacd7
KH		 4392 detect_info->found |= found;
4393 return 1;
fa42c37f
KH		 4394}
4395
4ed46869
KH		 4396/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
4397 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
fa42c37f 4398
b73bfc1c 4399static void
df7492f9 4400decode_coding_sjis (coding)
4ed46869 4401 struct coding_system *coding;
4ed46869 4402{
8f924df7
KH		 4403 const unsigned char *src = coding->source + coding->consumed;
4404 const unsigned char *src_end = coding->source + coding->src_bytes;
4405 const unsigned char *src_base;
69a80ea3
KH		 4406 int *charbuf = coding->charbuf + coding->charbuf_used;
4407 int *charbuf_end
4408 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 4409 int consumed_chars = 0, consumed_chars_base;
4410 int multibytep = coding->src_multibyte;
4411 struct charset *charset_roman, *charset_kanji, *charset_kana;
57a47f8a 4412 struct charset *charset_kanji2;
24a73b0a 4413 Lisp_Object attrs, charset_list, val;
ff0dacd7
KH		 4414 int char_offset = coding->produced_char;
4415 int last_offset = char_offset;
4416 int last_id = charset_ascii;
119852e7
KH		 4417 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
4418 int byte_after_cr = -1;
a5d301df 4419
24a73b0a 4420 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4421
4422 val = charset_list;
4423 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
89528eb3 4424 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4425 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4426 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 4427
b73bfc1c 4428 while (1)
4ed46869 4429 {
df7492f9 4430 int c, c1;
24a73b0a 4431 struct charset *charset;
fa42c37f 4432
b73bfc1c 4433 src_base = src;
df7492f9 4434 consumed_chars_base = consumed_chars;
fa42c37f 4435
df7492f9 4436 if (charbuf >= charbuf_end)
b71f6f73
KH		 4437 {
4438 if (byte_after_cr >= 0)
4439 src_base--;
4440 break;
4441 }
df7492f9 4442
119852e7
KH		 4443 if (byte_after_cr >= 0)
4444 c = byte_after_cr, byte_after_cr = -1;
4445 else
4446 ONE_MORE_BYTE (c);
065e3595
KH		 4447 if (c < 0)
4448 goto invalid_code;
24a73b0a 4449 if (c < 0x80)
119852e7
KH		 4450 {
4451 if (eol_crlf && c == '\r')
4452 ONE_MORE_BYTE (byte_after_cr);
4453 charset = charset_roman;
4454 }
57a47f8a 4455 else if (c == 0x80 || c == 0xA0)
8e921c4b 4456 goto invalid_code;
57a47f8a
KH		 4457 else if (c >= 0xA1 && c <= 0xDF)
4458 {
4459 /* SJIS -> JISX0201-Kana */
4460 c &= 0x7F;
4461 charset = charset_kana;
4462 }
4463 else if (c <= 0xEF)
df7492f9 4464 {
57a47f8a
KH		 4465 /* SJIS -> JISX0208 */
4466 ONE_MORE_BYTE (c1);
4467 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4468 goto invalid_code;
57a47f8a
KH		 4469 c = (c << 8) | c1;
4470 SJIS_TO_JIS (c);
4471 charset = charset_kanji;
4472 }
4473 else if (c <= 0xFC && charset_kanji2)
4474 {
c6876370 4475 /* SJIS -> JISX0213-2 */
57a47f8a
KH		 4476 ONE_MORE_BYTE (c1);
4477 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4478 goto invalid_code;
57a47f8a
KH		 4479 c = (c << 8) | c1;
4480 SJIS_TO_JIS2 (c);
4481 charset = charset_kanji2;
df7492f9 4482 }
57a47f8a
KH		 4483 else
4484 goto invalid_code;
24a73b0a
KH		 4485 if (charset->id != charset_ascii
4486 && last_id != charset->id)
4487 {
4488 if (last_id != charset_ascii)
69a80ea3 4489 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4490 last_id = charset->id;
4491 last_offset = char_offset;
4492 }
4493 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4494 *charbuf++ = c;
ff0dacd7 4495 char_offset++;
df7492f9 4496 continue;
b73bfc1c 4497
df7492f9
KH		 4498 invalid_code:
4499 src = src_base;
4500 consumed_chars = consumed_chars_base;
4501 ONE_MORE_BYTE (c);
065e3595 4502 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4503 char_offset++;
df7492f9
KH		 4504 coding->errors++;
4505 }
fa42c37f 4506
df7492f9 4507 no_more_source:
ff0dacd7 4508 if (last_id != charset_ascii)
69a80ea3 4509 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4510 coding->consumed_char += consumed_chars_base;
4511 coding->consumed = src_base - coding->source;
4512 coding->charbuf_used = charbuf - coding->charbuf;
fa42c37f
KH		 4513}
4514
b73bfc1c 4515static void
df7492f9 4516decode_coding_big5 (coding)
4ed46869 4517 struct coding_system *coding;
4ed46869 4518{
8f924df7
KH		 4519 const unsigned char *src = coding->source + coding->consumed;
4520 const unsigned char *src_end = coding->source + coding->src_bytes;
4521 const unsigned char *src_base;
69a80ea3
KH		 4522 int *charbuf = coding->charbuf + coding->charbuf_used;
4523 int *charbuf_end
4524 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 4525 int consumed_chars = 0, consumed_chars_base;
4526 int multibytep = coding->src_multibyte;
4527 struct charset *charset_roman, *charset_big5;
24a73b0a 4528 Lisp_Object attrs, charset_list, val;
ff0dacd7
KH		 4529 int char_offset = coding->produced_char;
4530 int last_offset = char_offset;
4531 int last_id = charset_ascii;
119852e7
KH		 4532 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
4533 int byte_after_cr = -1;
df7492f9 4534
24a73b0a 4535 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4536 val = charset_list;
4537 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4538 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4539
b73bfc1c 4540 while (1)
4ed46869 4541 {
df7492f9 4542 int c, c1;
24a73b0a 4543 struct charset *charset;
b73bfc1c
KH		 4544
4545 src_base = src;
df7492f9
KH		 4546 consumed_chars_base = consumed_chars;
4547
4548 if (charbuf >= charbuf_end)
b71f6f73
KH		 4549 {
4550 if (byte_after_cr >= 0)
4551 src_base--;
4552 break;
4553 }
df7492f9 4554
119852e7 4555 if (byte_after_cr >= 0)
14daee73 4556 c = byte_after_cr, byte_after_cr = -1;
119852e7
KH		 4557 else
4558 ONE_MORE_BYTE (c);
b73bfc1c 4559
065e3595
KH		 4560 if (c < 0)
4561 goto invalid_code;
24a73b0a 4562 if (c < 0x80)
119852e7 4563 {
14daee73 4564 if (eol_crlf && c == '\r')
119852e7
KH		 4565 ONE_MORE_BYTE (byte_after_cr);
4566 charset = charset_roman;
4567 }
24a73b0a 4568 else
4ed46869 4569 {
24a73b0a
KH		 4570 /* BIG5 -> Big5 */
4571 if (c < 0xA1 || c > 0xFE)
4572 goto invalid_code;
4573 ONE_MORE_BYTE (c1);
4574 if (c1 < 0x40 || (c1 > 0x7E && c1 < 0xA1) || c1 > 0xFE)
4575 goto invalid_code;
4576 c = c << 8 | c1;
4577 charset = charset_big5;
4ed46869 4578 }
24a73b0a
KH		 4579 if (charset->id != charset_ascii
4580 && last_id != charset->id)
df7492f9 4581 {
24a73b0a 4582 if (last_id != charset_ascii)
69a80ea3 4583 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4584 last_id = charset->id;
4585 last_offset = char_offset;
4ed46869 4586 }
24a73b0a 4587 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4588 *charbuf++ = c;
ff0dacd7 4589 char_offset++;
fb88bf2d
KH		 4590 continue;
4591
df7492f9 4592 invalid_code:
4ed46869 4593 src = src_base;
df7492f9
KH		 4594 consumed_chars = consumed_chars_base;
4595 ONE_MORE_BYTE (c);
065e3595 4596 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4597 char_offset++;
df7492f9 4598 coding->errors++;
fb88bf2d 4599 }
d46c5b12 4600
df7492f9 4601 no_more_source:
ff0dacd7 4602 if (last_id != charset_ascii)
69a80ea3 4603 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4604 coding->consumed_char += consumed_chars_base;
4605 coding->consumed = src_base - coding->source;
4606 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 4607}
4608
4609/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
b73bfc1c
KH		 4610 This function can encode charsets `ascii', `katakana-jisx0201',
4611 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
4612 are sure that all these charsets are registered as official charset
4ed46869
KH		 4613 (i.e. do not have extended leading-codes). Characters of other
4614 charsets are produced without any encoding. If SJIS_P is 1, encode
4615 SJIS text, else encode BIG5 text. */
4616
df7492f9
KH		 4617static int
4618encode_coding_sjis (coding)
4ed46869 4619 struct coding_system *coding;
4ed46869 4620{
df7492f9
KH		 4621 int multibytep = coding->dst_multibyte;
4622 int *charbuf = coding->charbuf;
4623 int *charbuf_end = charbuf + coding->charbuf_used;
4624 unsigned char *dst = coding->destination + coding->produced;
4625 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4626 int safe_room = 4;
4627 int produced_chars = 0;
24a73b0a 4628 Lisp_Object attrs, charset_list, val;
df7492f9
KH		 4629 int ascii_compatible;
4630 struct charset *charset_roman, *charset_kanji, *charset_kana;
57a47f8a 4631 struct charset *charset_kanji2;
df7492f9 4632 int c;
a5d301df 4633
24a73b0a 4634 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4635 val = charset_list;
4636 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4637 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4638 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4639 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4640
df7492f9 4641 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
93dec019 4642
df7492f9
KH		 4643 while (charbuf < charbuf_end)
4644 {
4645 ASSURE_DESTINATION (safe_room);
4646 c = *charbuf++;
b73bfc1c 4647 /* Now encode the character C. */
df7492f9
KH		 4648 if (ASCII_CHAR_P (c) && ascii_compatible)
4649 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4650 else if (CHAR_BYTE8_P (c))
4651 {
4652 c = CHAR_TO_BYTE8 (c);
4653 EMIT_ONE_BYTE (c);
4654 }
df7492f9 4655 else
b73bfc1c 4656 {
df7492f9
KH		 4657 unsigned code;
4658 struct charset *charset = char_charset (c, charset_list, &code);
4659
4660 if (!charset)
4ed46869 4661 {
41cbe562 4662 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4663 {
41cbe562
KH		 4664 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4665 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4666 }
41cbe562 4667 else
b73bfc1c 4668 {
41cbe562
KH		 4669 c = coding->default_char;
4670 charset = char_charset (c, charset_list, &code);
b73bfc1c 4671 }
b73bfc1c 4672 }
df7492f9
KH		 4673 if (code == CHARSET_INVALID_CODE (charset))
4674 abort ();
4675 if (charset == charset_kanji)
4676 {
4677 int c1, c2;
4678 JIS_TO_SJIS (code);
4679 c1 = code >> 8, c2 = code & 0xFF;
4680 EMIT_TWO_BYTES (c1, c2);
4681 }
4682 else if (charset == charset_kana)
4683 EMIT_ONE_BYTE (code | 0x80);
57a47f8a
KH		 4684 else if (charset_kanji2 && charset == charset_kanji2)
4685 {
4686 int c1, c2;
4687
4688 c1 = code >> 8;
4689 if (c1 == 0x21 || (c1 >= 0x23 && c1 < 0x25)
4690 || (c1 >= 0x2C && c1 <= 0x2F) || c1 >= 0x6E)
4691 {
4692 JIS_TO_SJIS2 (code);
4693 c1 = code >> 8, c2 = code & 0xFF;
4694 EMIT_TWO_BYTES (c1, c2);
4695 }
4696 else
4697 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4698 }
df7492f9
KH		 4699 else
4700 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4701 }
4702 }
065e3595 4703 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4704 coding->produced_char += produced_chars;
4705 coding->produced = dst - coding->destination;
4706 return 0;
4707}
4708
4709static int
4710encode_coding_big5 (coding)
4711 struct coding_system *coding;
4712{
4713 int multibytep = coding->dst_multibyte;
4714 int *charbuf = coding->charbuf;
4715 int *charbuf_end = charbuf + coding->charbuf_used;
4716 unsigned char *dst = coding->destination + coding->produced;
4717 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4718 int safe_room = 4;
4719 int produced_chars = 0;
24a73b0a 4720 Lisp_Object attrs, charset_list, val;
df7492f9
KH		 4721 int ascii_compatible;
4722 struct charset *charset_roman, *charset_big5;
4723 int c;
4724
24a73b0a 4725 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4726 val = charset_list;
4727 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4728 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4729 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4730
4731 while (charbuf < charbuf_end)
4732 {
4733 ASSURE_DESTINATION (safe_room);
4734 c = *charbuf++;
4735 /* Now encode the character C. */
4736 if (ASCII_CHAR_P (c) && ascii_compatible)
4737 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4738 else if (CHAR_BYTE8_P (c))
4739 {
4740 c = CHAR_TO_BYTE8 (c);
4741 EMIT_ONE_BYTE (c);
b73bfc1c
KH		 4742 }
4743 else
4744 {
df7492f9
KH		 4745 unsigned code;
4746 struct charset *charset = char_charset (c, charset_list, &code);
4747
4748 if (! charset)
b73bfc1c 4749 {
41cbe562 4750 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4751 {
41cbe562
KH		 4752 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4753 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4754 }
41cbe562 4755 else
0eecad43 4756 {
41cbe562
KH		 4757 c = coding->default_char;
4758 charset = char_charset (c, charset_list, &code);
0eecad43 4759 }
4ed46869 4760 }
df7492f9
KH		 4761 if (code == CHARSET_INVALID_CODE (charset))
4762 abort ();
4763 if (charset == charset_big5)
b73bfc1c 4764 {
df7492f9
KH		 4765 int c1, c2;
4766
4767 c1 = code >> 8, c2 = code & 0xFF;
4768 EMIT_TWO_BYTES (c1, c2);
b73bfc1c 4769 }
df7492f9
KH		 4770 else
4771 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4ed46869 4772 }
4ed46869 4773 }
065e3595 4774 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4775 coding->produced_char += produced_chars;
4776 coding->produced = dst - coding->destination;
4777 return 0;
4ed46869
KH		 4778}
4779
4780\f
df7492f9 4781/*** 10. CCL handlers ***/
1397dc18
KH		 4782
4783/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4784 Check if a text is encoded in a coding system of which
4785 encoder/decoder are written in CCL program. If it is, return
df7492f9 4786 CATEGORY_MASK_CCL, else return 0. */
1397dc18 4787
0a28aafb 4788static int
ff0dacd7 4789detect_coding_ccl (coding, detect_info)
df7492f9 4790 struct coding_system *coding;
ff0dacd7 4791 struct coding_detection_info *detect_info;
1397dc18 4792{
065e3595 4793 const unsigned char *src = coding->source, *src_base;
8f924df7 4794 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4795 int multibytep = coding->src_multibyte;
4796 int consumed_chars = 0;
4797 int found = 0;
0e219d54 4798 unsigned char *valids;
df7492f9
KH		 4799 int head_ascii = coding->head_ascii;
4800 Lisp_Object attrs;
4801
ff0dacd7
KH		 4802 detect_info->checked |= CATEGORY_MASK_CCL;
4803
df7492f9 4804 coding = &coding_categories[coding_category_ccl];
0e219d54 4805 valids = CODING_CCL_VALIDS (coding);
df7492f9
KH		 4806 attrs = CODING_ID_ATTRS (coding->id);
4807 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
4808 src += head_ascii;
1397dc18 4809
b73bfc1c 4810 while (1)
1397dc18 4811 {
df7492f9 4812 int c;
065e3595
KH		 4813
4814 src_base = src;
df7492f9 4815 ONE_MORE_BYTE (c);
065e3595 4816 if (c < 0 || ! valids[c])
df7492f9 4817 break;
ff0dacd7
KH		 4818 if ((valids[c] > 1))
4819 found = CATEGORY_MASK_CCL;
df7492f9 4820 }
ff0dacd7 4821 detect_info->rejected |= CATEGORY_MASK_CCL;
df7492f9
KH		 4822 return 0;
4823
4824 no_more_source:
ff0dacd7
KH		 4825 detect_info->found |= found;
4826 return 1;
df7492f9
KH		 4827}
4828
4829static void
4830decode_coding_ccl (coding)
4831 struct coding_system *coding;
4832{
7c78e542 4833 const unsigned char *src = coding->source + coding->consumed;
8f924df7 4834 const unsigned char *src_end = coding->source + coding->src_bytes;
69a80ea3
KH		 4835 int *charbuf = coding->charbuf + coding->charbuf_used;
4836 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 4837 int consumed_chars = 0;
4838 int multibytep = coding->src_multibyte;
4839 struct ccl_program ccl;
4840 int source_charbuf[1024];
4841 int source_byteidx[1024];
24a73b0a 4842 Lisp_Object attrs, charset_list;
df7492f9 4843
24a73b0a 4844 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4845 setup_ccl_program (&ccl, CODING_CCL_DECODER (coding));
4846
4847 while (src < src_end)
4848 {
7c78e542 4849 const unsigned char *p = src;
df7492f9
KH		 4850 int *source, *source_end;
4851 int i = 0;
4852
4853 if (multibytep)
4854 while (i < 1024 && p < src_end)
4855 {
4856 source_byteidx[i] = p - src;
4857 source_charbuf[i++] = STRING_CHAR_ADVANCE (p);
4858 }
4859 else
4860 while (i < 1024 && p < src_end)
4861 source_charbuf[i++] = *p++;
8f924df7 4862
df7492f9
KH		 4863 if (p == src_end && coding->mode & CODING_MODE_LAST_BLOCK)
4864 ccl.last_block = 1;
4865
4866 source = source_charbuf;
4867 source_end = source + i;
4868 while (source < source_end)
4869 {
4870 ccl_driver (&ccl, source, charbuf,
8dcbea82
KH		 4871 source_end - source, charbuf_end - charbuf,
4872 charset_list);
df7492f9
KH		 4873 source += ccl.consumed;
4874 charbuf += ccl.produced;
4875 if (ccl.status != CCL_STAT_SUSPEND_BY_DST)
4876 break;
4877 }
4878 if (source < source_end)
4879 src += source_byteidx[source - source_charbuf];
4880 else
4881 src = p;
4882 consumed_chars += source - source_charbuf;
4883
4884 if (ccl.status != CCL_STAT_SUSPEND_BY_SRC
4885 && ccl.status != CODING_RESULT_INSUFFICIENT_SRC)
4886 break;
4887 }
4888
4889 switch (ccl.status)
4890 {
4891 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 4892 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 4893 break;
4894 case CCL_STAT_SUSPEND_BY_DST:
4895 break;
4896 case CCL_STAT_QUIT:
4897 case CCL_STAT_INVALID_CMD:
065e3595 4898 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 4899 break;
4900 default:
065e3595 4901 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4902 break;
4903 }
4904 coding->consumed_char += consumed_chars;
4905 coding->consumed = src - coding->source;
4906 coding->charbuf_used = charbuf - coding->charbuf;
4907}
4908
4909static int
4910encode_coding_ccl (coding)
4911 struct coding_system *coding;
4912{
4913 struct ccl_program ccl;
4914 int multibytep = coding->dst_multibyte;
4915 int *charbuf = coding->charbuf;
4916 int *charbuf_end = charbuf + coding->charbuf_used;
4917 unsigned char *dst = coding->destination + coding->produced;
4918 unsigned char *dst_end = coding->destination + coding->dst_bytes;
df7492f9
KH		 4919 int destination_charbuf[1024];
4920 int i, produced_chars = 0;
24a73b0a 4921 Lisp_Object attrs, charset_list;
df7492f9 4922
24a73b0a 4923 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4924 setup_ccl_program (&ccl, CODING_CCL_ENCODER (coding));
4925
4926 ccl.last_block = coding->mode & CODING_MODE_LAST_BLOCK;
4927 ccl.dst_multibyte = coding->dst_multibyte;
4928
8cffd3e7 4929 while (charbuf < charbuf_end)
df7492f9 4930 {
df7492f9 4931 ccl_driver (&ccl, charbuf, destination_charbuf,
8cffd3e7 4932 charbuf_end - charbuf, 1024, charset_list);
df7492f9 4933 if (multibytep)
8cffd3e7
KH		 4934 {
4935 ASSURE_DESTINATION (ccl.produced * 2);
4936 for (i = 0; i < ccl.produced; i++)
4937 EMIT_ONE_BYTE (destination_charbuf[i] & 0xFF);
4938 }
df7492f9
KH		 4939 else
4940 {
8cffd3e7 4941 ASSURE_DESTINATION (ccl.produced);
3ed051d4 4942 for (i = 0; i < ccl.produced; i++)
df7492f9
KH		 4943 *dst++ = destination_charbuf[i] & 0xFF;
4944 produced_chars += ccl.produced;
4945 }
8cffd3e7
KH		 4946 charbuf += ccl.consumed;
4947 if (ccl.status == CCL_STAT_QUIT
4948 || ccl.status == CCL_STAT_INVALID_CMD)
4949 break;
df7492f9
KH		 4950 }
4951
4952 switch (ccl.status)
4953 {
4954 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 4955 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 4956 break;
4957 case CCL_STAT_SUSPEND_BY_DST:
065e3595 4958 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 4959 break;
4960 case CCL_STAT_QUIT:
4961 case CCL_STAT_INVALID_CMD:
065e3595 4962 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 4963 break;
4964 default:
065e3595 4965 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 4966 break;
1397dc18 4967 }
df7492f9
KH		 4968
4969 coding->produced_char += produced_chars;
4970 coding->produced = dst - coding->destination;
4971 return 0;
1397dc18
KH		 4972}
4973
df7492f9 4974
1397dc18 4975\f
df7492f9 4976/*** 10, 11. no-conversion handlers ***/
4ed46869 4977
b73bfc1c 4978/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
4ed46869 4979
b73bfc1c 4980static void
df7492f9 4981decode_coding_raw_text (coding)
4ed46869 4982 struct coding_system *coding;
4ed46869 4983{
119852e7
KH		 4984 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
4985
df7492f9 4986 coding->chars_at_source = 1;
119852e7
KH		 4987 coding->consumed_char = coding->src_chars;
4988 coding->consumed = coding->src_bytes;
4989 if (eol_crlf && coding->source[coding->src_bytes - 1] == '\r')
4990 {
4991 coding->consumed_char--;
4992 coding->consumed--;
4993 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
4994 }
4995 else
4996 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 4997}
4ed46869 4998
df7492f9
KH		 4999static int
5000encode_coding_raw_text (coding)
5001 struct coding_system *coding;
5002{
5003 int multibytep = coding->dst_multibyte;
5004 int *charbuf = coding->charbuf;
5005 int *charbuf_end = coding->charbuf + coding->charbuf_used;
5006 unsigned char *dst = coding->destination + coding->produced;
5007 unsigned char *dst_end = coding->destination + coding->dst_bytes;
a0ed9b27 5008 int produced_chars = 0;
b73bfc1c
KH		 5009 int c;
5010
df7492f9 5011 if (multibytep)
b73bfc1c 5012 {
df7492f9 5013 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
4ed46869 5014
df7492f9
KH		 5015 if (coding->src_multibyte)
5016 while (charbuf < charbuf_end)
5017 {
5018 ASSURE_DESTINATION (safe_room);
5019 c = *charbuf++;
5020 if (ASCII_CHAR_P (c))
5021 EMIT_ONE_ASCII_BYTE (c);
5022 else if (CHAR_BYTE8_P (c))
5023 {
5024 c = CHAR_TO_BYTE8 (c);
5025 EMIT_ONE_BYTE (c);
5026 }
5027 else
5028 {
5029 unsigned char str[MAX_MULTIBYTE_LENGTH], *p0 = str, *p1 = str;
93dec019 5030
df7492f9
KH		 5031 CHAR_STRING_ADVANCE (c, p1);
5032 while (p0 < p1)
9d123124
KH		 5033 {
5034 EMIT_ONE_BYTE (*p0);
5035 p0++;
5036 }
df7492f9
KH		 5037 }
5038 }
b73bfc1c 5039 else
df7492f9
KH		 5040 while (charbuf < charbuf_end)
5041 {
5042 ASSURE_DESTINATION (safe_room);
5043 c = *charbuf++;
5044 EMIT_ONE_BYTE (c);
5045 }
5046 }
5047 else
4ed46869 5048 {
df7492f9 5049 if (coding->src_multibyte)
d46c5b12 5050 {
df7492f9
KH		 5051 int safe_room = MAX_MULTIBYTE_LENGTH;
5052
5053 while (charbuf < charbuf_end)
d46c5b12 5054 {
df7492f9
KH		 5055 ASSURE_DESTINATION (safe_room);
5056 c = *charbuf++;
5057 if (ASCII_CHAR_P (c))
5058 *dst++ = c;
5059 else if (CHAR_BYTE8_P (c))
5060 *dst++ = CHAR_TO_BYTE8 (c);
b73bfc1c 5061 else
df7492f9 5062 CHAR_STRING_ADVANCE (c, dst);
d46c5b12
KH		 5063 }
5064 }
df7492f9
KH		 5065 else
5066 {
5067 ASSURE_DESTINATION (charbuf_end - charbuf);
5068 while (charbuf < charbuf_end && dst < dst_end)
5069 *dst++ = *charbuf++;
8f924df7 5070 }
319a3947 5071 produced_chars = dst - (coding->destination + coding->produced);
4ed46869 5072 }
065e3595 5073 record_conversion_result (coding, CODING_RESULT_SUCCESS);
a0ed9b27 5074 coding->produced_char += produced_chars;
df7492f9
KH		 5075 coding->produced = dst - coding->destination;
5076 return 0;
4ed46869
KH		 5077}
5078
ff0dacd7
KH		 5079/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
5080 Check if a text is encoded in a charset-based coding system. If it
5081 is, return 1, else return 0. */
5082
0a28aafb 5083static int
ff0dacd7 5084detect_coding_charset (coding, detect_info)
df7492f9 5085 struct coding_system *coding;
ff0dacd7 5086 struct coding_detection_info *detect_info;
1397dc18 5087{
065e3595 5088 const unsigned char *src = coding->source, *src_base;
8f924df7 5089 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 5090 int multibytep = coding->src_multibyte;
5091 int consumed_chars = 0;
07295713 5092 Lisp_Object attrs, valids, name;
584948ac 5093 int found = 0;
716b3fa0 5094 int head_ascii = coding->head_ascii;
07295713 5095 int check_latin_extra = 0;
1397dc18 5096
ff0dacd7
KH		 5097 detect_info->checked |= CATEGORY_MASK_CHARSET;
5098
df7492f9
KH		 5099 coding = &coding_categories[coding_category_charset];
5100 attrs = CODING_ID_ATTRS (coding->id);
5101 valids = AREF (attrs, coding_attr_charset_valids);
07295713
KH		 5102 name = CODING_ID_NAME (coding->id);
5103 if (VECTORP (Vlatin_extra_code_table)
5104 && strcmp ((char *) SDATA (SYMBOL_NAME (name)), "iso-8859-"))
5105 check_latin_extra = 1;
df7492f9 5106 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
716b3fa0 5107 src += head_ascii;
1397dc18 5108
b73bfc1c 5109 while (1)
1397dc18 5110 {
df7492f9 5111 int c;
716b3fa0
KH		 5112 Lisp_Object val;
5113 struct charset *charset;
5114 int dim, idx;
1397dc18 5115
065e3595 5116 src_base = src;
df7492f9 5117 ONE_MORE_BYTE (c);
065e3595
KH		 5118 if (c < 0)
5119 continue;
716b3fa0
KH		 5120 val = AREF (valids, c);
5121 if (NILP (val))
df7492f9 5122 break;
584948ac 5123 if (c >= 0x80)
07295713
KH		 5124 {
5125 if (c < 0xA0
5126 && check_latin_extra
5127 && NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
5128 break;
5129 found = CATEGORY_MASK_CHARSET;
5130 }
716b3fa0
KH		 5131 if (INTEGERP (val))
5132 {
5133 charset = CHARSET_FROM_ID (XFASTINT (val));
5134 dim = CHARSET_DIMENSION (charset);
5135 for (idx = 1; idx < dim; idx++)
5136 {
5137 if (src == src_end)
5138 goto too_short;
5139 ONE_MORE_BYTE (c);
3ed051d4 5140 if (c < charset->code_space[(dim - 1 - idx) * 2]
716b3fa0
KH		 5141 || c > charset->code_space[(dim - 1 - idx) * 2 + 1])
5142 break;
5143 }
5144 if (idx < dim)
5145 break;
5146 }
5147 else
5148 {
5149 idx = 1;
5150 for (; CONSP (val); val = XCDR (val))
5151 {
5152 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
5153 dim = CHARSET_DIMENSION (charset);
5154 while (idx < dim)
5155 {
5156 if (src == src_end)
5157 goto too_short;
5158 ONE_MORE_BYTE (c);
5159 if (c < charset->code_space[(dim - 1 - idx) * 4]
5160 || c > charset->code_space[(dim - 1 - idx) * 4 + 1])
5161 break;
5162 idx++;
5163 }
5164 if (idx == dim)
5165 {
5166 val = Qnil;
5167 break;
5168 }
5169 }
5170 if (CONSP (val))
5171 break;
5172 }
df7492f9 5173 }
716b3fa0 5174 too_short:
ff0dacd7 5175 detect_info->rejected |= CATEGORY_MASK_CHARSET;
df7492f9 5176 return 0;
4ed46869 5177
df7492f9 5178 no_more_source:
ff0dacd7
KH		 5179 detect_info->found |= found;
5180 return 1;
df7492f9 5181}
b73bfc1c 5182
b73bfc1c 5183static void
df7492f9 5184decode_coding_charset (coding)
4ed46869 5185 struct coding_system *coding;
4ed46869 5186{
8f924df7
KH		 5187 const unsigned char *src = coding->source + coding->consumed;
5188 const unsigned char *src_end = coding->source + coding->src_bytes;
5189 const unsigned char *src_base;
69a80ea3
KH		 5190 int *charbuf = coding->charbuf + coding->charbuf_used;
5191 int *charbuf_end
5192 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 5193 int consumed_chars = 0, consumed_chars_base;
5194 int multibytep = coding->src_multibyte;
24a73b0a 5195 Lisp_Object attrs, charset_list, valids;
ff0dacd7
KH		 5196 int char_offset = coding->produced_char;
5197 int last_offset = char_offset;
5198 int last_id = charset_ascii;
119852e7
KH		 5199 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
5200 int byte_after_cr = -1;
df7492f9 5201
24a73b0a 5202 CODING_GET_INFO (coding, attrs, charset_list);
4eb6d3f1 5203 valids = AREF (attrs, coding_attr_charset_valids);
b73bfc1c 5204
df7492f9 5205 while (1)
4ed46869 5206 {
4eb6d3f1 5207 int c;
24a73b0a
KH		 5208 Lisp_Object val;
5209 struct charset *charset;
5210 int dim;
5211 int len = 1;
5212 unsigned code;
df7492f9
KH		 5213
5214 src_base = src;
5215 consumed_chars_base = consumed_chars;
b73bfc1c 5216
df7492f9 5217 if (charbuf >= charbuf_end)
b71f6f73
KH		 5218 {
5219 if (byte_after_cr >= 0)
5220 src_base--;
5221 break;
5222 }
df7492f9 5223
119852e7
KH		 5224 if (byte_after_cr >= 0)
5225 {
5226 c = byte_after_cr;
5227 byte_after_cr = -1;
5228 }
5229 else
5230 {
5231 ONE_MORE_BYTE (c);
5232 if (eol_crlf && c == '\r')
5233 ONE_MORE_BYTE (byte_after_cr);
5234 }
065e3595
KH		 5235 if (c < 0)
5236 goto invalid_code;
24a73b0a
KH		 5237 code = c;
5238
5239 val = AREF (valids, c);
1b17adfd 5240 if (! INTEGERP (val) && ! CONSP (val))
24a73b0a
KH		 5241 goto invalid_code;
5242 if (INTEGERP (val))
d46c5b12 5243 {
24a73b0a
KH		 5244 charset = CHARSET_FROM_ID (XFASTINT (val));
5245 dim = CHARSET_DIMENSION (charset);
5246 while (len < dim)
b73bfc1c 5247 {
24a73b0a
KH		 5248 ONE_MORE_BYTE (c);
5249 code = (code << 8) | c;
5250 len++;
b73bfc1c 5251 }
24a73b0a
KH		 5252 CODING_DECODE_CHAR (coding, src, src_base, src_end,
5253 charset, code, c);
d46c5b12 5254 }
df7492f9 5255 else
d46c5b12 5256 {
24a73b0a
KH		 5257 /* VAL is a list of charset IDs. It is assured that the
5258 list is sorted by charset dimensions (smaller one
5259 comes first). */
5260 while (CONSP (val))
4eb6d3f1 5261 {
24a73b0a 5262 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
c7c66a95 5263 dim = CHARSET_DIMENSION (charset);
f9d71dcd 5264 while (len < dim)
4eb6d3f1 5265 {
acb2a965
KH		 5266 ONE_MORE_BYTE (c);
5267 code = (code << 8) | c;
f9d71dcd 5268 len++;
4eb6d3f1 5269 }
24a73b0a
KH		 5270 CODING_DECODE_CHAR (coding, src, src_base,
5271 src_end, charset, code, c);
5272 if (c >= 0)
5273 break;
5274 val = XCDR (val);
ff0dacd7 5275 }
d46c5b12 5276 }
24a73b0a
KH		 5277 if (c < 0)
5278 goto invalid_code;
5279 if (charset->id != charset_ascii
5280 && last_id != charset->id)
5281 {
5282 if (last_id != charset_ascii)
69a80ea3 5283 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 5284 last_id = charset->id;
5285 last_offset = char_offset;
5286 }
5287
df7492f9 5288 *charbuf++ = c;
ff0dacd7 5289 char_offset++;
df7492f9
KH		 5290 continue;
5291
5292 invalid_code:
5293 src = src_base;
5294 consumed_chars = consumed_chars_base;
5295 ONE_MORE_BYTE (c);
065e3595 5296 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 5297 char_offset++;
df7492f9 5298 coding->errors++;
4ed46869
KH		 5299 }
5300
df7492f9 5301 no_more_source:
ff0dacd7 5302 if (last_id != charset_ascii)
69a80ea3 5303 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 5304 coding->consumed_char += consumed_chars_base;
5305 coding->consumed = src_base - coding->source;
5306 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 5307}
5308
df7492f9
KH		 5309static int
5310encode_coding_charset (coding)
4ed46869 5311 struct coding_system *coding;
4ed46869 5312{
df7492f9
KH		 5313 int multibytep = coding->dst_multibyte;
5314 int *charbuf = coding->charbuf;
5315 int *charbuf_end = charbuf + coding->charbuf_used;
5316 unsigned char *dst = coding->destination + coding->produced;
5317 unsigned char *dst_end = coding->destination + coding->dst_bytes;
5318 int safe_room = MAX_MULTIBYTE_LENGTH;
5319 int produced_chars = 0;
24a73b0a 5320 Lisp_Object attrs, charset_list;
df7492f9 5321 int ascii_compatible;
b73bfc1c 5322 int c;
b73bfc1c 5323
24a73b0a 5324 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 5325 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
fb88bf2d 5326
df7492f9 5327 while (charbuf < charbuf_end)
4ed46869 5328 {
4eb6d3f1 5329 struct charset *charset;
df7492f9 5330 unsigned code;
8f924df7 5331
df7492f9
KH		 5332 ASSURE_DESTINATION (safe_room);
5333 c = *charbuf++;
5334 if (ascii_compatible && ASCII_CHAR_P (c))
5335 EMIT_ONE_ASCII_BYTE (c);
16eafb5d 5336 else if (CHAR_BYTE8_P (c))
4ed46869 5337 {
16eafb5d
KH		 5338 c = CHAR_TO_BYTE8 (c);
5339 EMIT_ONE_BYTE (c);
d46c5b12 5340 }
d46c5b12 5341 else
b73bfc1c 5342 {
4eb6d3f1
KH		 5343 charset = char_charset (c, charset_list, &code);
5344 if (charset)
5345 {
5346 if (CHARSET_DIMENSION (charset) == 1)
5347 EMIT_ONE_BYTE (code);
5348 else if (CHARSET_DIMENSION (charset) == 2)
5349 EMIT_TWO_BYTES (code >> 8, code & 0xFF);
5350 else if (CHARSET_DIMENSION (charset) == 3)
5351 EMIT_THREE_BYTES (code >> 16, (code >> 8) & 0xFF, code & 0xFF);
5352 else
5353 EMIT_FOUR_BYTES (code >> 24, (code >> 16) & 0xFF,
5354 (code >> 8) & 0xFF, code & 0xFF);
5355 }
5356 else
41cbe562
KH		 5357 {
5358 if (coding->mode & CODING_MODE_SAFE_ENCODING)
5359 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
5360 else
5361 c = coding->default_char;
5362 EMIT_ONE_BYTE (c);
5363 }
4ed46869 5364 }
4ed46869
KH		 5365 }
5366
065e3595 5367 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 5368 coding->produced_char += produced_chars;
5369 coding->produced = dst - coding->destination;
5370 return 0;
4ed46869
KH		 5371}
5372
5373\f
1397dc18 5374/*** 7. C library functions ***/
4ed46869 5375
df7492f9
KH		 5376/* Setup coding context CODING from information about CODING_SYSTEM.
5377 If CODING_SYSTEM is nil, `no-conversion' is assumed. If
5378 CODING_SYSTEM is invalid, signal an error. */
4ed46869 5379
ec6d2bb8 5380void
e0e989f6
KH		 5381setup_coding_system (coding_system, coding)
5382 Lisp_Object coding_system;
4ed46869
KH		 5383 struct coding_system *coding;
5384{
df7492f9
KH		 5385 Lisp_Object attrs;
5386 Lisp_Object eol_type;
5387 Lisp_Object coding_type;
4608c386 5388 Lisp_Object val;
4ed46869 5389
df7492f9 5390 if (NILP (coding_system))
ae6f73fa 5391 coding_system = Qundecided;
c07c8e12 5392
df7492f9 5393 CHECK_CODING_SYSTEM_GET_ID (coding_system, coding->id);
1f5dbf34 5394
df7492f9
KH		 5395 attrs = CODING_ID_ATTRS (coding->id);
5396 eol_type = CODING_ID_EOL_TYPE (coding->id);
1f5dbf34 5397
df7492f9
KH		 5398 coding->mode = 0;
5399 coding->head_ascii = -1;
4a015c45
KH		 5400 if (VECTORP (eol_type))
5401 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5402 | CODING_REQUIRE_DETECTION_MASK);
5403 else if (! EQ (eol_type, Qunix))
5404 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5405 | CODING_REQUIRE_ENCODING_MASK);
5406 else
5407 coding->common_flags = 0;
5e5c78be
KH		 5408 if (! NILP (CODING_ATTR_POST_READ (attrs)))
5409 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5410 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
5411 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
8f924df7
KH		 5412 if (! NILP (CODING_ATTR_FOR_UNIBYTE (attrs)))
5413 coding->common_flags |= CODING_FOR_UNIBYTE_MASK;
1f5dbf34 5414
df7492f9 5415 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 5416 coding->max_charset_id = SCHARS (val) - 1;
5417 coding->safe_charsets = (char *) SDATA (val);
df7492f9 5418 coding->default_char = XINT (CODING_ATTR_DEFAULT_CHAR (attrs));
4608c386 5419
df7492f9
KH		 5420 coding_type = CODING_ATTR_TYPE (attrs);
5421 if (EQ (coding_type, Qundecided))
d46c5b12 5422 {
df7492f9
KH		 5423 coding->detector = NULL;
5424 coding->decoder = decode_coding_raw_text;
5425 coding->encoder = encode_coding_raw_text;
5426 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 5427 }
df7492f9 5428 else if (EQ (coding_type, Qiso_2022))
d46c5b12 5429 {
df7492f9
KH		 5430 int i;
5431 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5432
5433 /* Invoke graphic register 0 to plane 0. */
5434 CODING_ISO_INVOCATION (coding, 0) = 0;
5435 /* Invoke graphic register 1 to plane 1 if we can use 8-bit. */
5436 CODING_ISO_INVOCATION (coding, 1)
5437 = (flags & CODING_ISO_FLAG_SEVEN_BITS ? -1 : 1);
5438 /* Setup the initial status of designation. */
5439 for (i = 0; i < 4; i++)
5440 CODING_ISO_DESIGNATION (coding, i) = CODING_ISO_INITIAL (coding, i);
5441 /* Not single shifting initially. */
5442 CODING_ISO_SINGLE_SHIFTING (coding) = 0;
5443 /* Beginning of buffer should also be regarded as bol. */
5444 CODING_ISO_BOL (coding) = 1;
5445 coding->detector = detect_coding_iso_2022;
5446 coding->decoder = decode_coding_iso_2022;
5447 coding->encoder = encode_coding_iso_2022;
5448 if (flags & CODING_ISO_FLAG_SAFE)
5449 coding->mode |= CODING_MODE_SAFE_ENCODING;
d46c5b12 5450 coding->common_flags
df7492f9
KH		 5451 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5452 | CODING_REQUIRE_FLUSHING_MASK);
5453 if (flags & CODING_ISO_FLAG_COMPOSITION)
5454 coding->common_flags |= CODING_ANNOTATE_COMPOSITION_MASK;
ff0dacd7
KH		 5455 if (flags & CODING_ISO_FLAG_DESIGNATION)
5456 coding->common_flags |= CODING_ANNOTATE_CHARSET_MASK;
df7492f9
KH		 5457 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5458 {
5459 setup_iso_safe_charsets (attrs);
5460 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 5461 coding->max_charset_id = SCHARS (val) - 1;
5462 coding->safe_charsets = (char *) SDATA (val);
df7492f9
KH		 5463 }
5464 CODING_ISO_FLAGS (coding) = flags;
d46c5b12 5465 }
df7492f9 5466 else if (EQ (coding_type, Qcharset))
d46c5b12 5467 {
df7492f9
KH		 5468 coding->detector = detect_coding_charset;
5469 coding->decoder = decode_coding_charset;
5470 coding->encoder = encode_coding_charset;
d46c5b12 5471 coding->common_flags
df7492f9 5472 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
d46c5b12 5473 }
df7492f9 5474 else if (EQ (coding_type, Qutf_8))
d46c5b12 5475 {
a470d443
KH		 5476 val = AREF (attrs, coding_attr_utf_bom);
5477 CODING_UTF_8_BOM (coding) = (CONSP (val) ? utf_detect_bom
5478 : EQ (val, Qt) ? utf_with_bom
5479 : utf_without_bom);
df7492f9
KH		 5480 coding->detector = detect_coding_utf_8;
5481 coding->decoder = decode_coding_utf_8;
5482 coding->encoder = encode_coding_utf_8;
5483 coding->common_flags
5484 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
a470d443
KH		 5485 if (CODING_UTF_8_BOM (coding) == utf_detect_bom)
5486 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
df7492f9
KH		 5487 }
5488 else if (EQ (coding_type, Qutf_16))
5489 {
a470d443
KH		 5490 val = AREF (attrs, coding_attr_utf_bom);
5491 CODING_UTF_16_BOM (coding) = (CONSP (val) ? utf_detect_bom
5492 : EQ (val, Qt) ? utf_with_bom
5493 : utf_without_bom);
df7492f9 5494 val = AREF (attrs, coding_attr_utf_16_endian);
b49a1807 5495 CODING_UTF_16_ENDIAN (coding) = (EQ (val, Qbig) ? utf_16_big_endian
df7492f9 5496 : utf_16_little_endian);
e19c3639 5497 CODING_UTF_16_SURROGATE (coding) = 0;
df7492f9
KH		 5498 coding->detector = detect_coding_utf_16;
5499 coding->decoder = decode_coding_utf_16;
5500 coding->encoder = encode_coding_utf_16;
5501 coding->common_flags
5502 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
a470d443 5503 if (CODING_UTF_16_BOM (coding) == utf_detect_bom)
b49a1807 5504 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 5505 }
df7492f9 5506 else if (EQ (coding_type, Qccl))
4ed46869 5507 {
df7492f9
KH		 5508 coding->detector = detect_coding_ccl;
5509 coding->decoder = decode_coding_ccl;
5510 coding->encoder = encode_coding_ccl;
c952af22 5511 coding->common_flags
df7492f9
KH		 5512 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5513 | CODING_REQUIRE_FLUSHING_MASK);
5514 }
5515 else if (EQ (coding_type, Qemacs_mule))
5516 {
5517 coding->detector = detect_coding_emacs_mule;
5518 coding->decoder = decode_coding_emacs_mule;
5519 coding->encoder = encode_coding_emacs_mule;
c952af22 5520 coding->common_flags
df7492f9
KH		 5521 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5522 if (! NILP (AREF (attrs, coding_attr_emacs_mule_full))
5523 && ! EQ (CODING_ATTR_CHARSET_LIST (attrs), Vemacs_mule_charset_list))
5524 {
5525 Lisp_Object tail, safe_charsets;
5526 int max_charset_id = 0;
5527
5528 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5529 tail = XCDR (tail))
5530 if (max_charset_id < XFASTINT (XCAR (tail)))
5531 max_charset_id = XFASTINT (XCAR (tail));
5532 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
5533 make_number (255));
5534 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5535 tail = XCDR (tail))
8f924df7 5536 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 5537 coding->max_charset_id = max_charset_id;
8f924df7 5538 coding->safe_charsets = (char *) SDATA (safe_charsets);
df7492f9
KH		 5539 }
5540 }
5541 else if (EQ (coding_type, Qshift_jis))
5542 {
5543 coding->detector = detect_coding_sjis;
5544 coding->decoder = decode_coding_sjis;
5545 coding->encoder = encode_coding_sjis;
c952af22 5546 coding->common_flags
df7492f9
KH		 5547 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5548 }
5549 else if (EQ (coding_type, Qbig5))
5550 {
5551 coding->detector = detect_coding_big5;
5552 coding->decoder = decode_coding_big5;
5553 coding->encoder = encode_coding_big5;
c952af22 5554 coding->common_flags
df7492f9
KH		 5555 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5556 }
5557 else /* EQ (coding_type, Qraw_text) */
ec6d2bb8 5558 {
df7492f9
KH		 5559 coding->detector = NULL;
5560 coding->decoder = decode_coding_raw_text;
5561 coding->encoder = encode_coding_raw_text;
ea29edf2
KH		 5562 if (! EQ (eol_type, Qunix))
5563 {
5564 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5565 if (! VECTORP (eol_type))
5566 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
5567 }
5568
4ed46869 5569 }
4ed46869 5570
df7492f9 5571 return;
4ed46869
KH		 5572}
5573
0ff61e78
KH		 5574/* Return a list of charsets supported by CODING. */
5575
5576Lisp_Object
5577coding_charset_list (coding)
5578 struct coding_system *coding;
5579{
35befdaa 5580 Lisp_Object attrs, charset_list;
0ff61e78
KH		 5581
5582 CODING_GET_INFO (coding, attrs, charset_list);
5583 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5584 {
5585 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5586
5587 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5588 charset_list = Viso_2022_charset_list;
5589 }
5590 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5591 {
5592 charset_list = Vemacs_mule_charset_list;
5593 }
5594 return charset_list;
5595}
5596
5597
e9f91ece
KH		 5598/* Return a list of charsets supported by CODING-SYSTEM. */
5599
5600Lisp_Object
5601coding_system_charset_list (coding_system)
5602 Lisp_Object coding_system;
5603{
5604 int id;
5605 Lisp_Object attrs, charset_list;
5606
5607 CHECK_CODING_SYSTEM_GET_ID (coding_system, id);
5608 attrs = CODING_ID_ATTRS (id);
5609
5610 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5611 {
5612 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5613
5614 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5615 charset_list = Viso_2022_charset_list;
5616 else
5617 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
5618 }
5619 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5620 {
5621 charset_list = Vemacs_mule_charset_list;
5622 }
5623 else
5624 {
5625 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
5626 }
5627 return charset_list;
5628}
5629
5630
df7492f9
KH		 5631/* Return raw-text or one of its subsidiaries that has the same
5632 eol_type as CODING-SYSTEM. */
ec6d2bb8 5633
df7492f9
KH		 5634Lisp_Object
5635raw_text_coding_system (coding_system)
5636 Lisp_Object coding_system;
ec6d2bb8 5637{
0be8721c 5638 Lisp_Object spec, attrs;
df7492f9 5639 Lisp_Object eol_type, raw_text_eol_type;
ec6d2bb8 5640
d3e4cb56
KH		 5641 if (NILP (coding_system))
5642 return Qraw_text;
df7492f9
KH		 5643 spec = CODING_SYSTEM_SPEC (coding_system);
5644 attrs = AREF (spec, 0);
ec6d2bb8 5645
df7492f9
KH		 5646 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
5647 return coding_system;
ec6d2bb8 5648
df7492f9
KH		 5649 eol_type = AREF (spec, 2);
5650 if (VECTORP (eol_type))
5651 return Qraw_text;
5652 spec = CODING_SYSTEM_SPEC (Qraw_text);
5653 raw_text_eol_type = AREF (spec, 2);
5654 return (EQ (eol_type, Qunix) ? AREF (raw_text_eol_type, 0)
5655 : EQ (eol_type, Qdos) ? AREF (raw_text_eol_type, 1)
5656 : AREF (raw_text_eol_type, 2));
ec6d2bb8
KH		 5657}
5658
54f78171 5659
df7492f9
KH		 5660/* If CODING_SYSTEM doesn't specify end-of-line format but PARENT
5661 does, return one of the subsidiary that has the same eol-spec as
fcbcfb64
KH		 5662 PARENT. Otherwise, return CODING_SYSTEM. If PARENT is nil,
5663 inherit end-of-line format from the system's setting
5664 (system_eol_type). */
df7492f9
KH		 5665
5666Lisp_Object
5667coding_inherit_eol_type (coding_system, parent)
b74e4686 5668 Lisp_Object coding_system, parent;
54f78171 5669{
3e139625 5670 Lisp_Object spec, eol_type;
54f78171 5671
d3e4cb56
KH		 5672 if (NILP (coding_system))
5673 coding_system = Qraw_text;
df7492f9 5674 spec = CODING_SYSTEM_SPEC (coding_system);
df7492f9 5675 eol_type = AREF (spec, 2);
fcbcfb64 5676 if (VECTORP (eol_type))
df7492f9 5677 {
df7492f9
KH		 5678 Lisp_Object parent_eol_type;
5679
fcbcfb64
KH		 5680 if (! NILP (parent))
5681 {
5682 Lisp_Object parent_spec;
5683
4a015c45 5684 parent_spec = CODING_SYSTEM_SPEC (parent);
fcbcfb64
KH		 5685 parent_eol_type = AREF (parent_spec, 2);
5686 }
5687 else
5688 parent_eol_type = system_eol_type;
df7492f9
KH		 5689 if (EQ (parent_eol_type, Qunix))
5690 coding_system = AREF (eol_type, 0);
5691 else if (EQ (parent_eol_type, Qdos))
5692 coding_system = AREF (eol_type, 1);
5693 else if (EQ (parent_eol_type, Qmac))
5694 coding_system = AREF (eol_type, 2);
54f78171 5695 }
df7492f9 5696 return coding_system;
54f78171
KH		 5697}
5698
4ed46869
KH		 5699/* Emacs has a mechanism to automatically detect a coding system if it
5700 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
5701 it's impossible to distinguish some coding systems accurately
5702 because they use the same range of codes. So, at first, coding
5703 systems are categorized into 7, those are:
5704
0ef69138 5705 o coding-category-emacs-mule
4ed46869
KH		 5706
5707 The category for a coding system which has the same code range
5708 as Emacs' internal format. Assigned the coding-system (Lisp
0ef69138 5709 symbol) `emacs-mule' by default.
4ed46869
KH		 5710
5711 o coding-category-sjis
5712
5713 The category for a coding system which has the same code range
5714 as SJIS. Assigned the coding-system (Lisp
7717c392 5715 symbol) `japanese-shift-jis' by default.
4ed46869
KH		 5716
5717 o coding-category-iso-7
5718
5719 The category for a coding system which has the same code range
7717c392 5720 as ISO2022 of 7-bit environment. This doesn't use any locking
d46c5b12
KH		 5721 shift and single shift functions. This can encode/decode all
5722 charsets. Assigned the coding-system (Lisp symbol)
5723 `iso-2022-7bit' by default.
5724
5725 o coding-category-iso-7-tight
5726
5727 Same as coding-category-iso-7 except that this can
5728 encode/decode only the specified charsets.
4ed46869
KH		 5729
5730 o coding-category-iso-8-1
5731
5732 The category for a coding system which has the same code range
5733 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 5734 for DIMENSION1 charset. This doesn't use any locking shift
5735 and single shift functions. Assigned the coding-system (Lisp
5736 symbol) `iso-latin-1' by default.
4ed46869
KH		 5737
5738 o coding-category-iso-8-2
5739
5740 The category for a coding system which has the same code range
5741 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 5742 for DIMENSION2 charset. This doesn't use any locking shift
5743 and single shift functions. Assigned the coding-system (Lisp
5744 symbol) `japanese-iso-8bit' by default.
4ed46869 5745
7717c392 5746 o coding-category-iso-7-else
4ed46869
KH		 5747
5748 The category for a coding system which has the same code range
df7492f9 5749 as ISO2022 of 7-bit environemnt but uses locking shift or
7717c392
KH		 5750 single shift functions. Assigned the coding-system (Lisp
5751 symbol) `iso-2022-7bit-lock' by default.
5752
5753 o coding-category-iso-8-else
5754
5755 The category for a coding system which has the same code range
df7492f9 5756 as ISO2022 of 8-bit environemnt but uses locking shift or
7717c392
KH		 5757 single shift functions. Assigned the coding-system (Lisp
5758 symbol) `iso-2022-8bit-ss2' by default.
4ed46869
KH		 5759
5760 o coding-category-big5
5761
5762 The category for a coding system which has the same code range
5763 as BIG5. Assigned the coding-system (Lisp symbol)
e0e989f6 5764 `cn-big5' by default.
4ed46869 5765
fa42c37f
KH		 5766 o coding-category-utf-8
5767
5768 The category for a coding system which has the same code range
6e76ae91 5769 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
fa42c37f
KH		 5770 symbol) `utf-8' by default.
5771
5772 o coding-category-utf-16-be
5773
5774 The category for a coding system in which a text has an
5775 Unicode signature (cf. Unicode Standard) in the order of BIG
5776 endian at the head. Assigned the coding-system (Lisp symbol)
5777 `utf-16-be' by default.
5778
5779 o coding-category-utf-16-le
5780
5781 The category for a coding system in which a text has an
5782 Unicode signature (cf. Unicode Standard) in the order of
5783 LITTLE endian at the head. Assigned the coding-system (Lisp
5784 symbol) `utf-16-le' by default.
5785
1397dc18
KH		 5786 o coding-category-ccl
5787
5788 The category for a coding system of which encoder/decoder is
5789 written in CCL programs. The default value is nil, i.e., no
5790 coding system is assigned.
5791
4ed46869
KH		 5792 o coding-category-binary
5793
5794 The category for a coding system not categorized in any of the
5795 above. Assigned the coding-system (Lisp symbol)
e0e989f6 5796 `no-conversion' by default.
4ed46869
KH		 5797
5798 Each of them is a Lisp symbol and the value is an actual
df7492f9 5799 `coding-system's (this is also a Lisp symbol) assigned by a user.
4ed46869
KH		 5800 What Emacs does actually is to detect a category of coding system.
5801 Then, it uses a `coding-system' assigned to it. If Emacs can't
df7492f9 5802 decide only one possible category, it selects a category of the
4ed46869
KH		 5803 highest priority. Priorities of categories are also specified by a
5804 user in a Lisp variable `coding-category-list'.
5805
5806*/
5807
df7492f9
KH		 5808#define EOL_SEEN_NONE 0
5809#define EOL_SEEN_LF 1
5810#define EOL_SEEN_CR 2
5811#define EOL_SEEN_CRLF 4
66cfb530 5812
ff0dacd7
KH		 5813/* Detect how end-of-line of a text of length SRC_BYTES pointed by
5814 SOURCE is encoded. If CATEGORY is one of
5815 coding_category_utf_16_XXXX, assume that CR and LF are encoded by
5816 two-byte, else they are encoded by one-byte.
5817
5818 Return one of EOL_SEEN_XXX. */
4ed46869 5819
bc4bc72a 5820#define MAX_EOL_CHECK_COUNT 3
d46c5b12
KH		 5821
5822static int
89528eb3 5823detect_eol (source, src_bytes, category)
f6cbaf43 5824 const unsigned char *source;
df7492f9 5825 EMACS_INT src_bytes;
89528eb3 5826 enum coding_category category;
4ed46869 5827{
f6cbaf43 5828 const unsigned char *src = source, *src_end = src + src_bytes;
4ed46869 5829 unsigned char c;
df7492f9
KH		 5830 int total = 0;
5831 int eol_seen = EOL_SEEN_NONE;
4ed46869 5832
89528eb3 5833 if ((1 << category) & CATEGORY_MASK_UTF_16)
4ed46869 5834 {
df7492f9 5835 int msb, lsb;
fa42c37f 5836
89528eb3
KH		 5837 msb = category == (coding_category_utf_16_le
5838 | coding_category_utf_16_le_nosig);
df7492f9 5839 lsb = 1 - msb;
fa42c37f 5840
df7492f9 5841 while (src + 1 < src_end)
fa42c37f 5842 {
df7492f9
KH		 5843 c = src[lsb];
5844 if (src[msb] == 0 && (c == '\n' || c == '\r'))
fa42c37f 5845 {
df7492f9
KH		 5846 int this_eol;
5847
5848 if (c == '\n')
5849 this_eol = EOL_SEEN_LF;
5850 else if (src + 3 >= src_end
5851 || src[msb + 2] != 0
5852 || src[lsb + 2] != '\n')
5853 this_eol = EOL_SEEN_CR;
fa42c37f 5854 else
75f4f1ac
EZ		 5855 {
5856 this_eol = EOL_SEEN_CRLF;
5857 src += 2;
5858 }
df7492f9
KH		 5859
5860 if (eol_seen == EOL_SEEN_NONE)
5861 /* This is the first end-of-line. */
5862 eol_seen = this_eol;
5863 else if (eol_seen != this_eol)
fa42c37f 5864 {
75f4f1ac
EZ		 5865 /* The found type is different from what found before.
5866 Allow for stray ^M characters in DOS EOL files. */
5867 if (eol_seen == EOL_SEEN_CR && this_eol == EOL_SEEN_CRLF
5868 || eol_seen == EOL_SEEN_CRLF && this_eol == EOL_SEEN_CR)
5869 eol_seen = EOL_SEEN_CRLF;
5870 else
5871 {
5872 eol_seen = EOL_SEEN_LF;
5873 break;
5874 }
fa42c37f 5875 }
df7492f9
KH		 5876 if (++total == MAX_EOL_CHECK_COUNT)
5877 break;
fa42c37f 5878 }
df7492f9 5879 src += 2;
fa42c37f 5880 }
bcf26d6a 5881 }
d46c5b12 5882 else
c4825358 5883 {
df7492f9 5884 while (src < src_end)
27901516 5885 {
df7492f9
KH		 5886 c = *src++;
5887 if (c == '\n' || c == '\r')
5888 {
5889 int this_eol;
d46c5b12 5890
df7492f9
KH		 5891 if (c == '\n')
5892 this_eol = EOL_SEEN_LF;
5893 else if (src >= src_end || *src != '\n')
5894 this_eol = EOL_SEEN_CR;
5895 else
5896 this_eol = EOL_SEEN_CRLF, src++;
d46c5b12 5897
df7492f9
KH		 5898 if (eol_seen == EOL_SEEN_NONE)
5899 /* This is the first end-of-line. */
5900 eol_seen = this_eol;
5901 else if (eol_seen != this_eol)
5902 {
75f4f1ac
EZ		 5903 /* The found type is different from what found before.
5904 Allow for stray ^M characters in DOS EOL files. */
5905 if (eol_seen == EOL_SEEN_CR && this_eol == EOL_SEEN_CRLF
5906 || eol_seen == EOL_SEEN_CRLF && this_eol == EOL_SEEN_CR)
5907 eol_seen = EOL_SEEN_CRLF;
5908 else
5909 {
5910 eol_seen = EOL_SEEN_LF;
5911 break;
5912 }
df7492f9
KH		 5913 }
5914 if (++total == MAX_EOL_CHECK_COUNT)
5915 break;
5916 }
5917 }
73be902c 5918 }
df7492f9 5919 return eol_seen;
73be902c
KH		 5920}
5921
df7492f9 5922
24a73b0a 5923static Lisp_Object
df7492f9
KH		 5924adjust_coding_eol_type (coding, eol_seen)
5925 struct coding_system *coding;
5926 int eol_seen;
73be902c 5927{
0be8721c 5928 Lisp_Object eol_type;
8f924df7 5929
df7492f9
KH		 5930 eol_type = CODING_ID_EOL_TYPE (coding->id);
5931 if (eol_seen & EOL_SEEN_LF)
24a73b0a
KH		 5932 {
5933 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 0));
5934 eol_type = Qunix;
5935 }
6f197c07 5936 else if (eol_seen & EOL_SEEN_CRLF)
24a73b0a
KH		 5937 {
5938 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 1));
5939 eol_type = Qdos;
5940 }
6f197c07 5941 else if (eol_seen & EOL_SEEN_CR)
24a73b0a
KH		 5942 {
5943 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 2));
5944 eol_type = Qmac;
5945 }
5946 return eol_type;
d46c5b12 5947}
4ed46869 5948
df7492f9
KH		 5949/* Detect how a text specified in CODING is encoded. If a coding
5950 system is detected, update fields of CODING by the detected coding
5951 system. */
0a28aafb 5952
df7492f9
KH		 5953void
5954detect_coding (coding)
d46c5b12 5955 struct coding_system *coding;
d46c5b12 5956{
8f924df7 5957 const unsigned char *src, *src_end;
d46c5b12 5958
df7492f9
KH		 5959 coding->consumed = coding->consumed_char = 0;
5960 coding->produced = coding->produced_char = 0;
5961 coding_set_source (coding);
1c3478b0 5962
df7492f9 5963 src_end = coding->source + coding->src_bytes;
c0e16b14 5964 coding->head_ascii = 0;
1c3478b0 5965
df7492f9
KH		 5966 /* If we have not yet decided the text encoding type, detect it
5967 now. */
5968 if (EQ (CODING_ATTR_TYPE (CODING_ID_ATTRS (coding->id)), Qundecided))
b73bfc1c 5969 {
df7492f9 5970 int c, i;
6cb21a4f 5971 struct coding_detection_info detect_info;
2f3cbb32 5972 int null_byte_found = 0, eight_bit_found = 0;
df7492f9 5973
6cb21a4f 5974 detect_info.checked = detect_info.found = detect_info.rejected = 0;
2f3cbb32 5975 for (src = coding->source; src < src_end; src++)
d46c5b12 5976 {
df7492f9 5977 c = *src;
6cb21a4f 5978 if (c & 0x80)
6cb21a4f 5979 {
2f3cbb32 5980 eight_bit_found = 1;
2f3cbb32
KH		 5981 if (null_byte_found)
5982 break;
5983 }
5984 else if (c < 0x20)
5985 {
5986 if ((c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
5987 && ! inhibit_iso_escape_detection
5988 && ! detect_info.checked)
6cb21a4f 5989 {
2f3cbb32
KH		 5990 if (detect_coding_iso_2022 (coding, &detect_info))
5991 {
5992 /* We have scanned the whole data. */
5993 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
c0e16b14
KH		 5994 {
5995 /* We didn't find an 8-bit code. We may
5996 have found a null-byte, but it's very
5997 rare that a binary file confirm to
5998 ISO-2022. */
5999 src = src_end;
6000 coding->head_ascii = src - coding->source;
6001 }
6002 detect_info.rejected |= ~CATEGORY_MASK_ISO_ESCAPE;
2f3cbb32
KH		 6003 break;
6004 }
6005 }
97b1b294 6006 else if (! c && !inhibit_null_byte_detection)
2f3cbb32
KH		 6007 {
6008 null_byte_found = 1;
6009 if (eight_bit_found)
6010 break;
6cb21a4f 6011 }
c006c0c8
KH		 6012 if (! eight_bit_found)
6013 coding->head_ascii++;
6cb21a4f 6014 }
c006c0c8 6015 else if (! eight_bit_found)
c0e16b14 6016 coding->head_ascii++;
d46c5b12 6017 }
df7492f9 6018
2f3cbb32
KH		 6019 if (null_byte_found || eight_bit_found
6020 || coding->head_ascii < coding->src_bytes
6cb21a4f 6021 || detect_info.found)
d46c5b12 6022 {
ff0dacd7
KH		 6023 enum coding_category category;
6024 struct coding_system *this;
df7492f9 6025
6cb21a4f
KH		 6026 if (coding->head_ascii == coding->src_bytes)
6027 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
6028 for (i = 0; i < coding_category_raw_text; i++)
6029 {
6030 category = coding_priorities[i];
6031 this = coding_categories + category;
6032 if (detect_info.found & (1 << category))
24a73b0a 6033 break;
6cb21a4f
KH		 6034 }
6035 else
2f3cbb32
KH		 6036 {
6037 if (null_byte_found)
ff0dacd7 6038 {
2f3cbb32
KH		 6039 detect_info.checked |= ~CATEGORY_MASK_UTF_16;
6040 detect_info.rejected |= ~CATEGORY_MASK_UTF_16;
ff0dacd7 6041 }
2f3cbb32
KH		 6042 for (i = 0; i < coding_category_raw_text; i++)
6043 {
6044 category = coding_priorities[i];
6045 this = coding_categories + category;
6046 if (this->id < 0)
6047 {
6048 /* No coding system of this category is defined. */
6049 detect_info.rejected |= (1 << category);
6050 }
6051 else if (category >= coding_category_raw_text)
6052 continue;
6053 else if (detect_info.checked & (1 << category))
6054 {
6055 if (detect_info.found & (1 << category))
6056 break;
6057 }
6058 else if ((*(this->detector)) (coding, &detect_info)
6059 && detect_info.found & (1 << category))
6060 {
6061 if (category == coding_category_utf_16_auto)
6062 {
6063 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
6064 category = coding_category_utf_16_le;
6065 else
6066 category = coding_category_utf_16_be;
6067 }
6068 break;
6069 }
6070 }
2f3cbb32 6071 }
c0e16b14
KH		 6072
6073 if (i < coding_category_raw_text)
6074 setup_coding_system (CODING_ID_NAME (this->id), coding);
6075 else if (null_byte_found)
6076 setup_coding_system (Qno_conversion, coding);
6077 else if ((detect_info.rejected & CATEGORY_MASK_ANY)
6078 == CATEGORY_MASK_ANY)
6079 setup_coding_system (Qraw_text, coding);
6080 else if (detect_info.rejected)
6081 for (i = 0; i < coding_category_raw_text; i++)
6082 if (! (detect_info.rejected & (1 << coding_priorities[i])))
6083 {
6084 this = coding_categories + coding_priorities[i];
6085 setup_coding_system (CODING_ID_NAME (this->id), coding);
6086 break;
6087 }
d46c5b12 6088 }
b73bfc1c 6089 }
a470d443
KH		 6090 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
6091 == coding_category_utf_8_auto)
6092 {
6093 Lisp_Object coding_systems;
6094 struct coding_detection_info detect_info;
6095
6096 coding_systems
6097 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_bom);
6098 detect_info.found = detect_info.rejected = 0;
6099 coding->head_ascii = 0;
6100 if (CONSP (coding_systems)
6101 && detect_coding_utf_8 (coding, &detect_info))
6102 {
6103 if (detect_info.found & CATEGORY_MASK_UTF_8_SIG)
6104 setup_coding_system (XCAR (coding_systems), coding);
6105 else
6106 setup_coding_system (XCDR (coding_systems), coding);
6107 }
6108 }
24a73b0a
KH		 6109 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
6110 == coding_category_utf_16_auto)
b49a1807
KH		 6111 {
6112 Lisp_Object coding_systems;
6113 struct coding_detection_info detect_info;
6114
6115 coding_systems
a470d443 6116 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_bom);
b49a1807 6117 detect_info.found = detect_info.rejected = 0;
a470d443 6118 coding->head_ascii = 0;
b49a1807 6119 if (CONSP (coding_systems)
24a73b0a 6120 && detect_coding_utf_16 (coding, &detect_info))
b49a1807
KH		 6121 {
6122 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
6123 setup_coding_system (XCAR (coding_systems), coding);
24a73b0a 6124 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
b49a1807
KH		 6125 setup_coding_system (XCDR (coding_systems), coding);
6126 }
6127 }
4ed46869 6128}
4ed46869 6129
d46c5b12 6130
aaaf0b1e 6131static void
df7492f9 6132decode_eol (coding)
aaaf0b1e 6133 struct coding_system *coding;
aaaf0b1e 6134{
24a73b0a
KH		 6135 Lisp_Object eol_type;
6136 unsigned char *p, *pbeg, *pend;
3ed051d4 6137
24a73b0a
KH		 6138 eol_type = CODING_ID_EOL_TYPE (coding->id);
6139 if (EQ (eol_type, Qunix))
6140 return;
6141
6142 if (NILP (coding->dst_object))
6143 pbeg = coding->destination;
6144 else
6145 pbeg = BYTE_POS_ADDR (coding->dst_pos_byte);
6146 pend = pbeg + coding->produced;
6147
6148 if (VECTORP (eol_type))
aaaf0b1e 6149 {
df7492f9 6150 int eol_seen = EOL_SEEN_NONE;
4ed46869 6151
24a73b0a 6152 for (p = pbeg; p < pend; p++)
aaaf0b1e 6153 {
df7492f9
KH		 6154 if (*p == '\n')
6155 eol_seen |= EOL_SEEN_LF;
6156 else if (*p == '\r')
aaaf0b1e 6157 {
df7492f9 6158 if (p + 1 < pend && *(p + 1) == '\n')
aaaf0b1e 6159 {
df7492f9
KH		 6160 eol_seen |= EOL_SEEN_CRLF;
6161 p++;
aaaf0b1e 6162 }
aaaf0b1e 6163 else
df7492f9 6164 eol_seen |= EOL_SEEN_CR;
aaaf0b1e 6165 }
aaaf0b1e 6166 }
75f4f1ac
EZ		 6167 /* Handle DOS-style EOLs in a file with stray ^M characters. */
6168 if ((eol_seen & EOL_SEEN_CRLF) != 0
6169 && (eol_seen & EOL_SEEN_CR) != 0
6170 && (eol_seen & EOL_SEEN_LF) == 0)
6171 eol_seen = EOL_SEEN_CRLF;
6172 else if (eol_seen != EOL_SEEN_NONE
24a73b0a
KH		 6173 && eol_seen != EOL_SEEN_LF
6174 && eol_seen != EOL_SEEN_CRLF
6175 && eol_seen != EOL_SEEN_CR)
6176 eol_seen = EOL_SEEN_LF;
df7492f9 6177 if (eol_seen != EOL_SEEN_NONE)
24a73b0a 6178 eol_type = adjust_coding_eol_type (coding, eol_seen);
aaaf0b1e 6179 }
d46c5b12 6180
24a73b0a 6181 if (EQ (eol_type, Qmac))
27901516 6182 {
24a73b0a 6183 for (p = pbeg; p < pend; p++)
df7492f9
KH		 6184 if (*p == '\r')
6185 *p = '\n';
4ed46869 6186 }
24a73b0a 6187 else if (EQ (eol_type, Qdos))
df7492f9 6188 {
24a73b0a 6189 int n = 0;
b73bfc1c 6190
24a73b0a
KH		 6191 if (NILP (coding->dst_object))
6192 {
4347441b
KH		 6193 /* Start deleting '\r' from the tail to minimize the memory
6194 movement. */
24a73b0a
KH		 6195 for (p = pend - 2; p >= pbeg; p--)
6196 if (*p == '\r')
6197 {
6198 safe_bcopy ((char *) (p + 1), (char *) p, pend-- - p - 1);
6199 n++;
6200 }
6201 }
6202 else
6203 {
4347441b
KH		 6204 int pos_byte = coding->dst_pos_byte;
6205 int pos = coding->dst_pos;
6206 int pos_end = pos + coding->produced_char - 1;
6207
6208 while (pos < pos_end)
6209 {
6210 p = BYTE_POS_ADDR (pos_byte);
6211 if (*p == '\r' && p[1] == '\n')
6212 {
6213 del_range_2 (pos, pos_byte, pos + 1, pos_byte + 1, 0);
6214 n++;
6215 pos_end--;
6216 }
6217 pos++;
69b8522d
KH		 6218 if (coding->dst_multibyte)
6219 pos_byte += BYTES_BY_CHAR_HEAD (*p);
6220 else
6221 pos_byte++;
4347441b 6222 }
24a73b0a
KH		 6223 }
6224 coding->produced -= n;
6225 coding->produced_char -= n;
aaaf0b1e 6226 }
4ed46869
KH		 6227}
6228
7d64c6ad 6229
a6f87d34
KH		 6230/* Return a translation table (or list of them) from coding system
6231 attribute vector ATTRS for encoding (ENCODEP is nonzero) or
6232 decoding (ENCODEP is zero). */
7d64c6ad 6233
e6a54062 6234static Lisp_Object
09ee6fdd
KH		 6235get_translation_table (attrs, encodep, max_lookup)
6236 Lisp_Object attrs;
6237 int encodep, *max_lookup;
7d64c6ad
KH		 6238{
6239 Lisp_Object standard, translation_table;
09ee6fdd 6240 Lisp_Object val;
7d64c6ad
KH		 6241
6242 if (encodep)
6243 translation_table = CODING_ATTR_ENCODE_TBL (attrs),
6244 standard = Vstandard_translation_table_for_encode;
6245 else
6246 translation_table = CODING_ATTR_DECODE_TBL (attrs),
6247 standard = Vstandard_translation_table_for_decode;
7d64c6ad 6248 if (NILP (translation_table))
09ee6fdd
KH		 6249 translation_table = standard;
6250 else
a6f87d34 6251 {
09ee6fdd
KH		 6252 if (SYMBOLP (translation_table))
6253 translation_table = Fget (translation_table, Qtranslation_table);
6254 else if (CONSP (translation_table))
6255 {
6256 translation_table = Fcopy_sequence (translation_table);
6257 for (val = translation_table; CONSP (val); val = XCDR (val))
6258 if (SYMBOLP (XCAR (val)))
6259 XSETCAR (val, Fget (XCAR (val), Qtranslation_table));
6260 }
6261 if (CHAR_TABLE_P (standard))
6262 {
6263 if (CONSP (translation_table))
6264 translation_table = nconc2 (translation_table,
6265 Fcons (standard, Qnil));
6266 else
6267 translation_table = Fcons (translation_table,
6268 Fcons (standard, Qnil));
6269 }
a6f87d34 6270 }
2170c8f0
KH		 6271
6272 if (max_lookup)
09ee6fdd 6273 {
2170c8f0
KH		 6274 *max_lookup = 1;
6275 if (CHAR_TABLE_P (translation_table)
6276 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (translation_table)) > 1)
6277 {
6278 val = XCHAR_TABLE (translation_table)->extras[1];
6279 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
6280 *max_lookup = XFASTINT (val);
6281 }
6282 else if (CONSP (translation_table))
6283 {
6284 Lisp_Object tail, val;
09ee6fdd 6285
2170c8f0
KH		 6286 for (tail = translation_table; CONSP (tail); tail = XCDR (tail))
6287 if (CHAR_TABLE_P (XCAR (tail))
6288 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (XCAR (tail))) > 1)
6289 {
6290 val = XCHAR_TABLE (XCAR (tail))->extras[1];
6291 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
6292 *max_lookup = XFASTINT (val);
6293 }
6294 }
a6f87d34 6295 }
7d64c6ad
KH		 6296 return translation_table;
6297}
6298
09ee6fdd
KH		 6299#define LOOKUP_TRANSLATION_TABLE(table, c, trans) \
6300 do { \
6301 trans = Qnil; \
6302 if (CHAR_TABLE_P (table)) \
6303 { \
6304 trans = CHAR_TABLE_REF (table, c); \
6305 if (CHARACTERP (trans)) \
6306 c = XFASTINT (trans), trans = Qnil; \
6307 } \
6308 else if (CONSP (table)) \
6309 { \
6310 Lisp_Object tail; \
6311 \
6312 for (tail = table; CONSP (tail); tail = XCDR (tail)) \
6313 if (CHAR_TABLE_P (XCAR (tail))) \
6314 { \
6315 trans = CHAR_TABLE_REF (XCAR (tail), c); \
6316 if (CHARACTERP (trans)) \
6317 c = XFASTINT (trans), trans = Qnil; \
6318 else if (! NILP (trans)) \
6319 break; \
6320 } \
6321 } \
e6a54062
KH		 6322 } while (0)
6323
7d64c6ad 6324
69a80ea3
KH		 6325static Lisp_Object
6326get_translation (val, buf, buf_end, last_block, from_nchars, to_nchars)
6327 Lisp_Object val;
6328 int *buf, *buf_end;
6329 int last_block;
6330 int *from_nchars, *to_nchars;
6331{
433f7f87
KH		 6332 /* VAL is TO or (([FROM-CHAR ...] . TO) ...) where TO is TO-CHAR or
6333 [TO-CHAR ...]. */
69a80ea3
KH		 6334 if (CONSP (val))
6335 {
433f7f87 6336 Lisp_Object from, tail;
69a80ea3
KH		 6337 int i, len;
6338
433f7f87 6339 for (tail = val; CONSP (tail); tail = XCDR (tail))
69a80ea3 6340 {
433f7f87
KH		 6341 val = XCAR (tail);
6342 from = XCAR (val);
6343 len = ASIZE (from);
6344 for (i = 0; i < len; i++)
6345 {
6346 if (buf + i == buf_end)
6347 {
6348 if (! last_block)
6349 return Qt;
6350 break;
6351 }
6352 if (XINT (AREF (from, i)) != buf[i])
6353 break;
6354 }
6355 if (i == len)
6356 {
6357 val = XCDR (val);
6358 *from_nchars = len;
6359 break;
6360 }
69a80ea3 6361 }
433f7f87
KH		 6362 if (! CONSP (tail))
6363 return Qnil;
69a80ea3
KH		 6364 }
6365 if (VECTORP (val))
6366 *buf = XINT (AREF (val, 0)), *to_nchars = ASIZE (val);
6367 else
6368 *buf = XINT (val);
6369 return val;
6370}
6371
6372
d46c5b12 6373static int
69a80ea3 6374produce_chars (coding, translation_table, last_block)
df7492f9 6375 struct coding_system *coding;
69a80ea3
KH		 6376 Lisp_Object translation_table;
6377 int last_block;
4ed46869 6378{
df7492f9
KH		 6379 unsigned char *dst = coding->destination + coding->produced;
6380 unsigned char *dst_end = coding->destination + coding->dst_bytes;
119852e7
KH		 6381 EMACS_INT produced;
6382 EMACS_INT produced_chars = 0;
69a80ea3 6383 int carryover = 0;
4ed46869 6384
df7492f9 6385 if (! coding->chars_at_source)
4ed46869 6386 {
119852e7 6387 /* Source characters are in coding->charbuf. */
fba4576f
AS		 6388 int *buf = coding->charbuf;
6389 int *buf_end = buf + coding->charbuf_used;
4ed46869 6390
db274c7a
KH		 6391 if (EQ (coding->src_object, coding->dst_object))
6392 {
6393 coding_set_source (coding);
6394 dst_end = ((unsigned char *) coding->source) + coding->consumed;
6395 }
4ed46869 6396
df7492f9 6397 while (buf < buf_end)
4ed46869 6398 {
69a80ea3 6399 int c = *buf, i;
bc4bc72a 6400
df7492f9
KH		 6401 if (c >= 0)
6402 {
69a80ea3
KH		 6403 int from_nchars = 1, to_nchars = 1;
6404 Lisp_Object trans = Qnil;
6405
09ee6fdd 6406 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 6407 if (! NILP (trans))
69a80ea3
KH		 6408 {
6409 trans = get_translation (trans, buf, buf_end, last_block,
6410 &from_nchars, &to_nchars);
6411 if (EQ (trans, Qt))
6412 break;
6413 c = *buf;
6414 }
6415
6416 if (dst + MAX_MULTIBYTE_LENGTH * to_nchars > dst_end)
6417 {
6418 dst = alloc_destination (coding,
6419 buf_end - buf
6420 + MAX_MULTIBYTE_LENGTH * to_nchars,
6421 dst);
db274c7a
KH		 6422 if (EQ (coding->src_object, coding->dst_object))
6423 {
6424 coding_set_source (coding);
6425 dst_end = ((unsigned char *) coding->source) + coding->consumed;
6426 }
6427 else
6428 dst_end = coding->destination + coding->dst_bytes;
69a80ea3
KH		 6429 }
6430
433f7f87 6431 for (i = 0; i < to_nchars; i++)
69a80ea3 6432 {
433f7f87
KH		 6433 if (i > 0)
6434 c = XINT (AREF (trans, i));
69a80ea3
KH		 6435 if (coding->dst_multibyte
6436 || ! CHAR_BYTE8_P (c))
db274c7a 6437 CHAR_STRING_ADVANCE_NO_UNIFY (c, dst);
69a80ea3
KH		 6438 else
6439 *dst++ = CHAR_TO_BYTE8 (c);
6440 }
6441 produced_chars += to_nchars;
6442 *buf++ = to_nchars;
6443 while (--from_nchars > 0)
6444 *buf++ = 0;
d46c5b12 6445 }
df7492f9 6446 else
69a80ea3
KH		 6447 /* This is an annotation datum. (-C) is the length. */
6448 buf += -c;
4ed46869 6449 }
69a80ea3 6450 carryover = buf_end - buf;
4ed46869 6451 }
fa42c37f 6452 else
fa42c37f 6453 {
119852e7 6454 /* Source characters are at coding->source. */
8f924df7 6455 const unsigned char *src = coding->source;
119852e7 6456 const unsigned char *src_end = src + coding->consumed;
4ed46869 6457
db274c7a
KH		 6458 if (EQ (coding->dst_object, coding->src_object))
6459 dst_end = (unsigned char *) src;
df7492f9 6460 if (coding->src_multibyte != coding->dst_multibyte)
fa42c37f 6461 {
df7492f9 6462 if (coding->src_multibyte)
fa42c37f 6463 {
71c81426 6464 int multibytep = 1;
4533845d 6465 EMACS_INT consumed_chars = 0;
d46c5b12 6466
df7492f9
KH		 6467 while (1)
6468 {
8f924df7 6469 const unsigned char *src_base = src;
df7492f9 6470 int c;
b73bfc1c 6471
df7492f9 6472 ONE_MORE_BYTE (c);
119852e7 6473 if (dst == dst_end)
df7492f9 6474 {
119852e7
KH		 6475 if (EQ (coding->src_object, coding->dst_object))
6476 dst_end = (unsigned char *) src;
6477 if (dst == dst_end)
df7492f9 6478 {
119852e7
KH		 6479 EMACS_INT offset = src - coding->source;
6480
6481 dst = alloc_destination (coding, src_end - src + 1,
6482 dst);
6483 dst_end = coding->destination + coding->dst_bytes;
6484 coding_set_source (coding);
6485 src = coding->source + offset;
6486 src_end = coding->source + coding->src_bytes;
db274c7a
KH		 6487 if (EQ (coding->src_object, coding->dst_object))
6488 dst_end = (unsigned char *) src;
df7492f9 6489 }
df7492f9
KH		 6490 }
6491 *dst++ = c;
6492 produced_chars++;
6493 }
6494 no_more_source:
6495 ;
fa42c37f
KH		 6496 }
6497 else
df7492f9
KH		 6498 while (src < src_end)
6499 {
71c81426 6500 int multibytep = 1;
df7492f9 6501 int c = *src++;
b73bfc1c 6502
df7492f9
KH		 6503 if (dst >= dst_end - 1)
6504 {
2c78b7e1 6505 if (EQ (coding->src_object, coding->dst_object))
8f924df7 6506 dst_end = (unsigned char *) src;
2c78b7e1
KH		 6507 if (dst >= dst_end - 1)
6508 {
119852e7 6509 EMACS_INT offset = src - coding->source;
db274c7a 6510 EMACS_INT more_bytes;
119852e7 6511
db274c7a
KH		 6512 if (EQ (coding->src_object, coding->dst_object))
6513 more_bytes = ((src_end - src) / 2) + 2;
6514 else
6515 more_bytes = src_end - src + 2;
6516 dst = alloc_destination (coding, more_bytes, dst);
2c78b7e1
KH		 6517 dst_end = coding->destination + coding->dst_bytes;
6518 coding_set_source (coding);
119852e7 6519 src = coding->source + offset;
2c78b7e1 6520 src_end = coding->source + coding->src_bytes;
db274c7a
KH		 6521 if (EQ (coding->src_object, coding->dst_object))
6522 dst_end = (unsigned char *) src;
2c78b7e1 6523 }
df7492f9
KH		 6524 }
6525 EMIT_ONE_BYTE (c);
6526 }
d46c5b12 6527 }
df7492f9
KH		 6528 else
6529 {
6530 if (!EQ (coding->src_object, coding->dst_object))
fa42c37f 6531 {
119852e7 6532 EMACS_INT require = coding->src_bytes - coding->dst_bytes;
4ed46869 6533
df7492f9 6534 if (require > 0)
fa42c37f 6535 {
df7492f9
KH		 6536 EMACS_INT offset = src - coding->source;
6537
6538 dst = alloc_destination (coding, require, dst);
6539 coding_set_source (coding);
6540 src = coding->source + offset;
6541 src_end = coding->source + coding->src_bytes;
fa42c37f
KH		 6542 }
6543 }
119852e7 6544 produced_chars = coding->consumed_char;
df7492f9 6545 while (src < src_end)
14daee73 6546 *dst++ = *src++;
fa42c37f
KH		 6547 }
6548 }
6549
df7492f9 6550 produced = dst - (coding->destination + coding->produced);
284201e4 6551 if (BUFFERP (coding->dst_object) && produced_chars > 0)
df7492f9
KH		 6552 insert_from_gap (produced_chars, produced);
6553 coding->produced += produced;
6554 coding->produced_char += produced_chars;
69a80ea3 6555 return carryover;
fa42c37f
KH		 6556}
6557
ff0dacd7
KH		 6558/* Compose text in CODING->object according to the annotation data at
6559 CHARBUF. CHARBUF is an array:
6560 [ -LENGTH ANNOTATION_MASK FROM TO METHOD COMP_LEN [ COMPONENTS... ] ]
df7492f9 6561 */
4ed46869 6562
df7492f9 6563static INLINE void
69a80ea3 6564produce_composition (coding, charbuf, pos)
4ed46869 6565 struct coding_system *coding;
df7492f9 6566 int *charbuf;
69a80ea3 6567 EMACS_INT pos;
4ed46869 6568{
df7492f9 6569 int len;
69a80ea3 6570 EMACS_INT to;
df7492f9 6571 enum composition_method method;
df7492f9 6572 Lisp_Object components;
fa42c37f 6573
df7492f9 6574 len = -charbuf[0];
69a80ea3 6575 to = pos + charbuf[2];
9ffd559c
KH		 6576 if (to <= pos)
6577 return;
69a80ea3 6578 method = (enum composition_method) (charbuf[3]);
d46c5b12 6579
df7492f9
KH		 6580 if (method == COMPOSITION_RELATIVE)
6581 components = Qnil;
9ffd559c
KH		 6582 else if (method >= COMPOSITION_WITH_RULE
6583 && method <= COMPOSITION_WITH_RULE_ALTCHARS)
d46c5b12 6584 {
df7492f9
KH		 6585 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
6586 int i;
b73bfc1c 6587
69a80ea3
KH		 6588 len -= 4;
6589 charbuf += 4;
df7492f9 6590 for (i = 0; i < len; i++)
9ffd559c
KH		 6591 {
6592 args[i] = make_number (charbuf[i]);
f75c90a9 6593 if (charbuf[i] < 0)
9ffd559c
KH		 6594 return;
6595 }
df7492f9
KH		 6596 components = (method == COMPOSITION_WITH_ALTCHARS
6597 ? Fstring (len, args) : Fvector (len, args));
d46c5b12 6598 }
9ffd559c
KH		 6599 else
6600 return;
69a80ea3 6601 compose_text (pos, to, components, Qnil, coding->dst_object);
d46c5b12
KH		 6602}
6603
d46c5b12 6604
ff0dacd7
KH		 6605/* Put `charset' property on text in CODING->object according to
6606 the annotation data at CHARBUF. CHARBUF is an array:
69a80ea3 6607 [ -LENGTH ANNOTATION_MASK NCHARS CHARSET-ID ]
ff0dacd7 6608 */
d46c5b12 6609
ff0dacd7 6610static INLINE void
69a80ea3 6611produce_charset (coding, charbuf, pos)
d46c5b12 6612 struct coding_system *coding;
ff0dacd7 6613 int *charbuf;
69a80ea3 6614 EMACS_INT pos;
d46c5b12 6615{
69a80ea3
KH		 6616 EMACS_INT from = pos - charbuf[2];
6617 struct charset *charset = CHARSET_FROM_ID (charbuf[3]);
b73bfc1c 6618
69a80ea3 6619 Fput_text_property (make_number (from), make_number (pos),
ff0dacd7
KH		 6620 Qcharset, CHARSET_NAME (charset),
6621 coding->dst_object);
d46c5b12
KH		 6622}
6623
d46c5b12 6624
df7492f9
KH		 6625#define CHARBUF_SIZE 0x4000
6626
6627#define ALLOC_CONVERSION_WORK_AREA(coding) \
6628 do { \
6629 int size = CHARBUF_SIZE;; \
6630 \
6631 coding->charbuf = NULL; \
6632 while (size > 1024) \
6633 { \
6634 coding->charbuf = (int *) alloca (sizeof (int) * size); \
6635 if (coding->charbuf) \
6636 break; \
6637 size >>= 1; \
6638 } \
6639 if (! coding->charbuf) \
6640 { \
065e3595 6641 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_MEM); \
df7492f9
KH		 6642 return coding->result; \
6643 } \
6644 coding->charbuf_size = size; \
6645 } while (0)
4ed46869 6646
d46c5b12
KH		 6647
6648static void
69a80ea3 6649produce_annotation (coding, pos)
d46c5b12 6650 struct coding_system *coding;
69a80ea3 6651 EMACS_INT pos;
d46c5b12 6652{
df7492f9
KH		 6653 int *charbuf = coding->charbuf;
6654 int *charbuf_end = charbuf + coding->charbuf_used;
d46c5b12 6655
ff0dacd7
KH		 6656 if (NILP (coding->dst_object))
6657 return;
d46c5b12 6658
df7492f9 6659 while (charbuf < charbuf_end)
a84f1519 6660 {
df7492f9 6661 if (*charbuf >= 0)
69a80ea3 6662 pos += *charbuf++;
d46c5b12 6663 else
d46c5b12 6664 {
df7492f9 6665 int len = -*charbuf;
ff0dacd7 6666 switch (charbuf[1])
df7492f9
KH		 6667 {
6668 case CODING_ANNOTATE_COMPOSITION_MASK:
69a80ea3 6669 produce_composition (coding, charbuf, pos);
df7492f9 6670 break;
ff0dacd7 6671 case CODING_ANNOTATE_CHARSET_MASK:
69a80ea3 6672 produce_charset (coding, charbuf, pos);
ff0dacd7 6673 break;
df7492f9
KH		 6674 default:
6675 abort ();
6676 }
6677 charbuf += len;
d46c5b12 6678 }
a84f1519 6679 }
d46c5b12
KH		 6680}
6681
df7492f9
KH		 6682/* Decode the data at CODING->src_object into CODING->dst_object.
6683 CODING->src_object is a buffer, a string, or nil.
6684 CODING->dst_object is a buffer.
d46c5b12 6685
df7492f9
KH		 6686 If CODING->src_object is a buffer, it must be the current buffer.
6687 In this case, if CODING->src_pos is positive, it is a position of
6688 the source text in the buffer, otherwise, the source text is in the
6689 gap area of the buffer, and CODING->src_pos specifies the offset of
6690 the text from GPT (which must be the same as PT). If this is the
6691 same buffer as CODING->dst_object, CODING->src_pos must be
6692 negative.
d46c5b12 6693
b6828792 6694 If CODING->src_object is a string, CODING->src_pos is an index to
df7492f9 6695 that string.
d46c5b12 6696
df7492f9
KH		 6697 If CODING->src_object is nil, CODING->source must already point to
6698 the non-relocatable memory area. In this case, CODING->src_pos is
6699 an offset from CODING->source.
73be902c 6700
df7492f9
KH		 6701 The decoded data is inserted at the current point of the buffer
6702 CODING->dst_object.
6703*/
d46c5b12 6704
df7492f9
KH		 6705static int
6706decode_coding (coding)
d46c5b12 6707 struct coding_system *coding;
d46c5b12 6708{
df7492f9 6709 Lisp_Object attrs;
24a73b0a 6710 Lisp_Object undo_list;
7d64c6ad 6711 Lisp_Object translation_table;
69a80ea3
KH		 6712 int carryover;
6713 int i;
d46c5b12 6714
df7492f9
KH		 6715 if (BUFFERP (coding->src_object)
6716 && coding->src_pos > 0
6717 && coding->src_pos < GPT
6718 && coding->src_pos + coding->src_chars > GPT)
6719 move_gap_both (coding->src_pos, coding->src_pos_byte);
d46c5b12 6720
24a73b0a 6721 undo_list = Qt;
df7492f9 6722 if (BUFFERP (coding->dst_object))
1c3478b0 6723 {
df7492f9
KH		 6724 if (current_buffer != XBUFFER (coding->dst_object))
6725 set_buffer_internal (XBUFFER (coding->dst_object));
6726 if (GPT != PT)
6727 move_gap_both (PT, PT_BYTE);
24a73b0a
KH		 6728 undo_list = current_buffer->undo_list;
6729 current_buffer->undo_list = Qt;
1c3478b0
KH		 6730 }
6731
df7492f9
KH		 6732 coding->consumed = coding->consumed_char = 0;
6733 coding->produced = coding->produced_char = 0;
6734 coding->chars_at_source = 0;
065e3595 6735 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 6736 coding->errors = 0;
1c3478b0 6737
df7492f9
KH		 6738 ALLOC_CONVERSION_WORK_AREA (coding);
6739
6740 attrs = CODING_ID_ATTRS (coding->id);
2170c8f0 6741 translation_table = get_translation_table (attrs, 0, NULL);
df7492f9 6742
69a80ea3 6743 carryover = 0;
df7492f9 6744 do
b73bfc1c 6745 {
69a80ea3
KH		 6746 EMACS_INT pos = coding->dst_pos + coding->produced_char;
6747
df7492f9
KH		 6748 coding_set_source (coding);
6749 coding->annotated = 0;
69a80ea3 6750 coding->charbuf_used = carryover;
df7492f9 6751 (*(coding->decoder)) (coding);
df7492f9 6752 coding_set_destination (coding);
69a80ea3 6753 carryover = produce_chars (coding, translation_table, 0);
df7492f9 6754 if (coding->annotated)
69a80ea3
KH		 6755 produce_annotation (coding, pos);
6756 for (i = 0; i < carryover; i++)
6757 coding->charbuf[i]
6758 = coding->charbuf[coding->charbuf_used - carryover + i];
d46c5b12 6759 }
df7492f9 6760 while (coding->consumed < coding->src_bytes
54b367bb
KH		 6761 && (coding->result == CODING_RESULT_SUCCESS
6762 || coding->result == CODING_RESULT_INVALID_SRC));
d46c5b12 6763
69a80ea3
KH		 6764 if (carryover > 0)
6765 {
6766 coding_set_destination (coding);
6767 coding->charbuf_used = carryover;
6768 produce_chars (coding, translation_table, 1);
6769 }
6770
df7492f9
KH		 6771 coding->carryover_bytes = 0;
6772 if (coding->consumed < coding->src_bytes)
d46c5b12 6773 {
df7492f9 6774 int nbytes = coding->src_bytes - coding->consumed;
8f924df7 6775 const unsigned char *src;
df7492f9
KH		 6776
6777 coding_set_source (coding);
6778 coding_set_destination (coding);
6779 src = coding->source + coding->consumed;
6780
6781 if (coding->mode & CODING_MODE_LAST_BLOCK)
1c3478b0 6782 {
df7492f9
KH		 6783 /* Flush out unprocessed data as binary chars. We are sure
6784 that the number of data is less than the size of
6785 coding->charbuf. */
065e3595 6786 coding->charbuf_used = 0;
b2dab6c8
JR		 6787 coding->chars_at_source = 0;
6788
df7492f9 6789 while (nbytes-- > 0)
1c3478b0 6790 {
df7492f9 6791 int c = *src++;
98725083 6792
1c91457d
KH		 6793 if (c & 0x80)
6794 c = BYTE8_TO_CHAR (c);
6795 coding->charbuf[coding->charbuf_used++] = c;
1c3478b0 6796 }
f6cbaf43 6797 produce_chars (coding, Qnil, 1);
d46c5b12 6798 }
d46c5b12 6799 else
df7492f9
KH		 6800 {
6801 /* Record unprocessed bytes in coding->carryover. We are
6802 sure that the number of data is less than the size of
6803 coding->carryover. */
6804 unsigned char *p = coding->carryover;
6805
6806 coding->carryover_bytes = nbytes;
6807 while (nbytes-- > 0)
6808 *p++ = *src++;
1c3478b0 6809 }
df7492f9 6810 coding->consumed = coding->src_bytes;
b73bfc1c 6811 }
69f76525 6812
4347441b
KH		 6813 if (! EQ (CODING_ID_EOL_TYPE (coding->id), Qunix))
6814 decode_eol (coding);
24a73b0a
KH		 6815 if (BUFFERP (coding->dst_object))
6816 {
6817 current_buffer->undo_list = undo_list;
6818 record_insert (coding->dst_pos, coding->produced_char);
6819 }
73be902c 6820 return coding->result;
4ed46869
KH		 6821}
6822
aaaf0b1e 6823
e1c23804 6824/* Extract an annotation datum from a composition starting at POS and
ff0dacd7
KH		 6825 ending before LIMIT of CODING->src_object (buffer or string), store
6826 the data in BUF, set *STOP to a starting position of the next
6827 composition (if any) or to LIMIT, and return the address of the
6828 next element of BUF.
6829
6830 If such an annotation is not found, set *STOP to a starting
6831 position of a composition after POS (if any) or to LIMIT, and
6832 return BUF. */
6833
6834static INLINE int *
6835handle_composition_annotation (pos, limit, coding, buf, stop)
6836 EMACS_INT pos, limit;
aaaf0b1e 6837 struct coding_system *coding;
ff0dacd7
KH		 6838 int *buf;
6839 EMACS_INT *stop;
aaaf0b1e 6840{
ff0dacd7
KH		 6841 EMACS_INT start, end;
6842 Lisp_Object prop;
aaaf0b1e 6843
ff0dacd7
KH		 6844 if (! find_composition (pos, limit, &start, &end, &prop, coding->src_object)
6845 || end > limit)
6846 *stop = limit;
6847 else if (start > pos)
6848 *stop = start;
6849 else
aaaf0b1e 6850 {
ff0dacd7 6851 if (start == pos)
aaaf0b1e 6852 {
ff0dacd7
KH		 6853 /* We found a composition. Store the corresponding
6854 annotation data in BUF. */
6855 int *head = buf;
6856 enum composition_method method = COMPOSITION_METHOD (prop);
6857 int nchars = COMPOSITION_LENGTH (prop);
6858
69a80ea3 6859 ADD_COMPOSITION_DATA (buf, nchars, method);
ff0dacd7 6860 if (method != COMPOSITION_RELATIVE)
aaaf0b1e 6861 {
ff0dacd7
KH		 6862 Lisp_Object components;
6863 int len, i, i_byte;
6864
6865 components = COMPOSITION_COMPONENTS (prop);
6866 if (VECTORP (components))
aaaf0b1e 6867 {
ff0dacd7
KH		 6868 len = XVECTOR (components)->size;
6869 for (i = 0; i < len; i++)
6870 *buf++ = XINT (AREF (components, i));
aaaf0b1e 6871 }
ff0dacd7 6872 else if (STRINGP (components))
aaaf0b1e 6873 {
8f924df7 6874 len = SCHARS (components);
ff0dacd7
KH		 6875 i = i_byte = 0;
6876 while (i < len)
6877 {
6878 FETCH_STRING_CHAR_ADVANCE (*buf, components, i, i_byte);
6879 buf++;
6880 }
6881 }
6882 else if (INTEGERP (components))
6883 {
6884 len = 1;
6885 *buf++ = XINT (components);
6886 }
6887 else if (CONSP (components))
6888 {
6889 for (len = 0; CONSP (components);
6890 len++, components = XCDR (components))
6891 *buf++ = XINT (XCAR (components));
aaaf0b1e 6892 }
aaaf0b1e 6893 else
ff0dacd7
KH		 6894 abort ();
6895 *head -= len;
aaaf0b1e 6896 }
aaaf0b1e 6897 }
ff0dacd7
KH		 6898
6899 if (find_composition (end, limit, &start, &end, &prop,
6900 coding->src_object)
6901 && end <= limit)
6902 *stop = start;
6903 else
6904 *stop = limit;
aaaf0b1e 6905 }
ff0dacd7
KH		 6906 return buf;
6907}
6908
6909
e1c23804 6910/* Extract an annotation datum from a text property `charset' at POS of
ff0dacd7
KH		 6911 CODING->src_object (buffer of string), store the data in BUF, set
6912 *STOP to the position where the value of `charset' property changes
6913 (limiting by LIMIT), and return the address of the next element of
6914 BUF.
6915
6916 If the property value is nil, set *STOP to the position where the
6917 property value is non-nil (limiting by LIMIT), and return BUF. */
6918
6919static INLINE int *
6920handle_charset_annotation (pos, limit, coding, buf, stop)
6921 EMACS_INT pos, limit;
6922 struct coding_system *coding;
6923 int *buf;
6924 EMACS_INT *stop;
6925{
6926 Lisp_Object val, next;
6927 int id;
6928
6929 val = Fget_text_property (make_number (pos), Qcharset, coding->src_object);
6930 if (! NILP (val) && CHARSETP (val))
6931 id = XINT (CHARSET_SYMBOL_ID (val));
6932 else
6933 id = -1;
69a80ea3 6934 ADD_CHARSET_DATA (buf, 0, id);
ff0dacd7
KH		 6935 next = Fnext_single_property_change (make_number (pos), Qcharset,
6936 coding->src_object,
6937 make_number (limit));
6938 *stop = XINT (next);
6939 return buf;
6940}
6941
6942
df7492f9 6943static void
09ee6fdd 6944consume_chars (coding, translation_table, max_lookup)
df7492f9 6945 struct coding_system *coding;
433f7f87 6946 Lisp_Object translation_table;
09ee6fdd 6947 int max_lookup;
df7492f9
KH		 6948{
6949 int *buf = coding->charbuf;
ff0dacd7 6950 int *buf_end = coding->charbuf + coding->charbuf_size;
7c78e542 6951 const unsigned char *src = coding->source + coding->consumed;
4776e638 6952 const unsigned char *src_end = coding->source + coding->src_bytes;
ff0dacd7
KH		 6953 EMACS_INT pos = coding->src_pos + coding->consumed_char;
6954 EMACS_INT end_pos = coding->src_pos + coding->src_chars;
df7492f9
KH		 6955 int multibytep = coding->src_multibyte;
6956 Lisp_Object eol_type;
6957 int c;
ff0dacd7 6958 EMACS_INT stop, stop_composition, stop_charset;
09ee6fdd 6959 int *lookup_buf = NULL;
433f7f87
KH		 6960
6961 if (! NILP (translation_table))
09ee6fdd 6962 lookup_buf = alloca (sizeof (int) * max_lookup);
88993dfd 6963
df7492f9
KH		 6964 eol_type = CODING_ID_EOL_TYPE (coding->id);
6965 if (VECTORP (eol_type))
6966 eol_type = Qunix;
88993dfd 6967
df7492f9
KH		 6968 /* Note: composition handling is not yet implemented. */
6969 coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
ec6d2bb8 6970
0b5670c9
KH		 6971 if (NILP (coding->src_object))
6972 stop = stop_composition = stop_charset = end_pos;
ff0dacd7 6973 else
0b5670c9
KH		 6974 {
6975 if (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK)
6976 stop = stop_composition = pos;
6977 else
6978 stop = stop_composition = end_pos;
6979 if (coding->common_flags & CODING_ANNOTATE_CHARSET_MASK)
6980 stop = stop_charset = pos;
6981 else
6982 stop_charset = end_pos;
6983 }
ec6d2bb8 6984
24a73b0a 6985 /* Compensate for CRLF and conversion. */
ff0dacd7 6986 buf_end -= 1 + MAX_ANNOTATION_LENGTH;
df7492f9 6987 while (buf < buf_end)
aaaf0b1e 6988 {
433f7f87
KH		 6989 Lisp_Object trans;
6990
df7492f9 6991 if (pos == stop)
ec6d2bb8 6992 {
df7492f9
KH		 6993 if (pos == end_pos)
6994 break;
ff0dacd7
KH		 6995 if (pos == stop_composition)
6996 buf = handle_composition_annotation (pos, end_pos, coding,
6997 buf, &stop_composition);
6998 if (pos == stop_charset)
6999 buf = handle_charset_annotation (pos, end_pos, coding,
7000 buf, &stop_charset);
7001 stop = (stop_composition < stop_charset
7002 ? stop_composition : stop_charset);
df7492f9
KH		 7003 }
7004
7005 if (! multibytep)
4776e638 7006 {
d3e4cb56 7007 EMACS_INT bytes;
aaaf0b1e 7008
ea29edf2
KH		 7009 if (coding->encoder == encode_coding_raw_text)
7010 c = *src++, pos++;
7011 else if ((bytes = MULTIBYTE_LENGTH (src, src_end)) > 0)
db274c7a 7012 c = STRING_CHAR_ADVANCE_NO_UNIFY (src), pos += bytes;
4776e638 7013 else
f03caae0 7014 c = BYTE8_TO_CHAR (*src), src++, pos++;
4776e638 7015 }
df7492f9 7016 else
db274c7a 7017 c = STRING_CHAR_ADVANCE_NO_UNIFY (src), pos++;
df7492f9
KH		 7018 if ((c == '\r') && (coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
7019 c = '\n';
7020 if (! EQ (eol_type, Qunix))
aaaf0b1e 7021 {
df7492f9 7022 if (c == '\n')
aaaf0b1e 7023 {
df7492f9
KH		 7024 if (EQ (eol_type, Qdos))
7025 *buf++ = '\r';
7026 else
7027 c = '\r';
aaaf0b1e
KH		 7028 }
7029 }
433f7f87 7030
e6a54062 7031 trans = Qnil;
09ee6fdd 7032 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 7033 if (NILP (trans))
433f7f87
KH		 7034 *buf++ = c;
7035 else
7036 {
7037 int from_nchars = 1, to_nchars = 1;
7038 int *lookup_buf_end;
7039 const unsigned char *p = src;
7040 int i;
7041
7042 lookup_buf[0] = c;
7043 for (i = 1; i < max_lookup && p < src_end; i++)
7044 lookup_buf[i] = STRING_CHAR_ADVANCE (p);
7045 lookup_buf_end = lookup_buf + i;
7046 trans = get_translation (trans, lookup_buf, lookup_buf_end, 1,
7047 &from_nchars, &to_nchars);
7048 if (EQ (trans, Qt)
7049 || buf + to_nchars > buf_end)
7050 break;
7051 *buf++ = *lookup_buf;
7052 for (i = 1; i < to_nchars; i++)
7053 *buf++ = XINT (AREF (trans, i));
7054 for (i = 1; i < from_nchars; i++, pos++)
7055 src += MULTIBYTE_LENGTH_NO_CHECK (src);
7056 }
aaaf0b1e 7057 }
ec6d2bb8 7058
df7492f9
KH		 7059 coding->consumed = src - coding->source;
7060 coding->consumed_char = pos - coding->src_pos;
7061 coding->charbuf_used = buf - coding->charbuf;
7062 coding->chars_at_source = 0;
aaaf0b1e
KH		 7063}
7064
4ed46869 7065
df7492f9
KH		 7066/* Encode the text at CODING->src_object into CODING->dst_object.
7067 CODING->src_object is a buffer or a string.
7068 CODING->dst_object is a buffer or nil.
7069
7070 If CODING->src_object is a buffer, it must be the current buffer.
7071 In this case, if CODING->src_pos is positive, it is a position of
7072 the source text in the buffer, otherwise. the source text is in the
7073 gap area of the buffer, and coding->src_pos specifies the offset of
7074 the text from GPT (which must be the same as PT). If this is the
7075 same buffer as CODING->dst_object, CODING->src_pos must be
7076 negative and CODING should not have `pre-write-conversion'.
7077
7078 If CODING->src_object is a string, CODING should not have
7079 `pre-write-conversion'.
7080
7081 If CODING->dst_object is a buffer, the encoded data is inserted at
7082 the current point of that buffer.
7083
7084 If CODING->dst_object is nil, the encoded data is placed at the
7085 memory area specified by CODING->destination. */
7086
7087static int
7088encode_coding (coding)
4ed46869 7089 struct coding_system *coding;
4ed46869 7090{
df7492f9 7091 Lisp_Object attrs;
7d64c6ad 7092 Lisp_Object translation_table;
09ee6fdd 7093 int max_lookup;
9861e777 7094
df7492f9 7095 attrs = CODING_ID_ATTRS (coding->id);
ea29edf2
KH		 7096 if (coding->encoder == encode_coding_raw_text)
7097 translation_table = Qnil, max_lookup = 0;
7098 else
7099 translation_table = get_translation_table (attrs, 1, &max_lookup);
4ed46869 7100
df7492f9 7101 if (BUFFERP (coding->dst_object))
8844fa83 7102 {
df7492f9
KH		 7103 set_buffer_internal (XBUFFER (coding->dst_object));
7104 coding->dst_multibyte
7105 = ! NILP (current_buffer->enable_multibyte_characters);
8844fa83 7106 }
4ed46869 7107
b73bfc1c 7108 coding->consumed = coding->consumed_char = 0;
df7492f9 7109 coding->produced = coding->produced_char = 0;
065e3595 7110 record_conversion_result (coding, CODING_RESULT_SUCCESS);
b73bfc1c 7111 coding->errors = 0;
b73bfc1c 7112
df7492f9 7113 ALLOC_CONVERSION_WORK_AREA (coding);
4ed46869 7114
df7492f9
KH		 7115 do {
7116 coding_set_source (coding);
09ee6fdd 7117 consume_chars (coding, translation_table, max_lookup);
df7492f9
KH		 7118 coding_set_destination (coding);
7119 (*(coding->encoder)) (coding);
7120 } while (coding->consumed_char < coding->src_chars);
7121
284201e4 7122 if (BUFFERP (coding->dst_object) && coding->produced_char > 0)
df7492f9
KH		 7123 insert_from_gap (coding->produced_char, coding->produced);
7124
7125 return (coding->result);
ec6d2bb8
KH		 7126}
7127
fb88bf2d 7128
24a73b0a
KH		 7129/* Name (or base name) of work buffer for code conversion. */
7130static Lisp_Object Vcode_conversion_workbuf_name;
d46c5b12 7131
24a73b0a
KH		 7132/* A working buffer used by the top level conversion. Once it is
7133 created, it is never destroyed. It has the name
7134 Vcode_conversion_workbuf_name. The other working buffers are
7135 destroyed after the use is finished, and their names are modified
7136 versions of Vcode_conversion_workbuf_name. */
7137static Lisp_Object Vcode_conversion_reused_workbuf;
b73bfc1c 7138
24a73b0a
KH		 7139/* 1 iff Vcode_conversion_reused_workbuf is already in use. */
7140static int reused_workbuf_in_use;
4ed46869 7141
24a73b0a
KH		 7142
7143/* Return a working buffer of code convesion. MULTIBYTE specifies the
7144 multibyteness of returning buffer. */
b73bfc1c 7145
f6cbaf43 7146static Lisp_Object
24a73b0a 7147make_conversion_work_buffer (multibyte)
f6cbaf43 7148 int multibyte;
df7492f9 7149{
24a73b0a
KH		 7150 Lisp_Object name, workbuf;
7151 struct buffer *current;
4ed46869 7152
24a73b0a 7153 if (reused_workbuf_in_use++)
065e3595
KH		 7154 {
7155 name = Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name, Qnil);
7156 workbuf = Fget_buffer_create (name);
7157 }
df7492f9 7158 else
065e3595 7159 {
159bd5a2 7160 if (NILP (Fbuffer_live_p (Vcode_conversion_reused_workbuf)))
a993c7a1
KH		 7161 Vcode_conversion_reused_workbuf
7162 = Fget_buffer_create (Vcode_conversion_workbuf_name);
7163 workbuf = Vcode_conversion_reused_workbuf;
065e3595 7164 }
24a73b0a
KH		 7165 current = current_buffer;
7166 set_buffer_internal (XBUFFER (workbuf));
df36ff1f
CY		 7167 /* We can't allow modification hooks to run in the work buffer. For
7168 instance, directory_files_internal assumes that file decoding
7169 doesn't compile new regexps. */
7170 Fset (Fmake_local_variable (Qinhibit_modification_hooks), Qt);
3ed051d4 7171 Ferase_buffer ();
df7492f9 7172 current_buffer->undo_list = Qt;
24a73b0a 7173 current_buffer->enable_multibyte_characters = multibyte ? Qt : Qnil;
df7492f9 7174 set_buffer_internal (current);
24a73b0a 7175 return workbuf;
df7492f9 7176}
d46c5b12 7177
24a73b0a 7178
4776e638 7179static Lisp_Object
24a73b0a
KH		 7180code_conversion_restore (arg)
7181 Lisp_Object arg;
4776e638 7182{
24a73b0a 7183 Lisp_Object current, workbuf;
948bdcf3 7184 struct gcpro gcpro1;
24a73b0a 7185
948bdcf3 7186 GCPRO1 (arg);
24a73b0a
KH		 7187 current = XCAR (arg);
7188 workbuf = XCDR (arg);
7189 if (! NILP (workbuf))
7190 {
7191 if (EQ (workbuf, Vcode_conversion_reused_workbuf))
7192 reused_workbuf_in_use = 0;
7193 else if (! NILP (Fbuffer_live_p (workbuf)))
7194 Fkill_buffer (workbuf);
7195 }
7196 set_buffer_internal (XBUFFER (current));
948bdcf3 7197 UNGCPRO;
4776e638
KH		 7198 return Qnil;
7199}
b73bfc1c 7200
24a73b0a
KH		 7201Lisp_Object
7202code_conversion_save (with_work_buf, multibyte)
4776e638 7203 int with_work_buf, multibyte;
df7492f9 7204{
24a73b0a 7205 Lisp_Object workbuf = Qnil;
b73bfc1c 7206
4776e638 7207 if (with_work_buf)
24a73b0a
KH		 7208 workbuf = make_conversion_work_buffer (multibyte);
7209 record_unwind_protect (code_conversion_restore,
7210 Fcons (Fcurrent_buffer (), workbuf));
4776e638 7211 return workbuf;
df7492f9 7212}
d46c5b12 7213
df7492f9
KH		 7214int
7215decode_coding_gap (coding, chars, bytes)
7216 struct coding_system *coding;
7217 EMACS_INT chars, bytes;
7218{
7219 int count = specpdl_ptr - specpdl;
5e5c78be 7220 Lisp_Object attrs;
fb88bf2d 7221
24a73b0a 7222 code_conversion_save (0, 0);
ec6d2bb8 7223
24a73b0a 7224 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7225 coding->src_chars = chars;
7226 coding->src_bytes = bytes;
7227 coding->src_pos = -chars;
7228 coding->src_pos_byte = -bytes;
7229 coding->src_multibyte = chars < bytes;
24a73b0a 7230 coding->dst_object = coding->src_object;
df7492f9
KH		 7231 coding->dst_pos = PT;
7232 coding->dst_pos_byte = PT_BYTE;
71c81426 7233 coding->dst_multibyte = ! NILP (current_buffer->enable_multibyte_characters);
4ed46869 7234
df7492f9
KH		 7235 if (CODING_REQUIRE_DETECTION (coding))
7236 detect_coding (coding);
8f924df7 7237
9286b333 7238 coding->mode |= CODING_MODE_LAST_BLOCK;
287c57d7 7239 current_buffer->text->inhibit_shrinking = 1;
df7492f9 7240 decode_coding (coding);
287c57d7 7241 current_buffer->text->inhibit_shrinking = 0;
d46c5b12 7242
5e5c78be
KH		 7243 attrs = CODING_ID_ATTRS (coding->id);
7244 if (! NILP (CODING_ATTR_POST_READ (attrs)))
b73bfc1c 7245 {
5e5c78be
KH		 7246 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
7247 Lisp_Object val;
7248
7249 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
5e5c78be
KH		 7250 val = call1 (CODING_ATTR_POST_READ (attrs),
7251 make_number (coding->produced_char));
5e5c78be
KH		 7252 CHECK_NATNUM (val);
7253 coding->produced_char += Z - prev_Z;
7254 coding->produced += Z_BYTE - prev_Z_BYTE;
b73bfc1c 7255 }
4ed46869 7256
df7492f9 7257 unbind_to (count, Qnil);
b73bfc1c
KH		 7258 return coding->result;
7259}
52d41803 7260
4ed46869 7261int
df7492f9 7262encode_coding_gap (coding, chars, bytes)
4ed46869 7263 struct coding_system *coding;
df7492f9 7264 EMACS_INT chars, bytes;
4ed46869 7265{
df7492f9 7266 int count = specpdl_ptr - specpdl;
4ed46869 7267
24a73b0a 7268 code_conversion_save (0, 0);
4ed46869 7269
24a73b0a 7270 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7271 coding->src_chars = chars;
7272 coding->src_bytes = bytes;
7273 coding->src_pos = -chars;
7274 coding->src_pos_byte = -bytes;
7275 coding->src_multibyte = chars < bytes;
7276 coding->dst_object = coding->src_object;
7277 coding->dst_pos = PT;
7278 coding->dst_pos_byte = PT_BYTE;
4ed46869 7279
df7492f9 7280 encode_coding (coding);
b73bfc1c 7281
df7492f9
KH		 7282 unbind_to (count, Qnil);
7283 return coding->result;
7284}
4ed46869 7285
d46c5b12 7286
df7492f9
KH		 7287/* Decode the text in the range FROM/FROM_BYTE and TO/TO_BYTE in
7288 SRC_OBJECT into DST_OBJECT by coding context CODING.
b73bfc1c 7289
df7492f9 7290 SRC_OBJECT is a buffer, a string, or Qnil.
b73bfc1c 7291
df7492f9
KH		 7292 If it is a buffer, the text is at point of the buffer. FROM and TO
7293 are positions in the buffer.
b73bfc1c 7294
df7492f9
KH		 7295 If it is a string, the text is at the beginning of the string.
7296 FROM and TO are indices to the string.
4ed46869 7297
df7492f9
KH		 7298 If it is nil, the text is at coding->source. FROM and TO are
7299 indices to coding->source.
bb10be8b 7300
df7492f9 7301 DST_OBJECT is a buffer, Qt, or Qnil.
4ed46869 7302
df7492f9
KH		 7303 If it is a buffer, the decoded text is inserted at point of the
7304 buffer. If the buffer is the same as SRC_OBJECT, the source text
7305 is deleted.
4ed46869 7306
df7492f9
KH		 7307 If it is Qt, a string is made from the decoded text, and
7308 set in CODING->dst_object.
d46c5b12 7309
df7492f9 7310 If it is Qnil, the decoded text is stored at CODING->destination.
2cb26057 7311 The caller must allocate CODING->dst_bytes bytes at
df7492f9
KH		 7312 CODING->destination by xmalloc. If the decoded text is longer than
7313 CODING->dst_bytes, CODING->destination is relocated by xrealloc.
7314 */
d46c5b12 7315
df7492f9
KH		 7316void
7317decode_coding_object (coding, src_object, from, from_byte, to, to_byte,
7318 dst_object)
d46c5b12 7319 struct coding_system *coding;
df7492f9
KH		 7320 Lisp_Object src_object;
7321 EMACS_INT from, from_byte, to, to_byte;
7322 Lisp_Object dst_object;
d46c5b12 7323{
df7492f9
KH		 7324 int count = specpdl_ptr - specpdl;
7325 unsigned char *destination;
7326 EMACS_INT dst_bytes;
7327 EMACS_INT chars = to - from;
7328 EMACS_INT bytes = to_byte - from_byte;
7329 Lisp_Object attrs;
4776e638 7330 int saved_pt = -1, saved_pt_byte;
64cedb0c 7331 int need_marker_adjustment = 0;
b3bfad50 7332 Lisp_Object old_deactivate_mark;
d46c5b12 7333
b3bfad50 7334 old_deactivate_mark = Vdeactivate_mark;
93dec019 7335
df7492f9 7336 if (NILP (dst_object))
d46c5b12 7337 {
df7492f9
KH		 7338 destination = coding->destination;
7339 dst_bytes = coding->dst_bytes;
d46c5b12 7340 }
93dec019 7341
df7492f9
KH		 7342 coding->src_object = src_object;
7343 coding->src_chars = chars;
7344 coding->src_bytes = bytes;
7345 coding->src_multibyte = chars < bytes;
70ad9fc4 7346
df7492f9 7347 if (STRINGP (src_object))
d46c5b12 7348 {
df7492f9
KH		 7349 coding->src_pos = from;
7350 coding->src_pos_byte = from_byte;
d46c5b12 7351 }
df7492f9 7352 else if (BUFFERP (src_object))
88993dfd 7353 {
df7492f9
KH		 7354 set_buffer_internal (XBUFFER (src_object));
7355 if (from != GPT)
7356 move_gap_both (from, from_byte);
7357 if (EQ (src_object, dst_object))
fb88bf2d 7358 {
64cedb0c
KH		 7359 struct Lisp_Marker *tail;
7360
7361 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7362 {
7363 tail->need_adjustment
7364 = tail->charpos == (tail->insertion_type ? from : to);
7365 need_marker_adjustment |= tail->need_adjustment;
7366 }
4776e638 7367 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9 7368 TEMP_SET_PT_BOTH (from, from_byte);
f4a3cc44 7369 current_buffer->text->inhibit_shrinking = 1;
df7492f9
KH		 7370 del_range_both (from, from_byte, to, to_byte, 1);
7371 coding->src_pos = -chars;
7372 coding->src_pos_byte = -bytes;
fb88bf2d 7373 }
df7492f9 7374 else
fb88bf2d 7375 {
df7492f9
KH		 7376 coding->src_pos = from;
7377 coding->src_pos_byte = from_byte;
fb88bf2d 7378 }
88993dfd
KH		 7379 }
7380
df7492f9
KH		 7381 if (CODING_REQUIRE_DETECTION (coding))
7382 detect_coding (coding);
7383 attrs = CODING_ID_ATTRS (coding->id);
d46c5b12 7384
2cb26057
KH		 7385 if (EQ (dst_object, Qt)
7386 || (! NILP (CODING_ATTR_POST_READ (attrs))
7387 && NILP (dst_object)))
b73bfc1c 7388 {
a1567c45
SM		 7389 coding->dst_multibyte = !CODING_FOR_UNIBYTE (coding);
7390 coding->dst_object = code_conversion_save (1, coding->dst_multibyte);
df7492f9
KH		 7391 coding->dst_pos = BEG;
7392 coding->dst_pos_byte = BEG_BYTE;
b73bfc1c 7393 }
df7492f9 7394 else if (BUFFERP (dst_object))
d46c5b12 7395 {
24a73b0a 7396 code_conversion_save (0, 0);
df7492f9
KH		 7397 coding->dst_object = dst_object;
7398 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
7399 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
7400 coding->dst_multibyte
7401 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
d46c5b12
KH		 7402 }
7403 else
7404 {
24a73b0a 7405 code_conversion_save (0, 0);
df7492f9 7406 coding->dst_object = Qnil;
0154725e
SM		 7407 /* Most callers presume this will return a multibyte result, and they
7408 won't use `binary' or `raw-text' anyway, so let's not worry about
7409 CODING_FOR_UNIBYTE. */
bb555731 7410 coding->dst_multibyte = 1;
d46c5b12
KH		 7411 }
7412
df7492f9 7413 decode_coding (coding);
fa46990e 7414
df7492f9
KH		 7415 if (BUFFERP (coding->dst_object))
7416 set_buffer_internal (XBUFFER (coding->dst_object));
d46c5b12 7417
df7492f9 7418 if (! NILP (CODING_ATTR_POST_READ (attrs)))
d46c5b12 7419 {
b3bfad50 7420 struct gcpro gcpro1, gcpro2, gcpro3, gcpro4, gcpro5;
df7492f9 7421 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
2b4f9037 7422 Lisp_Object val;
d46c5b12 7423
c0cc7f7f 7424 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
b3bfad50
KH		 7425 GCPRO5 (coding->src_object, coding->dst_object, src_object, dst_object,
7426 old_deactivate_mark);
d4850d67
KH		 7427 val = safe_call1 (CODING_ATTR_POST_READ (attrs),
7428 make_number (coding->produced_char));
df7492f9
KH		 7429 UNGCPRO;
7430 CHECK_NATNUM (val);
7431 coding->produced_char += Z - prev_Z;
7432 coding->produced += Z_BYTE - prev_Z_BYTE;
d46c5b12 7433 }
de79a6a5 7434
df7492f9 7435 if (EQ (dst_object, Qt))
ec6d2bb8 7436 {
df7492f9
KH		 7437 coding->dst_object = Fbuffer_string ();
7438 }
7439 else if (NILP (dst_object) && BUFFERP (coding->dst_object))
7440 {
7441 set_buffer_internal (XBUFFER (coding->dst_object));
7442 if (dst_bytes < coding->produced)
7443 {
b3bfad50 7444 destination = xrealloc (destination, coding->produced);
df7492f9
KH		 7445 if (! destination)
7446 {
065e3595
KH		 7447 record_conversion_result (coding,
7448 CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 7449 unbind_to (count, Qnil);
7450 return;
7451 }
7452 if (BEGV < GPT && GPT < BEGV + coding->produced_char)
7453 move_gap_both (BEGV, BEGV_BYTE);
7454 bcopy (BEGV_ADDR, destination, coding->produced);
7455 coding->destination = destination;
d46c5b12 7456 }
ec6d2bb8 7457 }
b73bfc1c 7458
4776e638
KH		 7459 if (saved_pt >= 0)
7460 {
7461 /* This is the case of:
7462 (BUFFERP (src_object) && EQ (src_object, dst_object))
7463 As we have moved PT while replacing the original buffer
7464 contents, we must recover it now. */
7465 set_buffer_internal (XBUFFER (src_object));
f4a3cc44 7466 current_buffer->text->inhibit_shrinking = 0;
4776e638
KH		 7467 if (saved_pt < from)
7468 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7469 else if (saved_pt < from + chars)
7470 TEMP_SET_PT_BOTH (from, from_byte);
7471 else if (! NILP (current_buffer->enable_multibyte_characters))
7472 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7473 saved_pt_byte + (coding->produced - bytes));
7474 else
7475 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7476 saved_pt_byte + (coding->produced - bytes));
64cedb0c
KH		 7477
7478 if (need_marker_adjustment)
7479 {
7480 struct Lisp_Marker *tail;
7481
7482 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7483 if (tail->need_adjustment)
7484 {
7485 tail->need_adjustment = 0;
7486 if (tail->insertion_type)
7487 {
7488 tail->bytepos = from_byte;
7489 tail->charpos = from;
7490 }
7491 else
7492 {
7493 tail->bytepos = from_byte + coding->produced;
7494 tail->charpos
7495 = (NILP (current_buffer->enable_multibyte_characters)
7496 ? tail->bytepos : from + coding->produced_char);
7497 }
7498 }
7499 }
d46c5b12 7500 }
4776e638 7501
b3bfad50 7502 Vdeactivate_mark = old_deactivate_mark;
065e3595 7503 unbind_to (count, coding->dst_object);
d46c5b12
KH		 7504}
7505
d46c5b12 7506
df7492f9
KH		 7507void
7508encode_coding_object (coding, src_object, from, from_byte, to, to_byte,
7509 dst_object)
d46c5b12 7510 struct coding_system *coding;
df7492f9
KH		 7511 Lisp_Object src_object;
7512 EMACS_INT from, from_byte, to, to_byte;
7513 Lisp_Object dst_object;
d46c5b12 7514{
b73bfc1c 7515 int count = specpdl_ptr - specpdl;
df7492f9
KH		 7516 EMACS_INT chars = to - from;
7517 EMACS_INT bytes = to_byte - from_byte;
7518 Lisp_Object attrs;
4776e638 7519 int saved_pt = -1, saved_pt_byte;
64cedb0c 7520 int need_marker_adjustment = 0;
c02d943b 7521 int kill_src_buffer = 0;
b3bfad50 7522 Lisp_Object old_deactivate_mark;
df7492f9 7523
b3bfad50 7524 old_deactivate_mark = Vdeactivate_mark;
df7492f9
KH		 7525
7526 coding->src_object = src_object;
7527 coding->src_chars = chars;
7528 coding->src_bytes = bytes;
7529 coding->src_multibyte = chars < bytes;
7530
7531 attrs = CODING_ID_ATTRS (coding->id);
7532
64cedb0c
KH		 7533 if (EQ (src_object, dst_object))
7534 {
7535 struct Lisp_Marker *tail;
7536
7537 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7538 {
7539 tail->need_adjustment
7540 = tail->charpos == (tail->insertion_type ? from : to);
7541 need_marker_adjustment |= tail->need_adjustment;
7542 }
7543 }
7544
df7492f9 7545 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
6bac5b12 7546 {
24a73b0a 7547 coding->src_object = code_conversion_save (1, coding->src_multibyte);
df7492f9
KH		 7548 set_buffer_internal (XBUFFER (coding->src_object));
7549 if (STRINGP (src_object))
7550 insert_from_string (src_object, from, from_byte, chars, bytes, 0);
7551 else if (BUFFERP (src_object))
7552 insert_from_buffer (XBUFFER (src_object), from, chars, 0);
7553 else
7554 insert_1_both (coding->source + from, chars, bytes, 0, 0, 0);
d46c5b12 7555
df7492f9
KH		 7556 if (EQ (src_object, dst_object))
7557 {
7558 set_buffer_internal (XBUFFER (src_object));
4776e638 7559 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9
KH		 7560 del_range_both (from, from_byte, to, to_byte, 1);
7561 set_buffer_internal (XBUFFER (coding->src_object));
7562 }
7563
d4850d67
KH		 7564 {
7565 Lisp_Object args[3];
b3bfad50 7566 struct gcpro gcpro1, gcpro2, gcpro3, gcpro4, gcpro5;
d4850d67 7567
b3bfad50
KH		 7568 GCPRO5 (coding->src_object, coding->dst_object, src_object, dst_object,
7569 old_deactivate_mark);
d4850d67
KH		 7570 args[0] = CODING_ATTR_PRE_WRITE (attrs);
7571 args[1] = make_number (BEG);
7572 args[2] = make_number (Z);
7573 safe_call (3, args);
b3bfad50 7574 UNGCPRO;
d4850d67 7575 }
c02d943b
KH		 7576 if (XBUFFER (coding->src_object) != current_buffer)
7577 kill_src_buffer = 1;
ac87bbef 7578 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7579 if (BEG != GPT)
7580 move_gap_both (BEG, BEG_BYTE);
7581 coding->src_chars = Z - BEG;
7582 coding->src_bytes = Z_BYTE - BEG_BYTE;
7583 coding->src_pos = BEG;
7584 coding->src_pos_byte = BEG_BYTE;
7585 coding->src_multibyte = Z < Z_BYTE;
7586 }
7587 else if (STRINGP (src_object))
d46c5b12 7588 {
24a73b0a 7589 code_conversion_save (0, 0);
df7492f9
KH		 7590 coding->src_pos = from;
7591 coding->src_pos_byte = from_byte;
b73bfc1c 7592 }
df7492f9 7593 else if (BUFFERP (src_object))
b73bfc1c 7594 {
24a73b0a 7595 code_conversion_save (0, 0);
df7492f9 7596 set_buffer_internal (XBUFFER (src_object));
df7492f9 7597 if (EQ (src_object, dst_object))
d46c5b12 7598 {
4776e638 7599 saved_pt = PT, saved_pt_byte = PT_BYTE;
ff0dacd7
KH		 7600 coding->src_object = del_range_1 (from, to, 1, 1);
7601 coding->src_pos = 0;
7602 coding->src_pos_byte = 0;
d46c5b12 7603 }
df7492f9 7604 else
d46c5b12 7605 {
ff0dacd7
KH		 7606 if (from < GPT && to >= GPT)
7607 move_gap_both (from, from_byte);
df7492f9
KH		 7608 coding->src_pos = from;
7609 coding->src_pos_byte = from_byte;
d46c5b12 7610 }
d46c5b12 7611 }
4776e638 7612 else
24a73b0a 7613 code_conversion_save (0, 0);
d46c5b12 7614
df7492f9 7615 if (BUFFERP (dst_object))
88993dfd 7616 {
df7492f9 7617 coding->dst_object = dst_object;
28f67a95
KH		 7618 if (EQ (src_object, dst_object))
7619 {
7620 coding->dst_pos = from;
7621 coding->dst_pos_byte = from_byte;
7622 }
7623 else
7624 {
319a3947
KH		 7625 struct buffer *current = current_buffer;
7626
7627 set_buffer_temp (XBUFFER (dst_object));
7628 coding->dst_pos = PT;
7629 coding->dst_pos_byte = PT_BYTE;
7630 move_gap_both (coding->dst_pos, coding->dst_pos_byte);
7631 set_buffer_temp (current);
28f67a95 7632 }
df7492f9
KH		 7633 coding->dst_multibyte
7634 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
88993dfd 7635 }
df7492f9 7636 else if (EQ (dst_object, Qt))
d46c5b12 7637 {
df7492f9 7638 coding->dst_object = Qnil;
df7492f9 7639 coding->dst_bytes = coding->src_chars;
ac87bbef
KH		 7640 if (coding->dst_bytes == 0)
7641 coding->dst_bytes = 1;
7642 coding->destination = (unsigned char *) xmalloc (coding->dst_bytes);
df7492f9 7643 coding->dst_multibyte = 0;
d46c5b12
KH		 7644 }
7645 else
7646 {
df7492f9
KH		 7647 coding->dst_object = Qnil;
7648 coding->dst_multibyte = 0;
d46c5b12
KH		 7649 }
7650
df7492f9 7651 encode_coding (coding);
d46c5b12 7652
df7492f9 7653 if (EQ (dst_object, Qt))
d46c5b12 7654 {
df7492f9
KH		 7655 if (BUFFERP (coding->dst_object))
7656 coding->dst_object = Fbuffer_string ();
7657 else
d46c5b12 7658 {
df7492f9
KH		 7659 coding->dst_object
7660 = make_unibyte_string ((char *) coding->destination,
7661 coding->produced);
7662 xfree (coding->destination);
d46c5b12 7663 }
4ed46869 7664 }
d46c5b12 7665
4776e638
KH		 7666 if (saved_pt >= 0)
7667 {
7668 /* This is the case of:
7669 (BUFFERP (src_object) && EQ (src_object, dst_object))
7670 As we have moved PT while replacing the original buffer
7671 contents, we must recover it now. */
7672 set_buffer_internal (XBUFFER (src_object));
7673 if (saved_pt < from)
7674 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7675 else if (saved_pt < from + chars)
7676 TEMP_SET_PT_BOTH (from, from_byte);
7677 else if (! NILP (current_buffer->enable_multibyte_characters))
7678 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7679 saved_pt_byte + (coding->produced - bytes));
d46c5b12 7680 else
4776e638
KH		 7681 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7682 saved_pt_byte + (coding->produced - bytes));
64cedb0c
KH		 7683
7684 if (need_marker_adjustment)
7685 {
7686 struct Lisp_Marker *tail;
7687
7688 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7689 if (tail->need_adjustment)
7690 {
7691 tail->need_adjustment = 0;
7692 if (tail->insertion_type)
7693 {
7694 tail->bytepos = from_byte;
7695 tail->charpos = from;
7696 }
7697 else
7698 {
7699 tail->bytepos = from_byte + coding->produced;
7700 tail->charpos
7701 = (NILP (current_buffer->enable_multibyte_characters)
7702 ? tail->bytepos : from + coding->produced_char);
7703 }
7704 }
7705 }
4776e638
KH		 7706 }
7707
c02d943b
KH		 7708 if (kill_src_buffer)
7709 Fkill_buffer (coding->src_object);
b3bfad50
KH		 7710
7711 Vdeactivate_mark = old_deactivate_mark;
df7492f9 7712 unbind_to (count, Qnil);
b73bfc1c
KH		 7713}
7714
df7492f9 7715
b73bfc1c 7716Lisp_Object
df7492f9 7717preferred_coding_system ()
b73bfc1c 7718{
df7492f9 7719 int id = coding_categories[coding_priorities[0]].id;
2391eaa4 7720
df7492f9 7721 return CODING_ID_NAME (id);
4ed46869
KH		 7722}
7723
7724\f
7725#ifdef emacs
1397dc18 7726/*** 8. Emacs Lisp library functions ***/
4ed46869 7727
4ed46869 7728DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0,
48b0f3ae 7729 doc: /* Return t if OBJECT is nil or a coding-system.
df7492f9 7730See the documentation of `define-coding-system' for information
48b0f3ae 7731about coding-system objects. */)
d4a1d553
JB		 7732 (object)
7733 Lisp_Object object;
4ed46869 7734{
d4a1d553
JB		 7735 if (NILP (object)
7736 || CODING_SYSTEM_ID (object) >= 0)
44e8490d 7737 return Qt;
d4a1d553
JB		 7738 if (! SYMBOLP (object)
7739 || NILP (Fget (object, Qcoding_system_define_form)))
44e8490d
KH		 7740 return Qnil;
7741 return Qt;
4ed46869
KH		 7742}
7743
9d991de8
RS		 7744DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system,
7745 Sread_non_nil_coding_system, 1, 1, 0,
48b0f3ae
PJ		 7746 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
7747 (prompt)
4ed46869
KH		 7748 Lisp_Object prompt;
7749{
e0e989f6 7750 Lisp_Object val;
9d991de8
RS		 7751 do
7752 {
4608c386
KH		 7753 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
7754 Qt, Qnil, Qcoding_system_history, Qnil, Qnil);
9d991de8 7755 }
8f924df7 7756 while (SCHARS (val) == 0);
e0e989f6 7757 return (Fintern (val, Qnil));
4ed46869
KH		 7758}
7759
9b787f3e 7760DEFUN ("read-coding-system", Fread_coding_system, Sread_coding_system, 1, 2, 0,
48b0f3ae 7761 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT.
c7183fb8
GM		 7762If the user enters null input, return second argument DEFAULT-CODING-SYSTEM.
7763Ignores case when completing coding systems (all Emacs coding systems
7764are lower-case). */)
48b0f3ae 7765 (prompt, default_coding_system)
9b787f3e 7766 Lisp_Object prompt, default_coding_system;
4ed46869 7767{
f44d27ce 7768 Lisp_Object val;
c7183fb8
GM		 7769 int count = SPECPDL_INDEX ();
7770
9b787f3e 7771 if (SYMBOLP (default_coding_system))
57d25e6f 7772 default_coding_system = SYMBOL_NAME (default_coding_system);
c7183fb8 7773 specbind (Qcompletion_ignore_case, Qt);
4608c386 7774 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
9b787f3e
RS		 7775 Qt, Qnil, Qcoding_system_history,
7776 default_coding_system, Qnil);
c7183fb8 7777 unbind_to (count, Qnil);
8f924df7 7778 return (SCHARS (val) == 0 ? Qnil : Fintern (val, Qnil));
4ed46869
KH		 7779}
7780
7781DEFUN ("check-coding-system", Fcheck_coding_system, Scheck_coding_system,
7782 1, 1, 0,
48b0f3ae 7783 doc: /* Check validity of CODING-SYSTEM.
9ffd559c
KH		 7784If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
7785It is valid if it is nil or a symbol defined as a coding system by the
7786function `define-coding-system'. */)
df7492f9 7787 (coding_system)
4ed46869
KH		 7788 Lisp_Object coding_system;
7789{
44e8490d
KH		 7790 Lisp_Object define_form;
7791
7792 define_form = Fget (coding_system, Qcoding_system_define_form);
7793 if (! NILP (define_form))
7794 {
7795 Fput (coding_system, Qcoding_system_define_form, Qnil);
7796 safe_eval (define_form);
7797 }
4ed46869
KH		 7798 if (!NILP (Fcoding_system_p (coding_system)))
7799 return coding_system;
fcad4ec4 7800 xsignal1 (Qcoding_system_error, coding_system);
4ed46869 7801}
df7492f9 7802
3a73fa5d 7803\f
89528eb3
KH		 7804/* Detect how the bytes at SRC of length SRC_BYTES are encoded. If
7805 HIGHEST is nonzero, return the coding system of the highest
7806 priority among the detected coding systems. Otherwize return a
7807 list of detected coding systems sorted by their priorities. If
7808 MULTIBYTEP is nonzero, it is assumed that the bytes are in correct
7809 multibyte form but contains only ASCII and eight-bit chars.
7810 Otherwise, the bytes are raw bytes.
7811
7812 CODING-SYSTEM controls the detection as below:
7813
7814 If it is nil, detect both text-format and eol-format. If the
7815 text-format part of CODING-SYSTEM is already specified
7816 (e.g. `iso-latin-1'), detect only eol-format. If the eol-format
7817 part of CODING-SYSTEM is already specified (e.g. `undecided-unix'),
7818 detect only text-format. */
7819
d46c5b12 7820Lisp_Object
24a73b0a
KH		 7821detect_coding_system (src, src_chars, src_bytes, highest, multibytep,
7822 coding_system)
8f924df7 7823 const unsigned char *src;
13818c30
SM		 7824 EMACS_INT src_chars, src_bytes;
7825 int highest;
0a28aafb 7826 int multibytep;
df7492f9 7827 Lisp_Object coding_system;
4ed46869 7828{
8f924df7 7829 const unsigned char *src_end = src + src_bytes;
df7492f9 7830 Lisp_Object attrs, eol_type;
4533845d 7831 Lisp_Object val = Qnil;
df7492f9 7832 struct coding_system coding;
89528eb3 7833 int id;
ff0dacd7 7834 struct coding_detection_info detect_info;
24a73b0a 7835 enum coding_category base_category;
2f3cbb32 7836 int null_byte_found = 0, eight_bit_found = 0;
b73bfc1c 7837
df7492f9
KH		 7838 if (NILP (coding_system))
7839 coding_system = Qundecided;
7840 setup_coding_system (coding_system, &coding);
7841 attrs = CODING_ID_ATTRS (coding.id);
7842 eol_type = CODING_ID_EOL_TYPE (coding.id);
89528eb3 7843 coding_system = CODING_ATTR_BASE_NAME (attrs);
4ed46869 7844
df7492f9 7845 coding.source = src;
24a73b0a 7846 coding.src_chars = src_chars;
df7492f9
KH		 7847 coding.src_bytes = src_bytes;
7848 coding.src_multibyte = multibytep;
7849 coding.consumed = 0;
89528eb3 7850 coding.mode |= CODING_MODE_LAST_BLOCK;
c0e16b14 7851 coding.head_ascii = 0;
d46c5b12 7852
ff0dacd7 7853 detect_info.checked = detect_info.found = detect_info.rejected = 0;
d46c5b12 7854
89528eb3 7855 /* At first, detect text-format if necessary. */
24a73b0a
KH		 7856 base_category = XINT (CODING_ATTR_CATEGORY (attrs));
7857 if (base_category == coding_category_undecided)
4ed46869 7858 {
ff0dacd7
KH		 7859 enum coding_category category;
7860 struct coding_system *this;
7861 int c, i;
88993dfd 7862
24a73b0a 7863 /* Skip all ASCII bytes except for a few ISO2022 controls. */
2f3cbb32 7864 for (; src < src_end; src++)
4ed46869 7865 {
df7492f9 7866 c = *src;
6cb21a4f 7867 if (c & 0x80)
6cb21a4f 7868 {
2f3cbb32 7869 eight_bit_found = 1;
2f3cbb32
KH		 7870 if (null_byte_found)
7871 break;
7872 }
c0e16b14 7873 else if (c < 0x20)
2f3cbb32
KH		 7874 {
7875 if ((c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
7876 && ! inhibit_iso_escape_detection
7877 && ! detect_info.checked)
6cb21a4f 7878 {
2f3cbb32
KH		 7879 if (detect_coding_iso_2022 (&coding, &detect_info))
7880 {
7881 /* We have scanned the whole data. */
7882 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
c0e16b14
KH		 7883 {
7884 /* We didn't find an 8-bit code. We may
7885 have found a null-byte, but it's very
7886 rare that a binary file confirm to
7887 ISO-2022. */
7888 src = src_end;
7889 coding.head_ascii = src - coding.source;
7890 }
7891 detect_info.rejected |= ~CATEGORY_MASK_ISO_ESCAPE;
2f3cbb32
KH		 7892 break;
7893 }
7894 }
97b1b294 7895 else if (! c && !inhibit_null_byte_detection)
2f3cbb32
KH		 7896 {
7897 null_byte_found = 1;
7898 if (eight_bit_found)
7899 break;
6cb21a4f 7900 }
c006c0c8
KH		 7901 if (! eight_bit_found)
7902 coding.head_ascii++;
6cb21a4f 7903 }
c006c0c8 7904 else if (! eight_bit_found)
c0e16b14 7905 coding.head_ascii++;
4ed46869 7906 }
88993dfd 7907
2f3cbb32
KH		 7908 if (null_byte_found || eight_bit_found
7909 || coding.head_ascii < coding.src_bytes
6cb21a4f
KH		 7910 || detect_info.found)
7911 {
2f3cbb32 7912 if (coding.head_ascii == coding.src_bytes)
6cb21a4f
KH		 7913 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
7914 for (i = 0; i < coding_category_raw_text; i++)
ff0dacd7 7915 {
6cb21a4f 7916 category = coding_priorities[i];
c7266f4a 7917 this = coding_categories + category;
6cb21a4f 7918 if (detect_info.found & (1 << category))
ff0dacd7
KH		 7919 break;
7920 }
6cb21a4f 7921 else
2f3cbb32
KH		 7922 {
7923 if (null_byte_found)
7924 {
7925 detect_info.checked |= ~CATEGORY_MASK_UTF_16;
7926 detect_info.rejected |= ~CATEGORY_MASK_UTF_16;
7927 }
7928 for (i = 0; i < coding_category_raw_text; i++)
7929 {
7930 category = coding_priorities[i];
7931 this = coding_categories + category;
6cb21a4f 7932
2f3cbb32
KH		 7933 if (this->id < 0)
7934 {
7935 /* No coding system of this category is defined. */
7936 detect_info.rejected |= (1 << category);
7937 }
7938 else if (category >= coding_category_raw_text)
7939 continue;
7940 else if (detect_info.checked & (1 << category))
7941 {
7942 if (highest
7943 && (detect_info.found & (1 << category)))
6cb21a4f 7944 break;
2f3cbb32
KH		 7945 }
7946 else if ((*(this->detector)) (&coding, &detect_info)
7947 && highest
7948 && (detect_info.found & (1 << category)))
7949 {
7950 if (category == coding_category_utf_16_auto)
7951 {
7952 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
7953 category = coding_category_utf_16_le;
7954 else
7955 category = coding_category_utf_16_be;
7956 }
7957 break;
7958 }
7959 }
7960 }
6cb21a4f 7961 }
ec6d2bb8 7962
4cddb209
KH		 7963 if ((detect_info.rejected & CATEGORY_MASK_ANY) == CATEGORY_MASK_ANY
7964 || null_byte_found)
ec6d2bb8 7965 {
ff0dacd7 7966 detect_info.found = CATEGORY_MASK_RAW_TEXT;
4cddb209 7967 id = CODING_SYSTEM_ID (Qno_conversion);
89528eb3
KH		 7968 val = Fcons (make_number (id), Qnil);
7969 }
ff0dacd7 7970 else if (! detect_info.rejected && ! detect_info.found)
89528eb3 7971 {
ff0dacd7 7972 detect_info.found = CATEGORY_MASK_ANY;
89528eb3
KH		 7973 id = coding_categories[coding_category_undecided].id;
7974 val = Fcons (make_number (id), Qnil);
7975 }
7976 else if (highest)
7977 {
ff0dacd7 7978 if (detect_info.found)
ec6d2bb8 7979 {
ff0dacd7
KH		 7980 detect_info.found = 1 << category;
7981 val = Fcons (make_number (this->id), Qnil);
7982 }
7983 else
7984 for (i = 0; i < coding_category_raw_text; i++)
7985 if (! (detect_info.rejected & (1 << coding_priorities[i])))
7986 {
7987 detect_info.found = 1 << coding_priorities[i];
7988 id = coding_categories[coding_priorities[i]].id;
7989 val = Fcons (make_number (id), Qnil);
7990 break;
7991 }
7992 }
89528eb3
KH		 7993 else
7994 {
ff0dacd7
KH		 7995 int mask = detect_info.rejected | detect_info.found;
7996 int found = 0;
ec6d2bb8 7997
89528eb3 7998 for (i = coding_category_raw_text - 1; i >= 0; i--)
ff0dacd7
KH		 7999 {
8000 category = coding_priorities[i];
8001 if (! (mask & (1 << category)))
ec6d2bb8 8002 {
ff0dacd7
KH		 8003 found |= 1 << category;
8004 id = coding_categories[category].id;
c7266f4a
KH		 8005 if (id >= 0)
8006 val = Fcons (make_number (id), val);
ff0dacd7
KH		 8007 }
8008 }
8009 for (i = coding_category_raw_text - 1; i >= 0; i--)
8010 {
8011 category = coding_priorities[i];
8012 if (detect_info.found & (1 << category))
8013 {
8014 id = coding_categories[category].id;
8015 val = Fcons (make_number (id), val);
ec6d2bb8 8016 }
ec6d2bb8 8017 }
ff0dacd7 8018 detect_info.found |= found;
ec6d2bb8 8019 }
ec6d2bb8 8020 }
a470d443
KH		 8021 else if (base_category == coding_category_utf_8_auto)
8022 {
8023 if (detect_coding_utf_8 (&coding, &detect_info))
8024 {
8025 struct coding_system *this;
8026
8027 if (detect_info.found & CATEGORY_MASK_UTF_8_SIG)
8028 this = coding_categories + coding_category_utf_8_sig;
8029 else
8030 this = coding_categories + coding_category_utf_8_nosig;
8031 val = Fcons (make_number (this->id), Qnil);
8032 }
8033 }
24a73b0a
KH		 8034 else if (base_category == coding_category_utf_16_auto)
8035 {
8036 if (detect_coding_utf_16 (&coding, &detect_info))
8037 {
24a73b0a
KH		 8038 struct coding_system *this;
8039
8040 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
8041 this = coding_categories + coding_category_utf_16_le;
8042 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
8043 this = coding_categories + coding_category_utf_16_be;
8044 else if (detect_info.rejected & CATEGORY_MASK_UTF_16_LE_NOSIG)
8045 this = coding_categories + coding_category_utf_16_be_nosig;
8046 else
8047 this = coding_categories + coding_category_utf_16_le_nosig;
8048 val = Fcons (make_number (this->id), Qnil);
8049 }
8050 }
df7492f9
KH		 8051 else
8052 {
ff0dacd7 8053 detect_info.found = 1 << XINT (CODING_ATTR_CATEGORY (attrs));
89528eb3 8054 val = Fcons (make_number (coding.id), Qnil);
4ed46869 8055 }
df7492f9 8056
89528eb3 8057 /* Then, detect eol-format if necessary. */
df7492f9 8058 {
4533845d 8059 int normal_eol = -1, utf_16_be_eol = -1, utf_16_le_eol = -1;
df7492f9
KH		 8060 Lisp_Object tail;
8061
89528eb3
KH		 8062 if (VECTORP (eol_type))
8063 {
ff0dacd7 8064 if (detect_info.found & ~CATEGORY_MASK_UTF_16)
2f3cbb32
KH		 8065 {
8066 if (null_byte_found)
8067 normal_eol = EOL_SEEN_LF;
8068 else
8069 normal_eol = detect_eol (coding.source, src_bytes,
8070 coding_category_raw_text);
8071 }
ff0dacd7
KH		 8072 if (detect_info.found & (CATEGORY_MASK_UTF_16_BE
8073 | CATEGORY_MASK_UTF_16_BE_NOSIG))
89528eb3
KH		 8074 utf_16_be_eol = detect_eol (coding.source, src_bytes,
8075 coding_category_utf_16_be);
ff0dacd7
KH		 8076 if (detect_info.found & (CATEGORY_MASK_UTF_16_LE
8077 | CATEGORY_MASK_UTF_16_LE_NOSIG))
89528eb3
KH		 8078 utf_16_le_eol = detect_eol (coding.source, src_bytes,
8079 coding_category_utf_16_le);
8080 }
8081 else
8082 {
8083 if (EQ (eol_type, Qunix))
8084 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_LF;
8085 else if (EQ (eol_type, Qdos))
8086 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CRLF;
8087 else
8088 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CR;
8089 }
8090
df7492f9
KH		 8091 for (tail = val; CONSP (tail); tail = XCDR (tail))
8092 {
89528eb3 8093 enum coding_category category;
df7492f9 8094 int this_eol;
89528eb3
KH		 8095
8096 id = XINT (XCAR (tail));
8097 attrs = CODING_ID_ATTRS (id);
8098 category = XINT (CODING_ATTR_CATEGORY (attrs));
8099 eol_type = CODING_ID_EOL_TYPE (id);
df7492f9
KH		 8100 if (VECTORP (eol_type))
8101 {
89528eb3
KH		 8102 if (category == coding_category_utf_16_be
8103 || category == coding_category_utf_16_be_nosig)
8104 this_eol = utf_16_be_eol;
8105 else if (category == coding_category_utf_16_le
8106 || category == coding_category_utf_16_le_nosig)
8107 this_eol = utf_16_le_eol;
df7492f9 8108 else
89528eb3
KH		 8109 this_eol = normal_eol;
8110
df7492f9
KH		 8111 if (this_eol == EOL_SEEN_LF)
8112 XSETCAR (tail, AREF (eol_type, 0));
8113 else if (this_eol == EOL_SEEN_CRLF)
8114 XSETCAR (tail, AREF (eol_type, 1));
8115 else if (this_eol == EOL_SEEN_CR)
8116 XSETCAR (tail, AREF (eol_type, 2));
89528eb3
KH		 8117 else
8118 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9 8119 }
89528eb3
KH		 8120 else
8121 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9
KH		 8122 }
8123 }
ec6d2bb8 8124
4533845d 8125 return (highest ? (CONSP (val) ? XCAR (val) : Qnil) : val);
ec6d2bb8
KH		 8126}
8127
ec6d2bb8 8128
d46c5b12
KH		 8129DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region,
8130 2, 3, 0,
48b0f3ae
PJ		 8131 doc: /* Detect coding system of the text in the region between START and END.
8132Return a list of possible coding systems ordered by priority.
ec6d2bb8 8133
12e0131a 8134If only ASCII characters are found (except for such ISO-2022 control
d4a1d553
JB		 8135characters as ESC), it returns a list of single element `undecided'
8136or its subsidiary coding system according to a detected end-of-line
8137format.
ec6d2bb8 8138
48b0f3ae
PJ		 8139If optional argument HIGHEST is non-nil, return the coding system of
8140highest priority. */)
8141 (start, end, highest)
d46c5b12
KH		 8142 Lisp_Object start, end, highest;
8143{
8144 int from, to;
8145 int from_byte, to_byte;
ec6d2bb8 8146
b7826503
PJ		 8147 CHECK_NUMBER_COERCE_MARKER (start);
8148 CHECK_NUMBER_COERCE_MARKER (end);
ec6d2bb8 8149
d46c5b12
KH		 8150 validate_region (&start, &end);
8151 from = XINT (start), to = XINT (end);
8152 from_byte = CHAR_TO_BYTE (from);
8153 to_byte = CHAR_TO_BYTE (to);
ec6d2bb8 8154
d46c5b12
KH		 8155 if (from < GPT && to >= GPT)
8156 move_gap_both (to, to_byte);
c210f766 8157
d46c5b12 8158 return detect_coding_system (BYTE_POS_ADDR (from_byte),
24a73b0a 8159 to - from, to_byte - from_byte,
0a28aafb
KH		 8160 !NILP (highest),
8161 !NILP (current_buffer
df7492f9
KH		 8162 ->enable_multibyte_characters),
8163 Qnil);
ec6d2bb8
KH		 8164}
8165
d46c5b12
KH		 8166DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string,
8167 1, 2, 0,
48b0f3ae
PJ		 8168 doc: /* Detect coding system of the text in STRING.
8169Return a list of possible coding systems ordered by priority.
fb88bf2d 8170
12e0131a 8171If only ASCII characters are found (except for such ISO-2022 control
d4a1d553
JB		 8172characters as ESC), it returns a list of single element `undecided'
8173or its subsidiary coding system according to a detected end-of-line
8174format.
d46c5b12 8175
48b0f3ae
PJ		 8176If optional argument HIGHEST is non-nil, return the coding system of
8177highest priority. */)
8178 (string, highest)
d46c5b12
KH		 8179 Lisp_Object string, highest;
8180{
b7826503 8181 CHECK_STRING (string);
b73bfc1c 8182
24a73b0a
KH		 8183 return detect_coding_system (SDATA (string),
8184 SCHARS (string), SBYTES (string),
8f924df7 8185 !NILP (highest), STRING_MULTIBYTE (string),
df7492f9 8186 Qnil);
4ed46869 8187}
4ed46869 8188
b73bfc1c 8189
df7492f9
KH		 8190static INLINE int
8191char_encodable_p (c, attrs)
8192 int c;
8193 Lisp_Object attrs;
05e6f5dc 8194{
df7492f9 8195 Lisp_Object tail;
df7492f9 8196 struct charset *charset;
7d64c6ad 8197 Lisp_Object translation_table;
d46c5b12 8198
7d64c6ad 8199 translation_table = CODING_ATTR_TRANS_TBL (attrs);
a6f87d34 8200 if (! NILP (translation_table))
7d64c6ad 8201 c = translate_char (translation_table, c);
df7492f9
KH		 8202 for (tail = CODING_ATTR_CHARSET_LIST (attrs);
8203 CONSP (tail); tail = XCDR (tail))
e133c8fa 8204 {
df7492f9
KH		 8205 charset = CHARSET_FROM_ID (XINT (XCAR (tail)));
8206 if (CHAR_CHARSET_P (c, charset))
8207 break;
e133c8fa 8208 }
df7492f9 8209 return (! NILP (tail));
05e6f5dc 8210}
83fa074f 8211
fb88bf2d 8212
df7492f9
KH		 8213/* Return a list of coding systems that safely encode the text between
8214 START and END. If EXCLUDE is non-nil, it is a list of coding
8215 systems not to check. The returned list doesn't contain any such
48468dac 8216 coding systems. In any case, if the text contains only ASCII or is
df7492f9 8217 unibyte, return t. */
e077cc80 8218
df7492f9
KH		 8219DEFUN ("find-coding-systems-region-internal",
8220 Ffind_coding_systems_region_internal,
8221 Sfind_coding_systems_region_internal, 2, 3, 0,
8222 doc: /* Internal use only. */)
8223 (start, end, exclude)
8224 Lisp_Object start, end, exclude;
8225{
8226 Lisp_Object coding_attrs_list, safe_codings;
8227 EMACS_INT start_byte, end_byte;
7c78e542 8228 const unsigned char *p, *pbeg, *pend;
df7492f9
KH		 8229 int c;
8230 Lisp_Object tail, elt;
d46c5b12 8231
df7492f9
KH		 8232 if (STRINGP (start))
8233 {
8234 if (!STRING_MULTIBYTE (start)
8f924df7 8235 || SCHARS (start) == SBYTES (start))
df7492f9
KH		 8236 return Qt;
8237 start_byte = 0;
8f924df7 8238 end_byte = SBYTES (start);
df7492f9
KH		 8239 }
8240 else
d46c5b12 8241 {
df7492f9
KH		 8242 CHECK_NUMBER_COERCE_MARKER (start);
8243 CHECK_NUMBER_COERCE_MARKER (end);
8244 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
8245 args_out_of_range (start, end);
8246 if (NILP (current_buffer->enable_multibyte_characters))
8247 return Qt;
8248 start_byte = CHAR_TO_BYTE (XINT (start));
8249 end_byte = CHAR_TO_BYTE (XINT (end));
8250 if (XINT (end) - XINT (start) == end_byte - start_byte)
8251 return Qt;
d46c5b12 8252
e1c23804 8253 if (XINT (start) < GPT && XINT (end) > GPT)
d46c5b12 8254 {
e1c23804
DL		 8255 if ((GPT - XINT (start)) < (XINT (end) - GPT))
8256 move_gap_both (XINT (start), start_byte);
df7492f9 8257 else
e1c23804 8258 move_gap_both (XINT (end), end_byte);
d46c5b12
KH		 8259 }
8260 }
8261
df7492f9
KH		 8262 coding_attrs_list = Qnil;
8263 for (tail = Vcoding_system_list; CONSP (tail); tail = XCDR (tail))
8264 if (NILP (exclude)
8265 || NILP (Fmemq (XCAR (tail), exclude)))
8266 {
8267 Lisp_Object attrs;
d46c5b12 8268
df7492f9
KH		 8269 attrs = AREF (CODING_SYSTEM_SPEC (XCAR (tail)), 0);
8270 if (EQ (XCAR (tail), CODING_ATTR_BASE_NAME (attrs))
8271 && ! EQ (CODING_ATTR_TYPE (attrs), Qundecided))
7d64c6ad
KH		 8272 {
8273 ASET (attrs, coding_attr_trans_tbl,
2170c8f0 8274 get_translation_table (attrs, 1, NULL));
7d64c6ad
KH		 8275 coding_attrs_list = Fcons (attrs, coding_attrs_list);
8276 }
df7492f9 8277 }
d46c5b12 8278
df7492f9 8279 if (STRINGP (start))
8f924df7 8280 p = pbeg = SDATA (start);
df7492f9
KH		 8281 else
8282 p = pbeg = BYTE_POS_ADDR (start_byte);
8283 pend = p + (end_byte - start_byte);
b843d1ae 8284
df7492f9
KH		 8285 while (p < pend && ASCII_BYTE_P (*p)) p++;
8286 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
d46c5b12 8287
05e6f5dc 8288 while (p < pend)
72d1a715 8289 {
df7492f9
KH		 8290 if (ASCII_BYTE_P (*p))
8291 p++;
72d1a715
RS		 8292 else
8293 {
df7492f9 8294 c = STRING_CHAR_ADVANCE (p);
12410ef1 8295
df7492f9
KH		 8296 charset_map_loaded = 0;
8297 for (tail = coding_attrs_list; CONSP (tail);)
8298 {
8299 elt = XCAR (tail);
8300 if (NILP (elt))
8301 tail = XCDR (tail);
8302 else if (char_encodable_p (c, elt))
8303 tail = XCDR (tail);
8304 else if (CONSP (XCDR (tail)))
8305 {
8306 XSETCAR (tail, XCAR (XCDR (tail)));
8307 XSETCDR (tail, XCDR (XCDR (tail)));
8308 }
8309 else
8310 {
8311 XSETCAR (tail, Qnil);
8312 tail = XCDR (tail);
8313 }
8314 }
8315 if (charset_map_loaded)
8316 {
8317 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
05e6f5dc 8318
df7492f9 8319 if (STRINGP (start))
8f924df7 8320 pbeg = SDATA (start);
df7492f9
KH		 8321 else
8322 pbeg = BYTE_POS_ADDR (start_byte);
8323 p = pbeg + p_offset;
8324 pend = pbeg + pend_offset;
8325 }
8326 }
ec6d2bb8 8327 }
fb88bf2d 8328
988b3759 8329 safe_codings = list2 (Qraw_text, Qno_conversion);
df7492f9
KH		 8330 for (tail = coding_attrs_list; CONSP (tail); tail = XCDR (tail))
8331 if (! NILP (XCAR (tail)))
8332 safe_codings = Fcons (CODING_ATTR_BASE_NAME (XCAR (tail)), safe_codings);
ec6d2bb8 8333
05e6f5dc
KH		 8334 return safe_codings;
8335}
4956c225 8336
d46c5b12 8337
8f924df7
KH		 8338DEFUN ("unencodable-char-position", Funencodable_char_position,
8339 Sunencodable_char_position, 3, 5, 0,
8340 doc: /*
8341Return position of first un-encodable character in a region.
d4a1d553 8342START and END specify the region and CODING-SYSTEM specifies the
8f924df7 8343encoding to check. Return nil if CODING-SYSTEM does encode the region.
d46c5b12 8344
8f924df7
KH		 8345If optional 4th argument COUNT is non-nil, it specifies at most how
8346many un-encodable characters to search. In this case, the value is a
8347list of positions.
d46c5b12 8348
8f924df7
KH		 8349If optional 5th argument STRING is non-nil, it is a string to search
8350for un-encodable characters. In that case, START and END are indexes
8351to the string. */)
8352 (start, end, coding_system, count, string)
8353 Lisp_Object start, end, coding_system, count, string;
8354{
8355 int n;
8356 struct coding_system coding;
7d64c6ad 8357 Lisp_Object attrs, charset_list, translation_table;
8f924df7
KH		 8358 Lisp_Object positions;
8359 int from, to;
8360 const unsigned char *p, *stop, *pend;
8361 int ascii_compatible;
fb88bf2d 8362
8f924df7
KH		 8363 setup_coding_system (Fcheck_coding_system (coding_system), &coding);
8364 attrs = CODING_ID_ATTRS (coding.id);
8365 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
8366 return Qnil;
8367 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
8368 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2170c8f0 8369 translation_table = get_translation_table (attrs, 1, NULL);
fb88bf2d 8370
8f924df7
KH		 8371 if (NILP (string))
8372 {
8373 validate_region (&start, &end);
8374 from = XINT (start);
8375 to = XINT (end);
8376 if (NILP (current_buffer->enable_multibyte_characters)
8377 || (ascii_compatible
8378 && (to - from) == (CHAR_TO_BYTE (to) - (CHAR_TO_BYTE (from)))))
8379 return Qnil;
8380 p = CHAR_POS_ADDR (from);
8381 pend = CHAR_POS_ADDR (to);
8382 if (from < GPT && to >= GPT)
8383 stop = GPT_ADDR;
8384 else
8385 stop = pend;
8386 }
8387 else
8388 {
8389 CHECK_STRING (string);
8390 CHECK_NATNUM (start);
8391 CHECK_NATNUM (end);
8392 from = XINT (start);
8393 to = XINT (end);
8394 if (from > to
8395 || to > SCHARS (string))
8396 args_out_of_range_3 (string, start, end);
8397 if (! STRING_MULTIBYTE (string))
8398 return Qnil;
8399 p = SDATA (string) + string_char_to_byte (string, from);
8400 stop = pend = SDATA (string) + string_char_to_byte (string, to);
8401 if (ascii_compatible && (to - from) == (pend - p))
8402 return Qnil;
8403 }
f2558efd 8404
8f924df7
KH		 8405 if (NILP (count))
8406 n = 1;
8407 else
b73bfc1c 8408 {
8f924df7
KH		 8409 CHECK_NATNUM (count);
8410 n = XINT (count);
b73bfc1c
KH		 8411 }
8412
8f924df7
KH		 8413 positions = Qnil;
8414 while (1)
d46c5b12 8415 {
8f924df7 8416 int c;
ec6d2bb8 8417
8f924df7
KH		 8418 if (ascii_compatible)
8419 while (p < stop && ASCII_BYTE_P (*p))
8420 p++, from++;
8421 if (p >= stop)
0e79d667 8422 {
8f924df7
KH		 8423 if (p >= pend)
8424 break;
8425 stop = pend;
8426 p = GAP_END_ADDR;
0e79d667 8427 }
ec6d2bb8 8428
8f924df7
KH		 8429 c = STRING_CHAR_ADVANCE (p);
8430 if (! (ASCII_CHAR_P (c) && ascii_compatible)
7d64c6ad
KH		 8431 && ! char_charset (translate_char (translation_table, c),
8432 charset_list, NULL))
ec6d2bb8 8433 {
8f924df7
KH		 8434 positions = Fcons (make_number (from), positions);
8435 n--;
8436 if (n == 0)
8437 break;
ec6d2bb8
KH		 8438 }
8439
8f924df7
KH		 8440 from++;
8441 }
d46c5b12 8442
8f924df7
KH		 8443 return (NILP (count) ? Fcar (positions) : Fnreverse (positions));
8444}
d46c5b12 8445
d46c5b12 8446
df7492f9
KH		 8447DEFUN ("check-coding-systems-region", Fcheck_coding_systems_region,
8448 Scheck_coding_systems_region, 3, 3, 0,
8449 doc: /* Check if the region is encodable by coding systems.
d46c5b12 8450
df7492f9
KH		 8451START and END are buffer positions specifying the region.
8452CODING-SYSTEM-LIST is a list of coding systems to check.
d46c5b12 8453
df7492f9 8454The value is an alist ((CODING-SYSTEM POS0 POS1 ...) ...), where
d4a1d553 8455CODING-SYSTEM is a member of CODING-SYSTEM-LIST and can't encode the
df7492f9
KH		 8456whole region, POS0, POS1, ... are buffer positions where non-encodable
8457characters are found.
93dec019 8458
df7492f9
KH		 8459If all coding systems in CODING-SYSTEM-LIST can encode the region, the
8460value is nil.
93dec019 8461
df7492f9
KH		 8462START may be a string. In that case, check if the string is
8463encodable, and the value contains indices to the string instead of
8464buffer positions. END is ignored. */)
8465 (start, end, coding_system_list)
8466 Lisp_Object start, end, coding_system_list;
05e6f5dc 8467{
df7492f9
KH		 8468 Lisp_Object list;
8469 EMACS_INT start_byte, end_byte;
8470 int pos;
7c78e542 8471 const unsigned char *p, *pbeg, *pend;
df7492f9 8472 int c;
7d64c6ad 8473 Lisp_Object tail, elt, attrs;
70ad9fc4 8474
05e6f5dc
KH		 8475 if (STRINGP (start))
8476 {
df7492f9 8477 if (!STRING_MULTIBYTE (start)
8f924df7 8478 && SCHARS (start) != SBYTES (start))
df7492f9
KH		 8479 return Qnil;
8480 start_byte = 0;
8f924df7 8481 end_byte = SBYTES (start);
df7492f9 8482 pos = 0;
d46c5b12 8483 }
05e6f5dc 8484 else
b73bfc1c 8485 {
b7826503
PJ		 8486 CHECK_NUMBER_COERCE_MARKER (start);
8487 CHECK_NUMBER_COERCE_MARKER (end);
05e6f5dc
KH		 8488 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
8489 args_out_of_range (start, end);
8490 if (NILP (current_buffer->enable_multibyte_characters))
df7492f9
KH		 8491 return Qnil;
8492 start_byte = CHAR_TO_BYTE (XINT (start));
8493 end_byte = CHAR_TO_BYTE (XINT (end));
8494 if (XINT (end) - XINT (start) == end_byte - start_byte)
05e6f5dc 8495 return Qt;
df7492f9 8496
e1c23804 8497 if (XINT (start) < GPT && XINT (end) > GPT)
b73bfc1c 8498 {
e1c23804
DL		 8499 if ((GPT - XINT (start)) < (XINT (end) - GPT))
8500 move_gap_both (XINT (start), start_byte);
df7492f9 8501 else
e1c23804 8502 move_gap_both (XINT (end), end_byte);
b73bfc1c 8503 }
e1c23804 8504 pos = XINT (start);
b73bfc1c 8505 }
7553d0e1 8506
df7492f9
KH		 8507 list = Qnil;
8508 for (tail = coding_system_list; CONSP (tail); tail = XCDR (tail))
12410ef1 8509 {
df7492f9 8510 elt = XCAR (tail);
7d64c6ad 8511 attrs = AREF (CODING_SYSTEM_SPEC (elt), 0);
2170c8f0
KH		 8512 ASET (attrs, coding_attr_trans_tbl,
8513 get_translation_table (attrs, 1, NULL));
7d64c6ad 8514 list = Fcons (Fcons (elt, Fcons (attrs, Qnil)), list);
12410ef1
KH		 8515 }
8516
df7492f9 8517 if (STRINGP (start))
8f924df7 8518 p = pbeg = SDATA (start);
72d1a715 8519 else
df7492f9
KH		 8520 p = pbeg = BYTE_POS_ADDR (start_byte);
8521 pend = p + (end_byte - start_byte);
4ed46869 8522
df7492f9
KH		 8523 while (p < pend && ASCII_BYTE_P (*p)) p++, pos++;
8524 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
ec6d2bb8 8525
df7492f9 8526 while (p < pend)
d46c5b12 8527 {
df7492f9
KH		 8528 if (ASCII_BYTE_P (*p))
8529 p++;
e133c8fa 8530 else
05e6f5dc 8531 {
df7492f9
KH		 8532 c = STRING_CHAR_ADVANCE (p);
8533
8534 charset_map_loaded = 0;
8535 for (tail = list; CONSP (tail); tail = XCDR (tail))
8536 {
8537 elt = XCDR (XCAR (tail));
8538 if (! char_encodable_p (c, XCAR (elt)))
8539 XSETCDR (elt, Fcons (make_number (pos), XCDR (elt)));
8540 }
8541 if (charset_map_loaded)
8542 {
8543 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
8544
8545 if (STRINGP (start))
8f924df7 8546 pbeg = SDATA (start);
df7492f9
KH		 8547 else
8548 pbeg = BYTE_POS_ADDR (start_byte);
8549 p = pbeg + p_offset;
8550 pend = pbeg + pend_offset;
8551 }
05e6f5dc 8552 }
df7492f9 8553 pos++;
d46c5b12 8554 }
4ed46869 8555
df7492f9
KH		 8556 tail = list;
8557 list = Qnil;
8558 for (; CONSP (tail); tail = XCDR (tail))
ec6d2bb8 8559 {
df7492f9
KH		 8560 elt = XCAR (tail);
8561 if (CONSP (XCDR (XCDR (elt))))
8562 list = Fcons (Fcons (XCAR (elt), Fnreverse (XCDR (XCDR (elt)))),
8563 list);
ec6d2bb8 8564 }
2b4f9037 8565
df7492f9 8566 return list;
d46c5b12
KH		 8567}
8568
3fd9494b 8569
b73bfc1c 8570Lisp_Object
df7492f9
KH		 8571code_convert_region (start, end, coding_system, dst_object, encodep, norecord)
8572 Lisp_Object start, end, coding_system, dst_object;
8573 int encodep, norecord;
4ed46869 8574{
3a73fa5d 8575 struct coding_system coding;
df7492f9
KH		 8576 EMACS_INT from, from_byte, to, to_byte;
8577 Lisp_Object src_object;
4ed46869 8578
b7826503
PJ		 8579 CHECK_NUMBER_COERCE_MARKER (start);
8580 CHECK_NUMBER_COERCE_MARKER (end);
df7492f9
KH		 8581 if (NILP (coding_system))
8582 coding_system = Qno_conversion;
8583 else
8584 CHECK_CODING_SYSTEM (coding_system);
8585 src_object = Fcurrent_buffer ();
8586 if (NILP (dst_object))
8587 dst_object = src_object;
8588 else if (! EQ (dst_object, Qt))
8589 CHECK_BUFFER (dst_object);
3a73fa5d 8590
d46c5b12
KH		 8591 validate_region (&start, &end);
8592 from = XFASTINT (start);
df7492f9 8593 from_byte = CHAR_TO_BYTE (from);
d46c5b12 8594 to = XFASTINT (end);
df7492f9 8595 to_byte = CHAR_TO_BYTE (to);
764ca8da 8596
df7492f9
KH		 8597 setup_coding_system (coding_system, &coding);
8598 coding.mode |= CODING_MODE_LAST_BLOCK;
ec6d2bb8 8599
df7492f9
KH		 8600 if (encodep)
8601 encode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
8602 dst_object);
8603 else
8604 decode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
8605 dst_object);
8606 if (! norecord)
8607 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
ec6d2bb8 8608
df7492f9
KH		 8609 return (BUFFERP (dst_object)
8610 ? make_number (coding.produced_char)
8611 : coding.dst_object);
4031e2bf 8612}
78108bcd 8613
4ed46869 8614
4031e2bf 8615DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region,
df7492f9 8616 3, 4, "r\nzCoding system: ",
48b0f3ae 8617 doc: /* Decode the current region from the specified coding system.
df7492f9
KH		 8618When called from a program, takes four arguments:
8619 START, END, CODING-SYSTEM, and DESTINATION.
8620START and END are buffer positions.
8844fa83 8621
df7492f9 8622Optional 4th arguments DESTINATION specifies where the decoded text goes.
2354b80b 8623If nil, the region between START and END is replaced by the decoded text.
1560f91a
EZ		 8624If buffer, the decoded text is inserted in that buffer after point (point
8625does not move).
446dcd75 8626In those cases, the length of the decoded text is returned.
319a3947 8627If DESTINATION is t, the decoded text is returned.
8844fa83 8628
48b0f3ae
PJ		 8629This function sets `last-coding-system-used' to the precise coding system
8630used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
319a3947 8631not fully specified.) */)
df7492f9
KH		 8632 (start, end, coding_system, destination)
8633 Lisp_Object start, end, coding_system, destination;
4031e2bf 8634{
df7492f9 8635 return code_convert_region (start, end, coding_system, destination, 0, 0);
3a73fa5d 8636}
8844fa83 8637
3a73fa5d 8638DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region,
df7492f9
KH		 8639 3, 4, "r\nzCoding system: ",
8640 doc: /* Encode the current region by specified coding system.
d4a1d553
JB		 8641When called from a program, takes four arguments:
8642 START, END, CODING-SYSTEM and DESTINATION.
8643START and END are buffer positions.
d46c5b12 8644
df7492f9
KH		 8645Optional 4th arguments DESTINATION specifies where the encoded text goes.
8646If nil, the region between START and END is replace by the encoded text.
1560f91a
EZ		 8647If buffer, the encoded text is inserted in that buffer after point (point
8648does not move).
446dcd75 8649In those cases, the length of the encoded text is returned.
319a3947 8650If DESTINATION is t, the encoded text is returned.
2391eaa4 8651
48b0f3ae
PJ		 8652This function sets `last-coding-system-used' to the precise coding system
8653used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
319a3947 8654not fully specified.) */)
df7492f9
KH		 8655 (start, end, coding_system, destination)
8656 Lisp_Object start, end, coding_system, destination;
3a73fa5d 8657{
df7492f9 8658 return code_convert_region (start, end, coding_system, destination, 1, 0);
b73bfc1c
KH		 8659}
8660
8661Lisp_Object
df7492f9
KH		 8662code_convert_string (string, coding_system, dst_object,
8663 encodep, nocopy, norecord)
8664 Lisp_Object string, coding_system, dst_object;
8665 int encodep, nocopy, norecord;
b73bfc1c 8666{
4031e2bf 8667 struct coding_system coding;
df7492f9 8668 EMACS_INT chars, bytes;
ec6d2bb8 8669
b7826503 8670 CHECK_STRING (string);
d46c5b12 8671 if (NILP (coding_system))
4956c225 8672 {
df7492f9
KH		 8673 if (! norecord)
8674 Vlast_coding_system_used = Qno_conversion;
8675 if (NILP (dst_object))
8676 return (nocopy ? Fcopy_sequence (string) : string);
4956c225 8677 }
b73bfc1c 8678
df7492f9
KH		 8679 if (NILP (coding_system))
8680 coding_system = Qno_conversion;
8681 else
8682 CHECK_CODING_SYSTEM (coding_system);
8683 if (NILP (dst_object))
8684 dst_object = Qt;
8685 else if (! EQ (dst_object, Qt))
8686 CHECK_BUFFER (dst_object);
73be902c 8687
df7492f9 8688 setup_coding_system (coding_system, &coding);
d46c5b12 8689 coding.mode |= CODING_MODE_LAST_BLOCK;
8f924df7
KH		 8690 chars = SCHARS (string);
8691 bytes = SBYTES (string);
df7492f9
KH		 8692 if (encodep)
8693 encode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8694 else
8695 decode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8696 if (! norecord)
8697 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
73be902c 8698
df7492f9
KH		 8699 return (BUFFERP (dst_object)
8700 ? make_number (coding.produced_char)
8701 : coding.dst_object);
4ed46869 8702}
73be902c 8703
b73bfc1c 8704
ecec61c1 8705/* Encode or decode STRING according to CODING_SYSTEM.
ec6d2bb8 8706 Do not set Vlast_coding_system_used.
4ed46869 8707
ec6d2bb8
KH		 8708 This function is called only from macros DECODE_FILE and
8709 ENCODE_FILE, thus we ignore character composition. */
4ed46869 8710
ecec61c1
KH		 8711Lisp_Object
8712code_convert_string_norecord (string, coding_system, encodep)
8713 Lisp_Object string, coding_system;
8714 int encodep;
4ed46869 8715{
0be8721c 8716 return code_convert_string (string, coding_system, Qt, encodep, 0, 1);
4ed46869
KH		 8717}
8718
4ed46869 8719
df7492f9
KH		 8720DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string,
8721 2, 4, 0,
8722 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
8723
8724Optional third arg NOCOPY non-nil means it is OK to return STRING itself
8725if the decoding operation is trivial.
ecec61c1 8726
d4a1d553 8727Optional fourth arg BUFFER non-nil means that the decoded text is
1560f91a
EZ		 8728inserted in that buffer after point (point does not move). In this
8729case, the return value is the length of the decoded text.
ecec61c1 8730
df7492f9
KH		 8731This function sets `last-coding-system-used' to the precise coding system
8732used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
d4a1d553 8733not fully specified.) */)
df7492f9
KH		 8734 (string, coding_system, nocopy, buffer)
8735 Lisp_Object string, coding_system, nocopy, buffer;
4ed46869 8736{
df7492f9
KH		 8737 return code_convert_string (string, coding_system, buffer,
8738 0, ! NILP (nocopy), 0);
4ed46869
KH		 8739}
8740
df7492f9
KH		 8741DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
8742 2, 4, 0,
8743 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
8744
8745Optional third arg NOCOPY non-nil means it is OK to return STRING
8746itself if the encoding operation is trivial.
8747
d4a1d553 8748Optional fourth arg BUFFER non-nil means that the encoded text is
1560f91a
EZ		 8749inserted in that buffer after point (point does not move). In this
8750case, the return value is the length of the encoded text.
df7492f9
KH		 8751
8752This function sets `last-coding-system-used' to the precise coding system
8753used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
8754not fully specified.) */)
8755 (string, coding_system, nocopy, buffer)
8756 Lisp_Object string, coding_system, nocopy, buffer;
4ed46869 8757{
df7492f9 8758 return code_convert_string (string, coding_system, buffer,
c197f191 8759 1, ! NILP (nocopy), 1);
4ed46869 8760}
df7492f9 8761
3a73fa5d 8762\f
4ed46869 8763DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0,
48b0f3ae
PJ		 8764 doc: /* Decode a Japanese character which has CODE in shift_jis encoding.
8765Return the corresponding character. */)
8766 (code)
4ed46869 8767 Lisp_Object code;
4ed46869 8768{
df7492f9
KH		 8769 Lisp_Object spec, attrs, val;
8770 struct charset *charset_roman, *charset_kanji, *charset_kana, *charset;
8771 int c;
4ed46869 8772
df7492f9
KH		 8773 CHECK_NATNUM (code);
8774 c = XFASTINT (code);
8775 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8776 attrs = AREF (spec, 0);
4ed46869 8777
df7492f9
KH		 8778 if (ASCII_BYTE_P (c)
8779 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8780 return code;
4ed46869 8781
df7492f9
KH		 8782 val = CODING_ATTR_CHARSET_LIST (attrs);
8783 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
004068e4
KH		 8784 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8785 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 8786
df7492f9
KH		 8787 if (c <= 0x7F)
8788 charset = charset_roman;
8789 else if (c >= 0xA0 && c < 0xDF)
55ab7be3 8790 {
df7492f9
KH		 8791 charset = charset_kana;
8792 c -= 0x80;
4ed46869 8793 }
55ab7be3 8794 else
4ed46869 8795 {
004068e4 8796 int s1 = c >> 8, s2 = c & 0xFF;
df7492f9
KH		 8797
8798 if (s1 < 0x81 || (s1 > 0x9F && s1 < 0xE0) || s1 > 0xEF
8799 || s2 < 0x40 || s2 == 0x7F || s2 > 0xFC)
8800 error ("Invalid code: %d", code);
8801 SJIS_TO_JIS (c);
8802 charset = charset_kanji;
4ed46869 8803 }
df7492f9
KH		 8804 c = DECODE_CHAR (charset, c);
8805 if (c < 0)
8806 error ("Invalid code: %d", code);
8807 return make_number (c);
93dec019 8808}
4ed46869 8809
48b0f3ae 8810
4ed46869 8811DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0,
8acf0c0e 8812 doc: /* Encode a Japanese character CH to shift_jis encoding.
48b0f3ae
PJ		 8813Return the corresponding code in SJIS. */)
8814 (ch)
df7492f9 8815 Lisp_Object ch;
4ed46869 8816{
df7492f9
KH		 8817 Lisp_Object spec, attrs, charset_list;
8818 int c;
8819 struct charset *charset;
8820 unsigned code;
48b0f3ae 8821
df7492f9
KH		 8822 CHECK_CHARACTER (ch);
8823 c = XFASTINT (ch);
8824 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8825 attrs = AREF (spec, 0);
8826
8827 if (ASCII_CHAR_P (c)
8828 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8829 return ch;
8830
8831 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8832 charset = char_charset (c, charset_list, &code);
8833 if (code == CHARSET_INVALID_CODE (charset))
8834 error ("Can't encode by shift_jis encoding: %d", c);
8835 JIS_TO_SJIS (code);
8836
8837 return make_number (code);
4ed46869
KH		 8838}
8839
8840DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0,
48b0f3ae
PJ		 8841 doc: /* Decode a Big5 character which has CODE in BIG5 coding system.
8842Return the corresponding character. */)
8843 (code)
4ed46869 8844 Lisp_Object code;
d46c5b12 8845{
df7492f9
KH		 8846 Lisp_Object spec, attrs, val;
8847 struct charset *charset_roman, *charset_big5, *charset;
8848 int c;
6289dd10 8849
df7492f9
KH		 8850 CHECK_NATNUM (code);
8851 c = XFASTINT (code);
8852 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8853 attrs = AREF (spec, 0);
4ed46869 8854
df7492f9
KH		 8855 if (ASCII_BYTE_P (c)
8856 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8857 return code;
6289dd10 8858
df7492f9
KH		 8859 val = CODING_ATTR_CHARSET_LIST (attrs);
8860 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8861 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
c210f766 8862
df7492f9
KH		 8863 if (c <= 0x7F)
8864 charset = charset_roman;
c28a9453
KH		 8865 else
8866 {
df7492f9
KH		 8867 int b1 = c >> 8, b2 = c & 0x7F;
8868 if (b1 < 0xA1 || b1 > 0xFE
8869 || b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE)
8870 error ("Invalid code: %d", code);
8871 charset = charset_big5;
c28a9453 8872 }
df7492f9
KH		 8873 c = DECODE_CHAR (charset, (unsigned )c);
8874 if (c < 0)
8875 error ("Invalid code: %d", code);
8876 return make_number (c);
d46c5b12 8877}
6289dd10 8878
4ed46869 8879DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0,
8acf0c0e 8880 doc: /* Encode the Big5 character CH to BIG5 coding system.
48b0f3ae
PJ		 8881Return the corresponding character code in Big5. */)
8882 (ch)
4ed46869
KH		 8883 Lisp_Object ch;
8884{
df7492f9
KH		 8885 Lisp_Object spec, attrs, charset_list;
8886 struct charset *charset;
8887 int c;
8888 unsigned code;
8889
8890 CHECK_CHARACTER (ch);
8891 c = XFASTINT (ch);
8892 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8893 attrs = AREF (spec, 0);
8894 if (ASCII_CHAR_P (c)
8895 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8896 return ch;
8897
8898 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8899 charset = char_charset (c, charset_list, &code);
8900 if (code == CHARSET_INVALID_CODE (charset))
8901 error ("Can't encode by Big5 encoding: %d", c);
8902
8903 return make_number (code);
4ed46869 8904}
48b0f3ae 8905
3a73fa5d 8906\f
002fdb44 8907DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal,
68bba4e4 8908 Sset_terminal_coding_system_internal, 1, 2, 0,
48b0f3ae 8909 doc: /* Internal use only. */)
6ed8eeff 8910 (coding_system, terminal)
b74e4686 8911 Lisp_Object coding_system;
6ed8eeff 8912 Lisp_Object terminal;
4ed46869 8913{
6ed8eeff 8914 struct coding_system *terminal_coding = TERMINAL_TERMINAL_CODING (get_terminal (terminal, 1));
b7826503 8915 CHECK_SYMBOL (coding_system);
b8299c66 8916 setup_coding_system (Fcheck_coding_system (coding_system), terminal_coding);
70c22245 8917 /* We had better not send unsafe characters to terminal. */
c73bd236 8918 terminal_coding->mode |= CODING_MODE_SAFE_ENCODING;
df7492f9 8919 /* Characer composition should be disabled. */
c73bd236 8920 terminal_coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b8299c66
KL		 8921 terminal_coding->src_multibyte = 1;
8922 terminal_coding->dst_multibyte = 0;
4ed46869
KH		 8923 return Qnil;
8924}
8925
c4825358
KH		 8926DEFUN ("set-safe-terminal-coding-system-internal",
8927 Fset_safe_terminal_coding_system_internal,
48b0f3ae 8928 Sset_safe_terminal_coding_system_internal, 1, 1, 0,
ddb67bdc 8929 doc: /* Internal use only. */)
48b0f3ae 8930 (coding_system)
b74e4686 8931 Lisp_Object coding_system;
d46c5b12 8932{
b7826503 8933 CHECK_SYMBOL (coding_system);
c4825358
KH		 8934 setup_coding_system (Fcheck_coding_system (coding_system),
8935 &safe_terminal_coding);
df7492f9
KH		 8936 /* Characer composition should be disabled. */
8937 safe_terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b73bfc1c
KH		 8938 safe_terminal_coding.src_multibyte = 1;
8939 safe_terminal_coding.dst_multibyte = 0;
c4825358
KH		 8940 return Qnil;
8941}
4ed46869 8942
002fdb44 8943DEFUN ("terminal-coding-system", Fterminal_coding_system,
68bba4e4 8944 Sterminal_coding_system, 0, 1, 0,
6ed8eeff
KL		 8945 doc: /* Return coding system specified for terminal output on the given terminal.
8946TERMINAL may be a terminal id, a frame, or nil for the selected
8947frame's terminal device. */)
8948 (terminal)
8949 Lisp_Object terminal;
4ed46869 8950{
985773c9
MB		 8951 struct coding_system *terminal_coding
8952 = TERMINAL_TERMINAL_CODING (get_terminal (terminal, 1));
8953 Lisp_Object coding_system = CODING_ID_NAME (terminal_coding->id);
ae6f73fa 8954
ae6f73fa 8955 /* For backward compatibility, return nil if it is `undecided'. */
f75c90a9 8956 return (! EQ (coding_system, Qundecided) ? coding_system : Qnil);
4ed46869
KH		 8957}
8958
002fdb44 8959DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal,
68bba4e4 8960 Sset_keyboard_coding_system_internal, 1, 2, 0,
48b0f3ae 8961 doc: /* Internal use only. */)
6ed8eeff 8962 (coding_system, terminal)
4ed46869 8963 Lisp_Object coding_system;
6ed8eeff 8964 Lisp_Object terminal;
4ed46869 8965{
6ed8eeff 8966 struct terminal *t = get_terminal (terminal, 1);
b7826503 8967 CHECK_SYMBOL (coding_system);
df7492f9 8968 setup_coding_system (Fcheck_coding_system (coding_system),
c73bd236 8969 TERMINAL_KEYBOARD_CODING (t));
df7492f9 8970 /* Characer composition should be disabled. */
c73bd236
MB		 8971 TERMINAL_KEYBOARD_CODING (t)->common_flags
8972 &= ~CODING_ANNOTATE_COMPOSITION_MASK;
4ed46869
KH		 8973 return Qnil;
8974}
8975
8976DEFUN ("keyboard-coding-system",
985773c9 8977 Fkeyboard_coding_system, Skeyboard_coding_system, 0, 1, 0,
48b0f3ae 8978 doc: /* Return coding system specified for decoding keyboard input. */)
985773c9
MB		 8979 (terminal)
8980 Lisp_Object terminal;
4ed46869 8981{
985773c9
MB		 8982 return CODING_ID_NAME (TERMINAL_KEYBOARD_CODING
8983 (get_terminal (terminal, 1))->id);
4ed46869
KH		 8984}
8985
4ed46869 8986\f
a5d301df
KH		 8987DEFUN ("find-operation-coding-system", Ffind_operation_coding_system,
8988 Sfind_operation_coding_system, 1, MANY, 0,
48b0f3ae
PJ		 8989 doc: /* Choose a coding system for an operation based on the target name.
8990The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
8991DECODING-SYSTEM is the coding system to use for decoding
8992\(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
8993for encoding (in case OPERATION does encoding).
05e6f5dc 8994
48b0f3ae
PJ		 8995The first argument OPERATION specifies an I/O primitive:
8996 For file I/O, `insert-file-contents' or `write-region'.
8997 For process I/O, `call-process', `call-process-region', or `start-process'.
8998 For network I/O, `open-network-stream'.
05e6f5dc 8999
48b0f3ae
PJ		 9000The remaining arguments should be the same arguments that were passed
9001to the primitive. Depending on which primitive, one of those arguments
9002is selected as the TARGET. For example, if OPERATION does file I/O,
9003whichever argument specifies the file name is TARGET.
05e6f5dc 9004
48b0f3ae 9005TARGET has a meaning which depends on OPERATION:
b883cdb2 9006 For file I/O, TARGET is a file name (except for the special case below).
48b0f3ae 9007 For process I/O, TARGET is a process name.
d4a1d553 9008 For network I/O, TARGET is a service name or a port number.
05e6f5dc 9009
d4a1d553 9010This function looks up what is specified for TARGET in
48b0f3ae
PJ		 9011`file-coding-system-alist', `process-coding-system-alist',
9012or `network-coding-system-alist' depending on OPERATION.
9013They may specify a coding system, a cons of coding systems,
9014or a function symbol to call.
9015In the last case, we call the function with one argument,
9016which is a list of all the arguments given to this function.
1011c487
MB		 9017If the function can't decide a coding system, it can return
9018`undecided' so that the normal code-detection is performed.
48b0f3ae 9019
b883cdb2
MB		 9020If OPERATION is `insert-file-contents', the argument corresponding to
9021TARGET may be a cons (FILENAME . BUFFER). In that case, FILENAME is a
9022file name to look up, and BUFFER is a buffer that contains the file's
9023contents (not yet decoded). If `file-coding-system-alist' specifies a
9024function to call for FILENAME, that function should examine the
9025contents of BUFFER instead of reading the file.
9026
d918f936 9027usage: (find-operation-coding-system OPERATION ARGUMENTS...) */)
48b0f3ae 9028 (nargs, args)
4ed46869
KH		 9029 int nargs;
9030 Lisp_Object *args;
6b89e3aa 9031{
4ed46869
KH		 9032 Lisp_Object operation, target_idx, target, val;
9033 register Lisp_Object chain;
177c0ea7 9034
4ed46869
KH		 9035 if (nargs < 2)
9036 error ("Too few arguments");
9037 operation = args[0];
9038 if (!SYMBOLP (operation)
9039 || !INTEGERP (target_idx = Fget (operation, Qtarget_idx)))
3ed051d4 9040 error ("Invalid first argument");
4ed46869
KH		 9041 if (nargs < 1 + XINT (target_idx))
9042 error ("Too few arguments for operation: %s",
8f924df7 9043 SDATA (SYMBOL_NAME (operation)));
4ed46869
KH		 9044 target = args[XINT (target_idx) + 1];
9045 if (!(STRINGP (target)
091a0ff0
KH		 9046 || (EQ (operation, Qinsert_file_contents) && CONSP (target)
9047 && STRINGP (XCAR (target)) && BUFFERP (XCDR (target)))
4ed46869 9048 || (EQ (operation, Qopen_network_stream) && INTEGERP (target))))
df7492f9 9049 error ("Invalid %dth argument", XINT (target_idx) + 1);
091a0ff0
KH		 9050 if (CONSP (target))
9051 target = XCAR (target);
4ed46869 9052
2e34157c
RS		 9053 chain = ((EQ (operation, Qinsert_file_contents)
9054 || EQ (operation, Qwrite_region))
02ba4723 9055 ? Vfile_coding_system_alist
2e34157c 9056 : (EQ (operation, Qopen_network_stream)
02ba4723
KH		 9057 ? Vnetwork_coding_system_alist
9058 : Vprocess_coding_system_alist));
4ed46869
KH		 9059 if (NILP (chain))
9060 return Qnil;
9061
03699b14 9062 for (; CONSP (chain); chain = XCDR (chain))
6b89e3aa 9063 {
f44d27ce 9064 Lisp_Object elt;
6b89e3aa 9065
df7492f9 9066 elt = XCAR (chain);
4ed46869
KH		 9067 if (CONSP (elt)
9068 && ((STRINGP (target)
03699b14
KR		 9069 && STRINGP (XCAR (elt))
9070 && fast_string_match (XCAR (elt), target) >= 0)
9071 || (INTEGERP (target) && EQ (target, XCAR (elt)))))
6b89e3aa 9072 {
03699b14 9073 val = XCDR (elt);
b19fd4c5
KH		 9074 /* Here, if VAL is both a valid coding system and a valid
9075 function symbol, we return VAL as a coding system. */
02ba4723
KH		 9076 if (CONSP (val))
9077 return val;
9078 if (! SYMBOLP (val))
9079 return Qnil;
9080 if (! NILP (Fcoding_system_p (val)))
9081 return Fcons (val, val);
b19fd4c5 9082 if (! NILP (Ffboundp (val)))
6b89e3aa 9083 {
e2b97060
MB		 9084 /* We use call1 rather than safe_call1
9085 so as to get bug reports about functions called here
9086 which don't handle the current interface. */
9087 val = call1 (val, Flist (nargs, args));
b19fd4c5
KH		 9088 if (CONSP (val))
9089 return val;
9090 if (SYMBOLP (val) && ! NILP (Fcoding_system_p (val)))
9091 return Fcons (val, val);
6b89e3aa 9092 }
02ba4723 9093 return Qnil;
6b89e3aa
KH		 9094 }
9095 }
4ed46869 9096 return Qnil;
6b89e3aa
KH		 9097}
9098
df7492f9 9099DEFUN ("set-coding-system-priority", Fset_coding_system_priority,
a3f6ee6d 9100 Sset_coding_system_priority, 0, MANY, 0,
da7db224 9101 doc: /* Assign higher priority to the coding systems given as arguments.
d4a1d553 9102If multiple coding systems belong to the same category,
a3181084
DL		 9103all but the first one are ignored.
9104
d4a1d553 9105usage: (set-coding-system-priority &rest coding-systems) */)
df7492f9
KH		 9106 (nargs, args)
9107 int nargs;
9108 Lisp_Object *args;
9109{
9110 int i, j;
9111 int changed[coding_category_max];
9112 enum coding_category priorities[coding_category_max];
9113
9114 bzero (changed, sizeof changed);
6b89e3aa 9115
df7492f9 9116 for (i = j = 0; i < nargs; i++)
6b89e3aa 9117 {
df7492f9
KH		 9118 enum coding_category category;
9119 Lisp_Object spec, attrs;
6b89e3aa 9120
df7492f9
KH		 9121 CHECK_CODING_SYSTEM_GET_SPEC (args[i], spec);
9122 attrs = AREF (spec, 0);
9123 category = XINT (CODING_ATTR_CATEGORY (attrs));
9124 if (changed[category])
9125 /* Ignore this coding system because a coding system of the
9126 same category already had a higher priority. */
9127 continue;
9128 changed[category] = 1;
9129 priorities[j++] = category;
9130 if (coding_categories[category].id >= 0
9131 && ! EQ (args[i], CODING_ID_NAME (coding_categories[category].id)))
9132 setup_coding_system (args[i], &coding_categories[category]);
ff563fce 9133 Fset (AREF (Vcoding_category_table, category), args[i]);
df7492f9 9134 }
6b89e3aa 9135
df7492f9
KH		 9136 /* Now we have decided top J priorities. Reflect the order of the
9137 original priorities to the remaining priorities. */
6b89e3aa 9138
df7492f9 9139 for (i = j, j = 0; i < coding_category_max; i++, j++)
6b89e3aa 9140 {
df7492f9
KH		 9141 while (j < coding_category_max
9142 && changed[coding_priorities[j]])
9143 j++;
9144 if (j == coding_category_max)
9145 abort ();
9146 priorities[i] = coding_priorities[j];
9147 }
6b89e3aa 9148
df7492f9 9149 bcopy (priorities, coding_priorities, sizeof priorities);
177c0ea7 9150
ff563fce
KH		 9151 /* Update `coding-category-list'. */
9152 Vcoding_category_list = Qnil;
9153 for (i = coding_category_max - 1; i >= 0; i--)
9154 Vcoding_category_list
9155 = Fcons (AREF (Vcoding_category_table, priorities[i]),
9156 Vcoding_category_list);
6b89e3aa 9157
df7492f9 9158 return Qnil;
6b89e3aa
KH		 9159}
9160
df7492f9
KH		 9161DEFUN ("coding-system-priority-list", Fcoding_system_priority_list,
9162 Scoding_system_priority_list, 0, 1, 0,
da7db224
DL		 9163 doc: /* Return a list of coding systems ordered by their priorities.
9164HIGHESTP non-nil means just return the highest priority one. */)
df7492f9
KH		 9165 (highestp)
9166 Lisp_Object highestp;
d46c5b12
KH		 9167{
9168 int i;
df7492f9 9169 Lisp_Object val;
6b89e3aa 9170
df7492f9 9171 for (i = 0, val = Qnil; i < coding_category_max; i++)
d46c5b12 9172 {
df7492f9
KH		 9173 enum coding_category category = coding_priorities[i];
9174 int id = coding_categories[category].id;
9175 Lisp_Object attrs;
068a9dbd 9176
df7492f9
KH		 9177 if (id < 0)
9178 continue;
9179 attrs = CODING_ID_ATTRS (id);
9180 if (! NILP (highestp))
9181 return CODING_ATTR_BASE_NAME (attrs);
9182 val = Fcons (CODING_ATTR_BASE_NAME (attrs), val);
9183 }
9184 return Fnreverse (val);
9185}
068a9dbd 9186
f0064e1f 9187static char *suffixes[] = { "-unix", "-dos", "-mac" };
068a9dbd
KH		 9188
9189static Lisp_Object
df7492f9
KH		 9190make_subsidiaries (base)
9191 Lisp_Object base;
068a9dbd 9192{
df7492f9 9193 Lisp_Object subsidiaries;
8f924df7 9194 int base_name_len = SBYTES (SYMBOL_NAME (base));
df7492f9
KH		 9195 char *buf = (char *) alloca (base_name_len + 6);
9196 int i;
068a9dbd 9197
8f924df7 9198 bcopy (SDATA (SYMBOL_NAME (base)), buf, base_name_len);
df7492f9
KH		 9199 subsidiaries = Fmake_vector (make_number (3), Qnil);
9200 for (i = 0; i < 3; i++)
068a9dbd 9201 {
df7492f9
KH		 9202 bcopy (suffixes[i], buf + base_name_len, strlen (suffixes[i]) + 1);
9203 ASET (subsidiaries, i, intern (buf));
068a9dbd 9204 }
df7492f9 9205 return subsidiaries;
068a9dbd
KH		 9206}
9207
9208
df7492f9
KH		 9209DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal,
9210 Sdefine_coding_system_internal, coding_arg_max, MANY, 0,
1fcd6c8b
DL		 9211 doc: /* For internal use only.
9212usage: (define-coding-system-internal ...) */)
df7492f9
KH		 9213 (nargs, args)
9214 int nargs;
9215 Lisp_Object *args;
068a9dbd 9216{
df7492f9
KH		 9217 Lisp_Object name;
9218 Lisp_Object spec_vec; /* [ ATTRS ALIASE EOL_TYPE ] */
9219 Lisp_Object attrs; /* Vector of attributes. */
9220 Lisp_Object eol_type;
9221 Lisp_Object aliases;
9222 Lisp_Object coding_type, charset_list, safe_charsets;
9223 enum coding_category category;
9224 Lisp_Object tail, val;
9225 int max_charset_id = 0;
9226 int i;
068a9dbd 9227
df7492f9
KH		 9228 if (nargs < coding_arg_max)
9229 goto short_args;
068a9dbd 9230
df7492f9 9231 attrs = Fmake_vector (make_number (coding_attr_last_index), Qnil);
068a9dbd 9232
df7492f9
KH		 9233 name = args[coding_arg_name];
9234 CHECK_SYMBOL (name);
9235 CODING_ATTR_BASE_NAME (attrs) = name;
068a9dbd 9236
df7492f9
KH		 9237 val = args[coding_arg_mnemonic];
9238 if (! STRINGP (val))
9239 CHECK_CHARACTER (val);
9240 CODING_ATTR_MNEMONIC (attrs) = val;
068a9dbd 9241
df7492f9
KH		 9242 coding_type = args[coding_arg_coding_type];
9243 CHECK_SYMBOL (coding_type);
9244 CODING_ATTR_TYPE (attrs) = coding_type;
068a9dbd 9245
df7492f9
KH		 9246 charset_list = args[coding_arg_charset_list];
9247 if (SYMBOLP (charset_list))
9248 {
9249 if (EQ (charset_list, Qiso_2022))
9250 {
9251 if (! EQ (coding_type, Qiso_2022))
9252 error ("Invalid charset-list");
9253 charset_list = Viso_2022_charset_list;
9254 }
9255 else if (EQ (charset_list, Qemacs_mule))
9256 {
9257 if (! EQ (coding_type, Qemacs_mule))
9258 error ("Invalid charset-list");
9259 charset_list = Vemacs_mule_charset_list;
9260 }
9261 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
9262 if (max_charset_id < XFASTINT (XCAR (tail)))
9263 max_charset_id = XFASTINT (XCAR (tail));
9264 }
068a9dbd
KH		 9265 else
9266 {
df7492f9 9267 charset_list = Fcopy_sequence (charset_list);
985773c9 9268 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
068a9dbd 9269 {
df7492f9
KH		 9270 struct charset *charset;
9271
985773c9 9272 val = XCAR (tail);
df7492f9
KH		 9273 CHECK_CHARSET_GET_CHARSET (val, charset);
9274 if (EQ (coding_type, Qiso_2022)
9275 ? CHARSET_ISO_FINAL (charset) < 0
9276 : EQ (coding_type, Qemacs_mule)
9277 ? CHARSET_EMACS_MULE_ID (charset) < 0
9278 : 0)
9279 error ("Can't handle charset `%s'",
8f924df7 9280 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9 9281
8f924df7 9282 XSETCAR (tail, make_number (charset->id));
df7492f9
KH		 9283 if (max_charset_id < charset->id)
9284 max_charset_id = charset->id;
068a9dbd
KH		 9285 }
9286 }
df7492f9 9287 CODING_ATTR_CHARSET_LIST (attrs) = charset_list;
068a9dbd 9288
df7492f9
KH		 9289 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
9290 make_number (255));
9291 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8f924df7 9292 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 9293 CODING_ATTR_SAFE_CHARSETS (attrs) = safe_charsets;
068a9dbd 9294
584948ac 9295 CODING_ATTR_ASCII_COMPAT (attrs) = args[coding_arg_ascii_compatible_p];
3a73fa5d 9296
df7492f9 9297 val = args[coding_arg_decode_translation_table];
a6f87d34 9298 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 9299 CHECK_SYMBOL (val);
df7492f9 9300 CODING_ATTR_DECODE_TBL (attrs) = val;
3a73fa5d 9301
df7492f9 9302 val = args[coding_arg_encode_translation_table];
a6f87d34 9303 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 9304 CHECK_SYMBOL (val);
df7492f9 9305 CODING_ATTR_ENCODE_TBL (attrs) = val;
d46c5b12 9306
df7492f9
KH		 9307 val = args[coding_arg_post_read_conversion];
9308 CHECK_SYMBOL (val);
9309 CODING_ATTR_POST_READ (attrs) = val;
d46c5b12 9310
df7492f9
KH		 9311 val = args[coding_arg_pre_write_conversion];
9312 CHECK_SYMBOL (val);
9313 CODING_ATTR_PRE_WRITE (attrs) = val;
3a73fa5d 9314
df7492f9
KH		 9315 val = args[coding_arg_default_char];
9316 if (NILP (val))
9317 CODING_ATTR_DEFAULT_CHAR (attrs) = make_number (' ');
9318 else
9319 {
8f924df7 9320 CHECK_CHARACTER (val);
df7492f9
KH		 9321 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
9322 }
4031e2bf 9323
8f924df7
KH		 9324 val = args[coding_arg_for_unibyte];
9325 CODING_ATTR_FOR_UNIBYTE (attrs) = NILP (val) ? Qnil : Qt;
3a73fa5d 9326
df7492f9
KH		 9327 val = args[coding_arg_plist];
9328 CHECK_LIST (val);
9329 CODING_ATTR_PLIST (attrs) = val;
3a73fa5d 9330
df7492f9
KH		 9331 if (EQ (coding_type, Qcharset))
9332 {
c7c66a95
KH		 9333 /* Generate a lisp vector of 256 elements. Each element is nil,
9334 integer, or a list of charset IDs.
3a73fa5d 9335
c7c66a95
KH		 9336 If Nth element is nil, the byte code N is invalid in this
9337 coding system.
4ed46869 9338
c7c66a95
KH		 9339 If Nth element is a number NUM, N is the first byte of a
9340 charset whose ID is NUM.
4ed46869 9341
c7c66a95
KH		 9342 If Nth element is a list of charset IDs, N is the first byte
9343 of one of them. The list is sorted by dimensions of the
2bc515e4 9344 charsets. A charset of smaller dimension comes firtst. */
df7492f9 9345 val = Fmake_vector (make_number (256), Qnil);
4ed46869 9346
5c99c2e6 9347 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
df7492f9 9348 {
c7c66a95
KH		 9349 struct charset *charset = CHARSET_FROM_ID (XFASTINT (XCAR (tail)));
9350 int dim = CHARSET_DIMENSION (charset);
9351 int idx = (dim - 1) * 4;
4ed46869 9352
5c99c2e6 9353 if (CHARSET_ASCII_COMPATIBLE_P (charset))
584948ac 9354 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
4031e2bf 9355
15d143f7
KH		 9356 for (i = charset->code_space[idx];
9357 i <= charset->code_space[idx + 1]; i++)
9358 {
c7c66a95
KH		 9359 Lisp_Object tmp, tmp2;
9360 int dim2;
ec6d2bb8 9361
c7c66a95
KH		 9362 tmp = AREF (val, i);
9363 if (NILP (tmp))
9364 tmp = XCAR (tail);
9365 else if (NUMBERP (tmp))
9366 {
9367 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (tmp)));
9368 if (dim < dim2)
c7c66a95 9369 tmp = Fcons (XCAR (tail), Fcons (tmp, Qnil));
f9d71dcd
KH		 9370 else
9371 tmp = Fcons (tmp, Fcons (XCAR (tail), Qnil));
c7c66a95 9372 }
15d143f7 9373 else
c7c66a95
KH		 9374 {
9375 for (tmp2 = tmp; CONSP (tmp2); tmp2 = XCDR (tmp2))
9376 {
9377 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (XCAR (tmp2))));
9378 if (dim < dim2)
9379 break;
9380 }
9381 if (NILP (tmp2))
9382 tmp = nconc2 (tmp, Fcons (XCAR (tail), Qnil));
9383 else
9384 {
9385 XSETCDR (tmp2, Fcons (XCAR (tmp2), XCDR (tmp2)));
9386 XSETCAR (tmp2, XCAR (tail));
9387 }
9388 }
9389 ASET (val, i, tmp);
15d143f7 9390 }
df7492f9
KH		 9391 }
9392 ASET (attrs, coding_attr_charset_valids, val);
9393 category = coding_category_charset;
9394 }
9395 else if (EQ (coding_type, Qccl))
9396 {
9397 Lisp_Object valids;
ecec61c1 9398
df7492f9
KH		 9399 if (nargs < coding_arg_ccl_max)
9400 goto short_args;
ecec61c1 9401
df7492f9
KH		 9402 val = args[coding_arg_ccl_decoder];
9403 CHECK_CCL_PROGRAM (val);
9404 if (VECTORP (val))
9405 val = Fcopy_sequence (val);
9406 ASET (attrs, coding_attr_ccl_decoder, val);
ecec61c1 9407
df7492f9
KH		 9408 val = args[coding_arg_ccl_encoder];
9409 CHECK_CCL_PROGRAM (val);
9410 if (VECTORP (val))
9411 val = Fcopy_sequence (val);
9412 ASET (attrs, coding_attr_ccl_encoder, val);
ecec61c1 9413
df7492f9
KH		 9414 val = args[coding_arg_ccl_valids];
9415 valids = Fmake_string (make_number (256), make_number (0));
9416 for (tail = val; !NILP (tail); tail = Fcdr (tail))
9417 {
8dcbea82 9418 int from, to;
ecec61c1 9419
df7492f9
KH		 9420 val = Fcar (tail);
9421 if (INTEGERP (val))
8dcbea82
KH		 9422 {
9423 from = to = XINT (val);
9424 if (from < 0 || from > 255)
9425 args_out_of_range_3 (val, make_number (0), make_number (255));
9426 }
df7492f9
KH		 9427 else
9428 {
df7492f9 9429 CHECK_CONS (val);
8f924df7
KH		 9430 CHECK_NATNUM_CAR (val);
9431 CHECK_NATNUM_CDR (val);
df7492f9 9432 from = XINT (XCAR (val));
8f924df7 9433 if (from > 255)
8dcbea82
KH		 9434 args_out_of_range_3 (XCAR (val),
9435 make_number (0), make_number (255));
df7492f9 9436 to = XINT (XCDR (val));
8dcbea82
KH		 9437 if (to < from || to > 255)
9438 args_out_of_range_3 (XCDR (val),
9439 XCAR (val), make_number (255));
df7492f9 9440 }
8dcbea82 9441 for (i = from; i <= to; i++)
8f924df7 9442 SSET (valids, i, 1);
df7492f9
KH		 9443 }
9444 ASET (attrs, coding_attr_ccl_valids, valids);
4ed46869 9445
df7492f9 9446 category = coding_category_ccl;
55ab7be3 9447 }
df7492f9 9448 else if (EQ (coding_type, Qutf_16))
55ab7be3 9449 {
df7492f9 9450 Lisp_Object bom, endian;
4ed46869 9451
584948ac 9452 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
4ed46869 9453
df7492f9
KH		 9454 if (nargs < coding_arg_utf16_max)
9455 goto short_args;
4ed46869 9456
df7492f9
KH		 9457 bom = args[coding_arg_utf16_bom];
9458 if (! NILP (bom) && ! EQ (bom, Qt))
9459 {
9460 CHECK_CONS (bom);
8f924df7
KH		 9461 val = XCAR (bom);
9462 CHECK_CODING_SYSTEM (val);
9463 val = XCDR (bom);
9464 CHECK_CODING_SYSTEM (val);
df7492f9 9465 }
a470d443 9466 ASET (attrs, coding_attr_utf_bom, bom);
df7492f9
KH		 9467
9468 endian = args[coding_arg_utf16_endian];
b49a1807
KH		 9469 CHECK_SYMBOL (endian);
9470 if (NILP (endian))
9471 endian = Qbig;
9472 else if (! EQ (endian, Qbig) && ! EQ (endian, Qlittle))
8f924df7 9473 error ("Invalid endian: %s", SDATA (SYMBOL_NAME (endian)));
df7492f9
KH		 9474 ASET (attrs, coding_attr_utf_16_endian, endian);
9475
9476 category = (CONSP (bom)
9477 ? coding_category_utf_16_auto
9478 : NILP (bom)
b49a1807 9479 ? (EQ (endian, Qbig)
df7492f9
KH		 9480 ? coding_category_utf_16_be_nosig
9481 : coding_category_utf_16_le_nosig)
b49a1807 9482 : (EQ (endian, Qbig)
df7492f9
KH		 9483 ? coding_category_utf_16_be
9484 : coding_category_utf_16_le));
9485 }
9486 else if (EQ (coding_type, Qiso_2022))
9487 {
9488 Lisp_Object initial, reg_usage, request, flags;
4776e638 9489 int i;
1397dc18 9490
df7492f9
KH		 9491 if (nargs < coding_arg_iso2022_max)
9492 goto short_args;
9493
9494 initial = Fcopy_sequence (args[coding_arg_iso2022_initial]);
9495 CHECK_VECTOR (initial);
9496 for (i = 0; i < 4; i++)
9497 {
9498 val = Faref (initial, make_number (i));
9499 if (! NILP (val))
9500 {
584948ac
KH		 9501 struct charset *charset;
9502
9503 CHECK_CHARSET_GET_CHARSET (val, charset);
9504 ASET (initial, i, make_number (CHARSET_ID (charset)));
9505 if (i == 0 && CHARSET_ASCII_COMPATIBLE_P (charset))
9506 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9507 }
9508 else
9509 ASET (initial, i, make_number (-1));
9510 }
9511
9512 reg_usage = args[coding_arg_iso2022_reg_usage];
9513 CHECK_CONS (reg_usage);
8f924df7
KH		 9514 CHECK_NUMBER_CAR (reg_usage);
9515 CHECK_NUMBER_CDR (reg_usage);
df7492f9
KH		 9516
9517 request = Fcopy_sequence (args[coding_arg_iso2022_request]);
9518 for (tail = request; ! NILP (tail); tail = Fcdr (tail))
1397dc18 9519 {
df7492f9 9520 int id;
8f924df7 9521 Lisp_Object tmp;
df7492f9
KH		 9522
9523 val = Fcar (tail);
9524 CHECK_CONS (val);
8f924df7
KH		 9525 tmp = XCAR (val);
9526 CHECK_CHARSET_GET_ID (tmp, id);
9527 CHECK_NATNUM_CDR (val);
df7492f9
KH		 9528 if (XINT (XCDR (val)) >= 4)
9529 error ("Invalid graphic register number: %d", XINT (XCDR (val)));
8f924df7 9530 XSETCAR (val, make_number (id));
1397dc18 9531 }
4ed46869 9532
df7492f9
KH		 9533 flags = args[coding_arg_iso2022_flags];
9534 CHECK_NATNUM (flags);
9535 i = XINT (flags);
9536 if (EQ (args[coding_arg_charset_list], Qiso_2022))
9537 flags = make_number (i | CODING_ISO_FLAG_FULL_SUPPORT);
9538
9539 ASET (attrs, coding_attr_iso_initial, initial);
9540 ASET (attrs, coding_attr_iso_usage, reg_usage);
9541 ASET (attrs, coding_attr_iso_request, request);
9542 ASET (attrs, coding_attr_iso_flags, flags);
9543 setup_iso_safe_charsets (attrs);
9544
9545 if (i & CODING_ISO_FLAG_SEVEN_BITS)
9546 category = ((i & (CODING_ISO_FLAG_LOCKING_SHIFT
9547 | CODING_ISO_FLAG_SINGLE_SHIFT))
9548 ? coding_category_iso_7_else
9549 : EQ (args[coding_arg_charset_list], Qiso_2022)
9550 ? coding_category_iso_7
9551 : coding_category_iso_7_tight);
9552 else
9553 {
9554 int id = XINT (AREF (initial, 1));
9555
c6fb6e98 9556 category = (((i & CODING_ISO_FLAG_LOCKING_SHIFT)
df7492f9
KH		 9557 || EQ (args[coding_arg_charset_list], Qiso_2022)
9558 || id < 0)
9559 ? coding_category_iso_8_else
9560 : (CHARSET_DIMENSION (CHARSET_FROM_ID (id)) == 1)
9561 ? coding_category_iso_8_1
9562 : coding_category_iso_8_2);
9563 }
0ce7886f
KH		 9564 if (category != coding_category_iso_8_1
9565 && category != coding_category_iso_8_2)
9566 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
df7492f9
KH		 9567 }
9568 else if (EQ (coding_type, Qemacs_mule))
c28a9453 9569 {
df7492f9
KH		 9570 if (EQ (args[coding_arg_charset_list], Qemacs_mule))
9571 ASET (attrs, coding_attr_emacs_mule_full, Qt);
584948ac 9572 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9 9573 category = coding_category_emacs_mule;
c28a9453 9574 }
df7492f9 9575 else if (EQ (coding_type, Qshift_jis))
c28a9453 9576 {
df7492f9
KH		 9577
9578 struct charset *charset;
9579
7d64c6ad 9580 if (XINT (Flength (charset_list)) != 3
6e07c25f 9581 && XINT (Flength (charset_list)) != 4)
7d64c6ad 9582 error ("There should be three or four charsets");
df7492f9
KH		 9583
9584 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9585 if (CHARSET_DIMENSION (charset) != 1)
9586 error ("Dimension of charset %s is not one",
8f924df7 9587 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 9588 if (CHARSET_ASCII_COMPATIBLE_P (charset))
9589 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9590
9591 charset_list = XCDR (charset_list);
9592 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9593 if (CHARSET_DIMENSION (charset) != 1)
9594 error ("Dimension of charset %s is not one",
8f924df7 9595 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9
KH		 9596
9597 charset_list = XCDR (charset_list);
9598 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9599 if (CHARSET_DIMENSION (charset) != 2)
7d64c6ad
KH		 9600 error ("Dimension of charset %s is not two",
9601 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9602
9603 charset_list = XCDR (charset_list);
2b917a06
KH		 9604 if (! NILP (charset_list))
9605 {
9606 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9607 if (CHARSET_DIMENSION (charset) != 2)
9608 error ("Dimension of charset %s is not two",
9609 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9610 }
df7492f9
KH		 9611
9612 category = coding_category_sjis;
9613 Vsjis_coding_system = name;
c28a9453 9614 }
df7492f9
KH		 9615 else if (EQ (coding_type, Qbig5))
9616 {
9617 struct charset *charset;
4ed46869 9618
df7492f9
KH		 9619 if (XINT (Flength (charset_list)) != 2)
9620 error ("There should be just two charsets");
9621
9622 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9623 if (CHARSET_DIMENSION (charset) != 1)
9624 error ("Dimension of charset %s is not one",
8f924df7 9625 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 9626 if (CHARSET_ASCII_COMPATIBLE_P (charset))
9627 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9628
9629 charset_list = XCDR (charset_list);
9630 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9631 if (CHARSET_DIMENSION (charset) != 2)
9632 error ("Dimension of charset %s is not two",
8f924df7 9633 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
4ed46869 9634
df7492f9
KH		 9635 category = coding_category_big5;
9636 Vbig5_coding_system = name;
9637 }
9638 else if (EQ (coding_type, Qraw_text))
c28a9453 9639 {
584948ac
KH		 9640 category = coding_category_raw_text;
9641 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
c28a9453 9642 }
df7492f9 9643 else if (EQ (coding_type, Qutf_8))
4ed46869 9644 {
a470d443
KH		 9645 Lisp_Object bom;
9646
584948ac 9647 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
a470d443
KH		 9648
9649 if (nargs < coding_arg_utf8_max)
9650 goto short_args;
9651
9652 bom = args[coding_arg_utf8_bom];
9653 if (! NILP (bom) && ! EQ (bom, Qt))
9654 {
9655 CHECK_CONS (bom);
9656 val = XCAR (bom);
9657 CHECK_CODING_SYSTEM (val);
9658 val = XCDR (bom);
9659 CHECK_CODING_SYSTEM (val);
9660 }
9661 ASET (attrs, coding_attr_utf_bom, bom);
9662
9663 category = (CONSP (bom) ? coding_category_utf_8_auto
9664 : NILP (bom) ? coding_category_utf_8_nosig
9665 : coding_category_utf_8_sig);
4ed46869 9666 }
df7492f9
KH		 9667 else if (EQ (coding_type, Qundecided))
9668 category = coding_category_undecided;
4ed46869 9669 else
df7492f9 9670 error ("Invalid coding system type: %s",
8f924df7 9671 SDATA (SYMBOL_NAME (coding_type)));
4ed46869 9672
df7492f9 9673 CODING_ATTR_CATEGORY (attrs) = make_number (category);
01378f49
KH		 9674 CODING_ATTR_PLIST (attrs)
9675 = Fcons (QCcategory, Fcons (AREF (Vcoding_category_table, category),
9676 CODING_ATTR_PLIST (attrs)));
35befdaa 9677 CODING_ATTR_PLIST (attrs)
3ed051d4 9678 = Fcons (QCascii_compatible_p,
35befdaa
KH		 9679 Fcons (CODING_ATTR_ASCII_COMPAT (attrs),
9680 CODING_ATTR_PLIST (attrs)));
c4825358 9681
df7492f9
KH		 9682 eol_type = args[coding_arg_eol_type];
9683 if (! NILP (eol_type)
9684 && ! EQ (eol_type, Qunix)
9685 && ! EQ (eol_type, Qdos)
9686 && ! EQ (eol_type, Qmac))
9687 error ("Invalid eol-type");
4ed46869 9688
df7492f9 9689 aliases = Fcons (name, Qnil);
4ed46869 9690
df7492f9
KH		 9691 if (NILP (eol_type))
9692 {
9693 eol_type = make_subsidiaries (name);
9694 for (i = 0; i < 3; i++)
1397dc18 9695 {
df7492f9
KH		 9696 Lisp_Object this_spec, this_name, this_aliases, this_eol_type;
9697
9698 this_name = AREF (eol_type, i);
9699 this_aliases = Fcons (this_name, Qnil);
9700 this_eol_type = (i == 0 ? Qunix : i == 1 ? Qdos : Qmac);
9701 this_spec = Fmake_vector (make_number (3), attrs);
9702 ASET (this_spec, 1, this_aliases);
9703 ASET (this_spec, 2, this_eol_type);
9704 Fputhash (this_name, this_spec, Vcoding_system_hash_table);
9705 Vcoding_system_list = Fcons (this_name, Vcoding_system_list);
583f71ca
KH		 9706 val = Fassoc (Fsymbol_name (this_name), Vcoding_system_alist);
9707 if (NILP (val))
9708 Vcoding_system_alist
9709 = Fcons (Fcons (Fsymbol_name (this_name), Qnil),
9710 Vcoding_system_alist);
1397dc18 9711 }
d46c5b12 9712 }
4ed46869 9713
df7492f9
KH		 9714 spec_vec = Fmake_vector (make_number (3), attrs);
9715 ASET (spec_vec, 1, aliases);
9716 ASET (spec_vec, 2, eol_type);
48b0f3ae 9717
df7492f9
KH		 9718 Fputhash (name, spec_vec, Vcoding_system_hash_table);
9719 Vcoding_system_list = Fcons (name, Vcoding_system_list);
583f71ca
KH		 9720 val = Fassoc (Fsymbol_name (name), Vcoding_system_alist);
9721 if (NILP (val))
9722 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (name), Qnil),
9723 Vcoding_system_alist);
48b0f3ae 9724
df7492f9
KH		 9725 {
9726 int id = coding_categories[category].id;
48b0f3ae 9727
df7492f9
KH		 9728 if (id < 0 || EQ (name, CODING_ID_NAME (id)))
9729 setup_coding_system (name, &coding_categories[category]);
9730 }
48b0f3ae 9731
d46c5b12 9732 return Qnil;
48b0f3ae 9733
df7492f9
KH		 9734 short_args:
9735 return Fsignal (Qwrong_number_of_arguments,
9736 Fcons (intern ("define-coding-system-internal"),
9737 make_number (nargs)));
d46c5b12 9738}
4ed46869 9739
d6925f38 9740
a6f87d34
KH		 9741DEFUN ("coding-system-put", Fcoding_system_put, Scoding_system_put,
9742 3, 3, 0,
9743 doc: /* Change value in CODING-SYSTEM's property list PROP to VAL. */)
9744 (coding_system, prop, val)
9745 Lisp_Object coding_system, prop, val;
9746{
3dbe7859 9747 Lisp_Object spec, attrs;
a6f87d34
KH		 9748
9749 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9750 attrs = AREF (spec, 0);
9751 if (EQ (prop, QCmnemonic))
9752 {
9753 if (! STRINGP (val))
9754 CHECK_CHARACTER (val);
9755 CODING_ATTR_MNEMONIC (attrs) = val;
9756 }
2133e2d1 9757 else if (EQ (prop, QCdefault_char))
a6f87d34
KH		 9758 {
9759 if (NILP (val))
9760 val = make_number (' ');
9761 else
9762 CHECK_CHARACTER (val);
9763 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
9764 }
9765 else if (EQ (prop, QCdecode_translation_table))
9766 {
9767 if (! CHAR_TABLE_P (val) && ! CONSP (val))
9768 CHECK_SYMBOL (val);
9769 CODING_ATTR_DECODE_TBL (attrs) = val;
9770 }
9771 else if (EQ (prop, QCencode_translation_table))
9772 {
9773 if (! CHAR_TABLE_P (val) && ! CONSP (val))
9774 CHECK_SYMBOL (val);
9775 CODING_ATTR_ENCODE_TBL (attrs) = val;
9776 }
9777 else if (EQ (prop, QCpost_read_conversion))
9778 {
9779 CHECK_SYMBOL (val);
9780 CODING_ATTR_POST_READ (attrs) = val;
9781 }
9782 else if (EQ (prop, QCpre_write_conversion))
9783 {
9784 CHECK_SYMBOL (val);
9785 CODING_ATTR_PRE_WRITE (attrs) = val;
9786 }
35befdaa
KH		 9787 else if (EQ (prop, QCascii_compatible_p))
9788 {
9789 CODING_ATTR_ASCII_COMPAT (attrs) = val;
9790 }
a6f87d34
KH		 9791
9792 CODING_ATTR_PLIST (attrs)
9793 = Fplist_put (CODING_ATTR_PLIST (attrs), prop, val);
9794 return val;
9795}
9796
9797
df7492f9
KH		 9798DEFUN ("define-coding-system-alias", Fdefine_coding_system_alias,
9799 Sdefine_coding_system_alias, 2, 2, 0,
9800 doc: /* Define ALIAS as an alias for CODING-SYSTEM. */)
9801 (alias, coding_system)
9802 Lisp_Object alias, coding_system;
66cfb530 9803{
583f71ca 9804 Lisp_Object spec, aliases, eol_type, val;
4ed46869 9805
df7492f9
KH		 9806 CHECK_SYMBOL (alias);
9807 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9808 aliases = AREF (spec, 1);
d4a1d553 9809 /* ALIASES should be a list of length more than zero, and the first
d6925f38
KH		 9810 element is a base coding system. Append ALIAS at the tail of the
9811 list. */
df7492f9
KH		 9812 while (!NILP (XCDR (aliases)))
9813 aliases = XCDR (aliases);
8f924df7 9814 XSETCDR (aliases, Fcons (alias, Qnil));
4ed46869 9815
df7492f9
KH		 9816 eol_type = AREF (spec, 2);
9817 if (VECTORP (eol_type))
4ed46869 9818 {
df7492f9
KH		 9819 Lisp_Object subsidiaries;
9820 int i;
4ed46869 9821
df7492f9
KH		 9822 subsidiaries = make_subsidiaries (alias);
9823 for (i = 0; i < 3; i++)
9824 Fdefine_coding_system_alias (AREF (subsidiaries, i),
9825 AREF (eol_type, i));
4ed46869 9826 }
df7492f9
KH		 9827
9828 Fputhash (alias, spec, Vcoding_system_hash_table);
d6925f38 9829 Vcoding_system_list = Fcons (alias, Vcoding_system_list);
583f71ca
KH		 9830 val = Fassoc (Fsymbol_name (alias), Vcoding_system_alist);
9831 if (NILP (val))
9832 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (alias), Qnil),
9833 Vcoding_system_alist);
66cfb530 9834
4ed46869
KH		 9835 return Qnil;
9836}
9837
df7492f9
KH		 9838DEFUN ("coding-system-base", Fcoding_system_base, Scoding_system_base,
9839 1, 1, 0,
9840 doc: /* Return the base of CODING-SYSTEM.
da7db224 9841Any alias or subsidiary coding system is not a base coding system. */)
df7492f9
KH		 9842 (coding_system)
9843 Lisp_Object coding_system;
d46c5b12 9844{
df7492f9 9845 Lisp_Object spec, attrs;
d46c5b12 9846
df7492f9
KH		 9847 if (NILP (coding_system))
9848 return (Qno_conversion);
9849 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9850 attrs = AREF (spec, 0);
9851 return CODING_ATTR_BASE_NAME (attrs);
9852}
1397dc18 9853
df7492f9
KH		 9854DEFUN ("coding-system-plist", Fcoding_system_plist, Scoding_system_plist,
9855 1, 1, 0,
9856 doc: "Return the property list of CODING-SYSTEM.")
9857 (coding_system)
9858 Lisp_Object coding_system;
9859{
9860 Lisp_Object spec, attrs;
1397dc18 9861
df7492f9
KH		 9862 if (NILP (coding_system))
9863 coding_system = Qno_conversion;
9864 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9865 attrs = AREF (spec, 0);
9866 return CODING_ATTR_PLIST (attrs);
d46c5b12
KH		 9867}
9868
df7492f9
KH		 9869
9870DEFUN ("coding-system-aliases", Fcoding_system_aliases, Scoding_system_aliases,
9871 1, 1, 0,
da7db224 9872 doc: /* Return the list of aliases of CODING-SYSTEM. */)
df7492f9
KH		 9873 (coding_system)
9874 Lisp_Object coding_system;
66cfb530 9875{
df7492f9 9876 Lisp_Object spec;
84d60297 9877
df7492f9
KH		 9878 if (NILP (coding_system))
9879 coding_system = Qno_conversion;
9880 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
da7db224 9881 return AREF (spec, 1);
df7492f9 9882}
66cfb530 9883
df7492f9
KH		 9884DEFUN ("coding-system-eol-type", Fcoding_system_eol_type,
9885 Scoding_system_eol_type, 1, 1, 0,
9886 doc: /* Return eol-type of CODING-SYSTEM.
d4a1d553 9887An eol-type is an integer 0, 1, 2, or a vector of coding systems.
66cfb530 9888
df7492f9
KH		 9889Integer values 0, 1, and 2 indicate a format of end-of-line; LF, CRLF,
9890and CR respectively.
66cfb530 9891
df7492f9
KH		 9892A vector value indicates that a format of end-of-line should be
9893detected automatically. Nth element of the vector is the subsidiary
9894coding system whose eol-type is N. */)
6b89e3aa
KH		 9895 (coding_system)
9896 Lisp_Object coding_system;
9897{
df7492f9
KH		 9898 Lisp_Object spec, eol_type;
9899 int n;
6b89e3aa 9900
df7492f9
KH		 9901 if (NILP (coding_system))
9902 coding_system = Qno_conversion;
9903 if (! CODING_SYSTEM_P (coding_system))
9904 return Qnil;
9905 spec = CODING_SYSTEM_SPEC (coding_system);
9906 eol_type = AREF (spec, 2);
9907 if (VECTORP (eol_type))
9908 return Fcopy_sequence (eol_type);
9909 n = EQ (eol_type, Qunix) ? 0 : EQ (eol_type, Qdos) ? 1 : 2;
9910 return make_number (n);
6b89e3aa
KH		 9911}
9912
4ed46869
KH		 9913#endif /* emacs */
9914
9915\f
1397dc18 9916/*** 9. Post-amble ***/
4ed46869 9917
dfcf069d 9918void
4ed46869
KH		 9919init_coding_once ()
9920{
9921 int i;
9922
df7492f9
KH		 9923 for (i = 0; i < coding_category_max; i++)
9924 {
9925 coding_categories[i].id = -1;
9926 coding_priorities[i] = i;
9927 }
4ed46869
KH		 9928
9929 /* ISO2022 specific initialize routine. */
9930 for (i = 0; i < 0x20; i++)
b73bfc1c 9931 iso_code_class[i] = ISO_control_0;
4ed46869
KH		 9932 for (i = 0x21; i < 0x7F; i++)
9933 iso_code_class[i] = ISO_graphic_plane_0;
9934 for (i = 0x80; i < 0xA0; i++)
b73bfc1c 9935 iso_code_class[i] = ISO_control_1;
4ed46869
KH		 9936 for (i = 0xA1; i < 0xFF; i++)
9937 iso_code_class[i] = ISO_graphic_plane_1;
9938 iso_code_class[0x20] = iso_code_class[0x7F] = ISO_0x20_or_0x7F;
9939 iso_code_class[0xA0] = iso_code_class[0xFF] = ISO_0xA0_or_0xFF;
4ed46869
KH		 9940 iso_code_class[ISO_CODE_SO] = ISO_shift_out;
9941 iso_code_class[ISO_CODE_SI] = ISO_shift_in;
9942 iso_code_class[ISO_CODE_SS2_7] = ISO_single_shift_2_7;
9943 iso_code_class[ISO_CODE_ESC] = ISO_escape;
9944 iso_code_class[ISO_CODE_SS2] = ISO_single_shift_2;
9945 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3;
9946 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer;
9947
df7492f9
KH		 9948 for (i = 0; i < 256; i++)
9949 {
9950 emacs_mule_bytes[i] = 1;
9951 }
7c78e542
KH		 9952 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_11] = 3;
9953 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_12] = 3;
9954 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_21] = 4;
9955 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_22] = 4;
e0e989f6
KH		 9956}
9957
9958#ifdef emacs
9959
dfcf069d 9960void
e0e989f6
KH		 9961syms_of_coding ()
9962{
df7492f9 9963 staticpro (&Vcoding_system_hash_table);
8f924df7
KH		 9964 {
9965 Lisp_Object args[2];
9966 args[0] = QCtest;
9967 args[1] = Qeq;
9968 Vcoding_system_hash_table = Fmake_hash_table (2, args);
9969 }
df7492f9
KH		 9970
9971 staticpro (&Vsjis_coding_system);
9972 Vsjis_coding_system = Qnil;
e0e989f6 9973
df7492f9
KH		 9974 staticpro (&Vbig5_coding_system);
9975 Vbig5_coding_system = Qnil;
9976
24a73b0a
KH		 9977 staticpro (&Vcode_conversion_reused_workbuf);
9978 Vcode_conversion_reused_workbuf = Qnil;
9979
9980 staticpro (&Vcode_conversion_workbuf_name);
9981 Vcode_conversion_workbuf_name = build_string (" *code-conversion-work*");
e0e989f6 9982
24a73b0a 9983 reused_workbuf_in_use = 0;
df7492f9
KH		 9984
9985 DEFSYM (Qcharset, "charset");
9986 DEFSYM (Qtarget_idx, "target-idx");
9987 DEFSYM (Qcoding_system_history, "coding-system-history");
bb0115a2
RS		 9988 Fset (Qcoding_system_history, Qnil);
9989
9ce27fde 9990 /* Target FILENAME is the first argument. */
e0e989f6 9991 Fput (Qinsert_file_contents, Qtarget_idx, make_number (0));
9ce27fde 9992 /* Target FILENAME is the third argument. */
e0e989f6
KH		 9993 Fput (Qwrite_region, Qtarget_idx, make_number (2));
9994
df7492f9 9995 DEFSYM (Qcall_process, "call-process");
9ce27fde 9996 /* Target PROGRAM is the first argument. */
e0e989f6
KH		 9997 Fput (Qcall_process, Qtarget_idx, make_number (0));
9998
df7492f9 9999 DEFSYM (Qcall_process_region, "call-process-region");
9ce27fde 10000 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 10001 Fput (Qcall_process_region, Qtarget_idx, make_number (2));
10002
df7492f9 10003 DEFSYM (Qstart_process, "start-process");
9ce27fde 10004 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 10005 Fput (Qstart_process, Qtarget_idx, make_number (2));
10006
df7492f9 10007 DEFSYM (Qopen_network_stream, "open-network-stream");
9ce27fde 10008 /* Target SERVICE is the fourth argument. */
e0e989f6
KH		 10009 Fput (Qopen_network_stream, Qtarget_idx, make_number (3));
10010
df7492f9
KH		 10011 DEFSYM (Qcoding_system, "coding-system");
10012 DEFSYM (Qcoding_aliases, "coding-aliases");
4ed46869 10013
df7492f9
KH		 10014 DEFSYM (Qeol_type, "eol-type");
10015 DEFSYM (Qunix, "unix");
10016 DEFSYM (Qdos, "dos");
4ed46869 10017
df7492f9
KH		 10018 DEFSYM (Qbuffer_file_coding_system, "buffer-file-coding-system");
10019 DEFSYM (Qpost_read_conversion, "post-read-conversion");
10020 DEFSYM (Qpre_write_conversion, "pre-write-conversion");
10021 DEFSYM (Qdefault_char, "default-char");
10022 DEFSYM (Qundecided, "undecided");
10023 DEFSYM (Qno_conversion, "no-conversion");
10024 DEFSYM (Qraw_text, "raw-text");
4ed46869 10025
df7492f9 10026 DEFSYM (Qiso_2022, "iso-2022");
4ed46869 10027
df7492f9 10028 DEFSYM (Qutf_8, "utf-8");
8f924df7 10029 DEFSYM (Qutf_8_emacs, "utf-8-emacs");
27901516 10030
df7492f9 10031 DEFSYM (Qutf_16, "utf-16");
df7492f9
KH		 10032 DEFSYM (Qbig, "big");
10033 DEFSYM (Qlittle, "little");
27901516 10034
df7492f9
KH		 10035 DEFSYM (Qshift_jis, "shift-jis");
10036 DEFSYM (Qbig5, "big5");
4ed46869 10037
df7492f9 10038 DEFSYM (Qcoding_system_p, "coding-system-p");
4ed46869 10039
df7492f9 10040 DEFSYM (Qcoding_system_error, "coding-system-error");
4ed46869
KH		 10041 Fput (Qcoding_system_error, Qerror_conditions,
10042 Fcons (Qcoding_system_error, Fcons (Qerror, Qnil)));
10043 Fput (Qcoding_system_error, Qerror_message,
9ce27fde 10044 build_string ("Invalid coding system"));
4ed46869 10045
05e6f5dc
KH		 10046 /* Intern this now in case it isn't already done.
10047 Setting this variable twice is harmless.
10048 But don't staticpro it here--that is done in alloc.c. */
10049 Qchar_table_extra_slots = intern ("char-table-extra-slots");
70c22245 10050
df7492f9 10051 DEFSYM (Qtranslation_table, "translation-table");
433f7f87 10052 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (2));
df7492f9
KH		 10053 DEFSYM (Qtranslation_table_id, "translation-table-id");
10054 DEFSYM (Qtranslation_table_for_decode, "translation-table-for-decode");
10055 DEFSYM (Qtranslation_table_for_encode, "translation-table-for-encode");
1397dc18 10056
df7492f9 10057 DEFSYM (Qvalid_codes, "valid-codes");
9ce27fde 10058
df7492f9 10059 DEFSYM (Qemacs_mule, "emacs-mule");
d46c5b12 10060
01378f49 10061 DEFSYM (QCcategory, ":category");
a6f87d34 10062 DEFSYM (QCmnemonic, ":mnemonic");
2133e2d1 10063 DEFSYM (QCdefault_char, ":default-char");
a6f87d34
KH		 10064 DEFSYM (QCdecode_translation_table, ":decode-translation-table");
10065 DEFSYM (QCencode_translation_table, ":encode-translation-table");
10066 DEFSYM (QCpost_read_conversion, ":post-read-conversion");
10067 DEFSYM (QCpre_write_conversion, ":pre-write-conversion");
35befdaa 10068 DEFSYM (QCascii_compatible_p, ":ascii-compatible-p");
01378f49 10069
df7492f9
KH		 10070 Vcoding_category_table
10071 = Fmake_vector (make_number (coding_category_max), Qnil);
10072 staticpro (&Vcoding_category_table);
10073 /* Followings are target of code detection. */
10074 ASET (Vcoding_category_table, coding_category_iso_7,
10075 intern ("coding-category-iso-7"));
10076 ASET (Vcoding_category_table, coding_category_iso_7_tight,
10077 intern ("coding-category-iso-7-tight"));
10078 ASET (Vcoding_category_table, coding_category_iso_8_1,
10079 intern ("coding-category-iso-8-1"));
10080 ASET (Vcoding_category_table, coding_category_iso_8_2,
10081 intern ("coding-category-iso-8-2"));
10082 ASET (Vcoding_category_table, coding_category_iso_7_else,
10083 intern ("coding-category-iso-7-else"));
10084 ASET (Vcoding_category_table, coding_category_iso_8_else,
10085 intern ("coding-category-iso-8-else"));
a470d443
KH		 10086 ASET (Vcoding_category_table, coding_category_utf_8_auto,
10087 intern ("coding-category-utf-8-auto"));
10088 ASET (Vcoding_category_table, coding_category_utf_8_nosig,
df7492f9 10089 intern ("coding-category-utf-8"));
a470d443
KH		 10090 ASET (Vcoding_category_table, coding_category_utf_8_sig,
10091 intern ("coding-category-utf-8-sig"));
df7492f9
KH		 10092 ASET (Vcoding_category_table, coding_category_utf_16_be,
10093 intern ("coding-category-utf-16-be"));
ff563fce
KH		 10094 ASET (Vcoding_category_table, coding_category_utf_16_auto,
10095 intern ("coding-category-utf-16-auto"));
df7492f9
KH		 10096 ASET (Vcoding_category_table, coding_category_utf_16_le,
10097 intern ("coding-category-utf-16-le"));
10098 ASET (Vcoding_category_table, coding_category_utf_16_be_nosig,
10099 intern ("coding-category-utf-16-be-nosig"));
10100 ASET (Vcoding_category_table, coding_category_utf_16_le_nosig,
10101 intern ("coding-category-utf-16-le-nosig"));
10102 ASET (Vcoding_category_table, coding_category_charset,
10103 intern ("coding-category-charset"));
10104 ASET (Vcoding_category_table, coding_category_sjis,
10105 intern ("coding-category-sjis"));
10106 ASET (Vcoding_category_table, coding_category_big5,
10107 intern ("coding-category-big5"));
10108 ASET (Vcoding_category_table, coding_category_ccl,
10109 intern ("coding-category-ccl"));
10110 ASET (Vcoding_category_table, coding_category_emacs_mule,
10111 intern ("coding-category-emacs-mule"));
10112 /* Followings are NOT target of code detection. */
10113 ASET (Vcoding_category_table, coding_category_raw_text,
10114 intern ("coding-category-raw-text"));
10115 ASET (Vcoding_category_table, coding_category_undecided,
10116 intern ("coding-category-undecided"));
ecf488bc 10117
065e3595
KH		 10118 DEFSYM (Qinsufficient_source, "insufficient-source");
10119 DEFSYM (Qinconsistent_eol, "inconsistent-eol");
10120 DEFSYM (Qinvalid_source, "invalid-source");
10121 DEFSYM (Qinterrupted, "interrupted");
10122 DEFSYM (Qinsufficient_memory, "insufficient-memory");
44e8490d 10123 DEFSYM (Qcoding_system_define_form, "coding-system-define-form");
065e3595 10124
4ed46869
KH		 10125 defsubr (&Scoding_system_p);
10126 defsubr (&Sread_coding_system);
10127 defsubr (&Sread_non_nil_coding_system);
10128 defsubr (&Scheck_coding_system);
10129 defsubr (&Sdetect_coding_region);
d46c5b12 10130 defsubr (&Sdetect_coding_string);
05e6f5dc 10131 defsubr (&Sfind_coding_systems_region_internal);
068a9dbd 10132 defsubr (&Sunencodable_char_position);
df7492f9 10133 defsubr (&Scheck_coding_systems_region);
4ed46869
KH		 10134 defsubr (&Sdecode_coding_region);
10135 defsubr (&Sencode_coding_region);
10136 defsubr (&Sdecode_coding_string);
10137 defsubr (&Sencode_coding_string);
10138 defsubr (&Sdecode_sjis_char);
10139 defsubr (&Sencode_sjis_char);
10140 defsubr (&Sdecode_big5_char);
10141 defsubr (&Sencode_big5_char);
1ba9e4ab 10142 defsubr (&Sset_terminal_coding_system_internal);
c4825358 10143 defsubr (&Sset_safe_terminal_coding_system_internal);
4ed46869 10144 defsubr (&Sterminal_coding_system);
1ba9e4ab 10145 defsubr (&Sset_keyboard_coding_system_internal);
4ed46869 10146 defsubr (&Skeyboard_coding_system);
a5d301df 10147 defsubr (&Sfind_operation_coding_system);
df7492f9 10148 defsubr (&Sset_coding_system_priority);
6b89e3aa 10149 defsubr (&Sdefine_coding_system_internal);
df7492f9 10150 defsubr (&Sdefine_coding_system_alias);
a6f87d34 10151 defsubr (&Scoding_system_put);
df7492f9
KH		 10152 defsubr (&Scoding_system_base);
10153 defsubr (&Scoding_system_plist);
10154 defsubr (&Scoding_system_aliases);
10155 defsubr (&Scoding_system_eol_type);
10156 defsubr (&Scoding_system_priority_list);
4ed46869 10157
4608c386 10158 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list,
48b0f3ae
PJ		 10159 doc: /* List of coding systems.
10160
10161Do not alter the value of this variable manually. This variable should be
df7492f9 10162updated by the functions `define-coding-system' and
48b0f3ae 10163`define-coding-system-alias'. */);
4608c386
KH		 10164 Vcoding_system_list = Qnil;
10165
10166 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist,
48b0f3ae
PJ		 10167 doc: /* Alist of coding system names.
10168Each element is one element list of coding system name.
446dcd75 10169This variable is given to `completing-read' as COLLECTION argument.
48b0f3ae
PJ		 10170
10171Do not alter the value of this variable manually. This variable should be
10172updated by the functions `make-coding-system' and
10173`define-coding-system-alias'. */);
4608c386
KH		 10174 Vcoding_system_alist = Qnil;
10175
4ed46869 10176 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list,
48b0f3ae
PJ		 10177 doc: /* List of coding-categories (symbols) ordered by priority.
10178
10179On detecting a coding system, Emacs tries code detection algorithms
10180associated with each coding-category one by one in this order. When
10181one algorithm agrees with a byte sequence of source text, the coding
0ec31faf
KH		 10182system bound to the corresponding coding-category is selected.
10183
42205607 10184Don't modify this variable directly, but use `set-coding-priority'. */);
4ed46869
KH		 10185 {
10186 int i;
10187
10188 Vcoding_category_list = Qnil;
df7492f9 10189 for (i = coding_category_max - 1; i >= 0; i--)
4ed46869 10190 Vcoding_category_list
d46c5b12
KH		 10191 = Fcons (XVECTOR (Vcoding_category_table)->contents[i],
10192 Vcoding_category_list);
4ed46869
KH		 10193 }
10194
10195 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read,
48b0f3ae
PJ		 10196 doc: /* Specify the coding system for read operations.
10197It is useful to bind this variable with `let', but do not set it globally.
10198If the value is a coding system, it is used for decoding on read operation.
446dcd75
JB		 10199If not, an appropriate element is used from one of the coding system alists.
10200There are three such tables: `file-coding-system-alist',
48b0f3ae 10201`process-coding-system-alist', and `network-coding-system-alist'. */);
4ed46869
KH		 10202 Vcoding_system_for_read = Qnil;
10203
10204 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write,
48b0f3ae
PJ		 10205 doc: /* Specify the coding system for write operations.
10206Programs bind this variable with `let', but you should not set it globally.
10207If the value is a coding system, it is used for encoding of output,
10208when writing it to a file and when sending it to a file or subprocess.
10209
10210If this does not specify a coding system, an appropriate element
446dcd75
JB		 10211is used from one of the coding system alists.
10212There are three such tables: `file-coding-system-alist',
48b0f3ae
PJ		 10213`process-coding-system-alist', and `network-coding-system-alist'.
10214For output to files, if the above procedure does not specify a coding system,
10215the value of `buffer-file-coding-system' is used. */);
4ed46869
KH		 10216 Vcoding_system_for_write = Qnil;
10217
10218 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used,
df7492f9
KH		 10219 doc: /*
10220Coding system used in the latest file or process I/O. */);
4ed46869
KH		 10221 Vlast_coding_system_used = Qnil;
10222
065e3595
KH		 10223 DEFVAR_LISP ("last-code-conversion-error", &Vlast_code_conversion_error,
10224 doc: /*
10225Error status of the last code conversion.
10226
10227When an error was detected in the last code conversion, this variable
10228is set to one of the following symbols.
10229 `insufficient-source'
10230 `inconsistent-eol'
10231 `invalid-source'
10232 `interrupted'
10233 `insufficient-memory'
10234When no error was detected, the value doesn't change. So, to check
10235the error status of a code conversion by this variable, you must
10236explicitly set this variable to nil before performing code
10237conversion. */);
10238 Vlast_code_conversion_error = Qnil;
10239
9ce27fde 10240 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion,
df7492f9
KH		 10241 doc: /*
10242*Non-nil means always inhibit code conversion of end-of-line format.
48b0f3ae
PJ		 10243See info node `Coding Systems' and info node `Text and Binary' concerning
10244such conversion. */);
9ce27fde
KH		 10245 inhibit_eol_conversion = 0;
10246
ed29121d 10247 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system,
df7492f9
KH		 10248 doc: /*
10249Non-nil means process buffer inherits coding system of process output.
48b0f3ae
PJ		 10250Bind it to t if the process output is to be treated as if it were a file
10251read from some filesystem. */);
ed29121d
EZ		 10252 inherit_process_coding_system = 0;
10253
02ba4723 10254 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist,
df7492f9
KH		 10255 doc: /*
10256Alist to decide a coding system to use for a file I/O operation.
48b0f3ae
PJ		 10257The format is ((PATTERN . VAL) ...),
10258where PATTERN is a regular expression matching a file name,
10259VAL is a coding system, a cons of coding systems, or a function symbol.
10260If VAL is a coding system, it is used for both decoding and encoding
10261the file contents.
10262If VAL is a cons of coding systems, the car part is used for decoding,
10263and the cdr part is used for encoding.
10264If VAL is a function symbol, the function must return a coding system
2c53e699
KH		 10265or a cons of coding systems which are used as above. The function is
10266called with an argument that is a list of the arguments with which
5a0bbd9a
KH		 10267`find-operation-coding-system' was called. If the function can't decide
10268a coding system, it can return `undecided' so that the normal
10269code-detection is performed.
48b0f3ae
PJ		 10270
10271See also the function `find-operation-coding-system'
10272and the variable `auto-coding-alist'. */);
02ba4723
KH		 10273 Vfile_coding_system_alist = Qnil;
10274
10275 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist,
df7492f9
KH		 10276 doc: /*
10277Alist to decide a coding system to use for a process I/O operation.
48b0f3ae
PJ		 10278The format is ((PATTERN . VAL) ...),
10279where PATTERN is a regular expression matching a program name,
10280VAL is a coding system, a cons of coding systems, or a function symbol.
10281If VAL is a coding system, it is used for both decoding what received
10282from the program and encoding what sent to the program.
10283If VAL is a cons of coding systems, the car part is used for decoding,
10284and the cdr part is used for encoding.
10285If VAL is a function symbol, the function must return a coding system
10286or a cons of coding systems which are used as above.
10287
10288See also the function `find-operation-coding-system'. */);
02ba4723
KH		 10289 Vprocess_coding_system_alist = Qnil;
10290
10291 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist,
df7492f9
KH		 10292 doc: /*
10293Alist to decide a coding system to use for a network I/O operation.
48b0f3ae
PJ		 10294The format is ((PATTERN . VAL) ...),
10295where PATTERN is a regular expression matching a network service name
10296or is a port number to connect to,
10297VAL is a coding system, a cons of coding systems, or a function symbol.
10298If VAL is a coding system, it is used for both decoding what received
10299from the network stream and encoding what sent to the network stream.
10300If VAL is a cons of coding systems, the car part is used for decoding,
10301and the cdr part is used for encoding.
10302If VAL is a function symbol, the function must return a coding system
10303or a cons of coding systems which are used as above.
10304
10305See also the function `find-operation-coding-system'. */);
02ba4723 10306 Vnetwork_coding_system_alist = Qnil;
4ed46869 10307
68c45bf0 10308 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system,
75205970
RS		 10309 doc: /* Coding system to use with system messages.
10310Also used for decoding keyboard input on X Window system. */);
68c45bf0
PE		 10311 Vlocale_coding_system = Qnil;
10312
005f0d35 10313 /* The eol mnemonics are reset in startup.el system-dependently. */
7722baf9 10314 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix,
df7492f9
KH		 10315 doc: /*
10316*String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7722baf9 10317 eol_mnemonic_unix = build_string (":");
4ed46869 10318
7722baf9 10319 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos,
df7492f9
KH		 10320 doc: /*
10321*String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7722baf9 10322 eol_mnemonic_dos = build_string ("\\");
4ed46869 10323
7722baf9 10324 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac,
df7492f9
KH		 10325 doc: /*
10326*String displayed in mode line for MAC-like (CR) end-of-line format. */);
7722baf9 10327 eol_mnemonic_mac = build_string ("/");
4ed46869 10328
7722baf9 10329 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided,
df7492f9
KH		 10330 doc: /*
10331*String displayed in mode line when end-of-line format is not yet determined. */);
7722baf9 10332 eol_mnemonic_undecided = build_string (":");
4ed46869 10333
84fbb8a0 10334 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation,
df7492f9
KH		 10335 doc: /*
10336*Non-nil enables character translation while encoding and decoding. */);
84fbb8a0 10337 Venable_character_translation = Qt;
bdd9fb48 10338
f967223b 10339 DEFVAR_LISP ("standard-translation-table-for-decode",
48b0f3ae
PJ		 10340 &Vstandard_translation_table_for_decode,
10341 doc: /* Table for translating characters while decoding. */);
f967223b 10342 Vstandard_translation_table_for_decode = Qnil;
bdd9fb48 10343
f967223b 10344 DEFVAR_LISP ("standard-translation-table-for-encode",
48b0f3ae
PJ		 10345 &Vstandard_translation_table_for_encode,
10346 doc: /* Table for translating characters while encoding. */);
f967223b 10347 Vstandard_translation_table_for_encode = Qnil;
4ed46869 10348
df7492f9 10349 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_table,
48b0f3ae
PJ		 10350 doc: /* Alist of charsets vs revision numbers.
10351While encoding, if a charset (car part of an element) is found,
df7492f9
KH		 10352designate it with the escape sequence identifying revision (cdr part
10353of the element). */);
10354 Vcharset_revision_table = Qnil;
02ba4723
KH		 10355
10356 DEFVAR_LISP ("default-process-coding-system",
10357 &Vdefault_process_coding_system,
48b0f3ae
PJ		 10358 doc: /* Cons of coding systems used for process I/O by default.
10359The car part is used for decoding a process output,
10360the cdr part is used for encoding a text to be sent to a process. */);
02ba4723 10361 Vdefault_process_coding_system = Qnil;
c4825358 10362
3f003981 10363 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table,
df7492f9
KH		 10364 doc: /*
10365Table of extra Latin codes in the range 128..159 (inclusive).
48b0f3ae
PJ		 10366This is a vector of length 256.
10367If Nth element is non-nil, the existence of code N in a file
10368\(or output of subprocess) doesn't prevent it to be detected as
10369a coding system of ISO 2022 variant which has a flag
10370`accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
10371or reading output of a subprocess.
446dcd75 10372Only 128th through 159th elements have a meaning. */);
3f003981 10373 Vlatin_extra_code_table = Fmake_vector (make_number (256), Qnil);
d46c5b12
KH		 10374
10375 DEFVAR_LISP ("select-safe-coding-system-function",
10376 &Vselect_safe_coding_system_function,
df7492f9
KH		 10377 doc: /*
10378Function to call to select safe coding system for encoding a text.
48b0f3ae
PJ		 10379
10380If set, this function is called to force a user to select a proper
10381coding system which can encode the text in the case that a default
fdecf907
GM		 10382coding system used in each operation can't encode the text. The
10383function should take care that the buffer is not modified while
10384the coding system is being selected.
48b0f3ae
PJ		 10385
10386The default value is `select-safe-coding-system' (which see). */);
d46c5b12
KH		 10387 Vselect_safe_coding_system_function = Qnil;
10388
5d5bf4d8
KH		 10389 DEFVAR_BOOL ("coding-system-require-warning",
10390 &coding_system_require_warning,
10391 doc: /* Internal use only.
6b89e3aa
KH		 10392If non-nil, on writing a file, `select-safe-coding-system-function' is
10393called even if `coding-system-for-write' is non-nil. The command
10394`universal-coding-system-argument' binds this variable to t temporarily. */);
5d5bf4d8
KH		 10395 coding_system_require_warning = 0;
10396
10397
22ab2303 10398 DEFVAR_BOOL ("inhibit-iso-escape-detection",
74383408 10399 &inhibit_iso_escape_detection,
df7492f9 10400 doc: /*
97b1b294 10401If non-nil, Emacs ignores ISO-2022 escape sequences during code detection.
48b0f3ae 10402
97b1b294
EZ		 10403When Emacs reads text, it tries to detect how the text is encoded.
10404This code detection is sensitive to escape sequences. If Emacs sees
10405a valid ISO-2022 escape sequence, it assumes the text is encoded in one
10406of the ISO2022 encodings, and decodes text by the corresponding coding
10407system (e.g. `iso-2022-7bit').
48b0f3ae
PJ		 10408
10409However, there may be a case that you want to read escape sequences in
10410a file as is. In such a case, you can set this variable to non-nil.
97b1b294
EZ		 10411Then the code detection will ignore any escape sequences, and no text is
10412detected as encoded in some ISO-2022 encoding. The result is that all
48b0f3ae
PJ		 10413escape sequences become visible in a buffer.
10414
10415The default value is nil, and it is strongly recommended not to change
10416it. That is because many Emacs Lisp source files that contain
10417non-ASCII characters are encoded by the coding system `iso-2022-7bit'
10418in Emacs's distribution, and they won't be decoded correctly on
10419reading if you suppress escape sequence detection.
10420
10421The other way to read escape sequences in a file without decoding is
97b1b294 10422to explicitly specify some coding system that doesn't use ISO-2022
48b0f3ae 10423escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
74383408 10424 inhibit_iso_escape_detection = 0;
002fdb44 10425
97b1b294
EZ		 10426 DEFVAR_BOOL ("inhibit-null-byte-detection",
10427 &inhibit_null_byte_detection,
10428 doc: /* If non-nil, Emacs ignores null bytes on code detection.
10429By default, Emacs treats it as binary data, and does not attempt to
10430decode it. The effect is as if you specified `no-conversion' for
10431reading that text.
10432
10433Set this to non-nil when a regular text happens to include null bytes.
10434Examples are Index nodes of Info files and null-byte delimited output
10435from GNU Find and GNU Grep. Emacs will then ignore the null bytes and
10436decode text as usual. */);
10437 inhibit_null_byte_detection = 0;
10438
002fdb44 10439 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input,
15c8f9d1 10440 doc: /* Char table for translating self-inserting characters.
446dcd75 10441This is applied to the result of input methods, not their input.
8434d0b8
EZ		 10442See also `keyboard-translate-table'.
10443
10444Use of this variable for character code unification was rendered
10445obsolete in Emacs 23.1 and later, since Unicode is now the basis of
10446internal character representation. */);
002fdb44 10447 Vtranslation_table_for_input = Qnil;
8f924df7 10448
2c78b7e1
KH		 10449 {
10450 Lisp_Object args[coding_arg_max];
8f924df7 10451 Lisp_Object plist[16];
2c78b7e1
KH		 10452 int i;
10453
10454 for (i = 0; i < coding_arg_max; i++)
10455 args[i] = Qnil;
10456
10457 plist[0] = intern (":name");
10458 plist[1] = args[coding_arg_name] = Qno_conversion;
10459 plist[2] = intern (":mnemonic");
10460 plist[3] = args[coding_arg_mnemonic] = make_number ('=');
10461 plist[4] = intern (":coding-type");
10462 plist[5] = args[coding_arg_coding_type] = Qraw_text;
10463 plist[6] = intern (":ascii-compatible-p");
10464 plist[7] = args[coding_arg_ascii_compatible_p] = Qt;
10465 plist[8] = intern (":default-char");
10466 plist[9] = args[coding_arg_default_char] = make_number (0);
8f924df7
KH		 10467 plist[10] = intern (":for-unibyte");
10468 plist[11] = args[coding_arg_for_unibyte] = Qt;
10469 plist[12] = intern (":docstring");
10470 plist[13] = build_string ("Do no conversion.\n\
2c78b7e1
KH		 10471\n\
10472When you visit a file with this coding, the file is read into a\n\
10473unibyte buffer as is, thus each byte of a file is treated as a\n\
10474character.");
8f924df7
KH		 10475 plist[14] = intern (":eol-type");
10476 plist[15] = args[coding_arg_eol_type] = Qunix;
10477 args[coding_arg_plist] = Flist (16, plist);
2c78b7e1 10478 Fdefine_coding_system_internal (coding_arg_max, args);
ae6f73fa
KH		 10479
10480 plist[1] = args[coding_arg_name] = Qundecided;
10481 plist[3] = args[coding_arg_mnemonic] = make_number ('-');
10482 plist[5] = args[coding_arg_coding_type] = Qundecided;
10483 /* This is already set.
35befdaa 10484 plist[7] = args[coding_arg_ascii_compatible_p] = Qt; */
ae6f73fa
KH		 10485 plist[8] = intern (":charset-list");
10486 plist[9] = args[coding_arg_charset_list] = Fcons (Qascii, Qnil);
10487 plist[11] = args[coding_arg_for_unibyte] = Qnil;
10488 plist[13] = build_string ("No conversion on encoding, automatic conversion on decoding.");
10489 plist[15] = args[coding_arg_eol_type] = Qnil;
10490 args[coding_arg_plist] = Flist (16, plist);
10491 Fdefine_coding_system_internal (coding_arg_max, args);
2c78b7e1
KH		 10492 }
10493
2c78b7e1 10494 setup_coding_system (Qno_conversion, &safe_terminal_coding);
ff563fce
KH		 10495
10496 {
10497 int i;
10498
10499 for (i = 0; i < coding_category_max; i++)
10500 Fset (AREF (Vcoding_category_table, i), Qno_conversion);
10501 }
fcbcfb64
KH		 10502#if defined (MSDOS) || defined (WINDOWSNT)
10503 system_eol_type = Qdos;
10504#else
10505 system_eol_type = Qunix;
10506#endif
10507 staticpro (&system_eol_type);
4ed46869
KH		 10508}
10509
68c45bf0
PE		 10510char *
10511emacs_strerror (error_number)
10512 int error_number;
10513{
10514 char *str;
10515
ca9c0567 10516 synchronize_system_messages_locale ();
68c45bf0
PE		 10517 str = strerror (error_number);
10518
10519 if (! NILP (Vlocale_coding_system))
10520 {
10521 Lisp_Object dec = code_convert_string_norecord (build_string (str),
10522 Vlocale_coding_system,
10523 0);
d5db4077 10524 str = (char *) SDATA (dec);
68c45bf0
PE		 10525 }
10526
10527 return str;
10528}
10529
4ed46869 10530#endif /* emacs */
9ffd559c
KH		 10531
10532/* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
10533 (do not change this comment) */
Unnamed repository; edit this file 'description' to name the repository.