HCoop / bpt / emacs.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
(detect_coding): Preserve coding->mode.
[bpt/emacs.git] / src / coding.c
CommitLineData
9542cb1f 1/* Coding system handler (conversion, detection, etc).
aaef169d 2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
76b6f707 3 2006, 2007, 2008, 2009 Free Software Foundation, Inc.
7976eda0 4 Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
76b6f707 5 2005, 2006, 2007, 2008, 2009
ce03bf76
KH		 6 National Institute of Advanced Industrial Science and Technology (AIST)
7 Registration Number H14PRO021
8f924df7 8 Copyright (C) 2003
df7492f9
KH		 9 National Institute of Advanced Industrial Science and Technology (AIST)
10 Registration Number H13PRO009
4ed46869 11
369314dc
KH		 12This file is part of GNU Emacs.
13
9ec0b715 14GNU Emacs is free software: you can redistribute it and/or modify
369314dc 15it under the terms of the GNU General Public License as published by
9ec0b715
GM		 16the Free Software Foundation, either version 3 of the License, or
17(at your option) any later version.
4ed46869 18
369314dc
KH		 19GNU Emacs is distributed in the hope that it will be useful,
20but WITHOUT ANY WARRANTY; without even the implied warranty of
21MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
22GNU General Public License for more details.
4ed46869 23
369314dc 24You should have received a copy of the GNU General Public License
9ec0b715 25along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. */
4ed46869
KH		 26
27/*** TABLE OF CONTENTS ***
28
b73bfc1c 29 0. General comments
4ed46869 30 1. Preamble
df7492f9
KH		 31 2. Emacs' internal format (emacs-utf-8) handlers
32 3. UTF-8 handlers
33 4. UTF-16 handlers
34 5. Charset-base coding systems handlers
35 6. emacs-mule (old Emacs' internal format) handlers
36 7. ISO2022 handlers
37 8. Shift-JIS and BIG5 handlers
38 9. CCL handlers
39 10. C library functions
40 11. Emacs Lisp library functions
41 12. Postamble
4ed46869
KH		 42
43*/
44
df7492f9 45/*** 0. General comments ***
b73bfc1c
KH		 46
47
df7492f9 48CODING SYSTEM
4ed46869 49
5bad0796
DL		 50 A coding system is an object for an encoding mechanism that contains
51 information about how to convert byte sequences to character
e19c3639
KH		 52 sequences and vice versa. When we say "decode", it means converting
53 a byte sequence of a specific coding system into a character
54 sequence that is represented by Emacs' internal coding system
55 `emacs-utf-8', and when we say "encode", it means converting a
56 character sequence of emacs-utf-8 to a byte sequence of a specific
0ef69138 57 coding system.
4ed46869 58
e19c3639
KH		 59 In Emacs Lisp, a coding system is represented by a Lisp symbol. In
60 C level, a coding system is represented by a vector of attributes
5bad0796 61 stored in the hash table Vcharset_hash_table. The conversion from
e19c3639
KH		 62 coding system symbol to attributes vector is done by looking up
63 Vcharset_hash_table by the symbol.
4ed46869 64
e19c3639 65 Coding systems are classified into the following types depending on
5bad0796 66 the encoding mechanism. Here's a brief description of the types.
4ed46869 67
df7492f9
KH		 68 o UTF-8
69
70 o UTF-16
71
72 o Charset-base coding system
73
74 A coding system defined by one or more (coded) character sets.
5bad0796 75 Decoding and encoding are done by a code converter defined for each
df7492f9
KH		 76 character set.
77
5bad0796 78 o Old Emacs internal format (emacs-mule)
df7492f9 79
5bad0796 80 The coding system adopted by old versions of Emacs (20 and 21).
4ed46869 81
df7492f9 82 o ISO2022-base coding system
4ed46869
KH		 83
84 The most famous coding system for multiple character sets. X's
df7492f9
KH		 85 Compound Text, various EUCs (Extended Unix Code), and coding systems
86 used in the Internet communication such as ISO-2022-JP are all
87 variants of ISO2022.
4ed46869 88
df7492f9 89 o SJIS (or Shift-JIS or MS-Kanji-Code)
93dec019 90
4ed46869
KH		 91 A coding system to encode character sets: ASCII, JISX0201, and
92 JISX0208. Widely used for PC's in Japan. Details are described in
df7492f9 93 section 8.
4ed46869 94
df7492f9 95 o BIG5
4ed46869 96
df7492f9 97 A coding system to encode character sets: ASCII and Big5. Widely
cfb43547 98 used for Chinese (mainly in Taiwan and Hong Kong). Details are
df7492f9
KH		 99 described in section 8. In this file, when we write "big5" (all
100 lowercase), we mean the coding system, and when we write "Big5"
101 (capitalized), we mean the character set.
4ed46869 102
df7492f9 103 o CCL
27901516 104
5bad0796 105 If a user wants to decode/encode text encoded in a coding system
df7492f9
KH		 106 not listed above, he can supply a decoder and an encoder for it in
107 CCL (Code Conversion Language) programs. Emacs executes the CCL
108 program while decoding/encoding.
27901516 109
df7492f9 110 o Raw-text
4ed46869 111
5a936b46 112 A coding system for text containing raw eight-bit data. Emacs
5bad0796 113 treats each byte of source text as a character (except for
df7492f9 114 end-of-line conversion).
4ed46869 115
df7492f9
KH		 116 o No-conversion
117
118 Like raw text, but don't do end-of-line conversion.
4ed46869 119
4ed46869 120
df7492f9 121END-OF-LINE FORMAT
4ed46869 122
5bad0796 123 How text end-of-line is encoded depends on operating system. For
df7492f9 124 instance, Unix's format is just one byte of LF (line-feed) code,
f4dee582 125 whereas DOS's format is two-byte sequence of `carriage-return' and
d46c5b12
KH		 126 `line-feed' codes. MacOS's format is usually one byte of
127 `carriage-return'.
4ed46869 128
cfb43547 129 Since text character encoding and end-of-line encoding are
df7492f9
KH		 130 independent, any coding system described above can take any format
131 of end-of-line (except for no-conversion).
4ed46869 132
e19c3639
KH		 133STRUCT CODING_SYSTEM
134
135 Before using a coding system for code conversion (i.e. decoding and
136 encoding), we setup a structure of type `struct coding_system'.
137 This structure keeps various information about a specific code
5bad0796 138 conversion (e.g. the location of source and destination data).
4ed46869
KH		 139
140*/
141
df7492f9
KH		 142/* COMMON MACROS */
143
144
4ed46869
KH		 145/*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
146
df7492f9 147 These functions check if a byte sequence specified as a source in
ff0dacd7
KH		 148 CODING conforms to the format of XXX, and update the members of
149 DETECT_INFO.
df7492f9 150
ff0dacd7 151 Return 1 if the byte sequence conforms to XXX, otherwise return 0.
df7492f9
KH		 152
153 Below is the template of these functions. */
154
4ed46869 155#if 0
df7492f9 156static int
ff0dacd7 157detect_coding_XXX (coding, detect_info)
df7492f9 158 struct coding_system *coding;
ff0dacd7 159 struct coding_detection_info *detect_info;
4ed46869 160{
f1d34bca
MB		 161 const unsigned char *src = coding->source;
162 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 163 int multibytep = coding->src_multibyte;
ff0dacd7 164 int consumed_chars = 0;
df7492f9
KH		 165 int found = 0;
166 ...;
167
168 while (1)
169 {
170 /* Get one byte from the source. If the souce is exausted, jump
171 to no_more_source:. */
172 ONE_MORE_BYTE (c);
ff0dacd7
KH		 173
174 if (! __C_conforms_to_XXX___ (c))
175 break;
176 if (! __C_strongly_suggests_XXX__ (c))
177 found = CATEGORY_MASK_XXX;
df7492f9 178 }
ff0dacd7
KH		 179 /* The byte sequence is invalid for XXX. */
180 detect_info->rejected |= CATEGORY_MASK_XXX;
df7492f9 181 return 0;
ff0dacd7 182
df7492f9 183 no_more_source:
ff0dacd7
KH		 184 /* The source exausted successfully. */
185 detect_info->found |= found;
df7492f9 186 return 1;
4ed46869
KH		 187}
188#endif
189
190/*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
191
df7492f9
KH		 192 These functions decode a byte sequence specified as a source by
193 CODING. The resulting multibyte text goes to a place pointed to by
194 CODING->charbuf, the length of which should not exceed
195 CODING->charbuf_size;
d46c5b12 196
df7492f9
KH		 197 These functions set the information of original and decoded texts in
198 CODING->consumed, CODING->consumed_char, and CODING->charbuf_used.
199 They also set CODING->result to one of CODING_RESULT_XXX indicating
200 how the decoding is finished.
d46c5b12 201
df7492f9 202 Below is the template of these functions. */
d46c5b12 203
4ed46869 204#if 0
b73bfc1c 205static void
df7492f9 206decode_coding_XXXX (coding)
4ed46869 207 struct coding_system *coding;
4ed46869 208{
f1d34bca
MB		 209 const unsigned char *src = coding->source + coding->consumed;
210 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 211 /* SRC_BASE remembers the start position in source in each loop.
212 The loop will be exited when there's not enough source code, or
213 when there's no room in CHARBUF for a decoded character. */
f1d34bca 214 const unsigned char *src_base;
df7492f9 215 /* A buffer to produce decoded characters. */
69a80ea3
KH		 216 int *charbuf = coding->charbuf + coding->charbuf_used;
217 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 218 int multibytep = coding->src_multibyte;
219
220 while (1)
221 {
222 src_base = src;
223 if (charbuf < charbuf_end)
224 /* No more room to produce a decoded character. */
225 break;
226 ONE_MORE_BYTE (c);
227 /* Decode it. */
228 }
229
230 no_more_source:
231 if (src_base < src_end
232 && coding->mode & CODING_MODE_LAST_BLOCK)
233 /* If the source ends by partial bytes to construct a character,
234 treat them as eight-bit raw data. */
235 while (src_base < src_end && charbuf < charbuf_end)
236 *charbuf++ = *src_base++;
237 /* Remember how many bytes and characters we consumed. If the
238 source is multibyte, the bytes and chars are not identical. */
239 coding->consumed = coding->consumed_char = src_base - coding->source;
240 /* Remember how many characters we produced. */
241 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 242}
243#endif
244
245/*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
246
df7492f9
KH		 247 These functions encode SRC_BYTES length text at SOURCE of Emacs'
248 internal multibyte format by CODING. The resulting byte sequence
b73bfc1c
KH		 249 goes to a place pointed to by DESTINATION, the length of which
250 should not exceed DST_BYTES.
d46c5b12 251
df7492f9
KH		 252 These functions set the information of original and encoded texts in
253 the members produced, produced_char, consumed, and consumed_char of
254 the structure *CODING. They also set the member result to one of
255 CODING_RESULT_XXX indicating how the encoding finished.
d46c5b12 256
df7492f9
KH		 257 DST_BYTES zero means that source area and destination area are
258 overlapped, which means that we can produce a encoded text until it
259 reaches at the head of not-yet-encoded source text.
d46c5b12 260
df7492f9 261 Below is a template of these functions. */
4ed46869 262#if 0
b73bfc1c 263static void
df7492f9 264encode_coding_XXX (coding)
4ed46869 265 struct coding_system *coding;
4ed46869 266{
df7492f9
KH		 267 int multibytep = coding->dst_multibyte;
268 int *charbuf = coding->charbuf;
269 int *charbuf_end = charbuf->charbuf + coding->charbuf_used;
270 unsigned char *dst = coding->destination + coding->produced;
271 unsigned char *dst_end = coding->destination + coding->dst_bytes;
272 unsigned char *adjusted_dst_end = dst_end - _MAX_BYTES_PRODUCED_IN_LOOP_;
273 int produced_chars = 0;
274
275 for (; charbuf < charbuf_end && dst < adjusted_dst_end; charbuf++)
276 {
277 int c = *charbuf;
278 /* Encode C into DST, and increment DST. */
279 }
280 label_no_more_destination:
281 /* How many chars and bytes we produced. */
282 coding->produced_char += produced_chars;
283 coding->produced = dst - coding->destination;
4ed46869
KH		 284}
285#endif
286
4ed46869
KH		 287\f
288/*** 1. Preamble ***/
289
68c45bf0 290#include <config.h>
4ed46869
KH		 291#include <stdio.h>
292
4ed46869
KH		 293#include "lisp.h"
294#include "buffer.h"
df7492f9 295#include "character.h"
4ed46869
KH		 296#include "charset.h"
297#include "ccl.h"
df7492f9 298#include "composite.h"
4ed46869
KH		 299#include "coding.h"
300#include "window.h"
b8299c66
KL		 301#include "frame.h"
302#include "termhooks.h"
4ed46869 303
df7492f9 304Lisp_Object Vcoding_system_hash_table;
4ed46869 305
df7492f9 306Lisp_Object Qcoding_system, Qcoding_aliases, Qeol_type;
1965cb73
DL		 307Lisp_Object Qunix, Qdos;
308extern Lisp_Object Qmac; /* frame.c */
4ed46869
KH		 309Lisp_Object Qbuffer_file_coding_system;
310Lisp_Object Qpost_read_conversion, Qpre_write_conversion;
df7492f9 311Lisp_Object Qdefault_char;
27901516 312Lisp_Object Qno_conversion, Qundecided;
df7492f9 313Lisp_Object Qcharset, Qiso_2022, Qutf_8, Qutf_16, Qshift_jis, Qbig5;
b49a1807 314Lisp_Object Qbig, Qlittle;
bb0115a2 315Lisp_Object Qcoding_system_history;
1397dc18 316Lisp_Object Qvalid_codes;
2133e2d1 317Lisp_Object QCcategory, QCmnemonic, QCdefault_char;
a6f87d34
KH		 318Lisp_Object QCdecode_translation_table, QCencode_translation_table;
319Lisp_Object QCpost_read_conversion, QCpre_write_conversion;
35befdaa 320Lisp_Object QCascii_compatible_p;
4ed46869
KH		 321
322extern Lisp_Object Qinsert_file_contents, Qwrite_region;
387f6ba5 323Lisp_Object Qcall_process, Qcall_process_region;
4ed46869
KH		 324Lisp_Object Qstart_process, Qopen_network_stream;
325Lisp_Object Qtarget_idx;
326
065e3595
KH		 327Lisp_Object Qinsufficient_source, Qinconsistent_eol, Qinvalid_source;
328Lisp_Object Qinterrupted, Qinsufficient_memory;
329
c7183fb8
GM		 330extern Lisp_Object Qcompletion_ignore_case;
331
44e8490d
KH		 332/* If a symbol has this property, evaluate the value to define the
333 symbol as a coding system. */
334static Lisp_Object Qcoding_system_define_form;
335
5d5bf4d8
KH		 336int coding_system_require_warning;
337
d46c5b12
KH		 338Lisp_Object Vselect_safe_coding_system_function;
339
7722baf9
EZ		 340/* Mnemonic string for each format of end-of-line. */
341Lisp_Object eol_mnemonic_unix, eol_mnemonic_dos, eol_mnemonic_mac;
342/* Mnemonic string to indicate format of end-of-line is not yet
4ed46869 343 decided. */
7722baf9 344Lisp_Object eol_mnemonic_undecided;
4ed46869 345
fcbcfb64
KH		 346/* Format of end-of-line decided by system. This is Qunix on
347 Unix and Mac, Qdos on DOS/Windows.
348 This has an effect only for external encoding (i.e. for output to
349 file and process), not for in-buffer or Lisp string encoding. */
350static Lisp_Object system_eol_type;
351
4ed46869
KH		 352#ifdef emacs
353
4608c386
KH		 354Lisp_Object Vcoding_system_list, Vcoding_system_alist;
355
356Lisp_Object Qcoding_system_p, Qcoding_system_error;
4ed46869 357
d46c5b12
KH		 358/* Coding system emacs-mule and raw-text are for converting only
359 end-of-line format. */
360Lisp_Object Qemacs_mule, Qraw_text;
8f924df7 361Lisp_Object Qutf_8_emacs;
ecf488bc 362
4ed46869
KH		 363/* Coding-systems are handed between Emacs Lisp programs and C internal
364 routines by the following three variables. */
365/* Coding-system for reading files and receiving data from process. */
366Lisp_Object Vcoding_system_for_read;
367/* Coding-system for writing files and sending data to process. */
368Lisp_Object Vcoding_system_for_write;
369/* Coding-system actually used in the latest I/O. */
370Lisp_Object Vlast_coding_system_used;
065e3595
KH		 371/* Set to non-nil when an error is detected while code conversion. */
372Lisp_Object Vlast_code_conversion_error;
c4825358 373/* A vector of length 256 which contains information about special
94487c4e 374 Latin codes (especially for dealing with Microsoft codes). */
3f003981 375Lisp_Object Vlatin_extra_code_table;
c4825358 376
9ce27fde
KH		 377/* Flag to inhibit code conversion of end-of-line format. */
378int inhibit_eol_conversion;
379
74383408
KH		 380/* Flag to inhibit ISO2022 escape sequence detection. */
381int inhibit_iso_escape_detection;
382
97b1b294
EZ		 383/* Flag to inhibit detection of binary files through null bytes. */
384int inhibit_null_byte_detection;
385
ed29121d
EZ		 386/* Flag to make buffer-file-coding-system inherit from process-coding. */
387int inherit_process_coding_system;
388
c4825358
KH		 389/* Coding system to be used to encode text for terminal display when
390 terminal coding system is nil. */
391struct coding_system safe_terminal_coding;
392
02ba4723
KH		 393Lisp_Object Vfile_coding_system_alist;
394Lisp_Object Vprocess_coding_system_alist;
395Lisp_Object Vnetwork_coding_system_alist;
4ed46869 396
68c45bf0
PE		 397Lisp_Object Vlocale_coding_system;
398
4ed46869
KH		 399#endif /* emacs */
400
f967223b
KH		 401/* Flag to tell if we look up translation table on character code
402 conversion. */
84fbb8a0 403Lisp_Object Venable_character_translation;
f967223b
KH		 404/* Standard translation table to look up on decoding (reading). */
405Lisp_Object Vstandard_translation_table_for_decode;
406/* Standard translation table to look up on encoding (writing). */
407Lisp_Object Vstandard_translation_table_for_encode;
84fbb8a0 408
f967223b
KH		 409Lisp_Object Qtranslation_table;
410Lisp_Object Qtranslation_table_id;
411Lisp_Object Qtranslation_table_for_decode;
412Lisp_Object Qtranslation_table_for_encode;
4ed46869
KH		 413
414/* Alist of charsets vs revision number. */
df7492f9 415static Lisp_Object Vcharset_revision_table;
4ed46869 416
02ba4723
KH		 417/* Default coding systems used for process I/O. */
418Lisp_Object Vdefault_process_coding_system;
419
002fdb44
DL		 420/* Char table for translating Quail and self-inserting input. */
421Lisp_Object Vtranslation_table_for_input;
422
df7492f9
KH		 423/* Two special coding systems. */
424Lisp_Object Vsjis_coding_system;
425Lisp_Object Vbig5_coding_system;
426
df7492f9
KH		 427/* ISO2022 section */
428
429#define CODING_ISO_INITIAL(coding, reg) \
430 (XINT (AREF (AREF (CODING_ID_ATTRS ((coding)->id), \
431 coding_attr_iso_initial), \
432 reg)))
433
434
435#define CODING_ISO_REQUEST(coding, charset_id) \
436 ((charset_id <= (coding)->max_charset_id \
437 ? (coding)->safe_charsets[charset_id] \
438 : -1))
439
440
441#define CODING_ISO_FLAGS(coding) \
442 ((coding)->spec.iso_2022.flags)
443#define CODING_ISO_DESIGNATION(coding, reg) \
444 ((coding)->spec.iso_2022.current_designation[reg])
445#define CODING_ISO_INVOCATION(coding, plane) \
446 ((coding)->spec.iso_2022.current_invocation[plane])
447#define CODING_ISO_SINGLE_SHIFTING(coding) \
448 ((coding)->spec.iso_2022.single_shifting)
449#define CODING_ISO_BOL(coding) \
450 ((coding)->spec.iso_2022.bol)
451#define CODING_ISO_INVOKED_CHARSET(coding, plane) \
452 CODING_ISO_DESIGNATION ((coding), CODING_ISO_INVOCATION ((coding), (plane)))
453
454/* Control characters of ISO2022. */
455 /* code */ /* function */
456#define ISO_CODE_LF 0x0A /* line-feed */
457#define ISO_CODE_CR 0x0D /* carriage-return */
458#define ISO_CODE_SO 0x0E /* shift-out */
459#define ISO_CODE_SI 0x0F /* shift-in */
460#define ISO_CODE_SS2_7 0x19 /* single-shift-2 for 7-bit code */
461#define ISO_CODE_ESC 0x1B /* escape */
462#define ISO_CODE_SS2 0x8E /* single-shift-2 */
463#define ISO_CODE_SS3 0x8F /* single-shift-3 */
464#define ISO_CODE_CSI 0x9B /* control-sequence-introducer */
465
466/* All code (1-byte) of ISO2022 is classified into one of the
467 followings. */
468enum iso_code_class_type
469 {
470 ISO_control_0, /* Control codes in the range
471 0x00..0x1F and 0x7F, except for the
472 following 5 codes. */
df7492f9
KH		 473 ISO_shift_out, /* ISO_CODE_SO (0x0E) */
474 ISO_shift_in, /* ISO_CODE_SI (0x0F) */
475 ISO_single_shift_2_7, /* ISO_CODE_SS2_7 (0x19) */
476 ISO_escape, /* ISO_CODE_SO (0x1B) */
477 ISO_control_1, /* Control codes in the range
478 0x80..0x9F, except for the
479 following 3 codes. */
480 ISO_single_shift_2, /* ISO_CODE_SS2 (0x8E) */
481 ISO_single_shift_3, /* ISO_CODE_SS3 (0x8F) */
482 ISO_control_sequence_introducer, /* ISO_CODE_CSI (0x9B) */
483 ISO_0x20_or_0x7F, /* Codes of the values 0x20 or 0x7F. */
484 ISO_graphic_plane_0, /* Graphic codes in the range 0x21..0x7E. */
485 ISO_0xA0_or_0xFF, /* Codes of the values 0xA0 or 0xFF. */
486 ISO_graphic_plane_1 /* Graphic codes in the range 0xA1..0xFE. */
487 };
05e6f5dc 488
df7492f9
KH		 489/** The macros CODING_ISO_FLAG_XXX defines a flag bit of the
490 `iso-flags' attribute of an iso2022 coding system. */
05e6f5dc 491
df7492f9
KH		 492/* If set, produce long-form designation sequence (e.g. ESC $ ( A)
493 instead of the correct short-form sequence (e.g. ESC $ A). */
494#define CODING_ISO_FLAG_LONG_FORM 0x0001
93dec019 495
df7492f9
KH		 496/* If set, reset graphic planes and registers at end-of-line to the
497 initial state. */
498#define CODING_ISO_FLAG_RESET_AT_EOL 0x0002
05e6f5dc 499
df7492f9
KH		 500/* If set, reset graphic planes and registers before any control
501 characters to the initial state. */
502#define CODING_ISO_FLAG_RESET_AT_CNTL 0x0004
05e6f5dc 503
df7492f9
KH		 504/* If set, encode by 7-bit environment. */
505#define CODING_ISO_FLAG_SEVEN_BITS 0x0008
4ed46869 506
df7492f9
KH		 507/* If set, use locking-shift function. */
508#define CODING_ISO_FLAG_LOCKING_SHIFT 0x0010
b73bfc1c 509
df7492f9
KH		 510/* If set, use single-shift function. Overwrite
511 CODING_ISO_FLAG_LOCKING_SHIFT. */
512#define CODING_ISO_FLAG_SINGLE_SHIFT 0x0020
b73bfc1c 513
df7492f9
KH		 514/* If set, use designation escape sequence. */
515#define CODING_ISO_FLAG_DESIGNATION 0x0040
b73bfc1c 516
df7492f9
KH		 517/* If set, produce revision number sequence. */
518#define CODING_ISO_FLAG_REVISION 0x0080
b73bfc1c 519
df7492f9
KH		 520/* If set, produce ISO6429's direction specifying sequence. */
521#define CODING_ISO_FLAG_DIRECTION 0x0100
f4dee582 522
df7492f9
KH		 523/* If set, assume designation states are reset at beginning of line on
524 output. */
525#define CODING_ISO_FLAG_INIT_AT_BOL 0x0200
4ed46869 526
df7492f9
KH		 527/* If set, designation sequence should be placed at beginning of line
528 on output. */
529#define CODING_ISO_FLAG_DESIGNATE_AT_BOL 0x0400
aa72b389 530
df7492f9
KH		 531/* If set, do not encode unsafe charactes on output. */
532#define CODING_ISO_FLAG_SAFE 0x0800
aa72b389 533
df7492f9
KH		 534/* If set, extra latin codes (128..159) are accepted as a valid code
535 on input. */
536#define CODING_ISO_FLAG_LATIN_EXTRA 0x1000
aa72b389 537
df7492f9 538#define CODING_ISO_FLAG_COMPOSITION 0x2000
aa72b389 539
df7492f9 540#define CODING_ISO_FLAG_EUC_TW_SHIFT 0x4000
aa72b389 541
bf16eb23 542#define CODING_ISO_FLAG_USE_ROMAN 0x8000
aa72b389 543
bf16eb23 544#define CODING_ISO_FLAG_USE_OLDJIS 0x10000
aa72b389 545
bf16eb23 546#define CODING_ISO_FLAG_FULL_SUPPORT 0x100000
aa72b389 547
df7492f9
KH		 548/* A character to be produced on output if encoding of the original
549 character is prohibited by CODING_ISO_FLAG_SAFE. */
550#define CODING_INHIBIT_CHARACTER_SUBSTITUTION '?'
aa72b389 551
a470d443
KH		 552/* UTF-8 section */
553#define CODING_UTF_8_BOM(coding) \
554 ((coding)->spec.utf_8_bom)
4ed46869 555
df7492f9
KH		 556/* UTF-16 section */
557#define CODING_UTF_16_BOM(coding) \
558 ((coding)->spec.utf_16.bom)
4ed46869 559
df7492f9
KH		 560#define CODING_UTF_16_ENDIAN(coding) \
561 ((coding)->spec.utf_16.endian)
4ed46869 562
df7492f9
KH		 563#define CODING_UTF_16_SURROGATE(coding) \
564 ((coding)->spec.utf_16.surrogate)
4ed46869 565
4ed46869 566
df7492f9
KH		 567/* CCL section */
568#define CODING_CCL_DECODER(coding) \
569 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_decoder)
570#define CODING_CCL_ENCODER(coding) \
571 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_encoder)
572#define CODING_CCL_VALIDS(coding) \
8f924df7 573 (SDATA (AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_valids)))
4ed46869 574
5a936b46 575/* Index for each coding category in `coding_categories' */
4ed46869 576
df7492f9
KH		 577enum coding_category
578 {
579 coding_category_iso_7,
580 coding_category_iso_7_tight,
581 coding_category_iso_8_1,
582 coding_category_iso_8_2,
583 coding_category_iso_7_else,
584 coding_category_iso_8_else,
a470d443
KH		 585 coding_category_utf_8_auto,
586 coding_category_utf_8_nosig,
587 coding_category_utf_8_sig,
df7492f9
KH		 588 coding_category_utf_16_auto,
589 coding_category_utf_16_be,
590 coding_category_utf_16_le,
591 coding_category_utf_16_be_nosig,
592 coding_category_utf_16_le_nosig,
593 coding_category_charset,
594 coding_category_sjis,
595 coding_category_big5,
596 coding_category_ccl,
597 coding_category_emacs_mule,
598 /* All above are targets of code detection. */
599 coding_category_raw_text,
600 coding_category_undecided,
601 coding_category_max
602 };
603
604/* Definitions of flag bits used in detect_coding_XXXX. */
605#define CATEGORY_MASK_ISO_7 (1 << coding_category_iso_7)
606#define CATEGORY_MASK_ISO_7_TIGHT (1 << coding_category_iso_7_tight)
607#define CATEGORY_MASK_ISO_8_1 (1 << coding_category_iso_8_1)
608#define CATEGORY_MASK_ISO_8_2 (1 << coding_category_iso_8_2)
609#define CATEGORY_MASK_ISO_7_ELSE (1 << coding_category_iso_7_else)
610#define CATEGORY_MASK_ISO_8_ELSE (1 << coding_category_iso_8_else)
a470d443
KH		 611#define CATEGORY_MASK_UTF_8_AUTO (1 << coding_category_utf_8_auto)
612#define CATEGORY_MASK_UTF_8_NOSIG (1 << coding_category_utf_8_nosig)
613#define CATEGORY_MASK_UTF_8_SIG (1 << coding_category_utf_8_sig)
b49a1807 614#define CATEGORY_MASK_UTF_16_AUTO (1 << coding_category_utf_16_auto)
df7492f9
KH		 615#define CATEGORY_MASK_UTF_16_BE (1 << coding_category_utf_16_be)
616#define CATEGORY_MASK_UTF_16_LE (1 << coding_category_utf_16_le)
617#define CATEGORY_MASK_UTF_16_BE_NOSIG (1 << coding_category_utf_16_be_nosig)
618#define CATEGORY_MASK_UTF_16_LE_NOSIG (1 << coding_category_utf_16_le_nosig)
619#define CATEGORY_MASK_CHARSET (1 << coding_category_charset)
620#define CATEGORY_MASK_SJIS (1 << coding_category_sjis)
621#define CATEGORY_MASK_BIG5 (1 << coding_category_big5)
622#define CATEGORY_MASK_CCL (1 << coding_category_ccl)
623#define CATEGORY_MASK_EMACS_MULE (1 << coding_category_emacs_mule)
ff0dacd7 624#define CATEGORY_MASK_RAW_TEXT (1 << coding_category_raw_text)
df7492f9
KH		 625
626/* This value is returned if detect_coding_mask () find nothing other
627 than ASCII characters. */
628#define CATEGORY_MASK_ANY \
629 (CATEGORY_MASK_ISO_7 \
630 | CATEGORY_MASK_ISO_7_TIGHT \
631 | CATEGORY_MASK_ISO_8_1 \
632 | CATEGORY_MASK_ISO_8_2 \
633 | CATEGORY_MASK_ISO_7_ELSE \
634 | CATEGORY_MASK_ISO_8_ELSE \
a470d443
KH		 635 | CATEGORY_MASK_UTF_8_AUTO \
636 | CATEGORY_MASK_UTF_8_NOSIG \
637 | CATEGORY_MASK_UTF_8_SIG \
2f3cbb32 638 | CATEGORY_MASK_UTF_16_AUTO \
df7492f9
KH		 639 | CATEGORY_MASK_UTF_16_BE \
640 | CATEGORY_MASK_UTF_16_LE \
641 | CATEGORY_MASK_UTF_16_BE_NOSIG \
642 | CATEGORY_MASK_UTF_16_LE_NOSIG \
643 | CATEGORY_MASK_CHARSET \
644 | CATEGORY_MASK_SJIS \
645 | CATEGORY_MASK_BIG5 \
646 | CATEGORY_MASK_CCL \
647 | CATEGORY_MASK_EMACS_MULE)
648
649
650#define CATEGORY_MASK_ISO_7BIT \
651 (CATEGORY_MASK_ISO_7 | CATEGORY_MASK_ISO_7_TIGHT)
652
653#define CATEGORY_MASK_ISO_8BIT \
654 (CATEGORY_MASK_ISO_8_1 | CATEGORY_MASK_ISO_8_2)
655
656#define CATEGORY_MASK_ISO_ELSE \
657 (CATEGORY_MASK_ISO_7_ELSE | CATEGORY_MASK_ISO_8_ELSE)
658
659#define CATEGORY_MASK_ISO_ESCAPE \
660 (CATEGORY_MASK_ISO_7 \
661 | CATEGORY_MASK_ISO_7_TIGHT \
662 | CATEGORY_MASK_ISO_7_ELSE \
663 | CATEGORY_MASK_ISO_8_ELSE)
664
665#define CATEGORY_MASK_ISO \
666 ( CATEGORY_MASK_ISO_7BIT \
667 | CATEGORY_MASK_ISO_8BIT \
668 | CATEGORY_MASK_ISO_ELSE)
669
670#define CATEGORY_MASK_UTF_16 \
2f3cbb32
KH		 671 (CATEGORY_MASK_UTF_16_AUTO \
672 | CATEGORY_MASK_UTF_16_BE \
df7492f9
KH		 673 | CATEGORY_MASK_UTF_16_LE \
674 | CATEGORY_MASK_UTF_16_BE_NOSIG \
675 | CATEGORY_MASK_UTF_16_LE_NOSIG)
676
a470d443
KH		 677#define CATEGORY_MASK_UTF_8 \
678 (CATEGORY_MASK_UTF_8_AUTO \
679 | CATEGORY_MASK_UTF_8_NOSIG \
680 | CATEGORY_MASK_UTF_8_SIG)
df7492f9
KH		 681
682/* List of symbols `coding-category-xxx' ordered by priority. This
683 variable is exposed to Emacs Lisp. */
684static Lisp_Object Vcoding_category_list;
685
686/* Table of coding categories (Lisp symbols). This variable is for
687 internal use oly. */
688static Lisp_Object Vcoding_category_table;
689
690/* Table of coding-categories ordered by priority. */
691static enum coding_category coding_priorities[coding_category_max];
692
693/* Nth element is a coding context for the coding system bound to the
694 Nth coding category. */
695static struct coding_system coding_categories[coding_category_max];
696
df7492f9
KH		 697/*** Commonly used macros and functions ***/
698
699#ifndef min
700#define min(a, b) ((a) < (b) ? (a) : (b))
701#endif
702#ifndef max
703#define max(a, b) ((a) > (b) ? (a) : (b))
704#endif
4ed46869 705
24a73b0a
KH		 706#define CODING_GET_INFO(coding, attrs, charset_list) \
707 do { \
708 (attrs) = CODING_ID_ATTRS ((coding)->id); \
709 (charset_list) = CODING_ATTR_CHARSET_LIST (attrs); \
df7492f9 710 } while (0)
4ed46869 711
4ed46869 712
df7492f9
KH		 713/* Safely get one byte from the source text pointed by SRC which ends
714 at SRC_END, and set C to that byte. If there are not enough bytes
065e3595
KH		 715 in the source, it jumps to `no_more_source'. If multibytep is
716 nonzero, and a multibyte character is found at SRC, set C to the
717 negative value of the character code. The caller should declare
718 and set these variables appropriately in advance:
719 src, src_end, multibytep */
aa72b389 720
065e3595
KH		 721#define ONE_MORE_BYTE(c) \
722 do { \
723 if (src == src_end) \
724 { \
725 if (src_base < src) \
726 record_conversion_result \
727 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
728 goto no_more_source; \
729 } \
730 c = *src++; \
731 if (multibytep && (c & 0x80)) \
732 { \
733 if ((c & 0xFE) == 0xC0) \
734 c = ((c & 1) << 6) | *src++; \
735 else \
736 { \
35befdaa
KH		 737 src--; \
738 c = - string_char (src, &src, NULL); \
065e3595
KH		 739 record_conversion_result \
740 (coding, CODING_RESULT_INVALID_SRC); \
741 } \
742 } \
743 consumed_chars++; \
aa72b389
KH		 744 } while (0)
745
f56a4450 746/* Safely get two bytes from the source text pointed by SRC which ends
220eeac9
KH		 747 at SRC_END, and set C1 and C2 to those bytes while skipping the
748 heading multibyte characters. If there are not enough bytes in the
749 source, it jumps to `no_more_source'. If multibytep is nonzero and
750 a multibyte character is found for C2, set C2 to the negative value
751 of the character code. The caller should declare and set these
752 variables appropriately in advance:
f56a4450
KH		 753 src, src_end, multibytep
754 It is intended that this macro is used in detect_coding_utf_16. */
755
220eeac9
KH		 756#define TWO_MORE_BYTES(c1, c2) \
757 do { \
758 do { \
759 if (src == src_end) \
760 goto no_more_source; \
761 c1 = *src++; \
762 if (multibytep && (c1 & 0x80)) \
763 { \
764 if ((c1 & 0xFE) == 0xC0) \
765 c1 = ((c1 & 1) << 6) | *src++; \
766 else \
767 { \
768 src += BYTES_BY_CHAR_HEAD (c1) - 1; \
769 c1 = -1; \
770 } \
771 } \
772 } while (c1 < 0); \
773 if (src == src_end) \
774 goto no_more_source; \
775 c2 = *src++; \
776 if (multibytep && (c2 & 0x80)) \
777 { \
778 if ((c2 & 0xFE) == 0xC0) \
779 c2 = ((c2 & 1) << 6) | *src++; \
780 else \
781 c2 = -1; \
782 } \
f56a4450
KH		 783 } while (0)
784
aa72b389 785
065e3595
KH		 786#define ONE_MORE_BYTE_NO_CHECK(c) \
787 do { \
788 c = *src++; \
789 if (multibytep && (c & 0x80)) \
790 { \
791 if ((c & 0xFE) == 0xC0) \
792 c = ((c & 1) << 6) | *src++; \
793 else \
794 { \
35befdaa
KH		 795 src--; \
796 c = - string_char (src, &src, NULL); \
065e3595
KH		 797 record_conversion_result \
798 (coding, CODING_RESULT_INVALID_SRC); \
799 } \
800 } \
801 consumed_chars++; \
aa72b389
KH		 802 } while (0)
803
aa72b389 804
df7492f9
KH		 805/* Store a byte C in the place pointed by DST and increment DST to the
806 next free point, and increment PRODUCED_CHARS. The caller should
807 assure that C is 0..127, and declare and set the variable `dst'
808 appropriately in advance.
809*/
aa72b389
KH		 810
811
df7492f9
KH		 812#define EMIT_ONE_ASCII_BYTE(c) \
813 do { \
814 produced_chars++; \
815 *dst++ = (c); \
b6871cc7 816 } while (0)
aa72b389
KH		 817
818
df7492f9 819/* Like EMIT_ONE_ASCII_BYTE byt store two bytes; C1 and C2. */
aa72b389 820
df7492f9
KH		 821#define EMIT_TWO_ASCII_BYTES(c1, c2) \
822 do { \
823 produced_chars += 2; \
824 *dst++ = (c1), *dst++ = (c2); \
825 } while (0)
aa72b389
KH		 826
827
df7492f9
KH		 828/* Store a byte C in the place pointed by DST and increment DST to the
829 next free point, and increment PRODUCED_CHARS. If MULTIBYTEP is
830 nonzero, store in an appropriate multibyte from. The caller should
831 declare and set the variables `dst' and `multibytep' appropriately
832 in advance. */
833
834#define EMIT_ONE_BYTE(c) \
835 do { \
836 produced_chars++; \
837 if (multibytep) \
838 { \
839 int ch = (c); \
840 if (ch >= 0x80) \
841 ch = BYTE8_TO_CHAR (ch); \
842 CHAR_STRING_ADVANCE (ch, dst); \
843 } \
844 else \
845 *dst++ = (c); \
aa72b389 846 } while (0)
aa72b389 847
aa72b389 848
df7492f9 849/* Like EMIT_ONE_BYTE, but emit two bytes; C1 and C2. */
aa72b389 850
e19c3639
KH		 851#define EMIT_TWO_BYTES(c1, c2) \
852 do { \
853 produced_chars += 2; \
854 if (multibytep) \
855 { \
856 int ch; \
857 \
858 ch = (c1); \
859 if (ch >= 0x80) \
860 ch = BYTE8_TO_CHAR (ch); \
861 CHAR_STRING_ADVANCE (ch, dst); \
862 ch = (c2); \
863 if (ch >= 0x80) \
864 ch = BYTE8_TO_CHAR (ch); \
865 CHAR_STRING_ADVANCE (ch, dst); \
866 } \
867 else \
868 { \
869 *dst++ = (c1); \
870 *dst++ = (c2); \
871 } \
aa72b389
KH		 872 } while (0)
873
874
df7492f9
KH		 875#define EMIT_THREE_BYTES(c1, c2, c3) \
876 do { \
877 EMIT_ONE_BYTE (c1); \
878 EMIT_TWO_BYTES (c2, c3); \
879 } while (0)
aa72b389 880
aa72b389 881
df7492f9
KH		 882#define EMIT_FOUR_BYTES(c1, c2, c3, c4) \
883 do { \
884 EMIT_TWO_BYTES (c1, c2); \
885 EMIT_TWO_BYTES (c3, c4); \
886 } while (0)
aa72b389 887
aa72b389 888
f6cbaf43
KH		 889/* Prototypes for static functions. */
890static void record_conversion_result P_ ((struct coding_system *coding,
891 enum coding_result_code result));
892static int detect_coding_utf_8 P_ ((struct coding_system *,
893 struct coding_detection_info *info));
894static void decode_coding_utf_8 P_ ((struct coding_system *));
895static int encode_coding_utf_8 P_ ((struct coding_system *));
896
897static int detect_coding_utf_16 P_ ((struct coding_system *,
898 struct coding_detection_info *info));
899static void decode_coding_utf_16 P_ ((struct coding_system *));
900static int encode_coding_utf_16 P_ ((struct coding_system *));
901
902static int detect_coding_iso_2022 P_ ((struct coding_system *,
903 struct coding_detection_info *info));
904static void decode_coding_iso_2022 P_ ((struct coding_system *));
905static int encode_coding_iso_2022 P_ ((struct coding_system *));
906
907static int detect_coding_emacs_mule P_ ((struct coding_system *,
908 struct coding_detection_info *info));
909static void decode_coding_emacs_mule P_ ((struct coding_system *));
910static int encode_coding_emacs_mule P_ ((struct coding_system *));
911
912static int detect_coding_sjis P_ ((struct coding_system *,
913 struct coding_detection_info *info));
914static void decode_coding_sjis P_ ((struct coding_system *));
915static int encode_coding_sjis P_ ((struct coding_system *));
916
917static int detect_coding_big5 P_ ((struct coding_system *,
918 struct coding_detection_info *info));
919static void decode_coding_big5 P_ ((struct coding_system *));
920static int encode_coding_big5 P_ ((struct coding_system *));
921
922static int detect_coding_ccl P_ ((struct coding_system *,
923 struct coding_detection_info *info));
924static void decode_coding_ccl P_ ((struct coding_system *));
925static int encode_coding_ccl P_ ((struct coding_system *));
926
927static void decode_coding_raw_text P_ ((struct coding_system *));
928static int encode_coding_raw_text P_ ((struct coding_system *));
929
930static void coding_set_source P_ ((struct coding_system *));
931static void coding_set_destination P_ ((struct coding_system *));
932static void coding_alloc_by_realloc P_ ((struct coding_system *, EMACS_INT));
933static void coding_alloc_by_making_gap P_ ((struct coding_system *,
287c57d7 934 EMACS_INT, EMACS_INT));
f6cbaf43
KH		 935static unsigned char *alloc_destination P_ ((struct coding_system *,
936 EMACS_INT, unsigned char *));
937static void setup_iso_safe_charsets P_ ((Lisp_Object));
938static unsigned char *encode_designation_at_bol P_ ((struct coding_system *,
939 int *, int *,
940 unsigned char *));
941static int detect_eol P_ ((const unsigned char *,
942 EMACS_INT, enum coding_category));
943static Lisp_Object adjust_coding_eol_type P_ ((struct coding_system *, int));
944static void decode_eol P_ ((struct coding_system *));
945static Lisp_Object get_translation_table P_ ((Lisp_Object, int, int *));
946static Lisp_Object get_translation P_ ((Lisp_Object, int *, int *,
947 int, int *, int *));
948static int produce_chars P_ ((struct coding_system *, Lisp_Object, int));
949static INLINE void produce_composition P_ ((struct coding_system *, int *,
950 EMACS_INT));
951static INLINE void produce_charset P_ ((struct coding_system *, int *,
952 EMACS_INT));
953static void produce_annotation P_ ((struct coding_system *, EMACS_INT));
954static int decode_coding P_ ((struct coding_system *));
955static INLINE int *handle_composition_annotation P_ ((EMACS_INT, EMACS_INT,
3ed051d4 956 struct coding_system *,
f6cbaf43
KH		 957 int *, EMACS_INT *));
958static INLINE int *handle_charset_annotation P_ ((EMACS_INT, EMACS_INT,
959 struct coding_system *,
960 int *, EMACS_INT *));
961static void consume_chars P_ ((struct coding_system *, Lisp_Object, int));
962static int encode_coding P_ ((struct coding_system *));
963static Lisp_Object make_conversion_work_buffer P_ ((int));
964static Lisp_Object code_conversion_restore P_ ((Lisp_Object));
965static INLINE int char_encodable_p P_ ((int, Lisp_Object));
966static Lisp_Object make_subsidiaries P_ ((Lisp_Object));
967
065e3595
KH		 968static void
969record_conversion_result (struct coding_system *coding,
970 enum coding_result_code result)
971{
972 coding->result = result;
973 switch (result)
974 {
975 case CODING_RESULT_INSUFFICIENT_SRC:
976 Vlast_code_conversion_error = Qinsufficient_source;
977 break;
978 case CODING_RESULT_INCONSISTENT_EOL:
979 Vlast_code_conversion_error = Qinconsistent_eol;
980 break;
981 case CODING_RESULT_INVALID_SRC:
982 Vlast_code_conversion_error = Qinvalid_source;
983 break;
984 case CODING_RESULT_INTERRUPT:
985 Vlast_code_conversion_error = Qinterrupted;
986 break;
987 case CODING_RESULT_INSUFFICIENT_MEM:
988 Vlast_code_conversion_error = Qinsufficient_memory;
989 break;
35befdaa
KH		 990 default:
991 Vlast_code_conversion_error = intern ("Unknown error");
065e3595
KH		 992 }
993}
994
df7492f9
KH		 995#define CODING_DECODE_CHAR(coding, src, src_base, src_end, charset, code, c) \
996 do { \
997 charset_map_loaded = 0; \
998 c = DECODE_CHAR (charset, code); \
999 if (charset_map_loaded) \
1000 { \
8f924df7 1001 const unsigned char *orig = coding->source; \
df7492f9
KH		 1002 EMACS_INT offset; \
1003 \
1004 coding_set_source (coding); \
1005 offset = coding->source - orig; \
1006 src += offset; \
1007 src_base += offset; \
1008 src_end += offset; \
1009 } \
aa72b389
KH		 1010 } while (0)
1011
1012
119852e7
KH		 1013/* If there are at least BYTES length of room at dst, allocate memory
1014 for coding->destination and update dst and dst_end. We don't have
1015 to take care of coding->source which will be relocated. It is
1016 handled by calling coding_set_source in encode_coding. */
1017
df7492f9
KH		 1018#define ASSURE_DESTINATION(bytes) \
1019 do { \
1020 if (dst + (bytes) >= dst_end) \
1021 { \
1022 int more_bytes = charbuf_end - charbuf + (bytes); \
1023 \
1024 dst = alloc_destination (coding, more_bytes, dst); \
1025 dst_end = coding->destination + coding->dst_bytes; \
1026 } \
1027 } while (0)
aa72b389 1028
aa72b389 1029
db274c7a
KH		 1030/* Store multibyte form of the character C in P, and advance P to the
1031 end of the multibyte form. This is like CHAR_STRING_ADVANCE but it
1032 never calls MAYBE_UNIFY_CHAR. */
1033
1034#define CHAR_STRING_ADVANCE_NO_UNIFY(c, p) \
1035 do { \
1036 if ((c) <= MAX_1_BYTE_CHAR) \
1037 *(p)++ = (c); \
1038 else if ((c) <= MAX_2_BYTE_CHAR) \
1039 *(p)++ = (0xC0 | ((c) >> 6)), \
1040 *(p)++ = (0x80 | ((c) & 0x3F)); \
1041 else if ((c) <= MAX_3_BYTE_CHAR) \
1042 *(p)++ = (0xE0 | ((c) >> 12)), \
1043 *(p)++ = (0x80 | (((c) >> 6) & 0x3F)), \
1044 *(p)++ = (0x80 | ((c) & 0x3F)); \
1045 else if ((c) <= MAX_4_BYTE_CHAR) \
1046 *(p)++ = (0xF0 | (c >> 18)), \
1047 *(p)++ = (0x80 | ((c >> 12) & 0x3F)), \
1048 *(p)++ = (0x80 | ((c >> 6) & 0x3F)), \
1049 *(p)++ = (0x80 | (c & 0x3F)); \
1050 else if ((c) <= MAX_5_BYTE_CHAR) \
1051 *(p)++ = 0xF8, \
1052 *(p)++ = (0x80 | ((c >> 18) & 0x0F)), \
1053 *(p)++ = (0x80 | ((c >> 12) & 0x3F)), \
1054 *(p)++ = (0x80 | ((c >> 6) & 0x3F)), \
1055 *(p)++ = (0x80 | (c & 0x3F)); \
1056 else \
1057 (p) += BYTE8_STRING ((c) - 0x3FFF80, p); \
1058 } while (0)
1059
1060
1061/* Return the character code of character whose multibyte form is at
1062 P, and advance P to the end of the multibyte form. This is like
1063 STRING_CHAR_ADVANCE, but it never calls MAYBE_UNIFY_CHAR. */
1064
1065#define STRING_CHAR_ADVANCE_NO_UNIFY(p) \
1066 (!((p)[0] & 0x80) \
1067 ? *(p)++ \
1068 : ! ((p)[0] & 0x20) \
1069 ? ((p) += 2, \
1070 ((((p)[-2] & 0x1F) << 6) \
1071 | ((p)[-1] & 0x3F) \
1072 | ((unsigned char) ((p)[-2]) < 0xC2 ? 0x3FFF80 : 0))) \
1073 : ! ((p)[0] & 0x10) \
1074 ? ((p) += 3, \
1075 ((((p)[-3] & 0x0F) << 12) \
1076 | (((p)[-2] & 0x3F) << 6) \
1077 | ((p)[-1] & 0x3F))) \
1078 : ! ((p)[0] & 0x08) \
1079 ? ((p) += 4, \
1080 ((((p)[-4] & 0xF) << 18) \
1081 | (((p)[-3] & 0x3F) << 12) \
1082 | (((p)[-2] & 0x3F) << 6) \
1083 | ((p)[-1] & 0x3F))) \
1084 : ((p) += 5, \
1085 ((((p)[-4] & 0x3F) << 18) \
1086 | (((p)[-3] & 0x3F) << 12) \
1087 | (((p)[-2] & 0x3F) << 6) \
1088 | ((p)[-1] & 0x3F))))
1089
aa72b389 1090
df7492f9
KH		 1091static void
1092coding_set_source (coding)
aa72b389 1093 struct coding_system *coding;
aa72b389 1094{
df7492f9
KH		 1095 if (BUFFERP (coding->src_object))
1096 {
2cb26057 1097 struct buffer *buf = XBUFFER (coding->src_object);
aa72b389 1098
df7492f9 1099 if (coding->src_pos < 0)
2cb26057 1100 coding->source = BUF_GAP_END_ADDR (buf) + coding->src_pos_byte;
df7492f9 1101 else
2cb26057 1102 coding->source = BUF_BYTE_ADDRESS (buf, coding->src_pos_byte);
aa72b389 1103 }
df7492f9 1104 else if (STRINGP (coding->src_object))
aa72b389 1105 {
8f924df7 1106 coding->source = SDATA (coding->src_object) + coding->src_pos_byte;
aa72b389 1107 }
df7492f9
KH		 1108 else
1109 /* Otherwise, the source is C string and is never relocated
1110 automatically. Thus we don't have to update anything. */
1111 ;
1112}
aa72b389 1113
df7492f9
KH		 1114static void
1115coding_set_destination (coding)
1116 struct coding_system *coding;
1117{
1118 if (BUFFERP (coding->dst_object))
aa72b389 1119 {
df7492f9 1120 if (coding->src_pos < 0)
aa72b389 1121 {
13818c30 1122 coding->destination = BEG_ADDR + coding->dst_pos_byte - BEG_BYTE;
28f67a95
KH		 1123 coding->dst_bytes = (GAP_END_ADDR
1124 - (coding->src_bytes - coding->consumed)
1125 - coding->destination);
aa72b389 1126 }
df7492f9 1127 else
28f67a95
KH		 1128 {
1129 /* We are sure that coding->dst_pos_byte is before the gap
1130 of the buffer. */
1131 coding->destination = (BUF_BEG_ADDR (XBUFFER (coding->dst_object))
13818c30 1132 + coding->dst_pos_byte - BEG_BYTE);
28f67a95
KH		 1133 coding->dst_bytes = (BUF_GAP_END_ADDR (XBUFFER (coding->dst_object))
1134 - coding->destination);
1135 }
df7492f9
KH		 1136 }
1137 else
1138 /* Otherwise, the destination is C string and is never relocated
1139 automatically. Thus we don't have to update anything. */
1140 ;
1141}
1142
1143
1144static void
1145coding_alloc_by_realloc (coding, bytes)
1146 struct coding_system *coding;
1147 EMACS_INT bytes;
1148{
1149 coding->destination = (unsigned char *) xrealloc (coding->destination,
1150 coding->dst_bytes + bytes);
1151 coding->dst_bytes += bytes;
1152}
1153
1154static void
db274c7a 1155coding_alloc_by_making_gap (coding, gap_head_used, bytes)
df7492f9 1156 struct coding_system *coding;
db274c7a 1157 EMACS_INT gap_head_used, bytes;
df7492f9 1158{
db274c7a 1159 if (EQ (coding->src_object, coding->dst_object))
df7492f9 1160 {
db274c7a
KH		 1161 /* The gap may contain the produced data at the head and not-yet
1162 consumed data at the tail. To preserve those data, we at
1163 first make the gap size to zero, then increase the gap
1164 size. */
1165 EMACS_INT add = GAP_SIZE;
1166
1167 GPT += gap_head_used, GPT_BYTE += gap_head_used;
1168 GAP_SIZE = 0; ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
df7492f9
KH		 1169 make_gap (bytes);
1170 GAP_SIZE += add; ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
db274c7a 1171 GPT -= gap_head_used, GPT_BYTE -= gap_head_used;
df7492f9 1172 }
730fff51 1173 else
df7492f9 1174 {
2c78b7e1
KH		 1175 Lisp_Object this_buffer;
1176
1177 this_buffer = Fcurrent_buffer ();
df7492f9
KH		 1178 set_buffer_internal (XBUFFER (coding->dst_object));
1179 make_gap (bytes);
1180 set_buffer_internal (XBUFFER (this_buffer));
aa72b389 1181 }
df7492f9 1182}
8f924df7 1183
df7492f9
KH		 1184
1185static unsigned char *
1186alloc_destination (coding, nbytes, dst)
1187 struct coding_system *coding;
3e139625 1188 EMACS_INT nbytes;
df7492f9
KH		 1189 unsigned char *dst;
1190{
1191 EMACS_INT offset = dst - coding->destination;
1192
1193 if (BUFFERP (coding->dst_object))
db274c7a
KH		 1194 {
1195 struct buffer *buf = XBUFFER (coding->dst_object);
1196
1197 coding_alloc_by_making_gap (coding, dst - BUF_GPT_ADDR (buf), nbytes);
1198 }
aa72b389 1199 else
df7492f9 1200 coding_alloc_by_realloc (coding, nbytes);
065e3595 1201 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1202 coding_set_destination (coding);
1203 dst = coding->destination + offset;
1204 return dst;
1205}
aa72b389 1206
ff0dacd7
KH		 1207/** Macros for annotations. */
1208
1209/* Maximum length of annotation data (sum of annotations for
1210 composition and charset). */
69a80ea3 1211#define MAX_ANNOTATION_LENGTH (4 + (MAX_COMPOSITION_COMPONENTS * 2) - 1 + 4)
ff0dacd7
KH		 1212
1213/* An annotation data is stored in the array coding->charbuf in this
1214 format:
69a80ea3 1215 [ -LENGTH ANNOTATION_MASK NCHARS ... ]
ff0dacd7
KH		 1216 LENGTH is the number of elements in the annotation.
1217 ANNOTATION_MASK is one of CODING_ANNOTATE_XXX_MASK.
69a80ea3 1218 NCHARS is the number of characters in the text annotated.
ff0dacd7
KH		 1219
1220 The format of the following elements depend on ANNOTATION_MASK.
1221
1222 In the case of CODING_ANNOTATE_COMPOSITION_MASK, these elements
1223 follows:
1224 ... METHOD [ COMPOSITION-COMPONENTS ... ]
1225 METHOD is one of enum composition_method.
1226 Optionnal COMPOSITION-COMPONENTS are characters and composition
1227 rules.
1228
1229 In the case of CODING_ANNOTATE_CHARSET_MASK, one element CHARSET-ID
1230 follows. */
1231
69a80ea3 1232#define ADD_ANNOTATION_DATA(buf, len, mask, nchars) \
ff0dacd7
KH		 1233 do { \
1234 *(buf)++ = -(len); \
1235 *(buf)++ = (mask); \
69a80ea3 1236 *(buf)++ = (nchars); \
ff0dacd7
KH		 1237 coding->annotated = 1; \
1238 } while (0);
1239
69a80ea3
KH		 1240#define ADD_COMPOSITION_DATA(buf, nchars, method) \
1241 do { \
1242 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_COMPOSITION_MASK, nchars); \
1243 *buf++ = method; \
ff0dacd7
KH		 1244 } while (0)
1245
1246
69a80ea3
KH		 1247#define ADD_CHARSET_DATA(buf, nchars, id) \
1248 do { \
1249 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_CHARSET_MASK, nchars); \
1250 *buf++ = id; \
ff0dacd7
KH		 1251 } while (0)
1252
df7492f9
KH		 1253\f
1254/*** 2. Emacs' internal format (emacs-utf-8) ***/
1255
1256
1257
1258\f
1259/*** 3. UTF-8 ***/
1260
1261/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1262 Check if a text is encoded in UTF-8. If it is, return 1, else
1263 return 0. */
df7492f9
KH		 1264
1265#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
1266#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
1267#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
1268#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
1269#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
1270#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
1271
a470d443
KH		 1272#define UTF_BOM 0xFEFF
1273#define UTF_8_BOM_1 0xEF
1274#define UTF_8_BOM_2 0xBB
1275#define UTF_8_BOM_3 0xBF
1276
df7492f9 1277static int
ff0dacd7 1278detect_coding_utf_8 (coding, detect_info)
df7492f9 1279 struct coding_system *coding;
ff0dacd7 1280 struct coding_detection_info *detect_info;
df7492f9 1281{
065e3595 1282 const unsigned char *src = coding->source, *src_base;
8f924df7 1283 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 1284 int multibytep = coding->src_multibyte;
1285 int consumed_chars = 0;
a470d443 1286 int bom_found = 0;
df7492f9
KH		 1287 int found = 0;
1288
ff0dacd7 1289 detect_info->checked |= CATEGORY_MASK_UTF_8;
df7492f9
KH		 1290 /* A coding system of this category is always ASCII compatible. */
1291 src += coding->head_ascii;
1292
1293 while (1)
aa72b389 1294 {
df7492f9 1295 int c, c1, c2, c3, c4;
aa72b389 1296
065e3595 1297 src_base = src;
df7492f9 1298 ONE_MORE_BYTE (c);
065e3595 1299 if (c < 0 || UTF_8_1_OCTET_P (c))
df7492f9
KH		 1300 continue;
1301 ONE_MORE_BYTE (c1);
065e3595 1302 if (c1 < 0 || ! UTF_8_EXTRA_OCTET_P (c1))
df7492f9
KH		 1303 break;
1304 if (UTF_8_2_OCTET_LEADING_P (c))
aa72b389 1305 {
a470d443 1306 found = 1;
df7492f9 1307 continue;
aa72b389 1308 }
df7492f9 1309 ONE_MORE_BYTE (c2);
065e3595 1310 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1311 break;
1312 if (UTF_8_3_OCTET_LEADING_P (c))
aa72b389 1313 {
a470d443
KH		 1314 found = 1;
1315 if (src_base == coding->source
1316 && c == UTF_8_BOM_1 && c1 == UTF_8_BOM_2 && c2 == UTF_8_BOM_3)
1317 bom_found = 1;
df7492f9 1318 continue;
aa72b389 1319 }
df7492f9 1320 ONE_MORE_BYTE (c3);
065e3595 1321 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1322 break;
1323 if (UTF_8_4_OCTET_LEADING_P (c))
aa72b389 1324 {
a470d443 1325 found = 1;
df7492f9
KH		 1326 continue;
1327 }
1328 ONE_MORE_BYTE (c4);
065e3595 1329 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1330 break;
1331 if (UTF_8_5_OCTET_LEADING_P (c))
1332 {
a470d443 1333 found = 1;
df7492f9
KH		 1334 continue;
1335 }
1336 break;
aa72b389 1337 }
ff0dacd7 1338 detect_info->rejected |= CATEGORY_MASK_UTF_8;
df7492f9 1339 return 0;
aa72b389 1340
df7492f9 1341 no_more_source:
065e3595 1342 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
aa72b389 1343 {
ff0dacd7 1344 detect_info->rejected |= CATEGORY_MASK_UTF_8;
89528eb3 1345 return 0;
aa72b389 1346 }
a470d443
KH		 1347 if (bom_found)
1348 {
1349 /* The first character 0xFFFE doesn't necessarily mean a BOM. */
1350 detect_info->found |= CATEGORY_MASK_UTF_8_SIG | CATEGORY_MASK_UTF_8_NOSIG;
1351 }
1352 else
1353 {
1354 detect_info->rejected |= CATEGORY_MASK_UTF_8_SIG;
0e17387a
KH		 1355 if (found)
1356 detect_info->found |= CATEGORY_MASK_UTF_8_NOSIG;
a470d443 1357 }
ff0dacd7 1358 return 1;
aa72b389
KH		 1359}
1360
4ed46869 1361
b73bfc1c 1362static void
df7492f9 1363decode_coding_utf_8 (coding)
b73bfc1c 1364 struct coding_system *coding;
b73bfc1c 1365{
8f924df7
KH		 1366 const unsigned char *src = coding->source + coding->consumed;
1367 const unsigned char *src_end = coding->source + coding->src_bytes;
1368 const unsigned char *src_base;
69a80ea3
KH		 1369 int *charbuf = coding->charbuf + coding->charbuf_used;
1370 int *charbuf_end = coding->charbuf + coding->charbuf_size;
453b38f0 1371 int consumed_chars = 0, consumed_chars_base = 0;
df7492f9 1372 int multibytep = coding->src_multibyte;
a470d443 1373 enum utf_bom_type bom = CODING_UTF_8_BOM (coding);
24a73b0a 1374 Lisp_Object attr, charset_list;
119852e7
KH		 1375 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
1376 int byte_after_cr = -1;
4ed46869 1377
24a73b0a 1378 CODING_GET_INFO (coding, attr, charset_list);
df7492f9 1379
a470d443
KH		 1380 if (bom != utf_without_bom)
1381 {
1382 int c1, c2, c3;
1383
1384 src_base = src;
1385 ONE_MORE_BYTE (c1);
1386 if (! UTF_8_3_OCTET_LEADING_P (c1))
1387 src = src_base;
1388 else
1389 {
159bd5a2 1390 ONE_MORE_BYTE (c2);
a470d443
KH		 1391 if (! UTF_8_EXTRA_OCTET_P (c2))
1392 src = src_base;
1393 else
1394 {
159bd5a2 1395 ONE_MORE_BYTE (c3);
a470d443
KH		 1396 if (! UTF_8_EXTRA_OCTET_P (c3))
1397 src = src_base;
1398 else
1399 {
1400 if ((c1 != UTF_8_BOM_1)
1401 || (c2 != UTF_8_BOM_2) || (c3 != UTF_8_BOM_3))
1402 src = src_base;
1403 else
1404 CODING_UTF_8_BOM (coding) = utf_without_bom;
1405 }
1406 }
1407 }
1408 }
1409 CODING_UTF_8_BOM (coding) = utf_without_bom;
1410
1411
1412
df7492f9 1413 while (1)
b73bfc1c 1414 {
df7492f9 1415 int c, c1, c2, c3, c4, c5;
ec6d2bb8 1416
df7492f9
KH		 1417 src_base = src;
1418 consumed_chars_base = consumed_chars;
4af310db 1419
df7492f9 1420 if (charbuf >= charbuf_end)
b71f6f73
KH		 1421 {
1422 if (byte_after_cr >= 0)
1423 src_base--;
1424 break;
1425 }
df7492f9 1426
119852e7
KH		 1427 if (byte_after_cr >= 0)
1428 c1 = byte_after_cr, byte_after_cr = -1;
1429 else
1430 ONE_MORE_BYTE (c1);
065e3595
KH		 1431 if (c1 < 0)
1432 {
1433 c = - c1;
1434 }
1435 else if (UTF_8_1_OCTET_P(c1))
df7492f9 1436 {
119852e7
KH		 1437 if (eol_crlf && c1 == '\r')
1438 ONE_MORE_BYTE (byte_after_cr);
df7492f9 1439 c = c1;
4af310db 1440 }
df7492f9 1441 else
4af310db 1442 {
df7492f9 1443 ONE_MORE_BYTE (c2);
065e3595 1444 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1445 goto invalid_code;
1446 if (UTF_8_2_OCTET_LEADING_P (c1))
4af310db 1447 {
b0edb2c5
DL		 1448 c = ((c1 & 0x1F) << 6) | (c2 & 0x3F);
1449 /* Reject overlong sequences here and below. Encoders
1450 producing them are incorrect, they can be misleading,
1451 and they mess up read/write invariance. */
1452 if (c < 128)
1453 goto invalid_code;
4af310db 1454 }
df7492f9 1455 else
aa72b389 1456 {
df7492f9 1457 ONE_MORE_BYTE (c3);
065e3595 1458 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1459 goto invalid_code;
1460 if (UTF_8_3_OCTET_LEADING_P (c1))
b0edb2c5
DL		 1461 {
1462 c = (((c1 & 0xF) << 12)
1463 | ((c2 & 0x3F) << 6) | (c3 & 0x3F));
72fe1301
DL		 1464 if (c < 0x800
1465 || (c >= 0xd800 && c < 0xe000)) /* surrogates (invalid) */
b0edb2c5
DL		 1466 goto invalid_code;
1467 }
df7492f9
KH		 1468 else
1469 {
1470 ONE_MORE_BYTE (c4);
065e3595 1471 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1472 goto invalid_code;
1473 if (UTF_8_4_OCTET_LEADING_P (c1))
b0edb2c5 1474 {
df7492f9
KH		 1475 c = (((c1 & 0x7) << 18) | ((c2 & 0x3F) << 12)
1476 | ((c3 & 0x3F) << 6) | (c4 & 0x3F));
b0edb2c5
DL		 1477 if (c < 0x10000)
1478 goto invalid_code;
1479 }
df7492f9
KH		 1480 else
1481 {
1482 ONE_MORE_BYTE (c5);
065e3595 1483 if (c5 < 0 || ! UTF_8_EXTRA_OCTET_P (c5))
df7492f9
KH		 1484 goto invalid_code;
1485 if (UTF_8_5_OCTET_LEADING_P (c1))
1486 {
1487 c = (((c1 & 0x3) << 24) | ((c2 & 0x3F) << 18)
1488 | ((c3 & 0x3F) << 12) | ((c4 & 0x3F) << 6)
1489 | (c5 & 0x3F));
b0edb2c5 1490 if ((c > MAX_CHAR) || (c < 0x200000))
df7492f9
KH		 1491 goto invalid_code;
1492 }
1493 else
1494 goto invalid_code;
1495 }
1496 }
aa72b389 1497 }
b73bfc1c 1498 }
df7492f9
KH		 1499
1500 *charbuf++ = c;
1501 continue;
1502
1503 invalid_code:
1504 src = src_base;
1505 consumed_chars = consumed_chars_base;
1506 ONE_MORE_BYTE (c);
1507 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
1508 coding->errors++;
aa72b389
KH		 1509 }
1510
df7492f9
KH		 1511 no_more_source:
1512 coding->consumed_char += consumed_chars_base;
1513 coding->consumed = src_base - coding->source;
1514 coding->charbuf_used = charbuf - coding->charbuf;
1515}
1516
1517
1518static int
1519encode_coding_utf_8 (coding)
1520 struct coding_system *coding;
1521{
1522 int multibytep = coding->dst_multibyte;
1523 int *charbuf = coding->charbuf;
1524 int *charbuf_end = charbuf + coding->charbuf_used;
1525 unsigned char *dst = coding->destination + coding->produced;
1526 unsigned char *dst_end = coding->destination + coding->dst_bytes;
e19c3639 1527 int produced_chars = 0;
df7492f9
KH		 1528 int c;
1529
a470d443
KH		 1530 if (CODING_UTF_8_BOM (coding) == utf_with_bom)
1531 {
1532 ASSURE_DESTINATION (3);
1533 EMIT_THREE_BYTES (UTF_8_BOM_1, UTF_8_BOM_2, UTF_8_BOM_3);
1534 CODING_UTF_8_BOM (coding) = utf_without_bom;
1535 }
1536
df7492f9 1537 if (multibytep)
aa72b389 1538 {
df7492f9
KH		 1539 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
1540
1541 while (charbuf < charbuf_end)
b73bfc1c 1542 {
df7492f9 1543 unsigned char str[MAX_MULTIBYTE_LENGTH], *p, *pend = str;
8f924df7 1544
df7492f9
KH		 1545 ASSURE_DESTINATION (safe_room);
1546 c = *charbuf++;
28f67a95
KH		 1547 if (CHAR_BYTE8_P (c))
1548 {
1549 c = CHAR_TO_BYTE8 (c);
1550 EMIT_ONE_BYTE (c);
1551 }
1552 else
1553 {
db274c7a 1554 CHAR_STRING_ADVANCE_NO_UNIFY (c, pend);
28f67a95
KH		 1555 for (p = str; p < pend; p++)
1556 EMIT_ONE_BYTE (*p);
1557 }
b73bfc1c 1558 }
aa72b389 1559 }
df7492f9
KH		 1560 else
1561 {
1562 int safe_room = MAX_MULTIBYTE_LENGTH;
1563
1564 while (charbuf < charbuf_end)
b73bfc1c 1565 {
df7492f9
KH		 1566 ASSURE_DESTINATION (safe_room);
1567 c = *charbuf++;
f03caae0
KH		 1568 if (CHAR_BYTE8_P (c))
1569 *dst++ = CHAR_TO_BYTE8 (c);
1570 else
db274c7a 1571 CHAR_STRING_ADVANCE_NO_UNIFY (c, dst);
df7492f9 1572 produced_chars++;
4ed46869
KH		 1573 }
1574 }
065e3595 1575 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1576 coding->produced_char += produced_chars;
1577 coding->produced = dst - coding->destination;
1578 return 0;
4ed46869
KH		 1579}
1580
b73bfc1c 1581
df7492f9 1582/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1583 Check if a text is encoded in one of UTF-16 based coding systems.
1584 If it is, return 1, else return 0. */
aa72b389 1585
df7492f9
KH		 1586#define UTF_16_HIGH_SURROGATE_P(val) \
1587 (((val) & 0xFC00) == 0xD800)
1588
1589#define UTF_16_LOW_SURROGATE_P(val) \
1590 (((val) & 0xFC00) == 0xDC00)
93dec019 1591
df7492f9
KH		 1592#define UTF_16_INVALID_P(val) \
1593 (((val) == 0xFFFE) \
1594 || ((val) == 0xFFFF) \
1595 || UTF_16_LOW_SURROGATE_P (val))
aa72b389 1596
aa72b389 1597
df7492f9 1598static int
ff0dacd7 1599detect_coding_utf_16 (coding, detect_info)
aa72b389 1600 struct coding_system *coding;
ff0dacd7 1601 struct coding_detection_info *detect_info;
aa72b389 1602{
8f924df7
KH		 1603 const unsigned char *src = coding->source, *src_base = src;
1604 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 1605 int multibytep = coding->src_multibyte;
1606 int consumed_chars = 0;
1607 int c1, c2;
aa72b389 1608
ff0dacd7 1609 detect_info->checked |= CATEGORY_MASK_UTF_16;
ff0dacd7 1610 if (coding->mode & CODING_MODE_LAST_BLOCK
24a73b0a 1611 && (coding->src_chars & 1))
ff0dacd7
KH		 1612 {
1613 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1614 return 0;
1615 }
24a73b0a 1616
f56a4450 1617 TWO_MORE_BYTES (c1, c2);
df7492f9 1618 if ((c1 == 0xFF) && (c2 == 0xFE))
aa72b389 1619 {
b49a1807
KH		 1620 detect_info->found |= (CATEGORY_MASK_UTF_16_LE
1621 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1622 detect_info->rejected |= (CATEGORY_MASK_UTF_16_BE
1623 | CATEGORY_MASK_UTF_16_BE_NOSIG
1624 | CATEGORY_MASK_UTF_16_LE_NOSIG);
aa72b389 1625 }
df7492f9 1626 else if ((c1 == 0xFE) && (c2 == 0xFF))
ff0dacd7 1627 {
b49a1807
KH		 1628 detect_info->found |= (CATEGORY_MASK_UTF_16_BE
1629 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1630 detect_info->rejected |= (CATEGORY_MASK_UTF_16_LE
1631 | CATEGORY_MASK_UTF_16_BE_NOSIG
1632 | CATEGORY_MASK_UTF_16_LE_NOSIG);
1633 }
220eeac9 1634 else if (c2 < 0)
f56a4450
KH		 1635 {
1636 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1637 return 0;
1638 }
2f3cbb32 1639 else
24a73b0a 1640 {
2f3cbb32
KH		 1641 /* We check the dispersion of Eth and Oth bytes where E is even and
1642 O is odd. If both are high, we assume binary data.*/
1643 unsigned char e[256], o[256];
1644 unsigned e_num = 1, o_num = 1;
1645
1646 memset (e, 0, 256);
1647 memset (o, 0, 256);
1648 e[c1] = 1;
1649 o[c2] = 1;
1650
24a73b0a
KH		 1651 detect_info->rejected
1652 |= (CATEGORY_MASK_UTF_16_BE | CATEGORY_MASK_UTF_16_LE);
2f3cbb32
KH		 1653
1654 while (1)
1655 {
f56a4450 1656 TWO_MORE_BYTES (c1, c2);
220eeac9 1657 if (c2 < 0)
f56a4450 1658 break;
2f3cbb32
KH		 1659 if (! e[c1])
1660 {
1661 e[c1] = 1;
1662 e_num++;
1663 if (e_num >= 128)
1664 break;
1665 }
1666 if (! o[c2])
1667 {
1668 o[c1] = 1;
1669 o_num++;
1670 if (o_num >= 128)
1671 break;
1672 }
1673 }
1674 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1675 return 0;
ff0dacd7 1676 }
2f3cbb32 1677
df7492f9 1678 no_more_source:
ff0dacd7 1679 return 1;
df7492f9 1680}
aa72b389 1681
df7492f9
KH		 1682static void
1683decode_coding_utf_16 (coding)
1684 struct coding_system *coding;
1685{
8f924df7
KH		 1686 const unsigned char *src = coding->source + coding->consumed;
1687 const unsigned char *src_end = coding->source + coding->src_bytes;
1688 const unsigned char *src_base;
69a80ea3
KH		 1689 int *charbuf = coding->charbuf + coding->charbuf_used;
1690 int *charbuf_end = coding->charbuf + coding->charbuf_size;
3a8406e1 1691 int consumed_chars = 0, consumed_chars_base = 0;
df7492f9 1692 int multibytep = coding->src_multibyte;
a470d443 1693 enum utf_bom_type bom = CODING_UTF_16_BOM (coding);
df7492f9
KH		 1694 enum utf_16_endian_type endian = CODING_UTF_16_ENDIAN (coding);
1695 int surrogate = CODING_UTF_16_SURROGATE (coding);
24a73b0a 1696 Lisp_Object attr, charset_list;
119852e7
KH		 1697 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
1698 int byte_after_cr1 = -1, byte_after_cr2 = -1;
df7492f9 1699
24a73b0a 1700 CODING_GET_INFO (coding, attr, charset_list);
df7492f9 1701
a470d443 1702 if (bom == utf_with_bom)
aa72b389 1703 {
df7492f9 1704 int c, c1, c2;
4af310db 1705
aa72b389 1706 src_base = src;
df7492f9
KH		 1707 ONE_MORE_BYTE (c1);
1708 ONE_MORE_BYTE (c2);
e19c3639 1709 c = (c1 << 8) | c2;
aa72b389 1710
b49a1807
KH		 1711 if (endian == utf_16_big_endian
1712 ? c != 0xFEFF : c != 0xFFFE)
aa72b389 1713 {
b49a1807
KH		 1714 /* The first two bytes are not BOM. Treat them as bytes
1715 for a normal character. */
1716 src = src_base;
1717 coding->errors++;
aa72b389 1718 }
a470d443 1719 CODING_UTF_16_BOM (coding) = utf_without_bom;
b49a1807 1720 }
a470d443 1721 else if (bom == utf_detect_bom)
b49a1807
KH		 1722 {
1723 /* We have already tried to detect BOM and failed in
1724 detect_coding. */
a470d443 1725 CODING_UTF_16_BOM (coding) = utf_without_bom;
df7492f9 1726 }
aa72b389 1727
df7492f9
KH		 1728 while (1)
1729 {
1730 int c, c1, c2;
1731
1732 src_base = src;
1733 consumed_chars_base = consumed_chars;
1734
1735 if (charbuf + 2 >= charbuf_end)
b71f6f73
KH		 1736 {
1737 if (byte_after_cr1 >= 0)
1738 src_base -= 2;
1739 break;
1740 }
df7492f9 1741
119852e7
KH		 1742 if (byte_after_cr1 >= 0)
1743 c1 = byte_after_cr1, byte_after_cr1 = -1;
1744 else
1745 ONE_MORE_BYTE (c1);
065e3595
KH		 1746 if (c1 < 0)
1747 {
1748 *charbuf++ = -c1;
1749 continue;
1750 }
119852e7
KH		 1751 if (byte_after_cr2 >= 0)
1752 c2 = byte_after_cr2, byte_after_cr2 = -1;
1753 else
1754 ONE_MORE_BYTE (c2);
065e3595
KH		 1755 if (c2 < 0)
1756 {
1757 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
1758 *charbuf++ = -c2;
1759 continue;
1760 }
df7492f9 1761 c = (endian == utf_16_big_endian
e19c3639 1762 ? ((c1 << 8) | c2) : ((c2 << 8) | c1));
119852e7 1763
df7492f9 1764 if (surrogate)
fd3ae0b9 1765 {
df7492f9 1766 if (! UTF_16_LOW_SURROGATE_P (c))
fd3ae0b9 1767 {
df7492f9
KH		 1768 if (endian == utf_16_big_endian)
1769 c1 = surrogate >> 8, c2 = surrogate & 0xFF;
1770 else
1771 c1 = surrogate & 0xFF, c2 = surrogate >> 8;
1772 *charbuf++ = c1;
1773 *charbuf++ = c2;
1774 coding->errors++;
1775 if (UTF_16_HIGH_SURROGATE_P (c))
1776 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
fd3ae0b9 1777 else
df7492f9 1778 *charbuf++ = c;
fd3ae0b9
KH		 1779 }
1780 else
df7492f9
KH		 1781 {
1782 c = ((surrogate - 0xD800) << 10) | (c - 0xDC00);
1783 CODING_UTF_16_SURROGATE (coding) = surrogate = 0;
29f7ffd0 1784 *charbuf++ = 0x10000 + c;
df7492f9 1785 }
fd3ae0b9 1786 }
aa72b389 1787 else
df7492f9
KH		 1788 {
1789 if (UTF_16_HIGH_SURROGATE_P (c))
1790 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1791 else
119852e7
KH		 1792 {
1793 if (eol_crlf && c == '\r')
1794 {
1795 ONE_MORE_BYTE (byte_after_cr1);
1796 ONE_MORE_BYTE (byte_after_cr2);
1797 }
1798 *charbuf++ = c;
1799 }
8f924df7 1800 }
aa72b389 1801 }
df7492f9
KH		 1802
1803 no_more_source:
1804 coding->consumed_char += consumed_chars_base;
1805 coding->consumed = src_base - coding->source;
1806 coding->charbuf_used = charbuf - coding->charbuf;
aa72b389 1807}
b73bfc1c 1808
df7492f9
KH		 1809static int
1810encode_coding_utf_16 (coding)
1811 struct coding_system *coding;
1812{
1813 int multibytep = coding->dst_multibyte;
1814 int *charbuf = coding->charbuf;
1815 int *charbuf_end = charbuf + coding->charbuf_used;
1816 unsigned char *dst = coding->destination + coding->produced;
1817 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1818 int safe_room = 8;
a470d443 1819 enum utf_bom_type bom = CODING_UTF_16_BOM (coding);
df7492f9
KH		 1820 int big_endian = CODING_UTF_16_ENDIAN (coding) == utf_16_big_endian;
1821 int produced_chars = 0;
24a73b0a 1822 Lisp_Object attrs, charset_list;
df7492f9 1823 int c;
4ed46869 1824
24a73b0a 1825 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 1826
a470d443 1827 if (bom != utf_without_bom)
df7492f9
KH		 1828 {
1829 ASSURE_DESTINATION (safe_room);
1830 if (big_endian)
df7492f9 1831 EMIT_TWO_BYTES (0xFE, 0xFF);
880cf180
KH		 1832 else
1833 EMIT_TWO_BYTES (0xFF, 0xFE);
a470d443 1834 CODING_UTF_16_BOM (coding) = utf_without_bom;
df7492f9
KH		 1835 }
1836
1837 while (charbuf < charbuf_end)
1838 {
1839 ASSURE_DESTINATION (safe_room);
1840 c = *charbuf++;
e19c3639
KH		 1841 if (c >= MAX_UNICODE_CHAR)
1842 c = coding->default_char;
df7492f9
KH		 1843
1844 if (c < 0x10000)
1845 {
1846 if (big_endian)
1847 EMIT_TWO_BYTES (c >> 8, c & 0xFF);
1848 else
1849 EMIT_TWO_BYTES (c & 0xFF, c >> 8);
1850 }
1851 else
1852 {
1853 int c1, c2;
1854
1855 c -= 0x10000;
1856 c1 = (c >> 10) + 0xD800;
1857 c2 = (c & 0x3FF) + 0xDC00;
1858 if (big_endian)
1859 EMIT_FOUR_BYTES (c1 >> 8, c1 & 0xFF, c2 >> 8, c2 & 0xFF);
1860 else
1861 EMIT_FOUR_BYTES (c1 & 0xFF, c1 >> 8, c2 & 0xFF, c2 >> 8);
1862 }
1863 }
065e3595 1864 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1865 coding->produced = dst - coding->destination;
1866 coding->produced_char += produced_chars;
1867 return 0;
1868}
1869
1870\f
1871/*** 6. Old Emacs' internal format (emacs-mule) ***/
1872
1873/* Emacs' internal format for representation of multiple character
1874 sets is a kind of multi-byte encoding, i.e. characters are
1875 represented by variable-length sequences of one-byte codes.
1876
1877 ASCII characters and control characters (e.g. `tab', `newline') are
1878 represented by one-byte sequences which are their ASCII codes, in
1879 the range 0x00 through 0x7F.
1880
1881 8-bit characters of the range 0x80..0x9F are represented by
1882 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
1883 code + 0x20).
1884
1885 8-bit characters of the range 0xA0..0xFF are represented by
1886 one-byte sequences which are their 8-bit code.
1887
1888 The other characters are represented by a sequence of `base
1889 leading-code', optional `extended leading-code', and one or two
1890 `position-code's. The length of the sequence is determined by the
1891 base leading-code. Leading-code takes the range 0x81 through 0x9D,
1892 whereas extended leading-code and position-code take the range 0xA0
1893 through 0xFF. See `charset.h' for more details about leading-code
1894 and position-code.
1895
1896 --- CODE RANGE of Emacs' internal format ---
1897 character set range
1898 ------------- -----
1899 ascii 0x00..0x7F
1900 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
1901 eight-bit-graphic 0xA0..0xBF
1902 ELSE 0x81..0x9D + [0xA0..0xFF]+
1903 ---------------------------------------------
1904
1905 As this is the internal character representation, the format is
1906 usually not used externally (i.e. in a file or in a data sent to a
1907 process). But, it is possible to have a text externally in this
1908 format (i.e. by encoding by the coding system `emacs-mule').
1909
1910 In that case, a sequence of one-byte codes has a slightly different
1911 form.
1912
1913 At first, all characters in eight-bit-control are represented by
1914 one-byte sequences which are their 8-bit code.
1915
1916 Next, character composition data are represented by the byte
1917 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
1918 where,
1919 METHOD is 0xF0 plus one of composition method (enum
1920 composition_method),
1921
1922 BYTES is 0xA0 plus a byte length of this composition data,
1923
1924 CHARS is 0x20 plus a number of characters composed by this
1925 data,
1926
1927 COMPONENTs are characters of multibye form or composition
1928 rules encoded by two-byte of ASCII codes.
1929
1930 In addition, for backward compatibility, the following formats are
1931 also recognized as composition data on decoding.
1932
1933 0x80 MSEQ ...
1934 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
1935
1936 Here,
1937 MSEQ is a multibyte form but in these special format:
1938 ASCII: 0xA0 ASCII_CODE+0x80,
1939 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
1940 RULE is a one byte code of the range 0xA0..0xF0 that
1941 represents a composition rule.
1942 */
1943
1944char emacs_mule_bytes[256];
1945
df7492f9 1946int
ff0dacd7 1947emacs_mule_char (coding, src, nbytes, nchars, id)
df7492f9 1948 struct coding_system *coding;
065e3595 1949 const unsigned char *src;
ff0dacd7 1950 int *nbytes, *nchars, *id;
df7492f9 1951{
8f924df7
KH		 1952 const unsigned char *src_end = coding->source + coding->src_bytes;
1953 const unsigned char *src_base = src;
df7492f9 1954 int multibytep = coding->src_multibyte;
df7492f9
KH		 1955 struct charset *charset;
1956 unsigned code;
1957 int c;
1958 int consumed_chars = 0;
1959
1960 ONE_MORE_BYTE (c);
065e3595 1961 if (c < 0)
df7492f9 1962 {
065e3595
KH		 1963 c = -c;
1964 charset = emacs_mule_charset[0];
1965 }
1966 else
1967 {
4d41e8b7
KH		 1968 if (c >= 0xA0)
1969 {
b3af4b28 1970 /* Old style component character of a composition. */
4d41e8b7
KH		 1971 if (c == 0xA0)
1972 {
1973 ONE_MORE_BYTE (c);
1974 c -= 0x80;
1975 }
1976 else
1977 c -= 0x20;
1978 }
1979
065e3595 1980 switch (emacs_mule_bytes[c])
b73bfc1c 1981 {
065e3595 1982 case 2:
df7492f9
KH		 1983 if (! (charset = emacs_mule_charset[c]))
1984 goto invalid_code;
1985 ONE_MORE_BYTE (c);
9ffd559c 1986 if (c < 0xA0)
065e3595 1987 goto invalid_code;
df7492f9 1988 code = c & 0x7F;
065e3595
KH		 1989 break;
1990
1991 case 3:
1992 if (c == EMACS_MULE_LEADING_CODE_PRIVATE_11
1993 || c == EMACS_MULE_LEADING_CODE_PRIVATE_12)
1994 {
1995 ONE_MORE_BYTE (c);
9ffd559c 1996 if (c < 0xA0 || ! (charset = emacs_mule_charset[c]))
065e3595
KH		 1997 goto invalid_code;
1998 ONE_MORE_BYTE (c);
9ffd559c 1999 if (c < 0xA0)
065e3595
KH		 2000 goto invalid_code;
2001 code = c & 0x7F;
2002 }
2003 else
2004 {
2005 if (! (charset = emacs_mule_charset[c]))
2006 goto invalid_code;
2007 ONE_MORE_BYTE (c);
9ffd559c 2008 if (c < 0xA0)
065e3595
KH		 2009 goto invalid_code;
2010 code = (c & 0x7F) << 8;
2011 ONE_MORE_BYTE (c);
9ffd559c 2012 if (c < 0xA0)
065e3595
KH		 2013 goto invalid_code;
2014 code |= c & 0x7F;
2015 }
2016 break;
2017
2018 case 4:
2019 ONE_MORE_BYTE (c);
2020 if (c < 0 || ! (charset = emacs_mule_charset[c]))
df7492f9
KH		 2021 goto invalid_code;
2022 ONE_MORE_BYTE (c);
9ffd559c 2023 if (c < 0xA0)
065e3595 2024 goto invalid_code;
781d7a48 2025 code = (c & 0x7F) << 8;
df7492f9 2026 ONE_MORE_BYTE (c);
9ffd559c 2027 if (c < 0xA0)
065e3595 2028 goto invalid_code;
df7492f9 2029 code |= c & 0x7F;
065e3595 2030 break;
df7492f9 2031
065e3595
KH		 2032 case 1:
2033 code = c;
2034 charset = CHARSET_FROM_ID (ASCII_BYTE_P (code)
2035 ? charset_ascii : charset_eight_bit);
2036 break;
df7492f9 2037
065e3595
KH		 2038 default:
2039 abort ();
2040 }
2041 c = DECODE_CHAR (charset, code);
2042 if (c < 0)
2043 goto invalid_code;
df7492f9 2044 }
df7492f9
KH		 2045 *nbytes = src - src_base;
2046 *nchars = consumed_chars;
ff0dacd7
KH		 2047 if (id)
2048 *id = charset->id;
df7492f9
KH		 2049 return c;
2050
2051 no_more_source:
2052 return -2;
2053
2054 invalid_code:
2055 return -1;
2056}
2057
2058
2059/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 2060 Check if a text is encoded in `emacs-mule'. If it is, return 1,
2061 else return 0. */
df7492f9
KH		 2062
2063static int
ff0dacd7 2064detect_coding_emacs_mule (coding, detect_info)
df7492f9 2065 struct coding_system *coding;
ff0dacd7 2066 struct coding_detection_info *detect_info;
df7492f9 2067{
065e3595 2068 const unsigned char *src = coding->source, *src_base;
8f924df7 2069 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 2070 int multibytep = coding->src_multibyte;
2071 int consumed_chars = 0;
2072 int c;
2073 int found = 0;
2074
ff0dacd7 2075 detect_info->checked |= CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 2076 /* A coding system of this category is always ASCII compatible. */
2077 src += coding->head_ascii;
2078
2079 while (1)
2080 {
065e3595 2081 src_base = src;
df7492f9 2082 ONE_MORE_BYTE (c);
065e3595
KH		 2083 if (c < 0)
2084 continue;
df7492f9
KH		 2085 if (c == 0x80)
2086 {
2087 /* Perhaps the start of composite character. We simple skip
2088 it because analyzing it is too heavy for detecting. But,
2089 at least, we check that the composite character
3ed051d4 2090 constitutes of more than 4 bytes. */
8f924df7 2091 const unsigned char *src_base;
df7492f9
KH		 2092
2093 repeat:
2094 src_base = src;
2095 do
2096 {
2097 ONE_MORE_BYTE (c);
2098 }
2099 while (c >= 0xA0);
2100
2101 if (src - src_base <= 4)
2102 break;
ff0dacd7 2103 found = CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 2104 if (c == 0x80)
2105 goto repeat;
b73bfc1c 2106 }
df7492f9
KH		 2107
2108 if (c < 0x80)
b73bfc1c 2109 {
df7492f9
KH		 2110 if (c < 0x20
2111 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO))
2112 break;
2113 }
2114 else
2115 {
0e219d54 2116 int more_bytes = emacs_mule_bytes[*src_base] - 1;
df7492f9 2117
0e219d54 2118 while (more_bytes > 0)
df7492f9
KH		 2119 {
2120 ONE_MORE_BYTE (c);
0e219d54
KH		 2121 if (c < 0xA0)
2122 {
2123 src--; /* Unread the last byte. */
2124 break;
2125 }
2126 more_bytes--;
df7492f9 2127 }
0e219d54 2128 if (more_bytes != 0)
df7492f9 2129 break;
ff0dacd7 2130 found = CATEGORY_MASK_EMACS_MULE;
4ed46869
KH		 2131 }
2132 }
ff0dacd7 2133 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 2134 return 0;
2135
2136 no_more_source:
065e3595 2137 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 2138 {
ff0dacd7 2139 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
89528eb3
KH		 2140 return 0;
2141 }
ff0dacd7
KH		 2142 detect_info->found |= found;
2143 return 1;
4ed46869
KH		 2144}
2145
b73bfc1c 2146
df7492f9
KH		 2147/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
2148
2149/* Decode a character represented as a component of composition
2150 sequence of Emacs 20/21 style at SRC. Set C to that character and
2151 update SRC to the head of next character (or an encoded composition
2152 rule). If SRC doesn't points a composition component, set C to -1.
2153 If SRC points an invalid byte sequence, global exit by a return
2154 value 0. */
2155
2156#define DECODE_EMACS_MULE_COMPOSITION_CHAR(buf) \
f937a7db 2157 do \
df7492f9
KH		 2158 { \
2159 int c; \
2160 int nbytes, nchars; \
2161 \
2162 if (src == src_end) \
2163 break; \
ff0dacd7 2164 c = emacs_mule_char (coding, src, &nbytes, &nchars, NULL);\
df7492f9
KH		 2165 if (c < 0) \
2166 { \
2167 if (c == -2) \
2168 break; \
2169 goto invalid_code; \
2170 } \
2171 *buf++ = c; \
2172 src += nbytes; \
2173 consumed_chars += nchars; \
2174 } \
f937a7db 2175 while (0)
df7492f9
KH		 2176
2177
2178/* Decode a composition rule represented as a component of composition
781d7a48
KH		 2179 sequence of Emacs 20 style at SRC. Store the decoded rule in *BUF,
2180 and increment BUF. If SRC points an invalid byte sequence, set C
2181 to -1. */
df7492f9 2182
781d7a48 2183#define DECODE_EMACS_MULE_COMPOSITION_RULE_20(buf) \
df7492f9
KH		 2184 do { \
2185 int c, gref, nref; \
2186 \
781d7a48 2187 if (src >= src_end) \
df7492f9
KH		 2188 goto invalid_code; \
2189 ONE_MORE_BYTE_NO_CHECK (c); \
4d41e8b7 2190 c -= 0xA0; \
df7492f9
KH		 2191 if (c < 0 || c >= 81) \
2192 goto invalid_code; \
2193 \
2194 gref = c / 9, nref = c % 9; \
2195 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
2196 } while (0)
2197
2198
781d7a48
KH		 2199/* Decode a composition rule represented as a component of composition
2200 sequence of Emacs 21 style at SRC. Store the decoded rule in *BUF,
2201 and increment BUF. If SRC points an invalid byte sequence, set C
2202 to -1. */
2203
2204#define DECODE_EMACS_MULE_COMPOSITION_RULE_21(buf) \
2205 do { \
2206 int gref, nref; \
2207 \
2208 if (src + 1>= src_end) \
2209 goto invalid_code; \
2210 ONE_MORE_BYTE_NO_CHECK (gref); \
2211 gref -= 0x20; \
2212 ONE_MORE_BYTE_NO_CHECK (nref); \
2213 nref -= 0x20; \
2214 if (gref < 0 || gref >= 81 \
2215 || nref < 0 || nref >= 81) \
2216 goto invalid_code; \
2217 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
2218 } while (0)
2219
2220
df7492f9 2221#define DECODE_EMACS_MULE_21_COMPOSITION(c) \
aa72b389 2222 do { \
df7492f9 2223 /* Emacs 21 style format. The first three bytes at SRC are \
781d7a48 2224 (METHOD - 0xF2), (BYTES - 0xA0), (CHARS - 0xA0), where BYTES is \
df7492f9
KH		 2225 the byte length of this composition information, CHARS is the \
2226 number of characters composed by this composition. */ \
781d7a48
KH		 2227 enum composition_method method = c - 0xF2; \
2228 int *charbuf_base = charbuf; \
df7492f9
KH		 2229 int consumed_chars_limit; \
2230 int nbytes, nchars; \
2231 \
2232 ONE_MORE_BYTE (c); \
065e3595
KH		 2233 if (c < 0) \
2234 goto invalid_code; \
df7492f9
KH		 2235 nbytes = c - 0xA0; \
2236 if (nbytes < 3) \
2237 goto invalid_code; \
2238 ONE_MORE_BYTE (c); \
065e3595
KH		 2239 if (c < 0) \
2240 goto invalid_code; \
df7492f9 2241 nchars = c - 0xA0; \
69a80ea3 2242 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
df7492f9
KH		 2243 consumed_chars_limit = consumed_chars_base + nbytes; \
2244 if (method != COMPOSITION_RELATIVE) \
aa72b389 2245 { \
df7492f9
KH		 2246 int i = 0; \
2247 while (consumed_chars < consumed_chars_limit) \
aa72b389 2248 { \
df7492f9 2249 if (i % 2 && method != COMPOSITION_WITH_ALTCHARS) \
781d7a48 2250 DECODE_EMACS_MULE_COMPOSITION_RULE_21 (charbuf); \
df7492f9
KH		 2251 else \
2252 DECODE_EMACS_MULE_COMPOSITION_CHAR (charbuf); \
781d7a48 2253 i++; \
aa72b389 2254 } \
df7492f9
KH		 2255 if (consumed_chars < consumed_chars_limit) \
2256 goto invalid_code; \
781d7a48 2257 charbuf_base[0] -= i; \
aa72b389
KH		 2258 } \
2259 } while (0)
93dec019 2260
aa72b389 2261
d959f512
KH		 2262#define DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION(c) \
2263 do { \
2264 /* Emacs 20 style format for relative composition. */ \
2265 /* Store multibyte form of characters to be composed. */ \
2266 enum composition_method method = COMPOSITION_RELATIVE; \
2267 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
2268 int *buf = components; \
2269 int i, j; \
2270 \
2271 src = src_base; \
2272 ONE_MORE_BYTE (c); /* skip 0x80 */ \
2273 for (i = 0; *src >= 0xA0 && i < MAX_COMPOSITION_COMPONENTS; i++) \
2274 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2275 if (i < 2) \
2276 goto invalid_code; \
2277 ADD_COMPOSITION_DATA (charbuf, i, method); \
2278 for (j = 0; j < i; j++) \
2279 *charbuf++ = components[j]; \
df7492f9
KH		 2280 } while (0)
2281
2282
2283#define DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION(c) \
2284 do { \
2285 /* Emacs 20 style format for rule-base composition. */ \
2286 /* Store multibyte form of characters to be composed. */ \
ff0dacd7 2287 enum composition_method method = COMPOSITION_WITH_RULE; \
4d41e8b7 2288 int *charbuf_base = charbuf; \
df7492f9
KH		 2289 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
2290 int *buf = components; \
2291 int i, j; \
4d41e8b7 2292 \
df7492f9 2293 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
4d41e8b7 2294 for (i = 1; i < MAX_COMPOSITION_COMPONENTS; i++) \
df7492f9 2295 { \
4d41e8b7
KH		 2296 if (*src < 0xA0) \
2297 break; \
781d7a48 2298 DECODE_EMACS_MULE_COMPOSITION_RULE_20 (buf); \
df7492f9
KH		 2299 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2300 } \
4d41e8b7 2301 if (i <= 1 || (buf - components) % 2 == 0) \
df7492f9 2302 goto invalid_code; \
4d41e8b7 2303 if (charbuf + i + (i / 2) + 1 >= charbuf_end) \
df7492f9 2304 goto no_more_source; \
4d41e8b7
KH		 2305 ADD_COMPOSITION_DATA (charbuf, i, method); \
2306 i = i * 2 - 1; \
df7492f9
KH		 2307 for (j = 0; j < i; j++) \
2308 *charbuf++ = components[j]; \
4d41e8b7 2309 charbuf_base[0] -= i; \
df7492f9
KH		 2310 for (j = 0; j < i; j += 2) \
2311 *charbuf++ = components[j]; \
2312 } while (0)
2313
aa72b389
KH		 2314
2315static void
df7492f9 2316decode_coding_emacs_mule (coding)
aa72b389 2317 struct coding_system *coding;
aa72b389 2318{
8f924df7
KH		 2319 const unsigned char *src = coding->source + coding->consumed;
2320 const unsigned char *src_end = coding->source + coding->src_bytes;
2321 const unsigned char *src_base;
69a80ea3
KH		 2322 int *charbuf = coding->charbuf + coding->charbuf_used;
2323 int *charbuf_end
2324 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9 2325 int consumed_chars = 0, consumed_chars_base;
df7492f9 2326 int multibytep = coding->src_multibyte;
24a73b0a 2327 Lisp_Object attrs, charset_list;
ff0dacd7
KH		 2328 int char_offset = coding->produced_char;
2329 int last_offset = char_offset;
2330 int last_id = charset_ascii;
119852e7
KH		 2331 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
2332 int byte_after_cr = -1;
aa72b389 2333
24a73b0a 2334 CODING_GET_INFO (coding, attrs, charset_list);
aa72b389 2335
aa72b389
KH		 2336 while (1)
2337 {
df7492f9
KH		 2338 int c;
2339
aa72b389 2340 src_base = src;
df7492f9
KH		 2341 consumed_chars_base = consumed_chars;
2342
2343 if (charbuf >= charbuf_end)
b71f6f73
KH		 2344 {
2345 if (byte_after_cr >= 0)
2346 src_base--;
2347 break;
2348 }
aa72b389 2349
119852e7
KH		 2350 if (byte_after_cr >= 0)
2351 c = byte_after_cr, byte_after_cr = -1;
2352 else
2353 ONE_MORE_BYTE (c);
065e3595
KH		 2354 if (c < 0)
2355 {
2356 *charbuf++ = -c;
2357 char_offset++;
2358 }
2359 else if (c < 0x80)
aa72b389 2360 {
119852e7
KH		 2361 if (eol_crlf && c == '\r')
2362 ONE_MORE_BYTE (byte_after_cr);
df7492f9
KH		 2363 *charbuf++ = c;
2364 char_offset++;
aa72b389 2365 }
df7492f9
KH		 2366 else if (c == 0x80)
2367 {
df7492f9 2368 ONE_MORE_BYTE (c);
065e3595
KH		 2369 if (c < 0)
2370 goto invalid_code;
781d7a48
KH		 2371 if (c - 0xF2 >= COMPOSITION_RELATIVE
2372 && c - 0xF2 <= COMPOSITION_WITH_RULE_ALTCHARS)
df7492f9
KH		 2373 DECODE_EMACS_MULE_21_COMPOSITION (c);
2374 else if (c < 0xC0)
2375 DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION (c);
2376 else if (c == 0xFF)
2377 DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION (c);
2378 else
2379 goto invalid_code;
2380 }
2381 else if (c < 0xA0 && emacs_mule_bytes[c] > 1)
2382 {
2383 int nbytes, nchars;
ff0dacd7
KH		 2384 int id;
2385
781d7a48
KH		 2386 src = src_base;
2387 consumed_chars = consumed_chars_base;
ff0dacd7 2388 c = emacs_mule_char (coding, src, &nbytes, &nchars, &id);
df7492f9
KH		 2389 if (c < 0)
2390 {
2391 if (c == -2)
2392 break;
2393 goto invalid_code;
2394 }
ff0dacd7
KH		 2395 if (last_id != id)
2396 {
2397 if (last_id != charset_ascii)
69a80ea3 2398 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
ff0dacd7
KH		 2399 last_id = id;
2400 last_offset = char_offset;
2401 }
df7492f9 2402 *charbuf++ = c;
781d7a48
KH		 2403 src += nbytes;
2404 consumed_chars += nchars;
df7492f9
KH		 2405 char_offset++;
2406 }
4d41e8b7
KH		 2407 else
2408 goto invalid_code;
df7492f9
KH		 2409 continue;
2410
2411 invalid_code:
2412 src = src_base;
2413 consumed_chars = consumed_chars_base;
2414 ONE_MORE_BYTE (c);
2415 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 2416 char_offset++;
df7492f9
KH		 2417 coding->errors++;
2418 }
2419
2420 no_more_source:
ff0dacd7 2421 if (last_id != charset_ascii)
69a80ea3 2422 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 2423 coding->consumed_char += consumed_chars_base;
2424 coding->consumed = src_base - coding->source;
2425 coding->charbuf_used = charbuf - coding->charbuf;
2426}
2427
2428
2429#define EMACS_MULE_LEADING_CODES(id, codes) \
2430 do { \
2431 if (id < 0xA0) \
2432 codes[0] = id, codes[1] = 0; \
2433 else if (id < 0xE0) \
2434 codes[0] = 0x9A, codes[1] = id; \
2435 else if (id < 0xF0) \
2436 codes[0] = 0x9B, codes[1] = id; \
2437 else if (id < 0xF5) \
2438 codes[0] = 0x9C, codes[1] = id; \
2439 else \
2440 codes[0] = 0x9D, codes[1] = id; \
2441 } while (0);
2442
aa72b389 2443
df7492f9
KH		 2444static int
2445encode_coding_emacs_mule (coding)
2446 struct coding_system *coding;
2447{
2448 int multibytep = coding->dst_multibyte;
2449 int *charbuf = coding->charbuf;
2450 int *charbuf_end = charbuf + coding->charbuf_used;
2451 unsigned char *dst = coding->destination + coding->produced;
2452 unsigned char *dst_end = coding->destination + coding->dst_bytes;
2453 int safe_room = 8;
df7492f9 2454 int produced_chars = 0;
24a73b0a 2455 Lisp_Object attrs, charset_list;
df7492f9 2456 int c;
ff0dacd7 2457 int preferred_charset_id = -1;
df7492f9 2458
24a73b0a 2459 CODING_GET_INFO (coding, attrs, charset_list);
eccb6815
KH		 2460 if (! EQ (charset_list, Vemacs_mule_charset_list))
2461 {
2462 CODING_ATTR_CHARSET_LIST (attrs)
2463 = charset_list = Vemacs_mule_charset_list;
2464 }
df7492f9
KH		 2465
2466 while (charbuf < charbuf_end)
2467 {
2468 ASSURE_DESTINATION (safe_room);
2469 c = *charbuf++;
ff0dacd7
KH		 2470
2471 if (c < 0)
2472 {
2473 /* Handle an annotation. */
2474 switch (*charbuf)
2475 {
2476 case CODING_ANNOTATE_COMPOSITION_MASK:
2477 /* Not yet implemented. */
2478 break;
2479 case CODING_ANNOTATE_CHARSET_MASK:
2480 preferred_charset_id = charbuf[3];
2481 if (preferred_charset_id >= 0
2482 && NILP (Fmemq (make_number (preferred_charset_id),
2483 charset_list)))
2484 preferred_charset_id = -1;
2485 break;
2486 default:
2487 abort ();
2488 }
2489 charbuf += -c - 1;
2490 continue;
2491 }
2492
df7492f9
KH		 2493 if (ASCII_CHAR_P (c))
2494 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 2495 else if (CHAR_BYTE8_P (c))
2496 {
2497 c = CHAR_TO_BYTE8 (c);
2498 EMIT_ONE_BYTE (c);
2499 }
df7492f9 2500 else
aa72b389 2501 {
df7492f9
KH		 2502 struct charset *charset;
2503 unsigned code;
2504 int dimension;
2505 int emacs_mule_id;
2506 unsigned char leading_codes[2];
2507
ff0dacd7
KH		 2508 if (preferred_charset_id >= 0)
2509 {
2510 charset = CHARSET_FROM_ID (preferred_charset_id);
905ca9d2
KH		 2511 if (CHAR_CHARSET_P (c, charset))
2512 code = ENCODE_CHAR (charset, c);
2513 else
2514 charset = char_charset (c, charset_list, &code);
ff0dacd7
KH		 2515 }
2516 else
2517 charset = char_charset (c, charset_list, &code);
df7492f9
KH		 2518 if (! charset)
2519 {
2520 c = coding->default_char;
2521 if (ASCII_CHAR_P (c))
2522 {
2523 EMIT_ONE_ASCII_BYTE (c);
2524 continue;
2525 }
2526 charset = char_charset (c, charset_list, &code);
2527 }
2528 dimension = CHARSET_DIMENSION (charset);
2529 emacs_mule_id = CHARSET_EMACS_MULE_ID (charset);
2530 EMACS_MULE_LEADING_CODES (emacs_mule_id, leading_codes);
2531 EMIT_ONE_BYTE (leading_codes[0]);
2532 if (leading_codes[1])
2533 EMIT_ONE_BYTE (leading_codes[1]);
2534 if (dimension == 1)
1fa663f9 2535 EMIT_ONE_BYTE (code | 0x80);
aa72b389 2536 else
df7492f9 2537 {
1fa663f9 2538 code |= 0x8080;
df7492f9
KH		 2539 EMIT_ONE_BYTE (code >> 8);
2540 EMIT_ONE_BYTE (code & 0xFF);
2541 }
aa72b389 2542 }
aa72b389 2543 }
065e3595 2544 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 2545 coding->produced_char += produced_chars;
2546 coding->produced = dst - coding->destination;
2547 return 0;
aa72b389 2548}
b73bfc1c 2549
4ed46869 2550\f
df7492f9 2551/*** 7. ISO2022 handlers ***/
4ed46869
KH		 2552
2553/* The following note describes the coding system ISO2022 briefly.
39787efd 2554 Since the intention of this note is to help understand the
5a936b46 2555 functions in this file, some parts are NOT ACCURATE or are OVERLY
39787efd 2556 SIMPLIFIED. For thorough understanding, please refer to the
5a936b46 2557 original document of ISO2022. This is equivalent to the standard
cfb43547 2558 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
4ed46869
KH		 2559
2560 ISO2022 provides many mechanisms to encode several character sets
cfb43547 2561 in 7-bit and 8-bit environments. For 7-bit environments, all text
39787efd
KH		 2562 is encoded using bytes less than 128. This may make the encoded
2563 text a little bit longer, but the text passes more easily through
cfb43547 2564 several types of gateway, some of which strip off the MSB (Most
8ca3766a 2565 Significant Bit).
b73bfc1c 2566
cfb43547
DL		 2567 There are two kinds of character sets: control character sets and
2568 graphic character sets. The former contain control characters such
4ed46869 2569 as `newline' and `escape' to provide control functions (control
39787efd 2570 functions are also provided by escape sequences). The latter
cfb43547 2571 contain graphic characters such as 'A' and '-'. Emacs recognizes
4ed46869
KH		 2572 two control character sets and many graphic character sets.
2573
2574 Graphic character sets are classified into one of the following
39787efd
KH		 2575 four classes, according to the number of bytes (DIMENSION) and
2576 number of characters in one dimension (CHARS) of the set:
2577 - DIMENSION1_CHARS94
2578 - DIMENSION1_CHARS96
2579 - DIMENSION2_CHARS94
2580 - DIMENSION2_CHARS96
2581
2582 In addition, each character set is assigned an identification tag,
cfb43547 2583 unique for each set, called the "final character" (denoted as <F>
39787efd
KH		 2584 hereafter). The <F> of each character set is decided by ECMA(*)
2585 when it is registered in ISO. The code range of <F> is 0x30..0x7F
2586 (0x30..0x3F are for private use only).
4ed46869
KH		 2587
2588 Note (*): ECMA = European Computer Manufacturers Association
2589
cfb43547 2590 Here are examples of graphic character sets [NAME(<F>)]:
4ed46869
KH		 2591 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
2592 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
2593 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
2594 o DIMENSION2_CHARS96 -- none for the moment
2595
39787efd 2596 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
4ed46869
KH		 2597 C0 [0x00..0x1F] -- control character plane 0
2598 GL [0x20..0x7F] -- graphic character plane 0
2599 C1 [0x80..0x9F] -- control character plane 1
2600 GR [0xA0..0xFF] -- graphic character plane 1
2601
2602 A control character set is directly designated and invoked to C0 or
39787efd
KH		 2603 C1 by an escape sequence. The most common case is that:
2604 - ISO646's control character set is designated/invoked to C0, and
2605 - ISO6429's control character set is designated/invoked to C1,
2606 and usually these designations/invocations are omitted in encoded
2607 text. In a 7-bit environment, only C0 can be used, and a control
2608 character for C1 is encoded by an appropriate escape sequence to
2609 fit into the environment. All control characters for C1 are
2610 defined to have corresponding escape sequences.
4ed46869
KH		 2611
2612 A graphic character set is at first designated to one of four
2613 graphic registers (G0 through G3), then these graphic registers are
2614 invoked to GL or GR. These designations and invocations can be
2615 done independently. The most common case is that G0 is invoked to
39787efd
KH		 2616 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
2617 these invocations and designations are omitted in encoded text.
2618 In a 7-bit environment, only GL can be used.
4ed46869 2619
39787efd
KH		 2620 When a graphic character set of CHARS94 is invoked to GL, codes
2621 0x20 and 0x7F of the GL area work as control characters SPACE and
2622 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
2623 be used.
4ed46869
KH		 2624
2625 There are two ways of invocation: locking-shift and single-shift.
2626 With locking-shift, the invocation lasts until the next different
39787efd
KH		 2627 invocation, whereas with single-shift, the invocation affects the
2628 following character only and doesn't affect the locking-shift
2629 state. Invocations are done by the following control characters or
2630 escape sequences:
4ed46869
KH		 2631
2632 ----------------------------------------------------------------------
39787efd 2633 abbrev function cntrl escape seq description
4ed46869 2634 ----------------------------------------------------------------------
39787efd
KH		 2635 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
2636 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
2637 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
2638 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
2639 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
2640 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
2641 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
2642 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
2643 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
4ed46869 2644 ----------------------------------------------------------------------
39787efd
KH		 2645 (*) These are not used by any known coding system.
2646
2647 Control characters for these functions are defined by macros
2648 ISO_CODE_XXX in `coding.h'.
4ed46869 2649
39787efd 2650 Designations are done by the following escape sequences:
4ed46869
KH		 2651 ----------------------------------------------------------------------
2652 escape sequence description
2653 ----------------------------------------------------------------------
2654 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
2655 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
2656 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
2657 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
2658 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
2659 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
2660 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
2661 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
2662 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
2663 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
2664 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
2665 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
2666 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
2667 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
2668 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
2669 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
2670 ----------------------------------------------------------------------
2671
2672 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
39787efd 2673 of dimension 1, chars 94, and final character <F>, etc...
4ed46869
KH		 2674
2675 Note (*): Although these designations are not allowed in ISO2022,
2676 Emacs accepts them on decoding, and produces them on encoding
39787efd 2677 CHARS96 character sets in a coding system which is characterized as
4ed46869
KH		 2678 7-bit environment, non-locking-shift, and non-single-shift.
2679
2680 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
df7492f9 2681 '(' must be omitted. We refer to this as "short-form" hereafter.
4ed46869 2682
cfb43547 2683 Now you may notice that there are a lot of ways of encoding the
39787efd
KH		 2684 same multilingual text in ISO2022. Actually, there exist many
2685 coding systems such as Compound Text (used in X11's inter client
8ca3766a
DL		 2686 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
2687 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
4ed46869
KH		 2688 localized platforms), and all of these are variants of ISO2022.
2689
2690 In addition to the above, Emacs handles two more kinds of escape
2691 sequences: ISO6429's direction specification and Emacs' private
2692 sequence for specifying character composition.
2693
39787efd 2694 ISO6429's direction specification takes the following form:
4ed46869
KH		 2695 o CSI ']' -- end of the current direction
2696 o CSI '0' ']' -- end of the current direction
2697 o CSI '1' ']' -- start of left-to-right text
2698 o CSI '2' ']' -- start of right-to-left text
2699 The control character CSI (0x9B: control sequence introducer) is
39787efd
KH		 2700 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
2701
2702 Character composition specification takes the following form:
ec6d2bb8
KH		 2703 o ESC '0' -- start relative composition
2704 o ESC '1' -- end composition
2705 o ESC '2' -- start rule-base composition (*)
2706 o ESC '3' -- start relative composition with alternate chars (**)
2707 o ESC '4' -- start rule-base composition with alternate chars (**)
b73bfc1c 2708 Since these are not standard escape sequences of any ISO standard,
cfb43547 2709 the use of them with these meanings is restricted to Emacs only.
ec6d2bb8 2710
5a936b46
DL		 2711 (*) This form is used only in Emacs 20.7 and older versions,
2712 but newer versions can safely decode it.
cfb43547 2713 (**) This form is used only in Emacs 21.1 and newer versions,
5a936b46 2714 and older versions can't decode it.
ec6d2bb8 2715
cfb43547 2716 Here's a list of example usages of these composition escape
b73bfc1c 2717 sequences (categorized by `enum composition_method').
ec6d2bb8 2718
b73bfc1c 2719 COMPOSITION_RELATIVE:
ec6d2bb8 2720 ESC 0 CHAR [ CHAR ] ESC 1
8ca3766a 2721 COMPOSITION_WITH_RULE:
ec6d2bb8 2722 ESC 2 CHAR [ RULE CHAR ] ESC 1
b73bfc1c 2723 COMPOSITION_WITH_ALTCHARS:
ec6d2bb8 2724 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
b73bfc1c 2725 COMPOSITION_WITH_RULE_ALTCHARS:
ec6d2bb8 2726 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
4ed46869
KH		 2727
2728enum iso_code_class_type iso_code_class[256];
2729
df7492f9
KH		 2730#define SAFE_CHARSET_P(coding, id) \
2731 ((id) <= (coding)->max_charset_id \
2732 && (coding)->safe_charsets[id] >= 0)
2733
2734
2735#define SHIFT_OUT_OK(category) \
2736 (CODING_ISO_INITIAL (&coding_categories[category], 1) >= 0)
2737
2738static void
f0064e1f
DL		 2739setup_iso_safe_charsets (attrs)
2740 Lisp_Object attrs;
df7492f9
KH		 2741{
2742 Lisp_Object charset_list, safe_charsets;
2743 Lisp_Object request;
2744 Lisp_Object reg_usage;
2745 Lisp_Object tail;
2746 int reg94, reg96;
2747 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
2748 int max_charset_id;
2749
2750 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2751 if ((flags & CODING_ISO_FLAG_FULL_SUPPORT)
2752 && ! EQ (charset_list, Viso_2022_charset_list))
2753 {
2754 CODING_ATTR_CHARSET_LIST (attrs)
2755 = charset_list = Viso_2022_charset_list;
2756 ASET (attrs, coding_attr_safe_charsets, Qnil);
2757 }
2758
2759 if (STRINGP (AREF (attrs, coding_attr_safe_charsets)))
2760 return;
2761
2762 max_charset_id = 0;
2763 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2764 {
2765 int id = XINT (XCAR (tail));
2766 if (max_charset_id < id)
2767 max_charset_id = id;
2768 }
d46c5b12 2769
df7492f9
KH		 2770 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
2771 make_number (255));
2772 request = AREF (attrs, coding_attr_iso_request);
2773 reg_usage = AREF (attrs, coding_attr_iso_usage);
2774 reg94 = XINT (XCAR (reg_usage));
2775 reg96 = XINT (XCDR (reg_usage));
2776
2777 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2778 {
2779 Lisp_Object id;
2780 Lisp_Object reg;
2781 struct charset *charset;
2782
2783 id = XCAR (tail);
2784 charset = CHARSET_FROM_ID (XINT (id));
bf16eb23 2785 reg = Fcdr (Fassq (id, request));
df7492f9 2786 if (! NILP (reg))
8f924df7 2787 SSET (safe_charsets, XINT (id), XINT (reg));
df7492f9
KH		 2788 else if (charset->iso_chars_96)
2789 {
2790 if (reg96 < 4)
8f924df7 2791 SSET (safe_charsets, XINT (id), reg96);
df7492f9
KH		 2792 }
2793 else
2794 {
2795 if (reg94 < 4)
8f924df7 2796 SSET (safe_charsets, XINT (id), reg94);
df7492f9
KH		 2797 }
2798 }
2799 ASET (attrs, coding_attr_safe_charsets, safe_charsets);
2800}
d46c5b12 2801
b6871cc7 2802
4ed46869 2803/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 2804 Check if a text is encoded in one of ISO-2022 based codig systems.
2805 If it is, return 1, else return 0. */
4ed46869 2806
0a28aafb 2807static int
ff0dacd7 2808detect_coding_iso_2022 (coding, detect_info)
df7492f9 2809 struct coding_system *coding;
ff0dacd7 2810 struct coding_detection_info *detect_info;
4ed46869 2811{
8f924df7
KH		 2812 const unsigned char *src = coding->source, *src_base = src;
2813 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 2814 int multibytep = coding->src_multibyte;
ff0dacd7 2815 int single_shifting = 0;
df7492f9
KH		 2816 int id;
2817 int c, c1;
2818 int consumed_chars = 0;
2819 int i;
ff0dacd7
KH		 2820 int rejected = 0;
2821 int found = 0;
cee53ed4 2822 int composition_count = -1;
ff0dacd7
KH		 2823
2824 detect_info->checked |= CATEGORY_MASK_ISO;
df7492f9
KH		 2825
2826 for (i = coding_category_iso_7; i <= coding_category_iso_8_else; i++)
2827 {
2828 struct coding_system *this = &(coding_categories[i]);
2829 Lisp_Object attrs, val;
2830
c6b278e7
KH		 2831 if (this->id < 0)
2832 continue;
df7492f9
KH		 2833 attrs = CODING_ID_ATTRS (this->id);
2834 if (CODING_ISO_FLAGS (this) & CODING_ISO_FLAG_FULL_SUPPORT
2835 && ! EQ (CODING_ATTR_SAFE_CHARSETS (attrs), Viso_2022_charset_list))
2836 setup_iso_safe_charsets (attrs);
2837 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 2838 this->max_charset_id = SCHARS (val) - 1;
2839 this->safe_charsets = (char *) SDATA (val);
df7492f9
KH		 2840 }
2841
2842 /* A coding system of this category is always ASCII compatible. */
2843 src += coding->head_ascii;
3f003981 2844
ff0dacd7 2845 while (rejected != CATEGORY_MASK_ISO)
4ed46869 2846 {
065e3595 2847 src_base = src;
df7492f9 2848 ONE_MORE_BYTE (c);
4ed46869
KH		 2849 switch (c)
2850 {
2851 case ISO_CODE_ESC:
74383408
KH		 2852 if (inhibit_iso_escape_detection)
2853 break;
f46869e4 2854 single_shifting = 0;
df7492f9 2855 ONE_MORE_BYTE (c);
d46c5b12 2856 if (c >= '(' && c <= '/')
4ed46869 2857 {
bf9cdd4e 2858 /* Designation sequence for a charset of dimension 1. */
df7492f9 2859 ONE_MORE_BYTE (c1);
d46c5b12 2860 if (c1 < ' ' || c1 >= 0x80
df7492f9 2861 || (id = iso_charset_table[0][c >= ','][c1]) < 0)
d46c5b12
KH		 2862 /* Invalid designation sequence. Just ignore. */
2863 break;
bf9cdd4e
KH		 2864 }
2865 else if (c == '$')
2866 {
2867 /* Designation sequence for a charset of dimension 2. */
df7492f9 2868 ONE_MORE_BYTE (c);
bf9cdd4e
KH		 2869 if (c >= '@' && c <= 'B')
2870 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
ff0dacd7 2871 id = iso_charset_table[1][0][c];
bf9cdd4e 2872 else if (c >= '(' && c <= '/')
bcf26d6a 2873 {
df7492f9 2874 ONE_MORE_BYTE (c1);
d46c5b12 2875 if (c1 < ' ' || c1 >= 0x80
df7492f9 2876 || (id = iso_charset_table[1][c >= ','][c1]) < 0)
d46c5b12
KH		 2877 /* Invalid designation sequence. Just ignore. */
2878 break;
bcf26d6a 2879 }
bf9cdd4e 2880 else
ff0dacd7 2881 /* Invalid designation sequence. Just ignore it. */
d46c5b12
KH		 2882 break;
2883 }
ae9ff118 2884 else if (c == 'N' || c == 'O')
d46c5b12 2885 {
ae9ff118 2886 /* ESC <Fe> for SS2 or SS3. */
ff0dacd7
KH		 2887 single_shifting = 1;
2888 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
d46c5b12 2889 break;
4ed46869 2890 }
cee53ed4
KH		 2891 else if (c == '1')
2892 {
2893 /* End of composition. */
2894 if (composition_count < 0
2895 || composition_count > MAX_COMPOSITION_COMPONENTS)
2896 /* Invalid */
2897 break;
2898 composition_count = -1;
2899 found |= CATEGORY_MASK_ISO;
2900 }
ec6d2bb8
KH		 2901 else if (c >= '0' && c <= '4')
2902 {
2903 /* ESC <Fp> for start/end composition. */
cee53ed4 2904 composition_count = 0;
ec6d2bb8
KH		 2905 break;
2906 }
bf9cdd4e 2907 else
df7492f9 2908 {
ff0dacd7 2909 /* Invalid escape sequence. Just ignore it. */
df7492f9
KH		 2910 break;
2911 }
d46c5b12
KH		 2912
2913 /* We found a valid designation sequence for CHARSET. */
ff0dacd7 2914 rejected |= CATEGORY_MASK_ISO_8BIT;
df7492f9
KH		 2915 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7],
2916 id))
ff0dacd7 2917 found |= CATEGORY_MASK_ISO_7;
d46c5b12 2918 else
ff0dacd7 2919 rejected |= CATEGORY_MASK_ISO_7;
df7492f9
KH		 2920 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_tight],
2921 id))
ff0dacd7 2922 found |= CATEGORY_MASK_ISO_7_TIGHT;
d46c5b12 2923 else
ff0dacd7 2924 rejected |= CATEGORY_MASK_ISO_7_TIGHT;
df7492f9
KH		 2925 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_else],
2926 id))
ff0dacd7 2927 found |= CATEGORY_MASK_ISO_7_ELSE;
ae9ff118 2928 else
ff0dacd7 2929 rejected |= CATEGORY_MASK_ISO_7_ELSE;
df7492f9
KH		 2930 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_8_else],
2931 id))
ff0dacd7 2932 found |= CATEGORY_MASK_ISO_8_ELSE;
ae9ff118 2933 else
ff0dacd7 2934 rejected |= CATEGORY_MASK_ISO_8_ELSE;
4ed46869
KH		 2935 break;
2936
4ed46869 2937 case ISO_CODE_SO:
d46c5b12 2938 case ISO_CODE_SI:
ff0dacd7 2939 /* Locking shift out/in. */
74383408
KH		 2940 if (inhibit_iso_escape_detection)
2941 break;
f46869e4 2942 single_shifting = 0;
ff0dacd7 2943 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
d46c5b12
KH		 2944 break;
2945
4ed46869 2946 case ISO_CODE_CSI:
ff0dacd7 2947 /* Control sequence introducer. */
f46869e4 2948 single_shifting = 0;
ff0dacd7
KH		 2949 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2950 found |= CATEGORY_MASK_ISO_8_ELSE;
2951 goto check_extra_latin;
2952
4ed46869
KH		 2953 case ISO_CODE_SS2:
2954 case ISO_CODE_SS3:
ff0dacd7
KH		 2955 /* Single shift. */
2956 if (inhibit_iso_escape_detection)
2957 break;
75e2a253 2958 single_shifting = 0;
ff0dacd7
KH		 2959 rejected |= CATEGORY_MASK_ISO_7BIT;
2960 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
2961 & CODING_ISO_FLAG_SINGLE_SHIFT)
75e2a253 2962 found |= CATEGORY_MASK_ISO_8_1, single_shifting = 1;
ff0dacd7
KH		 2963 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
2964 & CODING_ISO_FLAG_SINGLE_SHIFT)
75e2a253
KH		 2965 found |= CATEGORY_MASK_ISO_8_2, single_shifting = 1;
2966 if (single_shifting)
2967 break;
ff0dacd7 2968 goto check_extra_latin;
4ed46869
KH		 2969
2970 default:
065e3595
KH		 2971 if (c < 0)
2972 continue;
4ed46869 2973 if (c < 0x80)
f46869e4 2974 {
cee53ed4
KH		 2975 if (composition_count >= 0)
2976 composition_count++;
f46869e4
KH		 2977 single_shifting = 0;
2978 break;
2979 }
ff0dacd7 2980 if (c >= 0xA0)
c4825358 2981 {
ff0dacd7
KH		 2982 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2983 found |= CATEGORY_MASK_ISO_8_1;
f46869e4 2984 /* Check the length of succeeding codes of the range
ff0dacd7
KH		 2985 0xA0..0FF. If the byte length is even, we include
2986 CATEGORY_MASK_ISO_8_2 in `found'. We can check this
2987 only when we are not single shifting. */
2988 if (! single_shifting
2989 && ! (rejected & CATEGORY_MASK_ISO_8_2))
f46869e4 2990 {
e17de821 2991 int i = 1;
b73bfc1c
KH		 2992 while (src < src_end)
2993 {
df7492f9 2994 ONE_MORE_BYTE (c);
b73bfc1c
KH		 2995 if (c < 0xA0)
2996 break;
2997 i++;
2998 }
2999
3000 if (i & 1 && src < src_end)
cee53ed4
KH		 3001 {
3002 rejected |= CATEGORY_MASK_ISO_8_2;
3003 if (composition_count >= 0)
3004 composition_count += i;
3005 }
f46869e4 3006 else
cee53ed4
KH		 3007 {
3008 found |= CATEGORY_MASK_ISO_8_2;
3009 if (composition_count >= 0)
3010 composition_count += i / 2;
3011 }
f46869e4 3012 }
ff0dacd7 3013 break;
4ed46869 3014 }
ff0dacd7
KH		 3015 check_extra_latin:
3016 single_shifting = 0;
3017 if (! VECTORP (Vlatin_extra_code_table)
3018 || NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
3019 {
3020 rejected = CATEGORY_MASK_ISO;
3021 break;
3022 }
3023 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
3024 & CODING_ISO_FLAG_LATIN_EXTRA)
3025 found |= CATEGORY_MASK_ISO_8_1;
3026 else
3027 rejected |= CATEGORY_MASK_ISO_8_1;
75e2a253 3028 rejected |= CATEGORY_MASK_ISO_8_2;
4ed46869
KH		 3029 }
3030 }
ff0dacd7
KH		 3031 detect_info->rejected |= CATEGORY_MASK_ISO;
3032 return 0;
4ed46869 3033
df7492f9 3034 no_more_source:
ff0dacd7
KH		 3035 detect_info->rejected |= rejected;
3036 detect_info->found |= (found & ~rejected);
df7492f9 3037 return 1;
4ed46869 3038}
ec6d2bb8 3039
4ed46869 3040
134b9549
KH		 3041/* Set designation state into CODING. Set CHARS_96 to -1 if the
3042 escape sequence should be kept. */
df7492f9
KH		 3043#define DECODE_DESIGNATION(reg, dim, chars_96, final) \
3044 do { \
3045 int id, prev; \
3046 \
3047 if (final < '0' || final >= 128 \
3048 || ((id = ISO_CHARSET_TABLE (dim, chars_96, final)) < 0) \
3049 || !SAFE_CHARSET_P (coding, id)) \
3050 { \
3051 CODING_ISO_DESIGNATION (coding, reg) = -2; \
134b9549
KH		 3052 chars_96 = -1; \
3053 break; \
df7492f9
KH		 3054 } \
3055 prev = CODING_ISO_DESIGNATION (coding, reg); \
bf16eb23
KH		 3056 if (id == charset_jisx0201_roman) \
3057 { \
3058 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
3059 id = charset_ascii; \
3060 } \
3061 else if (id == charset_jisx0208_1978) \
3062 { \
3063 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
3064 id = charset_jisx0208; \
3065 } \
df7492f9
KH		 3066 CODING_ISO_DESIGNATION (coding, reg) = id; \
3067 /* If there was an invalid designation to REG previously, and this \
3068 designation is ASCII to REG, we should keep this designation \
3069 sequence. */ \
3070 if (prev == -2 && id == charset_ascii) \
134b9549 3071 chars_96 = -1; \
4ed46869
KH		 3072 } while (0)
3073
d46c5b12 3074
df7492f9
KH		 3075#define MAYBE_FINISH_COMPOSITION() \
3076 do { \
3077 int i; \
3078 if (composition_state == COMPOSING_NO) \
3079 break; \
3080 /* It is assured that we have enough room for producing \
3081 characters stored in the table `components'. */ \
3082 if (charbuf + component_idx > charbuf_end) \
3083 goto no_more_source; \
3084 composition_state = COMPOSING_NO; \
3085 if (method == COMPOSITION_RELATIVE \
3086 || method == COMPOSITION_WITH_ALTCHARS) \
3087 { \
3088 for (i = 0; i < component_idx; i++) \
3089 *charbuf++ = components[i]; \
3090 char_offset += component_idx; \
3091 } \
3092 else \
3093 { \
3094 for (i = 0; i < component_idx; i += 2) \
3095 *charbuf++ = components[i]; \
3096 char_offset += (component_idx / 2) + 1; \
3097 } \
3098 } while (0)
3099
d46c5b12 3100
aa72b389
KH		 3101/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
3102 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
3103 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
df7492f9
KH		 3104 ESC 3 : altchar composition : ESC 3 CHAR ... ESC 0 CHAR ... ESC 1
3105 ESC 4 : alt&rule composition : ESC 4 CHAR RULE ... CHAR ESC 0 CHAR ... ESC 1
aa72b389 3106 */
ec6d2bb8 3107
df7492f9
KH		 3108#define DECODE_COMPOSITION_START(c1) \
3109 do { \
3110 if (c1 == '0' \
781d7a48 3111 && composition_state == COMPOSING_COMPONENT_RULE) \
df7492f9
KH		 3112 { \
3113 component_len = component_idx; \
3114 composition_state = COMPOSING_CHAR; \
3115 } \
3116 else \
3117 { \
8f924df7 3118 const unsigned char *p; \
df7492f9
KH		 3119 \
3120 MAYBE_FINISH_COMPOSITION (); \
3121 if (charbuf + MAX_COMPOSITION_COMPONENTS > charbuf_end) \
3122 goto no_more_source; \
3123 for (p = src; p < src_end - 1; p++) \
3124 if (*p == ISO_CODE_ESC && p[1] == '1') \
3125 break; \
3126 if (p == src_end - 1) \
3127 { \
cee53ed4
KH		 3128 if (coding->mode & CODING_MODE_LAST_BLOCK) \
3129 goto invalid_code; \
9286b333
KH		 3130 /* The current composition doesn't end in the current \
3131 source. */ \
3132 record_conversion_result \
3133 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
df7492f9
KH		 3134 goto no_more_source; \
3135 } \
3136 \
3137 /* This is surely the start of a composition. */ \
3138 method = (c1 == '0' ? COMPOSITION_RELATIVE \
3139 : c1 == '2' ? COMPOSITION_WITH_RULE \
3140 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
3141 : COMPOSITION_WITH_RULE_ALTCHARS); \
3142 composition_state = (c1 <= '2' ? COMPOSING_CHAR \
3143 : COMPOSING_COMPONENT_CHAR); \
3144 component_idx = component_len = 0; \
3145 } \
ec6d2bb8
KH		 3146 } while (0)
3147
ec6d2bb8 3148
df7492f9
KH		 3149/* Handle compositoin end sequence ESC 1. */
3150
3151#define DECODE_COMPOSITION_END() \
ec6d2bb8 3152 do { \
df7492f9
KH		 3153 int nchars = (component_len > 0 ? component_idx - component_len \
3154 : method == COMPOSITION_RELATIVE ? component_idx \
3155 : (component_idx + 1) / 2); \
3156 int i; \
3157 int *saved_charbuf = charbuf; \
3158 \
69a80ea3 3159 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
df7492f9 3160 if (method != COMPOSITION_RELATIVE) \
ec6d2bb8 3161 { \
df7492f9
KH		 3162 if (component_len == 0) \
3163 for (i = 0; i < component_idx; i++) \
3164 *charbuf++ = components[i]; \
3165 else \
3166 for (i = 0; i < component_len; i++) \
3167 *charbuf++ = components[i]; \
3168 *saved_charbuf = saved_charbuf - charbuf; \
ec6d2bb8 3169 } \
df7492f9
KH		 3170 if (method == COMPOSITION_WITH_RULE) \
3171 for (i = 0; i < component_idx; i += 2, char_offset++) \
3172 *charbuf++ = components[i]; \
ec6d2bb8 3173 else \
df7492f9
KH		 3174 for (i = component_len; i < component_idx; i++, char_offset++) \
3175 *charbuf++ = components[i]; \
3176 coding->annotated = 1; \
3177 composition_state = COMPOSING_NO; \
ec6d2bb8
KH		 3178 } while (0)
3179
df7492f9 3180
ec6d2bb8
KH		 3181/* Decode a composition rule from the byte C1 (and maybe one more byte
3182 from SRC) and store one encoded composition rule in
3183 coding->cmp_data. */
3184
3185#define DECODE_COMPOSITION_RULE(c1) \
3186 do { \
ec6d2bb8
KH		 3187 (c1) -= 32; \
3188 if (c1 < 81) /* old format (before ver.21) */ \
3189 { \
3190 int gref = (c1) / 9; \
3191 int nref = (c1) % 9; \
3192 if (gref == 4) gref = 10; \
3193 if (nref == 4) nref = 10; \
df7492f9 3194 c1 = COMPOSITION_ENCODE_RULE (gref, nref); \
ec6d2bb8 3195 } \
b73bfc1c 3196 else if (c1 < 93) /* new format (after ver.21) */ \
ec6d2bb8
KH		 3197 { \
3198 ONE_MORE_BYTE (c2); \
df7492f9 3199 c1 = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
ec6d2bb8 3200 } \
df7492f9
KH		 3201 else \
3202 c1 = 0; \
ec6d2bb8 3203 } while (0)
88993dfd 3204
d46c5b12 3205
4ed46869
KH		 3206/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3207
b73bfc1c 3208static void
df7492f9 3209decode_coding_iso_2022 (coding)
4ed46869 3210 struct coding_system *coding;
4ed46869 3211{
8f924df7
KH		 3212 const unsigned char *src = coding->source + coding->consumed;
3213 const unsigned char *src_end = coding->source + coding->src_bytes;
3214 const unsigned char *src_base;
69a80ea3 3215 int *charbuf = coding->charbuf + coding->charbuf_used;
ff0dacd7 3216 int *charbuf_end
69a80ea3 3217 = coding->charbuf + coding->charbuf_size - 4 - MAX_ANNOTATION_LENGTH;
df7492f9 3218 int consumed_chars = 0, consumed_chars_base;
df7492f9 3219 int multibytep = coding->src_multibyte;
4ed46869 3220 /* Charsets invoked to graphic plane 0 and 1 respectively. */
df7492f9
KH		 3221 int charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3222 int charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
134b9549 3223 int charset_id_2, charset_id_3;
df7492f9
KH		 3224 struct charset *charset;
3225 int c;
3226 /* For handling composition sequence. */
3227#define COMPOSING_NO 0
3228#define COMPOSING_CHAR 1
3229#define COMPOSING_RULE 2
3230#define COMPOSING_COMPONENT_CHAR 3
3231#define COMPOSING_COMPONENT_RULE 4
3232
3233 int composition_state = COMPOSING_NO;
3234 enum composition_method method;
3235 int components[MAX_COMPOSITION_COMPONENTS * 2 + 1];
3236 int component_idx;
3237 int component_len;
24a73b0a 3238 Lisp_Object attrs, charset_list;
ff0dacd7
KH		 3239 int char_offset = coding->produced_char;
3240 int last_offset = char_offset;
3241 int last_id = charset_ascii;
119852e7
KH		 3242 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
3243 int byte_after_cr = -1;
df7492f9 3244
24a73b0a 3245 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 3246 setup_iso_safe_charsets (attrs);
287c57d7
KH		 3247 /* Charset list may have been changed. */
3248 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
3249 coding->safe_charsets = (char *) SDATA (CODING_ATTR_SAFE_CHARSETS(attrs));
b73bfc1c
KH		 3250
3251 while (1)
4ed46869 3252 {
463f5630 3253 int c1, c2;
b73bfc1c
KH		 3254
3255 src_base = src;
df7492f9
KH		 3256 consumed_chars_base = consumed_chars;
3257
3258 if (charbuf >= charbuf_end)
b71f6f73
KH		 3259 {
3260 if (byte_after_cr >= 0)
3261 src_base--;
3262 break;
3263 }
df7492f9 3264
119852e7
KH		 3265 if (byte_after_cr >= 0)
3266 c1 = byte_after_cr, byte_after_cr = -1;
3267 else
3268 ONE_MORE_BYTE (c1);
065e3595
KH		 3269 if (c1 < 0)
3270 goto invalid_code;
4ed46869 3271
98725083 3272 /* We produce at most one character. */
4ed46869
KH		 3273 switch (iso_code_class [c1])
3274 {
3275 case ISO_0x20_or_0x7F:
df7492f9 3276 if (composition_state != COMPOSING_NO)
ec6d2bb8 3277 {
df7492f9
KH		 3278 if (composition_state == COMPOSING_RULE
3279 || composition_state == COMPOSING_COMPONENT_RULE)
3280 {
cee53ed4
KH		 3281 if (component_idx < MAX_COMPOSITION_COMPONENTS * 2 + 1)
3282 {
3283 DECODE_COMPOSITION_RULE (c1);
3284 components[component_idx++] = c1;
3285 composition_state--;
3286 continue;
3287 }
3288 /* Too long composition. */
3289 MAYBE_FINISH_COMPOSITION ();
df7492f9 3290 }
4ed46869 3291 }
df7492f9
KH		 3292 if (charset_id_0 < 0
3293 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_0)))
781d7a48
KH		 3294 /* This is SPACE or DEL. */
3295 charset = CHARSET_FROM_ID (charset_ascii);
3296 else
3297 charset = CHARSET_FROM_ID (charset_id_0);
3298 break;
4ed46869
KH		 3299
3300 case ISO_graphic_plane_0:
781d7a48 3301 if (composition_state != COMPOSING_NO)
b73bfc1c 3302 {
781d7a48
KH		 3303 if (composition_state == COMPOSING_RULE
3304 || composition_state == COMPOSING_COMPONENT_RULE)
3305 {
cee53ed4
KH		 3306 if (component_idx < MAX_COMPOSITION_COMPONENTS * 2 + 1)
3307 {
3308 DECODE_COMPOSITION_RULE (c1);
3309 components[component_idx++] = c1;
3310 composition_state--;
3311 continue;
3312 }
3313 MAYBE_FINISH_COMPOSITION ();
781d7a48 3314 }
b73bfc1c 3315 }
134b9549
KH		 3316 if (charset_id_0 < 0)
3317 charset = CHARSET_FROM_ID (charset_ascii);
3318 else
3319 charset = CHARSET_FROM_ID (charset_id_0);
4ed46869
KH		 3320 break;
3321
3322 case ISO_0xA0_or_0xFF:
df7492f9
KH		 3323 if (charset_id_1 < 0
3324 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_1))
3325 || CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS)
3326 goto invalid_code;
4ed46869
KH		 3327 /* This is a graphic character, we fall down ... */
3328
3329 case ISO_graphic_plane_1:
df7492f9
KH		 3330 if (charset_id_1 < 0)
3331 goto invalid_code;
3332 charset = CHARSET_FROM_ID (charset_id_1);
4ed46869
KH		 3333 break;
3334
df7492f9 3335 case ISO_control_0:
119852e7
KH		 3336 if (eol_crlf && c1 == '\r')
3337 ONE_MORE_BYTE (byte_after_cr);
df7492f9
KH		 3338 MAYBE_FINISH_COMPOSITION ();
3339 charset = CHARSET_FROM_ID (charset_ascii);
4ed46869
KH		 3340 break;
3341
df7492f9
KH		 3342 case ISO_control_1:
3343 MAYBE_FINISH_COMPOSITION ();
3344 goto invalid_code;
3345
4ed46869 3346 case ISO_shift_out:
df7492f9
KH		 3347 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3348 || CODING_ISO_DESIGNATION (coding, 1) < 0)
3349 goto invalid_code;
3350 CODING_ISO_INVOCATION (coding, 0) = 1;
3351 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3352 continue;
4ed46869
KH		 3353
3354 case ISO_shift_in:
df7492f9
KH		 3355 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT))
3356 goto invalid_code;
3357 CODING_ISO_INVOCATION (coding, 0) = 0;
3358 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3359 continue;
4ed46869
KH		 3360
3361 case ISO_single_shift_2_7:
3362 case ISO_single_shift_2:
df7492f9
KH		 3363 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3364 goto invalid_code;
4ed46869
KH		 3365 /* SS2 is handled as an escape sequence of ESC 'N' */
3366 c1 = 'N';
3367 goto label_escape_sequence;
3368
3369 case ISO_single_shift_3:
df7492f9
KH		 3370 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3371 goto invalid_code;
4ed46869
KH		 3372 /* SS2 is handled as an escape sequence of ESC 'O' */
3373 c1 = 'O';
3374 goto label_escape_sequence;
3375
3376 case ISO_control_sequence_introducer:
3377 /* CSI is handled as an escape sequence of ESC '[' ... */
3378 c1 = '[';
3379 goto label_escape_sequence;
3380
3381 case ISO_escape:
3382 ONE_MORE_BYTE (c1);
3383 label_escape_sequence:
df7492f9 3384 /* Escape sequences handled here are invocation,
4ed46869
KH		 3385 designation, direction specification, and character
3386 composition specification. */
3387 switch (c1)
3388 {
3389 case '&': /* revision of following character set */
3390 ONE_MORE_BYTE (c1);
3391 if (!(c1 >= '@' && c1 <= '~'))
df7492f9 3392 goto invalid_code;
4ed46869
KH		 3393 ONE_MORE_BYTE (c1);
3394 if (c1 != ISO_CODE_ESC)
df7492f9 3395 goto invalid_code;
4ed46869
KH		 3396 ONE_MORE_BYTE (c1);
3397 goto label_escape_sequence;
3398
3399 case '$': /* designation of 2-byte character set */
df7492f9
KH		 3400 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3401 goto invalid_code;
134b9549
KH		 3402 {
3403 int reg, chars96;
3404
3405 ONE_MORE_BYTE (c1);
3406 if (c1 >= '@' && c1 <= 'B')
3407 { /* designation of JISX0208.1978, GB2312.1980,
88993dfd 3408 or JISX0208.1980 */
134b9549
KH		 3409 reg = 0, chars96 = 0;
3410 }
3411 else if (c1 >= 0x28 && c1 <= 0x2B)
3412 { /* designation of DIMENSION2_CHARS94 character set */
3413 reg = c1 - 0x28, chars96 = 0;
3414 ONE_MORE_BYTE (c1);
3415 }
3416 else if (c1 >= 0x2C && c1 <= 0x2F)
3417 { /* designation of DIMENSION2_CHARS96 character set */
3418 reg = c1 - 0x2C, chars96 = 1;
3419 ONE_MORE_BYTE (c1);
3420 }
3421 else
3422 goto invalid_code;
3423 DECODE_DESIGNATION (reg, 2, chars96, c1);
3424 /* We must update these variables now. */
3425 if (reg == 0)
3426 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3427 else if (reg == 1)
3428 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3429 if (chars96 < 0)
3430 goto invalid_code;
3431 }
b73bfc1c 3432 continue;
4ed46869
KH		 3433
3434 case 'n': /* invocation of locking-shift-2 */
df7492f9
KH		 3435 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3436 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3437 goto invalid_code;
3438 CODING_ISO_INVOCATION (coding, 0) = 2;
3439 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3440 continue;
4ed46869
KH		 3441
3442 case 'o': /* invocation of locking-shift-3 */
df7492f9
KH		 3443 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3444 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3445 goto invalid_code;
3446 CODING_ISO_INVOCATION (coding, 0) = 3;
3447 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3448 continue;
4ed46869
KH		 3449
3450 case 'N': /* invocation of single-shift-2 */
df7492f9
KH		 3451 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3452 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3453 goto invalid_code;
134b9549
KH		 3454 charset_id_2 = CODING_ISO_DESIGNATION (coding, 2);
3455 if (charset_id_2 < 0)
3456 charset = CHARSET_FROM_ID (charset_ascii);
3457 else
3458 charset = CHARSET_FROM_ID (charset_id_2);
b73bfc1c 3459 ONE_MORE_BYTE (c1);
e7046a18 3460 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3461 goto invalid_code;
4ed46869
KH		 3462 break;
3463
3464 case 'O': /* invocation of single-shift-3 */
df7492f9
KH		 3465 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3466 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3467 goto invalid_code;
134b9549
KH		 3468 charset_id_3 = CODING_ISO_DESIGNATION (coding, 3);
3469 if (charset_id_3 < 0)
3470 charset = CHARSET_FROM_ID (charset_ascii);
3471 else
3472 charset = CHARSET_FROM_ID (charset_id_3);
b73bfc1c 3473 ONE_MORE_BYTE (c1);
e7046a18 3474 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3475 goto invalid_code;
4ed46869
KH		 3476 break;
3477
ec6d2bb8 3478 case '0': case '2': case '3': case '4': /* start composition */
df7492f9
KH		 3479 if (! (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK))
3480 goto invalid_code;
ec6d2bb8 3481 DECODE_COMPOSITION_START (c1);
b73bfc1c 3482 continue;
4ed46869 3483
ec6d2bb8 3484 case '1': /* end composition */
df7492f9
KH		 3485 if (composition_state == COMPOSING_NO)
3486 goto invalid_code;
3487 DECODE_COMPOSITION_END ();
b73bfc1c 3488 continue;
4ed46869
KH		 3489
3490 case '[': /* specification of direction */
df7492f9
KH		 3491 if (! CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DIRECTION)
3492 goto invalid_code;
4ed46869 3493 /* For the moment, nested direction is not supported.
d46c5b12 3494 So, `coding->mode & CODING_MODE_DIRECTION' zero means
df7492f9 3495 left-to-right, and nozero means right-to-left. */
4ed46869
KH		 3496 ONE_MORE_BYTE (c1);
3497 switch (c1)
3498 {
3499 case ']': /* end of the current direction */
d46c5b12 3500 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869
KH		 3501
3502 case '0': /* end of the current direction */
3503 case '1': /* start of left-to-right direction */
3504 ONE_MORE_BYTE (c1);
3505 if (c1 == ']')
d46c5b12 3506 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869 3507 else
df7492f9 3508 goto invalid_code;
4ed46869
KH		 3509 break;
3510
3511 case '2': /* start of right-to-left direction */
3512 ONE_MORE_BYTE (c1);
3513 if (c1 == ']')
d46c5b12 3514 coding->mode |= CODING_MODE_DIRECTION;
4ed46869 3515 else
df7492f9 3516 goto invalid_code;
4ed46869
KH		 3517 break;
3518
3519 default:
df7492f9 3520 goto invalid_code;
4ed46869 3521 }
b73bfc1c 3522 continue;
4ed46869 3523
103e0180 3524 case '%':
103e0180
KH		 3525 ONE_MORE_BYTE (c1);
3526 if (c1 == '/')
3527 {
3528 /* CTEXT extended segment:
3529 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
3530 We keep these bytes as is for the moment.
3531 They may be decoded by post-read-conversion. */
3532 int dim, M, L;
4776e638 3533 int size;
8f924df7 3534
103e0180
KH		 3535 ONE_MORE_BYTE (dim);
3536 ONE_MORE_BYTE (M);
3537 ONE_MORE_BYTE (L);
3538 size = ((M - 128) * 128) + (L - 128);
4776e638
KH		 3539 if (charbuf + 8 + size > charbuf_end)
3540 goto break_loop;
3541 *charbuf++ = ISO_CODE_ESC;
3542 *charbuf++ = '%';
3543 *charbuf++ = '/';
3544 *charbuf++ = dim;
3545 *charbuf++ = BYTE8_TO_CHAR (M);
3546 *charbuf++ = BYTE8_TO_CHAR (L);
103e0180
KH		 3547 while (size-- > 0)
3548 {
3549 ONE_MORE_BYTE (c1);
4776e638 3550 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
103e0180 3551 }
103e0180
KH		 3552 }
3553 else if (c1 == 'G')
3554 {
103e0180
KH		 3555 /* XFree86 extension for embedding UTF-8 in CTEXT:
3556 ESC % G --UTF-8-BYTES-- ESC % @
3557 We keep these bytes as is for the moment.
3558 They may be decoded by post-read-conversion. */
4776e638
KH		 3559 int *p = charbuf;
3560
3561 if (p + 6 > charbuf_end)
3562 goto break_loop;
3563 *p++ = ISO_CODE_ESC;
3564 *p++ = '%';
3565 *p++ = 'G';
3566 while (p < charbuf_end)
103e0180
KH		 3567 {
3568 ONE_MORE_BYTE (c1);
3569 if (c1 == ISO_CODE_ESC
3570 && src + 1 < src_end
3571 && src[0] == '%'
3572 && src[1] == '@')
9ffd559c
KH		 3573 {
3574 src += 2;
3575 break;
3576 }
4776e638 3577 *p++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
103e0180 3578 }
4776e638
KH		 3579 if (p + 3 > charbuf_end)
3580 goto break_loop;
3581 *p++ = ISO_CODE_ESC;
3582 *p++ = '%';
3583 *p++ = '@';
3584 charbuf = p;
103e0180
KH		 3585 }
3586 else
4776e638 3587 goto invalid_code;
103e0180 3588 continue;
4776e638 3589 break;
103e0180 3590
4ed46869 3591 default:
df7492f9
KH		 3592 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3593 goto invalid_code;
134b9549
KH		 3594 {
3595 int reg, chars96;
3596
3597 if (c1 >= 0x28 && c1 <= 0x2B)
3598 { /* designation of DIMENSION1_CHARS94 character set */
3599 reg = c1 - 0x28, chars96 = 0;
3600 ONE_MORE_BYTE (c1);
3601 }
3602 else if (c1 >= 0x2C && c1 <= 0x2F)
3603 { /* designation of DIMENSION1_CHARS96 character set */
3604 reg = c1 - 0x2C, chars96 = 1;
3605 ONE_MORE_BYTE (c1);
3606 }
3607 else
3608 goto invalid_code;
3609 DECODE_DESIGNATION (reg, 1, chars96, c1);
3610 /* We must update these variables now. */
3611 if (reg == 0)
3612 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3613 else if (reg == 1)
3614 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3615 if (chars96 < 0)
3616 goto invalid_code;
3617 }
b73bfc1c 3618 continue;
4ed46869 3619 }
b73bfc1c 3620 }
4ed46869 3621
ff0dacd7
KH		 3622 if (charset->id != charset_ascii
3623 && last_id != charset->id)
3624 {
3625 if (last_id != charset_ascii)
69a80ea3 3626 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
ff0dacd7
KH		 3627 last_id = charset->id;
3628 last_offset = char_offset;
3629 }
3630
b73bfc1c 3631 /* Now we know CHARSET and 1st position code C1 of a character.
df7492f9
KH		 3632 Produce a decoded character while getting 2nd position code
3633 C2 if necessary. */
3634 c1 &= 0x7F;
3635 if (CHARSET_DIMENSION (charset) > 1)
b73bfc1c
KH		 3636 {
3637 ONE_MORE_BYTE (c2);
df7492f9 3638 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
b73bfc1c 3639 /* C2 is not in a valid range. */
df7492f9
KH		 3640 goto invalid_code;
3641 c1 = (c1 << 8) | (c2 & 0x7F);
3642 if (CHARSET_DIMENSION (charset) > 2)
3643 {
3644 ONE_MORE_BYTE (c2);
3645 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
3646 /* C2 is not in a valid range. */
3647 goto invalid_code;
3648 c1 = (c1 << 8) | (c2 & 0x7F);
3649 }
3650 }
3651
3652 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c1, c);
3653 if (c < 0)
3654 {
3655 MAYBE_FINISH_COMPOSITION ();
3656 for (; src_base < src; src_base++, char_offset++)
3657 {
3658 if (ASCII_BYTE_P (*src_base))
3659 *charbuf++ = *src_base;
3660 else
3661 *charbuf++ = BYTE8_TO_CHAR (*src_base);
3662 }
3663 }
3664 else if (composition_state == COMPOSING_NO)
3665 {
3666 *charbuf++ = c;
3667 char_offset++;
4ed46869 3668 }
df7492f9 3669 else
781d7a48 3670 {
cee53ed4
KH		 3671 if (component_idx < MAX_COMPOSITION_COMPONENTS * 2 + 1)
3672 {
3673 components[component_idx++] = c;
3674 if (method == COMPOSITION_WITH_RULE
3675 || (method == COMPOSITION_WITH_RULE_ALTCHARS
3676 && composition_state == COMPOSING_COMPONENT_CHAR))
3677 composition_state++;
3678 }
3679 else
3680 {
3681 MAYBE_FINISH_COMPOSITION ();
3682 *charbuf++ = c;
3683 char_offset++;
3684 }
4ed46869
KH		 3685 }
3686 continue;
3687
df7492f9
KH		 3688 invalid_code:
3689 MAYBE_FINISH_COMPOSITION ();
4ed46869 3690 src = src_base;
df7492f9
KH		 3691 consumed_chars = consumed_chars_base;
3692 ONE_MORE_BYTE (c);
065e3595 3693 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 3694 char_offset++;
df7492f9 3695 coding->errors++;
4776e638
KH		 3696 continue;
3697
3698 break_loop:
3699 break;
4ed46869 3700 }
fb88bf2d 3701
df7492f9 3702 no_more_source:
ff0dacd7 3703 if (last_id != charset_ascii)
69a80ea3 3704 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 3705 coding->consumed_char += consumed_chars_base;
3706 coding->consumed = src_base - coding->source;
3707 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 3708}
3709
b73bfc1c 3710
f4dee582 3711/* ISO2022 encoding stuff. */
4ed46869
KH		 3712
3713/*
f4dee582 3714 It is not enough to say just "ISO2022" on encoding, we have to
df7492f9 3715 specify more details. In Emacs, each coding system of ISO2022
4ed46869 3716 variant has the following specifications:
df7492f9 3717 1. Initial designation to G0 thru G3.
4ed46869
KH		 3718 2. Allows short-form designation?
3719 3. ASCII should be designated to G0 before control characters?
3720 4. ASCII should be designated to G0 at end of line?
3721 5. 7-bit environment or 8-bit environment?
3722 6. Use locking-shift?
3723 7. Use Single-shift?
3724 And the following two are only for Japanese:
3725 8. Use ASCII in place of JIS0201-1976-Roman?
3726 9. Use JISX0208-1983 in place of JISX0208-1978?
df7492f9
KH		 3727 These specifications are encoded in CODING_ISO_FLAGS (coding) as flag bits
3728 defined by macros CODING_ISO_FLAG_XXX. See `coding.h' for more
f4dee582 3729 details.
4ed46869
KH		 3730*/
3731
3732/* Produce codes (escape sequence) for designating CHARSET to graphic
b73bfc1c
KH		 3733 register REG at DST, and increment DST. If <final-char> of CHARSET is
3734 '@', 'A', or 'B' and the coding system CODING allows, produce
3735 designation sequence of short-form. */
4ed46869
KH		 3736
3737#define ENCODE_DESIGNATION(charset, reg, coding) \
3738 do { \
df7492f9 3739 unsigned char final_char = CHARSET_ISO_FINAL (charset); \
4ed46869
KH		 3740 char *intermediate_char_94 = "()*+"; \
3741 char *intermediate_char_96 = ",-./"; \
df7492f9
KH		 3742 int revision = -1; \
3743 int c; \
3744 \
3745 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_REVISION) \
c197f191 3746 revision = CHARSET_ISO_REVISION (charset); \
df7492f9
KH		 3747 \
3748 if (revision >= 0) \
70c22245 3749 { \
df7492f9
KH		 3750 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '&'); \
3751 EMIT_ONE_BYTE ('@' + revision); \
4ed46869 3752 } \
df7492f9 3753 EMIT_ONE_ASCII_BYTE (ISO_CODE_ESC); \
4ed46869
KH		 3754 if (CHARSET_DIMENSION (charset) == 1) \
3755 { \
df7492f9
KH		 3756 if (! CHARSET_ISO_CHARS_96 (charset)) \
3757 c = intermediate_char_94[reg]; \
4ed46869 3758 else \
df7492f9
KH		 3759 c = intermediate_char_96[reg]; \
3760 EMIT_ONE_ASCII_BYTE (c); \
4ed46869
KH		 3761 } \
3762 else \
3763 { \
df7492f9
KH		 3764 EMIT_ONE_ASCII_BYTE ('$'); \
3765 if (! CHARSET_ISO_CHARS_96 (charset)) \
4ed46869 3766 { \
df7492f9 3767 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LONG_FORM \
b73bfc1c
KH		 3768 || reg != 0 \
3769 || final_char < '@' || final_char > 'B') \
df7492f9 3770 EMIT_ONE_ASCII_BYTE (intermediate_char_94[reg]); \
4ed46869
KH		 3771 } \
3772 else \
df7492f9 3773 EMIT_ONE_ASCII_BYTE (intermediate_char_96[reg]); \
4ed46869 3774 } \
df7492f9
KH		 3775 EMIT_ONE_ASCII_BYTE (final_char); \
3776 \
3777 CODING_ISO_DESIGNATION (coding, reg) = CHARSET_ID (charset); \
4ed46869
KH		 3778 } while (0)
3779
df7492f9 3780
4ed46869
KH		 3781/* The following two macros produce codes (control character or escape
3782 sequence) for ISO2022 single-shift functions (single-shift-2 and
3783 single-shift-3). */
3784
df7492f9
KH		 3785#define ENCODE_SINGLE_SHIFT_2 \
3786 do { \
3787 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3788 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'N'); \
3789 else \
3790 EMIT_ONE_BYTE (ISO_CODE_SS2); \
3791 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 3792 } while (0)
3793
df7492f9
KH		 3794
3795#define ENCODE_SINGLE_SHIFT_3 \
3796 do { \
3797 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3798 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'O'); \
3799 else \
3800 EMIT_ONE_BYTE (ISO_CODE_SS3); \
3801 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 3802 } while (0)
3803
df7492f9 3804
4ed46869
KH		 3805/* The following four macros produce codes (control character or
3806 escape sequence) for ISO2022 locking-shift functions (shift-in,
3807 shift-out, locking-shift-2, and locking-shift-3). */
3808
df7492f9
KH		 3809#define ENCODE_SHIFT_IN \
3810 do { \
3811 EMIT_ONE_ASCII_BYTE (ISO_CODE_SI); \
3812 CODING_ISO_INVOCATION (coding, 0) = 0; \
4ed46869
KH		 3813 } while (0)
3814
df7492f9
KH		 3815
3816#define ENCODE_SHIFT_OUT \
3817 do { \
3818 EMIT_ONE_ASCII_BYTE (ISO_CODE_SO); \
3819 CODING_ISO_INVOCATION (coding, 0) = 1; \
4ed46869
KH		 3820 } while (0)
3821
df7492f9
KH		 3822
3823#define ENCODE_LOCKING_SHIFT_2 \
3824 do { \
3825 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3826 CODING_ISO_INVOCATION (coding, 0) = 2; \
4ed46869
KH		 3827 } while (0)
3828
df7492f9
KH		 3829
3830#define ENCODE_LOCKING_SHIFT_3 \
3831 do { \
3832 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3833 CODING_ISO_INVOCATION (coding, 0) = 3; \
4ed46869
KH		 3834 } while (0)
3835
df7492f9 3836
f4dee582
RS		 3837/* Produce codes for a DIMENSION1 character whose character set is
3838 CHARSET and whose position-code is C1. Designation and invocation
4ed46869
KH		 3839 sequences are also produced in advance if necessary. */
3840
6e85d753
KH		 3841#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
3842 do { \
df7492f9 3843 int id = CHARSET_ID (charset); \
bf16eb23
KH		 3844 \
3845 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
3846 && id == charset_ascii) \
3847 { \
3848 id = charset_jisx0201_roman; \
3849 charset = CHARSET_FROM_ID (id); \
3850 } \
3851 \
df7492f9 3852 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 3853 { \
df7492f9
KH		 3854 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3855 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753 3856 else \
df7492f9
KH		 3857 EMIT_ONE_BYTE (c1 | 0x80); \
3858 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 3859 break; \
3860 } \
df7492f9 3861 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 3862 { \
df7492f9 3863 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753
KH		 3864 break; \
3865 } \
df7492f9 3866 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 3867 { \
df7492f9 3868 EMIT_ONE_BYTE (c1 | 0x80); \
6e85d753
KH		 3869 break; \
3870 } \
6e85d753
KH		 3871 else \
3872 /* Since CHARSET is not yet invoked to any graphic planes, we \
3873 must invoke it, or, at first, designate it to some graphic \
3874 register. Then repeat the loop to actually produce the \
3875 character. */ \
df7492f9
KH		 3876 dst = encode_invocation_designation (charset, coding, dst, \
3877 &produced_chars); \
4ed46869
KH		 3878 } while (1)
3879
df7492f9 3880
f4dee582
RS		 3881/* Produce codes for a DIMENSION2 character whose character set is
3882 CHARSET and whose position-codes are C1 and C2. Designation and
4ed46869
KH		 3883 invocation codes are also produced in advance if necessary. */
3884
6e85d753
KH		 3885#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
3886 do { \
df7492f9 3887 int id = CHARSET_ID (charset); \
bf16eb23
KH		 3888 \
3889 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
3890 && id == charset_jisx0208) \
3891 { \
3892 id = charset_jisx0208_1978; \
3893 charset = CHARSET_FROM_ID (id); \
3894 } \
3895 \
df7492f9 3896 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 3897 { \
df7492f9
KH		 3898 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3899 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753 3900 else \
df7492f9
KH		 3901 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
3902 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 3903 break; \
3904 } \
df7492f9 3905 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 3906 { \
df7492f9 3907 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753
KH		 3908 break; \
3909 } \
df7492f9 3910 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 3911 { \
df7492f9 3912 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
6e85d753
KH		 3913 break; \
3914 } \
6e85d753
KH		 3915 else \
3916 /* Since CHARSET is not yet invoked to any graphic planes, we \
3917 must invoke it, or, at first, designate it to some graphic \
3918 register. Then repeat the loop to actually produce the \
3919 character. */ \
df7492f9
KH		 3920 dst = encode_invocation_designation (charset, coding, dst, \
3921 &produced_chars); \
4ed46869
KH		 3922 } while (1)
3923
05e6f5dc 3924
df7492f9
KH		 3925#define ENCODE_ISO_CHARACTER(charset, c) \
3926 do { \
3927 int code = ENCODE_CHAR ((charset),(c)); \
3928 \
3929 if (CHARSET_DIMENSION (charset) == 1) \
3930 ENCODE_ISO_CHARACTER_DIMENSION1 ((charset), code); \
3931 else \
3932 ENCODE_ISO_CHARACTER_DIMENSION2 ((charset), code >> 8, code & 0xFF); \
84fbb8a0 3933 } while (0)
bdd9fb48 3934
05e6f5dc 3935
4ed46869 3936/* Produce designation and invocation codes at a place pointed by DST
df7492f9 3937 to use CHARSET. The element `spec.iso_2022' of *CODING is updated.
4ed46869
KH		 3938 Return new DST. */
3939
3940unsigned char *
df7492f9
KH		 3941encode_invocation_designation (charset, coding, dst, p_nchars)
3942 struct charset *charset;
4ed46869
KH		 3943 struct coding_system *coding;
3944 unsigned char *dst;
df7492f9 3945 int *p_nchars;
4ed46869 3946{
df7492f9
KH		 3947 int multibytep = coding->dst_multibyte;
3948 int produced_chars = *p_nchars;
4ed46869 3949 int reg; /* graphic register number */
df7492f9 3950 int id = CHARSET_ID (charset);
4ed46869
KH		 3951
3952 /* At first, check designations. */
3953 for (reg = 0; reg < 4; reg++)
df7492f9 3954 if (id == CODING_ISO_DESIGNATION (coding, reg))
4ed46869
KH		 3955 break;
3956
3957 if (reg >= 4)
3958 {
3959 /* CHARSET is not yet designated to any graphic registers. */
3960 /* At first check the requested designation. */
df7492f9
KH		 3961 reg = CODING_ISO_REQUEST (coding, id);
3962 if (reg < 0)
1ba9e4ab
KH		 3963 /* Since CHARSET requests no special designation, designate it
3964 to graphic register 0. */
4ed46869
KH		 3965 reg = 0;
3966
3967 ENCODE_DESIGNATION (charset, reg, coding);
3968 }
3969
df7492f9
KH		 3970 if (CODING_ISO_INVOCATION (coding, 0) != reg
3971 && CODING_ISO_INVOCATION (coding, 1) != reg)
4ed46869
KH		 3972 {
3973 /* Since the graphic register REG is not invoked to any graphic
3974 planes, invoke it to graphic plane 0. */
3975 switch (reg)
3976 {
3977 case 0: /* graphic register 0 */
3978 ENCODE_SHIFT_IN;
3979 break;
3980
3981 case 1: /* graphic register 1 */
3982 ENCODE_SHIFT_OUT;
3983 break;
3984
3985 case 2: /* graphic register 2 */
df7492f9 3986 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 3987 ENCODE_SINGLE_SHIFT_2;
3988 else
3989 ENCODE_LOCKING_SHIFT_2;
3990 break;
3991
3992 case 3: /* graphic register 3 */
df7492f9 3993 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 3994 ENCODE_SINGLE_SHIFT_3;
3995 else
3996 ENCODE_LOCKING_SHIFT_3;
3997 break;
3998 }
3999 }
b73bfc1c 4000
df7492f9 4001 *p_nchars = produced_chars;
4ed46869
KH		 4002 return dst;
4003}
4004
df7492f9
KH		 4005/* The following three macros produce codes for indicating direction
4006 of text. */
4007#define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
ec6d2bb8 4008 do { \
df7492f9
KH		 4009 if (CODING_ISO_FLAGS (coding) == CODING_ISO_FLAG_SEVEN_BITS) \
4010 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '['); \
ec6d2bb8 4011 else \
df7492f9 4012 EMIT_ONE_BYTE (ISO_CODE_CSI); \
ec6d2bb8
KH		 4013 } while (0)
4014
ec6d2bb8 4015
df7492f9
KH		 4016#define ENCODE_DIRECTION_R2L() \
4017 do { \
4018 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
4019 EMIT_TWO_ASCII_BYTES ('2', ']'); \
ec6d2bb8
KH		 4020 } while (0)
4021
ec6d2bb8 4022
df7492f9 4023#define ENCODE_DIRECTION_L2R() \
ec6d2bb8 4024 do { \
df7492f9
KH		 4025 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
4026 EMIT_TWO_ASCII_BYTES ('0', ']'); \
ec6d2bb8 4027 } while (0)
4ed46869 4028
4ed46869
KH		 4029
4030/* Produce codes for designation and invocation to reset the graphic
4031 planes and registers to initial state. */
df7492f9
KH		 4032#define ENCODE_RESET_PLANE_AND_REGISTER() \
4033 do { \
4034 int reg; \
4035 struct charset *charset; \
4036 \
4037 if (CODING_ISO_INVOCATION (coding, 0) != 0) \
4038 ENCODE_SHIFT_IN; \
4039 for (reg = 0; reg < 4; reg++) \
4040 if (CODING_ISO_INITIAL (coding, reg) >= 0 \
4041 && (CODING_ISO_DESIGNATION (coding, reg) \
4042 != CODING_ISO_INITIAL (coding, reg))) \
4043 { \
4044 charset = CHARSET_FROM_ID (CODING_ISO_INITIAL (coding, reg)); \
4045 ENCODE_DESIGNATION (charset, reg, coding); \
4046 } \
4ed46869
KH		 4047 } while (0)
4048
df7492f9 4049
bdd9fb48 4050/* Produce designation sequences of charsets in the line started from
b73bfc1c 4051 SRC to a place pointed by DST, and return updated DST.
bdd9fb48
KH		 4052
4053 If the current block ends before any end-of-line, we may fail to
d46c5b12
KH		 4054 find all the necessary designations. */
4055
b73bfc1c 4056static unsigned char *
df7492f9 4057encode_designation_at_bol (coding, charbuf, charbuf_end, dst)
e0e989f6 4058 struct coding_system *coding;
df7492f9
KH		 4059 int *charbuf, *charbuf_end;
4060 unsigned char *dst;
e0e989f6 4061{
df7492f9 4062 struct charset *charset;
bdd9fb48
KH		 4063 /* Table of charsets to be designated to each graphic register. */
4064 int r[4];
df7492f9
KH		 4065 int c, found = 0, reg;
4066 int produced_chars = 0;
4067 int multibytep = coding->dst_multibyte;
4068 Lisp_Object attrs;
4069 Lisp_Object charset_list;
4070
4071 attrs = CODING_ID_ATTRS (coding->id);
4072 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
4073 if (EQ (charset_list, Qiso_2022))
4074 charset_list = Viso_2022_charset_list;
bdd9fb48
KH		 4075
4076 for (reg = 0; reg < 4; reg++)
4077 r[reg] = -1;
4078
b73bfc1c 4079 while (found < 4)
e0e989f6 4080 {
df7492f9
KH		 4081 int id;
4082
4083 c = *charbuf++;
b73bfc1c
KH		 4084 if (c == '\n')
4085 break;
df7492f9
KH		 4086 charset = char_charset (c, charset_list, NULL);
4087 id = CHARSET_ID (charset);
4088 reg = CODING_ISO_REQUEST (coding, id);
4089 if (reg >= 0 && r[reg] < 0)
bdd9fb48
KH		 4090 {
4091 found++;
df7492f9 4092 r[reg] = id;
bdd9fb48 4093 }
bdd9fb48
KH		 4094 }
4095
4096 if (found)
4097 {
4098 for (reg = 0; reg < 4; reg++)
4099 if (r[reg] >= 0
df7492f9
KH		 4100 && CODING_ISO_DESIGNATION (coding, reg) != r[reg])
4101 ENCODE_DESIGNATION (CHARSET_FROM_ID (r[reg]), reg, coding);
e0e989f6 4102 }
b73bfc1c
KH		 4103
4104 return dst;
e0e989f6
KH		 4105}
4106
4ed46869
KH		 4107/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
4108
df7492f9
KH		 4109static int
4110encode_coding_iso_2022 (coding)
4ed46869 4111 struct coding_system *coding;
4ed46869 4112{
df7492f9
KH		 4113 int multibytep = coding->dst_multibyte;
4114 int *charbuf = coding->charbuf;
4115 int *charbuf_end = charbuf + coding->charbuf_used;
4116 unsigned char *dst = coding->destination + coding->produced;
4117 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4118 int safe_room = 16;
4119 int bol_designation
4120 = (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL
4121 && CODING_ISO_BOL (coding));
4122 int produced_chars = 0;
4123 Lisp_Object attrs, eol_type, charset_list;
4124 int ascii_compatible;
b73bfc1c 4125 int c;
ff0dacd7 4126 int preferred_charset_id = -1;
05e6f5dc 4127
24a73b0a
KH		 4128 CODING_GET_INFO (coding, attrs, charset_list);
4129 eol_type = CODING_ID_EOL_TYPE (coding->id);
4130 if (VECTORP (eol_type))
4131 eol_type = Qunix;
4132
004068e4 4133 setup_iso_safe_charsets (attrs);
ff0dacd7 4134 /* Charset list may have been changed. */
287c57d7 4135 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8f924df7 4136 coding->safe_charsets = (char *) SDATA (CODING_ATTR_SAFE_CHARSETS(attrs));
0eecad43 4137
df7492f9 4138 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
bdd9fb48 4139
df7492f9 4140 while (charbuf < charbuf_end)
4ed46869 4141 {
df7492f9 4142 ASSURE_DESTINATION (safe_room);
b73bfc1c 4143
df7492f9 4144 if (bol_designation)
b73bfc1c 4145 {
df7492f9 4146 unsigned char *dst_prev = dst;
4ed46869 4147
bdd9fb48 4148 /* We have to produce designation sequences if any now. */
df7492f9
KH		 4149 dst = encode_designation_at_bol (coding, charbuf, charbuf_end, dst);
4150 bol_designation = 0;
4151 /* We are sure that designation sequences are all ASCII bytes. */
4152 produced_chars += dst - dst_prev;
e0e989f6
KH		 4153 }
4154
df7492f9 4155 c = *charbuf++;
ec6d2bb8 4156
ff0dacd7
KH		 4157 if (c < 0)
4158 {
4159 /* Handle an annotation. */
4160 switch (*charbuf)
ec6d2bb8 4161 {
ff0dacd7
KH		 4162 case CODING_ANNOTATE_COMPOSITION_MASK:
4163 /* Not yet implemented. */
4164 break;
4165 case CODING_ANNOTATE_CHARSET_MASK:
cf7dfdf5 4166 preferred_charset_id = charbuf[2];
ff0dacd7
KH		 4167 if (preferred_charset_id >= 0
4168 && NILP (Fmemq (make_number (preferred_charset_id),
4169 charset_list)))
4170 preferred_charset_id = -1;
4171 break;
4172 default:
4173 abort ();
4ed46869 4174 }
ff0dacd7
KH		 4175 charbuf += -c - 1;
4176 continue;
4ed46869 4177 }
ec6d2bb8 4178
b73bfc1c
KH		 4179 /* Now encode the character C. */
4180 if (c < 0x20 || c == 0x7F)
4181 {
df7492f9
KH		 4182 if (c == '\n'
4183 || (c == '\r' && EQ (eol_type, Qmac)))
19a8d9e0 4184 {
df7492f9
KH		 4185 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
4186 ENCODE_RESET_PLANE_AND_REGISTER ();
4187 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_INIT_AT_BOL)
b73bfc1c 4188 {
df7492f9
KH		 4189 int i;
4190
4191 for (i = 0; i < 4; i++)
4192 CODING_ISO_DESIGNATION (coding, i)
4193 = CODING_ISO_INITIAL (coding, i);
b73bfc1c 4194 }
df7492f9
KH		 4195 bol_designation
4196 = CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL;
19a8d9e0 4197 }
df7492f9
KH		 4198 else if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_CNTL)
4199 ENCODE_RESET_PLANE_AND_REGISTER ();
4200 EMIT_ONE_ASCII_BYTE (c);
4ed46869 4201 }
df7492f9 4202 else if (ASCII_CHAR_P (c))
88993dfd 4203 {
df7492f9
KH		 4204 if (ascii_compatible)
4205 EMIT_ONE_ASCII_BYTE (c);
93dec019 4206 else
19a8d9e0 4207 {
bf16eb23
KH		 4208 struct charset *charset = CHARSET_FROM_ID (charset_ascii);
4209 ENCODE_ISO_CHARACTER (charset, c);
19a8d9e0 4210 }
4ed46869 4211 }
16eafb5d 4212 else if (CHAR_BYTE8_P (c))
88993dfd 4213 {
16eafb5d
KH		 4214 c = CHAR_TO_BYTE8 (c);
4215 EMIT_ONE_BYTE (c);
88993dfd 4216 }
b73bfc1c 4217 else
df7492f9 4218 {
ff0dacd7 4219 struct charset *charset;
b73bfc1c 4220
ff0dacd7
KH		 4221 if (preferred_charset_id >= 0)
4222 {
4223 charset = CHARSET_FROM_ID (preferred_charset_id);
4224 if (! CHAR_CHARSET_P (c, charset))
4225 charset = char_charset (c, charset_list, NULL);
4226 }
4227 else
4228 charset = char_charset (c, charset_list, NULL);
df7492f9
KH		 4229 if (!charset)
4230 {
41cbe562
KH		 4231 if (coding->mode & CODING_MODE_SAFE_ENCODING)
4232 {
4233 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4234 charset = CHARSET_FROM_ID (charset_ascii);
4235 }
4236 else
4237 {
4238 c = coding->default_char;
4239 charset = char_charset (c, charset_list, NULL);
4240 }
df7492f9
KH		 4241 }
4242 ENCODE_ISO_CHARACTER (charset, c);
4243 }
84fbb8a0 4244 }
b73bfc1c 4245
df7492f9
KH		 4246 if (coding->mode & CODING_MODE_LAST_BLOCK
4247 && CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
4248 {
4249 ASSURE_DESTINATION (safe_room);
4250 ENCODE_RESET_PLANE_AND_REGISTER ();
4251 }
065e3595 4252 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4253 CODING_ISO_BOL (coding) = bol_designation;
4254 coding->produced_char += produced_chars;
4255 coding->produced = dst - coding->destination;
4256 return 0;
4ed46869
KH		 4257}
4258
4259\f
df7492f9 4260/*** 8,9. SJIS and BIG5 handlers ***/
4ed46869 4261
df7492f9 4262/* Although SJIS and BIG5 are not ISO's coding system, they are used
4ed46869
KH		 4263 quite widely. So, for the moment, Emacs supports them in the bare
4264 C code. But, in the future, they may be supported only by CCL. */
4265
4266/* SJIS is a coding system encoding three character sets: ASCII, right
4267 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
4268 as is. A character of charset katakana-jisx0201 is encoded by
4269 "position-code + 0x80". A character of charset japanese-jisx0208
4270 is encoded in 2-byte but two position-codes are divided and shifted
df7492f9 4271 so that it fit in the range below.
4ed46869
KH		 4272
4273 --- CODE RANGE of SJIS ---
4274 (character set) (range)
4275 ASCII 0x00 .. 0x7F
df7492f9 4276 KATAKANA-JISX0201 0xA0 .. 0xDF
c28a9453 4277 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
d14d03ac 4278 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
4ed46869
KH		 4279 -------------------------------
4280
4281*/
4282
4283/* BIG5 is a coding system encoding two character sets: ASCII and
4284 Big5. An ASCII character is encoded as is. Big5 is a two-byte
df7492f9 4285 character set and is encoded in two-byte.
4ed46869
KH		 4286
4287 --- CODE RANGE of BIG5 ---
4288 (character set) (range)
4289 ASCII 0x00 .. 0x7F
4290 Big5 (1st byte) 0xA1 .. 0xFE
4291 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
4292 --------------------------
4293
df7492f9 4294 */
4ed46869
KH		 4295
4296/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4297 Check if a text is encoded in SJIS. If it is, return
df7492f9 4298 CATEGORY_MASK_SJIS, else return 0. */
4ed46869 4299
0a28aafb 4300static int
ff0dacd7 4301detect_coding_sjis (coding, detect_info)
df7492f9 4302 struct coding_system *coding;
ff0dacd7 4303 struct coding_detection_info *detect_info;
4ed46869 4304{
065e3595 4305 const unsigned char *src = coding->source, *src_base;
8f924df7 4306 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4307 int multibytep = coding->src_multibyte;
4308 int consumed_chars = 0;
4309 int found = 0;
b73bfc1c 4310 int c;
df7492f9 4311
ff0dacd7 4312 detect_info->checked |= CATEGORY_MASK_SJIS;
df7492f9
KH		 4313 /* A coding system of this category is always ASCII compatible. */
4314 src += coding->head_ascii;
4ed46869 4315
b73bfc1c 4316 while (1)
4ed46869 4317 {
065e3595 4318 src_base = src;
df7492f9 4319 ONE_MORE_BYTE (c);
682169fe
KH		 4320 if (c < 0x80)
4321 continue;
df7492f9 4322 if ((c >= 0x81 && c <= 0x9F) || (c >= 0xE0 && c <= 0xEF))
4ed46869 4323 {
df7492f9 4324 ONE_MORE_BYTE (c);
682169fe 4325 if (c < 0x40 || c == 0x7F || c > 0xFC)
df7492f9 4326 break;
ff0dacd7 4327 found = CATEGORY_MASK_SJIS;
4ed46869 4328 }
df7492f9 4329 else if (c >= 0xA0 && c < 0xE0)
ff0dacd7 4330 found = CATEGORY_MASK_SJIS;
df7492f9
KH		 4331 else
4332 break;
4ed46869 4333 }
ff0dacd7 4334 detect_info->rejected |= CATEGORY_MASK_SJIS;
df7492f9
KH		 4335 return 0;
4336
4337 no_more_source:
065e3595 4338 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 4339 {
ff0dacd7 4340 detect_info->rejected |= CATEGORY_MASK_SJIS;
89528eb3 4341 return 0;
4ed46869 4342 }
ff0dacd7
KH		 4343 detect_info->found |= found;
4344 return 1;
4ed46869
KH		 4345}
4346
4347/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4348 Check if a text is encoded in BIG5. If it is, return
df7492f9 4349 CATEGORY_MASK_BIG5, else return 0. */
4ed46869 4350
0a28aafb 4351static int
ff0dacd7 4352detect_coding_big5 (coding, detect_info)
df7492f9 4353 struct coding_system *coding;
ff0dacd7 4354 struct coding_detection_info *detect_info;
4ed46869 4355{
065e3595 4356 const unsigned char *src = coding->source, *src_base;
8f924df7 4357 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4358 int multibytep = coding->src_multibyte;
4359 int consumed_chars = 0;
4360 int found = 0;
b73bfc1c 4361 int c;
fa42c37f 4362
ff0dacd7 4363 detect_info->checked |= CATEGORY_MASK_BIG5;
df7492f9
KH		 4364 /* A coding system of this category is always ASCII compatible. */
4365 src += coding->head_ascii;
fa42c37f 4366
b73bfc1c 4367 while (1)
fa42c37f 4368 {
065e3595 4369 src_base = src;
df7492f9
KH		 4370 ONE_MORE_BYTE (c);
4371 if (c < 0x80)
fa42c37f 4372 continue;
df7492f9 4373 if (c >= 0xA1)
fa42c37f 4374 {
df7492f9
KH		 4375 ONE_MORE_BYTE (c);
4376 if (c < 0x40 || (c >= 0x7F && c <= 0xA0))
fa42c37f 4377 return 0;
ff0dacd7 4378 found = CATEGORY_MASK_BIG5;
fa42c37f 4379 }
df7492f9
KH		 4380 else
4381 break;
fa42c37f 4382 }
ff0dacd7 4383 detect_info->rejected |= CATEGORY_MASK_BIG5;
fa42c37f 4384 return 0;
fa42c37f 4385
df7492f9 4386 no_more_source:
065e3595 4387 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 4388 {
ff0dacd7 4389 detect_info->rejected |= CATEGORY_MASK_BIG5;
89528eb3
KH		 4390 return 0;
4391 }
ff0dacd7
KH		 4392 detect_info->found |= found;
4393 return 1;
fa42c37f
KH		 4394}
4395
4ed46869
KH		 4396/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
4397 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
fa42c37f 4398
b73bfc1c 4399static void
df7492f9 4400decode_coding_sjis (coding)
4ed46869 4401 struct coding_system *coding;
4ed46869 4402{
8f924df7
KH		 4403 const unsigned char *src = coding->source + coding->consumed;
4404 const unsigned char *src_end = coding->source + coding->src_bytes;
4405 const unsigned char *src_base;
69a80ea3
KH		 4406 int *charbuf = coding->charbuf + coding->charbuf_used;
4407 int *charbuf_end
4408 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 4409 int consumed_chars = 0, consumed_chars_base;
4410 int multibytep = coding->src_multibyte;
4411 struct charset *charset_roman, *charset_kanji, *charset_kana;
57a47f8a 4412 struct charset *charset_kanji2;
24a73b0a 4413 Lisp_Object attrs, charset_list, val;
ff0dacd7
KH		 4414 int char_offset = coding->produced_char;
4415 int last_offset = char_offset;
4416 int last_id = charset_ascii;
119852e7
KH		 4417 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
4418 int byte_after_cr = -1;
a5d301df 4419
24a73b0a 4420 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4421
4422 val = charset_list;
4423 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
89528eb3 4424 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4425 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4426 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 4427
b73bfc1c 4428 while (1)
4ed46869 4429 {
df7492f9 4430 int c, c1;
24a73b0a 4431 struct charset *charset;
fa42c37f 4432
b73bfc1c 4433 src_base = src;
df7492f9 4434 consumed_chars_base = consumed_chars;
fa42c37f 4435
df7492f9 4436 if (charbuf >= charbuf_end)
b71f6f73
KH		 4437 {
4438 if (byte_after_cr >= 0)
4439 src_base--;
4440 break;
4441 }
df7492f9 4442
119852e7
KH		 4443 if (byte_after_cr >= 0)
4444 c = byte_after_cr, byte_after_cr = -1;
4445 else
4446 ONE_MORE_BYTE (c);
065e3595
KH		 4447 if (c < 0)
4448 goto invalid_code;
24a73b0a 4449 if (c < 0x80)
119852e7
KH		 4450 {
4451 if (eol_crlf && c == '\r')
4452 ONE_MORE_BYTE (byte_after_cr);
4453 charset = charset_roman;
4454 }
57a47f8a 4455 else if (c == 0x80 || c == 0xA0)
8e921c4b 4456 goto invalid_code;
57a47f8a
KH		 4457 else if (c >= 0xA1 && c <= 0xDF)
4458 {
4459 /* SJIS -> JISX0201-Kana */
4460 c &= 0x7F;
4461 charset = charset_kana;
4462 }
4463 else if (c <= 0xEF)
df7492f9 4464 {
57a47f8a
KH		 4465 /* SJIS -> JISX0208 */
4466 ONE_MORE_BYTE (c1);
4467 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4468 goto invalid_code;
57a47f8a
KH		 4469 c = (c << 8) | c1;
4470 SJIS_TO_JIS (c);
4471 charset = charset_kanji;
4472 }
4473 else if (c <= 0xFC && charset_kanji2)
4474 {
c6876370 4475 /* SJIS -> JISX0213-2 */
57a47f8a
KH		 4476 ONE_MORE_BYTE (c1);
4477 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4478 goto invalid_code;
57a47f8a
KH		 4479 c = (c << 8) | c1;
4480 SJIS_TO_JIS2 (c);
4481 charset = charset_kanji2;
df7492f9 4482 }
57a47f8a
KH		 4483 else
4484 goto invalid_code;
24a73b0a
KH		 4485 if (charset->id != charset_ascii
4486 && last_id != charset->id)
4487 {
4488 if (last_id != charset_ascii)
69a80ea3 4489 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4490 last_id = charset->id;
4491 last_offset = char_offset;
4492 }
4493 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4494 *charbuf++ = c;
ff0dacd7 4495 char_offset++;
df7492f9 4496 continue;
b73bfc1c 4497
df7492f9
KH		 4498 invalid_code:
4499 src = src_base;
4500 consumed_chars = consumed_chars_base;
4501 ONE_MORE_BYTE (c);
065e3595 4502 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4503 char_offset++;
df7492f9
KH		 4504 coding->errors++;
4505 }
fa42c37f 4506
df7492f9 4507 no_more_source:
ff0dacd7 4508 if (last_id != charset_ascii)
69a80ea3 4509 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4510 coding->consumed_char += consumed_chars_base;
4511 coding->consumed = src_base - coding->source;
4512 coding->charbuf_used = charbuf - coding->charbuf;
fa42c37f
KH		 4513}
4514
b73bfc1c 4515static void
df7492f9 4516decode_coding_big5 (coding)
4ed46869 4517 struct coding_system *coding;
4ed46869 4518{
8f924df7
KH		 4519 const unsigned char *src = coding->source + coding->consumed;
4520 const unsigned char *src_end = coding->source + coding->src_bytes;
4521 const unsigned char *src_base;
69a80ea3
KH		 4522 int *charbuf = coding->charbuf + coding->charbuf_used;
4523 int *charbuf_end
4524 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 4525 int consumed_chars = 0, consumed_chars_base;
4526 int multibytep = coding->src_multibyte;
4527 struct charset *charset_roman, *charset_big5;
24a73b0a 4528 Lisp_Object attrs, charset_list, val;
ff0dacd7
KH		 4529 int char_offset = coding->produced_char;
4530 int last_offset = char_offset;
4531 int last_id = charset_ascii;
119852e7
KH		 4532 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
4533 int byte_after_cr = -1;
df7492f9 4534
24a73b0a 4535 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4536 val = charset_list;
4537 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4538 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4539
b73bfc1c 4540 while (1)
4ed46869 4541 {
df7492f9 4542 int c, c1;
24a73b0a 4543 struct charset *charset;
b73bfc1c
KH		 4544
4545 src_base = src;
df7492f9
KH		 4546 consumed_chars_base = consumed_chars;
4547
4548 if (charbuf >= charbuf_end)
b71f6f73
KH		 4549 {
4550 if (byte_after_cr >= 0)
4551 src_base--;
4552 break;
4553 }
df7492f9 4554
119852e7 4555 if (byte_after_cr >= 0)
14daee73 4556 c = byte_after_cr, byte_after_cr = -1;
119852e7
KH		 4557 else
4558 ONE_MORE_BYTE (c);
b73bfc1c 4559
065e3595
KH		 4560 if (c < 0)
4561 goto invalid_code;
24a73b0a 4562 if (c < 0x80)
119852e7 4563 {
14daee73 4564 if (eol_crlf && c == '\r')
119852e7
KH		 4565 ONE_MORE_BYTE (byte_after_cr);
4566 charset = charset_roman;
4567 }
24a73b0a 4568 else
4ed46869 4569 {
24a73b0a
KH		 4570 /* BIG5 -> Big5 */
4571 if (c < 0xA1 || c > 0xFE)
4572 goto invalid_code;
4573 ONE_MORE_BYTE (c1);
4574 if (c1 < 0x40 || (c1 > 0x7E && c1 < 0xA1) || c1 > 0xFE)
4575 goto invalid_code;
4576 c = c << 8 | c1;
4577 charset = charset_big5;
4ed46869 4578 }
24a73b0a
KH		 4579 if (charset->id != charset_ascii
4580 && last_id != charset->id)
df7492f9 4581 {
24a73b0a 4582 if (last_id != charset_ascii)
69a80ea3 4583 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4584 last_id = charset->id;
4585 last_offset = char_offset;
4ed46869 4586 }
24a73b0a 4587 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4588 *charbuf++ = c;
ff0dacd7 4589 char_offset++;
fb88bf2d
KH		 4590 continue;
4591
df7492f9 4592 invalid_code:
4ed46869 4593 src = src_base;
df7492f9
KH		 4594 consumed_chars = consumed_chars_base;
4595 ONE_MORE_BYTE (c);
065e3595 4596 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4597 char_offset++;
df7492f9 4598 coding->errors++;
fb88bf2d 4599 }
d46c5b12 4600
df7492f9 4601 no_more_source:
ff0dacd7 4602 if (last_id != charset_ascii)
69a80ea3 4603 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4604 coding->consumed_char += consumed_chars_base;
4605 coding->consumed = src_base - coding->source;
4606 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 4607}
4608
4609/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
b73bfc1c
KH		 4610 This function can encode charsets `ascii', `katakana-jisx0201',
4611 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
4612 are sure that all these charsets are registered as official charset
4ed46869
KH		 4613 (i.e. do not have extended leading-codes). Characters of other
4614 charsets are produced without any encoding. If SJIS_P is 1, encode
4615 SJIS text, else encode BIG5 text. */
4616
df7492f9
KH		 4617static int
4618encode_coding_sjis (coding)
4ed46869 4619 struct coding_system *coding;
4ed46869 4620{
df7492f9
KH		 4621 int multibytep = coding->dst_multibyte;
4622 int *charbuf = coding->charbuf;
4623 int *charbuf_end = charbuf + coding->charbuf_used;
4624 unsigned char *dst = coding->destination + coding->produced;
4625 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4626 int safe_room = 4;
4627 int produced_chars = 0;
24a73b0a 4628 Lisp_Object attrs, charset_list, val;
df7492f9
KH		 4629 int ascii_compatible;
4630 struct charset *charset_roman, *charset_kanji, *charset_kana;
57a47f8a 4631 struct charset *charset_kanji2;
df7492f9 4632 int c;
a5d301df 4633
24a73b0a 4634 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4635 val = charset_list;
4636 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4637 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4638 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4639 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4640
df7492f9 4641 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
93dec019 4642
df7492f9
KH		 4643 while (charbuf < charbuf_end)
4644 {
4645 ASSURE_DESTINATION (safe_room);
4646 c = *charbuf++;
b73bfc1c 4647 /* Now encode the character C. */
df7492f9
KH		 4648 if (ASCII_CHAR_P (c) && ascii_compatible)
4649 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4650 else if (CHAR_BYTE8_P (c))
4651 {
4652 c = CHAR_TO_BYTE8 (c);
4653 EMIT_ONE_BYTE (c);
4654 }
df7492f9 4655 else
b73bfc1c 4656 {
df7492f9
KH		 4657 unsigned code;
4658 struct charset *charset = char_charset (c, charset_list, &code);
4659
4660 if (!charset)
4ed46869 4661 {
41cbe562 4662 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4663 {
41cbe562
KH		 4664 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4665 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4666 }
41cbe562 4667 else
b73bfc1c 4668 {
41cbe562
KH		 4669 c = coding->default_char;
4670 charset = char_charset (c, charset_list, &code);
b73bfc1c 4671 }
b73bfc1c 4672 }
df7492f9
KH		 4673 if (code == CHARSET_INVALID_CODE (charset))
4674 abort ();
4675 if (charset == charset_kanji)
4676 {
4677 int c1, c2;
4678 JIS_TO_SJIS (code);
4679 c1 = code >> 8, c2 = code & 0xFF;
4680 EMIT_TWO_BYTES (c1, c2);
4681 }
4682 else if (charset == charset_kana)
4683 EMIT_ONE_BYTE (code | 0x80);
57a47f8a
KH		 4684 else if (charset_kanji2 && charset == charset_kanji2)
4685 {
4686 int c1, c2;
4687
4688 c1 = code >> 8;
4689 if (c1 == 0x21 || (c1 >= 0x23 && c1 < 0x25)
4690 || (c1 >= 0x2C && c1 <= 0x2F) || c1 >= 0x6E)
4691 {
4692 JIS_TO_SJIS2 (code);
4693 c1 = code >> 8, c2 = code & 0xFF;
4694 EMIT_TWO_BYTES (c1, c2);
4695 }
4696 else
4697 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4698 }
df7492f9
KH		 4699 else
4700 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4701 }
4702 }
065e3595 4703 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4704 coding->produced_char += produced_chars;
4705 coding->produced = dst - coding->destination;
4706 return 0;
4707}
4708
4709static int
4710encode_coding_big5 (coding)
4711 struct coding_system *coding;
4712{
4713 int multibytep = coding->dst_multibyte;
4714 int *charbuf = coding->charbuf;
4715 int *charbuf_end = charbuf + coding->charbuf_used;
4716 unsigned char *dst = coding->destination + coding->produced;
4717 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4718 int safe_room = 4;
4719 int produced_chars = 0;
24a73b0a 4720 Lisp_Object attrs, charset_list, val;
df7492f9
KH		 4721 int ascii_compatible;
4722 struct charset *charset_roman, *charset_big5;
4723 int c;
4724
24a73b0a 4725 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4726 val = charset_list;
4727 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4728 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4729 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4730
4731 while (charbuf < charbuf_end)
4732 {
4733 ASSURE_DESTINATION (safe_room);
4734 c = *charbuf++;
4735 /* Now encode the character C. */
4736 if (ASCII_CHAR_P (c) && ascii_compatible)
4737 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4738 else if (CHAR_BYTE8_P (c))
4739 {
4740 c = CHAR_TO_BYTE8 (c);
4741 EMIT_ONE_BYTE (c);
b73bfc1c
KH		 4742 }
4743 else
4744 {
df7492f9
KH		 4745 unsigned code;
4746 struct charset *charset = char_charset (c, charset_list, &code);
4747
4748 if (! charset)
b73bfc1c 4749 {
41cbe562 4750 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4751 {
41cbe562
KH		 4752 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4753 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4754 }
41cbe562 4755 else
0eecad43 4756 {
41cbe562
KH		 4757 c = coding->default_char;
4758 charset = char_charset (c, charset_list, &code);
0eecad43 4759 }
4ed46869 4760 }
df7492f9
KH		 4761 if (code == CHARSET_INVALID_CODE (charset))
4762 abort ();
4763 if (charset == charset_big5)
b73bfc1c 4764 {
df7492f9
KH		 4765 int c1, c2;
4766
4767 c1 = code >> 8, c2 = code & 0xFF;
4768 EMIT_TWO_BYTES (c1, c2);
b73bfc1c 4769 }
df7492f9
KH		 4770 else
4771 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4ed46869 4772 }
4ed46869 4773 }
065e3595 4774 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4775 coding->produced_char += produced_chars;
4776 coding->produced = dst - coding->destination;
4777 return 0;
4ed46869
KH		 4778}
4779
4780\f
df7492f9 4781/*** 10. CCL handlers ***/
1397dc18
KH		 4782
4783/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4784 Check if a text is encoded in a coding system of which
4785 encoder/decoder are written in CCL program. If it is, return
df7492f9 4786 CATEGORY_MASK_CCL, else return 0. */
1397dc18 4787
0a28aafb 4788static int
ff0dacd7 4789detect_coding_ccl (coding, detect_info)
df7492f9 4790 struct coding_system *coding;
ff0dacd7 4791 struct coding_detection_info *detect_info;
1397dc18 4792{
065e3595 4793 const unsigned char *src = coding->source, *src_base;
8f924df7 4794 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4795 int multibytep = coding->src_multibyte;
4796 int consumed_chars = 0;
4797 int found = 0;
0e219d54 4798 unsigned char *valids;
df7492f9
KH		 4799 int head_ascii = coding->head_ascii;
4800 Lisp_Object attrs;
4801
ff0dacd7
KH		 4802 detect_info->checked |= CATEGORY_MASK_CCL;
4803
df7492f9 4804 coding = &coding_categories[coding_category_ccl];
0e219d54 4805 valids = CODING_CCL_VALIDS (coding);
df7492f9
KH		 4806 attrs = CODING_ID_ATTRS (coding->id);
4807 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
4808 src += head_ascii;
1397dc18 4809
b73bfc1c 4810 while (1)
1397dc18 4811 {
df7492f9 4812 int c;
065e3595
KH		 4813
4814 src_base = src;
df7492f9 4815 ONE_MORE_BYTE (c);
065e3595 4816 if (c < 0 || ! valids[c])
df7492f9 4817 break;
ff0dacd7
KH		 4818 if ((valids[c] > 1))
4819 found = CATEGORY_MASK_CCL;
df7492f9 4820 }
ff0dacd7 4821 detect_info->rejected |= CATEGORY_MASK_CCL;
df7492f9
KH		 4822 return 0;
4823
4824 no_more_source:
ff0dacd7
KH		 4825 detect_info->found |= found;
4826 return 1;
df7492f9
KH		 4827}
4828
4829static void
4830decode_coding_ccl (coding)
4831 struct coding_system *coding;
4832{
7c78e542 4833 const unsigned char *src = coding->source + coding->consumed;
8f924df7 4834 const unsigned char *src_end = coding->source + coding->src_bytes;
69a80ea3
KH		 4835 int *charbuf = coding->charbuf + coding->charbuf_used;
4836 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 4837 int consumed_chars = 0;
4838 int multibytep = coding->src_multibyte;
4839 struct ccl_program ccl;
4840 int source_charbuf[1024];
4841 int source_byteidx[1024];
24a73b0a 4842 Lisp_Object attrs, charset_list;
df7492f9 4843
24a73b0a 4844 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4845 setup_ccl_program (&ccl, CODING_CCL_DECODER (coding));
4846
4847 while (src < src_end)
4848 {
7c78e542 4849 const unsigned char *p = src;
df7492f9
KH		 4850 int *source, *source_end;
4851 int i = 0;
4852
4853 if (multibytep)
4854 while (i < 1024 && p < src_end)
4855 {
4856 source_byteidx[i] = p - src;
4857 source_charbuf[i++] = STRING_CHAR_ADVANCE (p);
4858 }
4859 else
4860 while (i < 1024 && p < src_end)
4861 source_charbuf[i++] = *p++;
8f924df7 4862
df7492f9
KH		 4863 if (p == src_end && coding->mode & CODING_MODE_LAST_BLOCK)
4864 ccl.last_block = 1;
4865
4866 source = source_charbuf;
4867 source_end = source + i;
4868 while (source < source_end)
4869 {
4870 ccl_driver (&ccl, source, charbuf,
8dcbea82
KH		 4871 source_end - source, charbuf_end - charbuf,
4872 charset_list);
df7492f9
KH		 4873 source += ccl.consumed;
4874 charbuf += ccl.produced;
4875 if (ccl.status != CCL_STAT_SUSPEND_BY_DST)
4876 break;
4877 }
4878 if (source < source_end)
4879 src += source_byteidx[source - source_charbuf];
4880 else
4881 src = p;
4882 consumed_chars += source - source_charbuf;
4883
4884 if (ccl.status != CCL_STAT_SUSPEND_BY_SRC
4885 && ccl.status != CODING_RESULT_INSUFFICIENT_SRC)
4886 break;
4887 }
4888
4889 switch (ccl.status)
4890 {
4891 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 4892 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 4893 break;
4894 case CCL_STAT_SUSPEND_BY_DST:
4895 break;
4896 case CCL_STAT_QUIT:
4897 case CCL_STAT_INVALID_CMD:
065e3595 4898 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 4899 break;
4900 default:
065e3595 4901 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4902 break;
4903 }
4904 coding->consumed_char += consumed_chars;
4905 coding->consumed = src - coding->source;
4906 coding->charbuf_used = charbuf - coding->charbuf;
4907}
4908
4909static int
4910encode_coding_ccl (coding)
4911 struct coding_system *coding;
4912{
4913 struct ccl_program ccl;
4914 int multibytep = coding->dst_multibyte;
4915 int *charbuf = coding->charbuf;
4916 int *charbuf_end = charbuf + coding->charbuf_used;
4917 unsigned char *dst = coding->destination + coding->produced;
4918 unsigned char *dst_end = coding->destination + coding->dst_bytes;
df7492f9
KH		 4919 int destination_charbuf[1024];
4920 int i, produced_chars = 0;
24a73b0a 4921 Lisp_Object attrs, charset_list;
df7492f9 4922
24a73b0a 4923 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4924 setup_ccl_program (&ccl, CODING_CCL_ENCODER (coding));
4925
4926 ccl.last_block = coding->mode & CODING_MODE_LAST_BLOCK;
4927 ccl.dst_multibyte = coding->dst_multibyte;
4928
8cffd3e7 4929 while (charbuf < charbuf_end)
df7492f9 4930 {
df7492f9 4931 ccl_driver (&ccl, charbuf, destination_charbuf,
8cffd3e7 4932 charbuf_end - charbuf, 1024, charset_list);
df7492f9 4933 if (multibytep)
8cffd3e7
KH		 4934 {
4935 ASSURE_DESTINATION (ccl.produced * 2);
4936 for (i = 0; i < ccl.produced; i++)
4937 EMIT_ONE_BYTE (destination_charbuf[i] & 0xFF);
4938 }
df7492f9
KH		 4939 else
4940 {
8cffd3e7 4941 ASSURE_DESTINATION (ccl.produced);
3ed051d4 4942 for (i = 0; i < ccl.produced; i++)
df7492f9
KH		 4943 *dst++ = destination_charbuf[i] & 0xFF;
4944 produced_chars += ccl.produced;
4945 }
8cffd3e7
KH		 4946 charbuf += ccl.consumed;
4947 if (ccl.status == CCL_STAT_QUIT
4948 || ccl.status == CCL_STAT_INVALID_CMD)
4949 break;
df7492f9
KH		 4950 }
4951
4952 switch (ccl.status)
4953 {
4954 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 4955 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 4956 break;
4957 case CCL_STAT_SUSPEND_BY_DST:
065e3595 4958 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 4959 break;
4960 case CCL_STAT_QUIT:
4961 case CCL_STAT_INVALID_CMD:
065e3595 4962 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 4963 break;
4964 default:
065e3595 4965 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 4966 break;
1397dc18 4967 }
df7492f9
KH		 4968
4969 coding->produced_char += produced_chars;
4970 coding->produced = dst - coding->destination;
4971 return 0;
1397dc18
KH		 4972}
4973
df7492f9 4974
1397dc18 4975\f
df7492f9 4976/*** 10, 11. no-conversion handlers ***/
4ed46869 4977
b73bfc1c 4978/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
4ed46869 4979
b73bfc1c 4980static void
df7492f9 4981decode_coding_raw_text (coding)
4ed46869 4982 struct coding_system *coding;
4ed46869 4983{
119852e7
KH		 4984 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
4985
df7492f9 4986 coding->chars_at_source = 1;
119852e7
KH		 4987 coding->consumed_char = coding->src_chars;
4988 coding->consumed = coding->src_bytes;
4989 if (eol_crlf && coding->source[coding->src_bytes - 1] == '\r')
4990 {
4991 coding->consumed_char--;
4992 coding->consumed--;
4993 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
4994 }
4995 else
4996 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 4997}
4ed46869 4998
df7492f9
KH		 4999static int
5000encode_coding_raw_text (coding)
5001 struct coding_system *coding;
5002{
5003 int multibytep = coding->dst_multibyte;
5004 int *charbuf = coding->charbuf;
5005 int *charbuf_end = coding->charbuf + coding->charbuf_used;
5006 unsigned char *dst = coding->destination + coding->produced;
5007 unsigned char *dst_end = coding->destination + coding->dst_bytes;
a0ed9b27 5008 int produced_chars = 0;
b73bfc1c
KH		 5009 int c;
5010
df7492f9 5011 if (multibytep)
b73bfc1c 5012 {
df7492f9 5013 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
4ed46869 5014
df7492f9
KH		 5015 if (coding->src_multibyte)
5016 while (charbuf < charbuf_end)
5017 {
5018 ASSURE_DESTINATION (safe_room);
5019 c = *charbuf++;
5020 if (ASCII_CHAR_P (c))
5021 EMIT_ONE_ASCII_BYTE (c);
5022 else if (CHAR_BYTE8_P (c))
5023 {
5024 c = CHAR_TO_BYTE8 (c);
5025 EMIT_ONE_BYTE (c);
5026 }
5027 else
5028 {
5029 unsigned char str[MAX_MULTIBYTE_LENGTH], *p0 = str, *p1 = str;
93dec019 5030
df7492f9
KH		 5031 CHAR_STRING_ADVANCE (c, p1);
5032 while (p0 < p1)
9d123124
KH		 5033 {
5034 EMIT_ONE_BYTE (*p0);
5035 p0++;
5036 }
df7492f9
KH		 5037 }
5038 }
b73bfc1c 5039 else
df7492f9
KH		 5040 while (charbuf < charbuf_end)
5041 {
5042 ASSURE_DESTINATION (safe_room);
5043 c = *charbuf++;
5044 EMIT_ONE_BYTE (c);
5045 }
5046 }
5047 else
4ed46869 5048 {
df7492f9 5049 if (coding->src_multibyte)
d46c5b12 5050 {
df7492f9
KH		 5051 int safe_room = MAX_MULTIBYTE_LENGTH;
5052
5053 while (charbuf < charbuf_end)
d46c5b12 5054 {
df7492f9
KH		 5055 ASSURE_DESTINATION (safe_room);
5056 c = *charbuf++;
5057 if (ASCII_CHAR_P (c))
5058 *dst++ = c;
5059 else if (CHAR_BYTE8_P (c))
5060 *dst++ = CHAR_TO_BYTE8 (c);
b73bfc1c 5061 else
df7492f9 5062 CHAR_STRING_ADVANCE (c, dst);
d46c5b12
KH		 5063 }
5064 }
df7492f9
KH		 5065 else
5066 {
5067 ASSURE_DESTINATION (charbuf_end - charbuf);
5068 while (charbuf < charbuf_end && dst < dst_end)
5069 *dst++ = *charbuf++;
8f924df7 5070 }
319a3947 5071 produced_chars = dst - (coding->destination + coding->produced);
4ed46869 5072 }
065e3595 5073 record_conversion_result (coding, CODING_RESULT_SUCCESS);
a0ed9b27 5074 coding->produced_char += produced_chars;
df7492f9
KH		 5075 coding->produced = dst - coding->destination;
5076 return 0;
4ed46869
KH		 5077}
5078
ff0dacd7
KH		 5079/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
5080 Check if a text is encoded in a charset-based coding system. If it
5081 is, return 1, else return 0. */
5082
0a28aafb 5083static int
ff0dacd7 5084detect_coding_charset (coding, detect_info)
df7492f9 5085 struct coding_system *coding;
ff0dacd7 5086 struct coding_detection_info *detect_info;
1397dc18 5087{
065e3595 5088 const unsigned char *src = coding->source, *src_base;
8f924df7 5089 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 5090 int multibytep = coding->src_multibyte;
5091 int consumed_chars = 0;
07295713 5092 Lisp_Object attrs, valids, name;
584948ac 5093 int found = 0;
716b3fa0 5094 int head_ascii = coding->head_ascii;
07295713 5095 int check_latin_extra = 0;
1397dc18 5096
ff0dacd7
KH		 5097 detect_info->checked |= CATEGORY_MASK_CHARSET;
5098
df7492f9
KH		 5099 coding = &coding_categories[coding_category_charset];
5100 attrs = CODING_ID_ATTRS (coding->id);
5101 valids = AREF (attrs, coding_attr_charset_valids);
07295713
KH		 5102 name = CODING_ID_NAME (coding->id);
5103 if (VECTORP (Vlatin_extra_code_table)
cb84a2be 5104 && strcmp ((char *) SDATA (SYMBOL_NAME (name)), "iso-8859-") == 0)
07295713 5105 check_latin_extra = 1;
df7492f9 5106 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
716b3fa0 5107 src += head_ascii;
1397dc18 5108
b73bfc1c 5109 while (1)
1397dc18 5110 {
df7492f9 5111 int c;
716b3fa0
KH		 5112 Lisp_Object val;
5113 struct charset *charset;
5114 int dim, idx;
1397dc18 5115
065e3595 5116 src_base = src;
df7492f9 5117 ONE_MORE_BYTE (c);
065e3595
KH		 5118 if (c < 0)
5119 continue;
716b3fa0
KH		 5120 val = AREF (valids, c);
5121 if (NILP (val))
df7492f9 5122 break;
584948ac 5123 if (c >= 0x80)
07295713
KH		 5124 {
5125 if (c < 0xA0
5126 && check_latin_extra
5127 && NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
5128 break;
5129 found = CATEGORY_MASK_CHARSET;
5130 }
716b3fa0
KH		 5131 if (INTEGERP (val))
5132 {
5133 charset = CHARSET_FROM_ID (XFASTINT (val));
5134 dim = CHARSET_DIMENSION (charset);
5135 for (idx = 1; idx < dim; idx++)
5136 {
5137 if (src == src_end)
5138 goto too_short;
5139 ONE_MORE_BYTE (c);
3ed051d4 5140 if (c < charset->code_space[(dim - 1 - idx) * 2]
716b3fa0
KH		 5141 || c > charset->code_space[(dim - 1 - idx) * 2 + 1])
5142 break;
5143 }
5144 if (idx < dim)
5145 break;
5146 }
5147 else
5148 {
5149 idx = 1;
5150 for (; CONSP (val); val = XCDR (val))
5151 {
5152 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
5153 dim = CHARSET_DIMENSION (charset);
5154 while (idx < dim)
5155 {
5156 if (src == src_end)
5157 goto too_short;
5158 ONE_MORE_BYTE (c);
5159 if (c < charset->code_space[(dim - 1 - idx) * 4]
5160 || c > charset->code_space[(dim - 1 - idx) * 4 + 1])
5161 break;
5162 idx++;
5163 }
5164 if (idx == dim)
5165 {
5166 val = Qnil;
5167 break;
5168 }
5169 }
5170 if (CONSP (val))
5171 break;
5172 }
df7492f9 5173 }
716b3fa0 5174 too_short:
ff0dacd7 5175 detect_info->rejected |= CATEGORY_MASK_CHARSET;
df7492f9 5176 return 0;
4ed46869 5177
df7492f9 5178 no_more_source:
ff0dacd7
KH		 5179 detect_info->found |= found;
5180 return 1;
df7492f9 5181}
b73bfc1c 5182
b73bfc1c 5183static void
df7492f9 5184decode_coding_charset (coding)
4ed46869 5185 struct coding_system *coding;
4ed46869 5186{
8f924df7
KH		 5187 const unsigned char *src = coding->source + coding->consumed;
5188 const unsigned char *src_end = coding->source + coding->src_bytes;
5189 const unsigned char *src_base;
69a80ea3
KH		 5190 int *charbuf = coding->charbuf + coding->charbuf_used;
5191 int *charbuf_end
5192 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 5193 int consumed_chars = 0, consumed_chars_base;
5194 int multibytep = coding->src_multibyte;
24a73b0a 5195 Lisp_Object attrs, charset_list, valids;
ff0dacd7
KH		 5196 int char_offset = coding->produced_char;
5197 int last_offset = char_offset;
5198 int last_id = charset_ascii;
119852e7
KH		 5199 int eol_crlf = EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
5200 int byte_after_cr = -1;
df7492f9 5201
24a73b0a 5202 CODING_GET_INFO (coding, attrs, charset_list);
4eb6d3f1 5203 valids = AREF (attrs, coding_attr_charset_valids);
b73bfc1c 5204
df7492f9 5205 while (1)
4ed46869 5206 {
4eb6d3f1 5207 int c;
24a73b0a
KH		 5208 Lisp_Object val;
5209 struct charset *charset;
5210 int dim;
5211 int len = 1;
5212 unsigned code;
df7492f9
KH		 5213
5214 src_base = src;
5215 consumed_chars_base = consumed_chars;
b73bfc1c 5216
df7492f9 5217 if (charbuf >= charbuf_end)
b71f6f73
KH		 5218 {
5219 if (byte_after_cr >= 0)
5220 src_base--;
5221 break;
5222 }
df7492f9 5223
119852e7
KH		 5224 if (byte_after_cr >= 0)
5225 {
5226 c = byte_after_cr;
5227 byte_after_cr = -1;
5228 }
5229 else
5230 {
5231 ONE_MORE_BYTE (c);
5232 if (eol_crlf && c == '\r')
5233 ONE_MORE_BYTE (byte_after_cr);
5234 }
065e3595
KH		 5235 if (c < 0)
5236 goto invalid_code;
24a73b0a
KH		 5237 code = c;
5238
5239 val = AREF (valids, c);
1b17adfd 5240 if (! INTEGERP (val) && ! CONSP (val))
24a73b0a
KH		 5241 goto invalid_code;
5242 if (INTEGERP (val))
d46c5b12 5243 {
24a73b0a
KH		 5244 charset = CHARSET_FROM_ID (XFASTINT (val));
5245 dim = CHARSET_DIMENSION (charset);
5246 while (len < dim)
b73bfc1c 5247 {
24a73b0a
KH		 5248 ONE_MORE_BYTE (c);
5249 code = (code << 8) | c;
5250 len++;
b73bfc1c 5251 }
24a73b0a
KH		 5252 CODING_DECODE_CHAR (coding, src, src_base, src_end,
5253 charset, code, c);
d46c5b12 5254 }
df7492f9 5255 else
d46c5b12 5256 {
24a73b0a
KH		 5257 /* VAL is a list of charset IDs. It is assured that the
5258 list is sorted by charset dimensions (smaller one
5259 comes first). */
5260 while (CONSP (val))
4eb6d3f1 5261 {
24a73b0a 5262 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
c7c66a95 5263 dim = CHARSET_DIMENSION (charset);
f9d71dcd 5264 while (len < dim)
4eb6d3f1 5265 {
acb2a965
KH		 5266 ONE_MORE_BYTE (c);
5267 code = (code << 8) | c;
f9d71dcd 5268 len++;
4eb6d3f1 5269 }
24a73b0a
KH		 5270 CODING_DECODE_CHAR (coding, src, src_base,
5271 src_end, charset, code, c);
5272 if (c >= 0)
5273 break;
5274 val = XCDR (val);
ff0dacd7 5275 }
d46c5b12 5276 }
24a73b0a
KH		 5277 if (c < 0)
5278 goto invalid_code;
5279 if (charset->id != charset_ascii
5280 && last_id != charset->id)
5281 {
5282 if (last_id != charset_ascii)
69a80ea3 5283 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 5284 last_id = charset->id;
5285 last_offset = char_offset;
5286 }
5287
df7492f9 5288 *charbuf++ = c;
ff0dacd7 5289 char_offset++;
df7492f9
KH		 5290 continue;
5291
5292 invalid_code:
5293 src = src_base;
5294 consumed_chars = consumed_chars_base;
5295 ONE_MORE_BYTE (c);
065e3595 5296 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 5297 char_offset++;
df7492f9 5298 coding->errors++;
4ed46869
KH		 5299 }
5300
df7492f9 5301 no_more_source:
ff0dacd7 5302 if (last_id != charset_ascii)
69a80ea3 5303 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 5304 coding->consumed_char += consumed_chars_base;
5305 coding->consumed = src_base - coding->source;
5306 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 5307}
5308
df7492f9
KH		 5309static int
5310encode_coding_charset (coding)
4ed46869 5311 struct coding_system *coding;
4ed46869 5312{
df7492f9
KH		 5313 int multibytep = coding->dst_multibyte;
5314 int *charbuf = coding->charbuf;
5315 int *charbuf_end = charbuf + coding->charbuf_used;
5316 unsigned char *dst = coding->destination + coding->produced;
5317 unsigned char *dst_end = coding->destination + coding->dst_bytes;
5318 int safe_room = MAX_MULTIBYTE_LENGTH;
5319 int produced_chars = 0;
24a73b0a 5320 Lisp_Object attrs, charset_list;
df7492f9 5321 int ascii_compatible;
b73bfc1c 5322 int c;
b73bfc1c 5323
24a73b0a 5324 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 5325 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
fb88bf2d 5326
df7492f9 5327 while (charbuf < charbuf_end)
4ed46869 5328 {
4eb6d3f1 5329 struct charset *charset;
df7492f9 5330 unsigned code;
8f924df7 5331
df7492f9
KH		 5332 ASSURE_DESTINATION (safe_room);
5333 c = *charbuf++;
5334 if (ascii_compatible && ASCII_CHAR_P (c))
5335 EMIT_ONE_ASCII_BYTE (c);
16eafb5d 5336 else if (CHAR_BYTE8_P (c))
4ed46869 5337 {
16eafb5d
KH		 5338 c = CHAR_TO_BYTE8 (c);
5339 EMIT_ONE_BYTE (c);
d46c5b12 5340 }
d46c5b12 5341 else
b73bfc1c 5342 {
4eb6d3f1
KH		 5343 charset = char_charset (c, charset_list, &code);
5344 if (charset)
5345 {
5346 if (CHARSET_DIMENSION (charset) == 1)
5347 EMIT_ONE_BYTE (code);
5348 else if (CHARSET_DIMENSION (charset) == 2)
5349 EMIT_TWO_BYTES (code >> 8, code & 0xFF);
5350 else if (CHARSET_DIMENSION (charset) == 3)
5351 EMIT_THREE_BYTES (code >> 16, (code >> 8) & 0xFF, code & 0xFF);
5352 else
5353 EMIT_FOUR_BYTES (code >> 24, (code >> 16) & 0xFF,
5354 (code >> 8) & 0xFF, code & 0xFF);
5355 }
5356 else
41cbe562
KH		 5357 {
5358 if (coding->mode & CODING_MODE_SAFE_ENCODING)
5359 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
5360 else
5361 c = coding->default_char;
5362 EMIT_ONE_BYTE (c);
5363 }
4ed46869 5364 }
4ed46869
KH		 5365 }
5366
065e3595 5367 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 5368 coding->produced_char += produced_chars;
5369 coding->produced = dst - coding->destination;
5370 return 0;
4ed46869
KH		 5371}
5372
5373\f
1397dc18 5374/*** 7. C library functions ***/
4ed46869 5375
df7492f9
KH		 5376/* Setup coding context CODING from information about CODING_SYSTEM.
5377 If CODING_SYSTEM is nil, `no-conversion' is assumed. If
5378 CODING_SYSTEM is invalid, signal an error. */
4ed46869 5379
ec6d2bb8 5380void
e0e989f6
KH		 5381setup_coding_system (coding_system, coding)
5382 Lisp_Object coding_system;
4ed46869
KH		 5383 struct coding_system *coding;
5384{
df7492f9
KH		 5385 Lisp_Object attrs;
5386 Lisp_Object eol_type;
5387 Lisp_Object coding_type;
4608c386 5388 Lisp_Object val;
4ed46869 5389
df7492f9 5390 if (NILP (coding_system))
ae6f73fa 5391 coding_system = Qundecided;
c07c8e12 5392
df7492f9 5393 CHECK_CODING_SYSTEM_GET_ID (coding_system, coding->id);
1f5dbf34 5394
df7492f9
KH		 5395 attrs = CODING_ID_ATTRS (coding->id);
5396 eol_type = CODING_ID_EOL_TYPE (coding->id);
1f5dbf34 5397
df7492f9
KH		 5398 coding->mode = 0;
5399 coding->head_ascii = -1;
4a015c45
KH		 5400 if (VECTORP (eol_type))
5401 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5402 | CODING_REQUIRE_DETECTION_MASK);
5403 else if (! EQ (eol_type, Qunix))
5404 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5405 | CODING_REQUIRE_ENCODING_MASK);
5406 else
5407 coding->common_flags = 0;
5e5c78be
KH		 5408 if (! NILP (CODING_ATTR_POST_READ (attrs)))
5409 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5410 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
5411 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
8f924df7
KH		 5412 if (! NILP (CODING_ATTR_FOR_UNIBYTE (attrs)))
5413 coding->common_flags |= CODING_FOR_UNIBYTE_MASK;
1f5dbf34 5414
df7492f9 5415 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 5416 coding->max_charset_id = SCHARS (val) - 1;
5417 coding->safe_charsets = (char *) SDATA (val);
df7492f9 5418 coding->default_char = XINT (CODING_ATTR_DEFAULT_CHAR (attrs));
4608c386 5419
df7492f9
KH		 5420 coding_type = CODING_ATTR_TYPE (attrs);
5421 if (EQ (coding_type, Qundecided))
d46c5b12 5422 {
df7492f9
KH		 5423 coding->detector = NULL;
5424 coding->decoder = decode_coding_raw_text;
5425 coding->encoder = encode_coding_raw_text;
5426 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 5427 }
df7492f9 5428 else if (EQ (coding_type, Qiso_2022))
d46c5b12 5429 {
df7492f9
KH		 5430 int i;
5431 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5432
5433 /* Invoke graphic register 0 to plane 0. */
5434 CODING_ISO_INVOCATION (coding, 0) = 0;
5435 /* Invoke graphic register 1 to plane 1 if we can use 8-bit. */
5436 CODING_ISO_INVOCATION (coding, 1)
5437 = (flags & CODING_ISO_FLAG_SEVEN_BITS ? -1 : 1);
5438 /* Setup the initial status of designation. */
5439 for (i = 0; i < 4; i++)
5440 CODING_ISO_DESIGNATION (coding, i) = CODING_ISO_INITIAL (coding, i);
5441 /* Not single shifting initially. */
5442 CODING_ISO_SINGLE_SHIFTING (coding) = 0;
5443 /* Beginning of buffer should also be regarded as bol. */
5444 CODING_ISO_BOL (coding) = 1;
5445 coding->detector = detect_coding_iso_2022;
5446 coding->decoder = decode_coding_iso_2022;
5447 coding->encoder = encode_coding_iso_2022;
5448 if (flags & CODING_ISO_FLAG_SAFE)
5449 coding->mode |= CODING_MODE_SAFE_ENCODING;
d46c5b12 5450 coding->common_flags
df7492f9
KH		 5451 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5452 | CODING_REQUIRE_FLUSHING_MASK);
5453 if (flags & CODING_ISO_FLAG_COMPOSITION)
5454 coding->common_flags |= CODING_ANNOTATE_COMPOSITION_MASK;
ff0dacd7
KH		 5455 if (flags & CODING_ISO_FLAG_DESIGNATION)
5456 coding->common_flags |= CODING_ANNOTATE_CHARSET_MASK;
df7492f9
KH		 5457 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5458 {
5459 setup_iso_safe_charsets (attrs);
5460 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 5461 coding->max_charset_id = SCHARS (val) - 1;
5462 coding->safe_charsets = (char *) SDATA (val);
df7492f9
KH		 5463 }
5464 CODING_ISO_FLAGS (coding) = flags;
d46c5b12 5465 }
df7492f9 5466 else if (EQ (coding_type, Qcharset))
d46c5b12 5467 {
df7492f9
KH		 5468 coding->detector = detect_coding_charset;
5469 coding->decoder = decode_coding_charset;
5470 coding->encoder = encode_coding_charset;
d46c5b12 5471 coding->common_flags
df7492f9 5472 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
d46c5b12 5473 }
df7492f9 5474 else if (EQ (coding_type, Qutf_8))
d46c5b12 5475 {
a470d443
KH		 5476 val = AREF (attrs, coding_attr_utf_bom);
5477 CODING_UTF_8_BOM (coding) = (CONSP (val) ? utf_detect_bom
5478 : EQ (val, Qt) ? utf_with_bom
5479 : utf_without_bom);
df7492f9
KH		 5480 coding->detector = detect_coding_utf_8;
5481 coding->decoder = decode_coding_utf_8;
5482 coding->encoder = encode_coding_utf_8;
5483 coding->common_flags
5484 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
a470d443
KH		 5485 if (CODING_UTF_8_BOM (coding) == utf_detect_bom)
5486 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
df7492f9
KH		 5487 }
5488 else if (EQ (coding_type, Qutf_16))
5489 {
a470d443
KH		 5490 val = AREF (attrs, coding_attr_utf_bom);
5491 CODING_UTF_16_BOM (coding) = (CONSP (val) ? utf_detect_bom
5492 : EQ (val, Qt) ? utf_with_bom
5493 : utf_without_bom);
df7492f9 5494 val = AREF (attrs, coding_attr_utf_16_endian);
b49a1807 5495 CODING_UTF_16_ENDIAN (coding) = (EQ (val, Qbig) ? utf_16_big_endian
df7492f9 5496 : utf_16_little_endian);
e19c3639 5497 CODING_UTF_16_SURROGATE (coding) = 0;
df7492f9
KH		 5498 coding->detector = detect_coding_utf_16;
5499 coding->decoder = decode_coding_utf_16;
5500 coding->encoder = encode_coding_utf_16;
5501 coding->common_flags
5502 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
a470d443 5503 if (CODING_UTF_16_BOM (coding) == utf_detect_bom)
b49a1807 5504 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 5505 }
df7492f9 5506 else if (EQ (coding_type, Qccl))
4ed46869 5507 {
df7492f9
KH		 5508 coding->detector = detect_coding_ccl;
5509 coding->decoder = decode_coding_ccl;
5510 coding->encoder = encode_coding_ccl;
c952af22 5511 coding->common_flags
df7492f9
KH		 5512 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5513 | CODING_REQUIRE_FLUSHING_MASK);
5514 }
5515 else if (EQ (coding_type, Qemacs_mule))
5516 {
5517 coding->detector = detect_coding_emacs_mule;
5518 coding->decoder = decode_coding_emacs_mule;
5519 coding->encoder = encode_coding_emacs_mule;
c952af22 5520 coding->common_flags
df7492f9
KH		 5521 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5522 if (! NILP (AREF (attrs, coding_attr_emacs_mule_full))
5523 && ! EQ (CODING_ATTR_CHARSET_LIST (attrs), Vemacs_mule_charset_list))
5524 {
5525 Lisp_Object tail, safe_charsets;
5526 int max_charset_id = 0;
5527
5528 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5529 tail = XCDR (tail))
5530 if (max_charset_id < XFASTINT (XCAR (tail)))
5531 max_charset_id = XFASTINT (XCAR (tail));
5532 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
5533 make_number (255));
5534 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5535 tail = XCDR (tail))
8f924df7 5536 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 5537 coding->max_charset_id = max_charset_id;
8f924df7 5538 coding->safe_charsets = (char *) SDATA (safe_charsets);
df7492f9
KH		 5539 }
5540 }
5541 else if (EQ (coding_type, Qshift_jis))
5542 {
5543 coding->detector = detect_coding_sjis;
5544 coding->decoder = decode_coding_sjis;
5545 coding->encoder = encode_coding_sjis;
c952af22 5546 coding->common_flags
df7492f9
KH		 5547 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5548 }
5549 else if (EQ (coding_type, Qbig5))
5550 {
5551 coding->detector = detect_coding_big5;
5552 coding->decoder = decode_coding_big5;
5553 coding->encoder = encode_coding_big5;
c952af22 5554 coding->common_flags
df7492f9
KH		 5555 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5556 }
5557 else /* EQ (coding_type, Qraw_text) */
ec6d2bb8 5558 {
df7492f9
KH		 5559 coding->detector = NULL;
5560 coding->decoder = decode_coding_raw_text;
5561 coding->encoder = encode_coding_raw_text;
ea29edf2
KH		 5562 if (! EQ (eol_type, Qunix))
5563 {
5564 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5565 if (! VECTORP (eol_type))
5566 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
5567 }
5568
4ed46869 5569 }
4ed46869 5570
df7492f9 5571 return;
4ed46869
KH		 5572}
5573
0ff61e78
KH		 5574/* Return a list of charsets supported by CODING. */
5575
5576Lisp_Object
5577coding_charset_list (coding)
5578 struct coding_system *coding;
5579{
35befdaa 5580 Lisp_Object attrs, charset_list;
0ff61e78
KH		 5581
5582 CODING_GET_INFO (coding, attrs, charset_list);
5583 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5584 {
5585 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5586
5587 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5588 charset_list = Viso_2022_charset_list;
5589 }
5590 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5591 {
5592 charset_list = Vemacs_mule_charset_list;
5593 }
5594 return charset_list;
5595}
5596
5597
e9f91ece
KH		 5598/* Return a list of charsets supported by CODING-SYSTEM. */
5599
5600Lisp_Object
5601coding_system_charset_list (coding_system)
5602 Lisp_Object coding_system;
5603{
5604 int id;
5605 Lisp_Object attrs, charset_list;
5606
5607 CHECK_CODING_SYSTEM_GET_ID (coding_system, id);
5608 attrs = CODING_ID_ATTRS (id);
5609
5610 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5611 {
5612 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5613
5614 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5615 charset_list = Viso_2022_charset_list;
5616 else
5617 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
5618 }
5619 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5620 {
5621 charset_list = Vemacs_mule_charset_list;
5622 }
5623 else
5624 {
5625 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
5626 }
5627 return charset_list;
5628}
5629
5630
df7492f9
KH		 5631/* Return raw-text or one of its subsidiaries that has the same
5632 eol_type as CODING-SYSTEM. */
ec6d2bb8 5633
df7492f9
KH		 5634Lisp_Object
5635raw_text_coding_system (coding_system)
5636 Lisp_Object coding_system;
ec6d2bb8 5637{
0be8721c 5638 Lisp_Object spec, attrs;
df7492f9 5639 Lisp_Object eol_type, raw_text_eol_type;
ec6d2bb8 5640
d3e4cb56
KH		 5641 if (NILP (coding_system))
5642 return Qraw_text;
df7492f9
KH		 5643 spec = CODING_SYSTEM_SPEC (coding_system);
5644 attrs = AREF (spec, 0);
ec6d2bb8 5645
df7492f9
KH		 5646 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
5647 return coding_system;
ec6d2bb8 5648
df7492f9
KH		 5649 eol_type = AREF (spec, 2);
5650 if (VECTORP (eol_type))
5651 return Qraw_text;
5652 spec = CODING_SYSTEM_SPEC (Qraw_text);
5653 raw_text_eol_type = AREF (spec, 2);
5654 return (EQ (eol_type, Qunix) ? AREF (raw_text_eol_type, 0)
5655 : EQ (eol_type, Qdos) ? AREF (raw_text_eol_type, 1)
5656 : AREF (raw_text_eol_type, 2));
ec6d2bb8
KH		 5657}
5658
54f78171 5659
df7492f9
KH		 5660/* If CODING_SYSTEM doesn't specify end-of-line format but PARENT
5661 does, return one of the subsidiary that has the same eol-spec as
fcbcfb64
KH		 5662 PARENT. Otherwise, return CODING_SYSTEM. If PARENT is nil,
5663 inherit end-of-line format from the system's setting
5664 (system_eol_type). */
df7492f9
KH		 5665
5666Lisp_Object
5667coding_inherit_eol_type (coding_system, parent)
b74e4686 5668 Lisp_Object coding_system, parent;
54f78171 5669{
3e139625 5670 Lisp_Object spec, eol_type;
54f78171 5671
d3e4cb56
KH		 5672 if (NILP (coding_system))
5673 coding_system = Qraw_text;
df7492f9 5674 spec = CODING_SYSTEM_SPEC (coding_system);
df7492f9 5675 eol_type = AREF (spec, 2);
fcbcfb64 5676 if (VECTORP (eol_type))
df7492f9 5677 {
df7492f9
KH		 5678 Lisp_Object parent_eol_type;
5679
fcbcfb64
KH		 5680 if (! NILP (parent))
5681 {
5682 Lisp_Object parent_spec;
5683
4a015c45 5684 parent_spec = CODING_SYSTEM_SPEC (parent);
fcbcfb64
KH		 5685 parent_eol_type = AREF (parent_spec, 2);
5686 }
5687 else
5688 parent_eol_type = system_eol_type;
df7492f9
KH		 5689 if (EQ (parent_eol_type, Qunix))
5690 coding_system = AREF (eol_type, 0);
5691 else if (EQ (parent_eol_type, Qdos))
5692 coding_system = AREF (eol_type, 1);
5693 else if (EQ (parent_eol_type, Qmac))
5694 coding_system = AREF (eol_type, 2);
54f78171 5695 }
df7492f9 5696 return coding_system;
54f78171
KH		 5697}
5698
4ed46869
KH		 5699/* Emacs has a mechanism to automatically detect a coding system if it
5700 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
5701 it's impossible to distinguish some coding systems accurately
5702 because they use the same range of codes. So, at first, coding
5703 systems are categorized into 7, those are:
5704
0ef69138 5705 o coding-category-emacs-mule
4ed46869
KH		 5706
5707 The category for a coding system which has the same code range
5708 as Emacs' internal format. Assigned the coding-system (Lisp
0ef69138 5709 symbol) `emacs-mule' by default.
4ed46869
KH		 5710
5711 o coding-category-sjis
5712
5713 The category for a coding system which has the same code range
5714 as SJIS. Assigned the coding-system (Lisp
7717c392 5715 symbol) `japanese-shift-jis' by default.
4ed46869
KH		 5716
5717 o coding-category-iso-7
5718
5719 The category for a coding system which has the same code range
7717c392 5720 as ISO2022 of 7-bit environment. This doesn't use any locking
d46c5b12
KH		 5721 shift and single shift functions. This can encode/decode all
5722 charsets. Assigned the coding-system (Lisp symbol)
5723 `iso-2022-7bit' by default.
5724
5725 o coding-category-iso-7-tight
5726
5727 Same as coding-category-iso-7 except that this can
5728 encode/decode only the specified charsets.
4ed46869
KH		 5729
5730 o coding-category-iso-8-1
5731
5732 The category for a coding system which has the same code range
5733 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 5734 for DIMENSION1 charset. This doesn't use any locking shift
5735 and single shift functions. Assigned the coding-system (Lisp
5736 symbol) `iso-latin-1' by default.
4ed46869
KH		 5737
5738 o coding-category-iso-8-2
5739
5740 The category for a coding system which has the same code range
5741 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 5742 for DIMENSION2 charset. This doesn't use any locking shift
5743 and single shift functions. Assigned the coding-system (Lisp
5744 symbol) `japanese-iso-8bit' by default.
4ed46869 5745
7717c392 5746 o coding-category-iso-7-else
4ed46869
KH		 5747
5748 The category for a coding system which has the same code range
df7492f9 5749 as ISO2022 of 7-bit environemnt but uses locking shift or
7717c392
KH		 5750 single shift functions. Assigned the coding-system (Lisp
5751 symbol) `iso-2022-7bit-lock' by default.
5752
5753 o coding-category-iso-8-else
5754
5755 The category for a coding system which has the same code range
df7492f9 5756 as ISO2022 of 8-bit environemnt but uses locking shift or
7717c392
KH		 5757 single shift functions. Assigned the coding-system (Lisp
5758 symbol) `iso-2022-8bit-ss2' by default.
4ed46869
KH		 5759
5760 o coding-category-big5
5761
5762 The category for a coding system which has the same code range
5763 as BIG5. Assigned the coding-system (Lisp symbol)
e0e989f6 5764 `cn-big5' by default.
4ed46869 5765
fa42c37f
KH		 5766 o coding-category-utf-8
5767
5768 The category for a coding system which has the same code range
6e76ae91 5769 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
fa42c37f
KH		 5770 symbol) `utf-8' by default.
5771
5772 o coding-category-utf-16-be
5773
5774 The category for a coding system in which a text has an
5775 Unicode signature (cf. Unicode Standard) in the order of BIG
5776 endian at the head. Assigned the coding-system (Lisp symbol)
5777 `utf-16-be' by default.
5778
5779 o coding-category-utf-16-le
5780
5781 The category for a coding system in which a text has an
5782 Unicode signature (cf. Unicode Standard) in the order of
5783 LITTLE endian at the head. Assigned the coding-system (Lisp
5784 symbol) `utf-16-le' by default.
5785
1397dc18
KH		 5786 o coding-category-ccl
5787
5788 The category for a coding system of which encoder/decoder is
5789 written in CCL programs. The default value is nil, i.e., no
5790 coding system is assigned.
5791
4ed46869
KH		 5792 o coding-category-binary
5793
5794 The category for a coding system not categorized in any of the
5795 above. Assigned the coding-system (Lisp symbol)
e0e989f6 5796 `no-conversion' by default.
4ed46869
KH		 5797
5798 Each of them is a Lisp symbol and the value is an actual
df7492f9 5799 `coding-system's (this is also a Lisp symbol) assigned by a user.
4ed46869
KH		 5800 What Emacs does actually is to detect a category of coding system.
5801 Then, it uses a `coding-system' assigned to it. If Emacs can't
df7492f9 5802 decide only one possible category, it selects a category of the
4ed46869
KH		 5803 highest priority. Priorities of categories are also specified by a
5804 user in a Lisp variable `coding-category-list'.
5805
5806*/
5807
df7492f9
KH		 5808#define EOL_SEEN_NONE 0
5809#define EOL_SEEN_LF 1
5810#define EOL_SEEN_CR 2
5811#define EOL_SEEN_CRLF 4
66cfb530 5812
ff0dacd7
KH		 5813/* Detect how end-of-line of a text of length SRC_BYTES pointed by
5814 SOURCE is encoded. If CATEGORY is one of
5815 coding_category_utf_16_XXXX, assume that CR and LF are encoded by
5816 two-byte, else they are encoded by one-byte.
5817
5818 Return one of EOL_SEEN_XXX. */
4ed46869 5819
bc4bc72a 5820#define MAX_EOL_CHECK_COUNT 3
d46c5b12
KH		 5821
5822static int
89528eb3 5823detect_eol (source, src_bytes, category)
f6cbaf43 5824 const unsigned char *source;
df7492f9 5825 EMACS_INT src_bytes;
89528eb3 5826 enum coding_category category;
4ed46869 5827{
f6cbaf43 5828 const unsigned char *src = source, *src_end = src + src_bytes;
4ed46869 5829 unsigned char c;
df7492f9
KH		 5830 int total = 0;
5831 int eol_seen = EOL_SEEN_NONE;
4ed46869 5832
89528eb3 5833 if ((1 << category) & CATEGORY_MASK_UTF_16)
4ed46869 5834 {
df7492f9 5835 int msb, lsb;
fa42c37f 5836
89528eb3
KH		 5837 msb = category == (coding_category_utf_16_le
5838 | coding_category_utf_16_le_nosig);
df7492f9 5839 lsb = 1 - msb;
fa42c37f 5840
df7492f9 5841 while (src + 1 < src_end)
fa42c37f 5842 {
df7492f9
KH		 5843 c = src[lsb];
5844 if (src[msb] == 0 && (c == '\n' || c == '\r'))
fa42c37f 5845 {
df7492f9
KH		 5846 int this_eol;
5847
5848 if (c == '\n')
5849 this_eol = EOL_SEEN_LF;
5850 else if (src + 3 >= src_end
5851 || src[msb + 2] != 0
5852 || src[lsb + 2] != '\n')
5853 this_eol = EOL_SEEN_CR;
fa42c37f 5854 else
75f4f1ac
EZ		 5855 {
5856 this_eol = EOL_SEEN_CRLF;
5857 src += 2;
5858 }
df7492f9
KH		 5859
5860 if (eol_seen == EOL_SEEN_NONE)
5861 /* This is the first end-of-line. */
5862 eol_seen = this_eol;
5863 else if (eol_seen != this_eol)
fa42c37f 5864 {
75f4f1ac
EZ		 5865 /* The found type is different from what found before.
5866 Allow for stray ^M characters in DOS EOL files. */
5867 if (eol_seen == EOL_SEEN_CR && this_eol == EOL_SEEN_CRLF
5868 || eol_seen == EOL_SEEN_CRLF && this_eol == EOL_SEEN_CR)
5869 eol_seen = EOL_SEEN_CRLF;
5870 else
5871 {
5872 eol_seen = EOL_SEEN_LF;
5873 break;
5874 }
fa42c37f 5875 }
df7492f9
KH		 5876 if (++total == MAX_EOL_CHECK_COUNT)
5877 break;
fa42c37f 5878 }
df7492f9 5879 src += 2;
fa42c37f 5880 }
bcf26d6a 5881 }
d46c5b12 5882 else
c4825358 5883 {
df7492f9 5884 while (src < src_end)
27901516 5885 {
df7492f9
KH		 5886 c = *src++;
5887 if (c == '\n' || c == '\r')
5888 {
5889 int this_eol;
d46c5b12 5890
df7492f9
KH		 5891 if (c == '\n')
5892 this_eol = EOL_SEEN_LF;
5893 else if (src >= src_end || *src != '\n')
5894 this_eol = EOL_SEEN_CR;
5895 else
5896 this_eol = EOL_SEEN_CRLF, src++;
d46c5b12 5897
df7492f9
KH		 5898 if (eol_seen == EOL_SEEN_NONE)
5899 /* This is the first end-of-line. */
5900 eol_seen = this_eol;
5901 else if (eol_seen != this_eol)
5902 {
75f4f1ac
EZ		 5903 /* The found type is different from what found before.
5904 Allow for stray ^M characters in DOS EOL files. */
5905 if (eol_seen == EOL_SEEN_CR && this_eol == EOL_SEEN_CRLF
5906 || eol_seen == EOL_SEEN_CRLF && this_eol == EOL_SEEN_CR)
5907 eol_seen = EOL_SEEN_CRLF;
5908 else
5909 {
5910 eol_seen = EOL_SEEN_LF;
5911 break;
5912 }
df7492f9
KH		 5913 }
5914 if (++total == MAX_EOL_CHECK_COUNT)
5915 break;
5916 }
5917 }
73be902c 5918 }
df7492f9 5919 return eol_seen;
73be902c
KH		 5920}
5921
df7492f9 5922
24a73b0a 5923static Lisp_Object
df7492f9
KH		 5924adjust_coding_eol_type (coding, eol_seen)
5925 struct coding_system *coding;
5926 int eol_seen;
73be902c 5927{
0be8721c 5928 Lisp_Object eol_type;
8f924df7 5929
df7492f9
KH		 5930 eol_type = CODING_ID_EOL_TYPE (coding->id);
5931 if (eol_seen & EOL_SEEN_LF)
24a73b0a
KH		 5932 {
5933 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 0));
5934 eol_type = Qunix;
5935 }
6f197c07 5936 else if (eol_seen & EOL_SEEN_CRLF)
24a73b0a
KH		 5937 {
5938 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 1));
5939 eol_type = Qdos;
5940 }
6f197c07 5941 else if (eol_seen & EOL_SEEN_CR)
24a73b0a
KH		 5942 {
5943 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 2));
5944 eol_type = Qmac;
5945 }
5946 return eol_type;
d46c5b12 5947}
4ed46869 5948
df7492f9
KH		 5949/* Detect how a text specified in CODING is encoded. If a coding
5950 system is detected, update fields of CODING by the detected coding
5951 system. */
0a28aafb 5952
df7492f9
KH		 5953void
5954detect_coding (coding)
d46c5b12 5955 struct coding_system *coding;
d46c5b12 5956{
8f924df7 5957 const unsigned char *src, *src_end;
73cce38d 5958 int saved_mode = coding->mode;
d46c5b12 5959
df7492f9
KH		 5960 coding->consumed = coding->consumed_char = 0;
5961 coding->produced = coding->produced_char = 0;
5962 coding_set_source (coding);
1c3478b0 5963
df7492f9 5964 src_end = coding->source + coding->src_bytes;
c0e16b14 5965 coding->head_ascii = 0;
1c3478b0 5966
df7492f9
KH		 5967 /* If we have not yet decided the text encoding type, detect it
5968 now. */
5969 if (EQ (CODING_ATTR_TYPE (CODING_ID_ATTRS (coding->id)), Qundecided))
b73bfc1c 5970 {
df7492f9 5971 int c, i;
6cb21a4f 5972 struct coding_detection_info detect_info;
2f3cbb32 5973 int null_byte_found = 0, eight_bit_found = 0;
df7492f9 5974
6cb21a4f 5975 detect_info.checked = detect_info.found = detect_info.rejected = 0;
2f3cbb32 5976 for (src = coding->source; src < src_end; src++)
d46c5b12 5977 {
df7492f9 5978 c = *src;
6cb21a4f 5979 if (c & 0x80)
6cb21a4f 5980 {
2f3cbb32 5981 eight_bit_found = 1;
2f3cbb32
KH		 5982 if (null_byte_found)
5983 break;
5984 }
5985 else if (c < 0x20)
5986 {
5987 if ((c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
5988 && ! inhibit_iso_escape_detection
5989 && ! detect_info.checked)
6cb21a4f 5990 {
2f3cbb32
KH		 5991 if (detect_coding_iso_2022 (coding, &detect_info))
5992 {
5993 /* We have scanned the whole data. */
5994 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
c0e16b14
KH		 5995 {
5996 /* We didn't find an 8-bit code. We may
5997 have found a null-byte, but it's very
5998 rare that a binary file confirm to
5999 ISO-2022. */
6000 src = src_end;
6001 coding->head_ascii = src - coding->source;
6002 }
6003 detect_info.rejected |= ~CATEGORY_MASK_ISO_ESCAPE;
2f3cbb32
KH		 6004 break;
6005 }
6006 }
97b1b294 6007 else if (! c && !inhibit_null_byte_detection)
2f3cbb32
KH		 6008 {
6009 null_byte_found = 1;
6010 if (eight_bit_found)
6011 break;
6cb21a4f 6012 }
c006c0c8
KH		 6013 if (! eight_bit_found)
6014 coding->head_ascii++;
6cb21a4f 6015 }
c006c0c8 6016 else if (! eight_bit_found)
c0e16b14 6017 coding->head_ascii++;
d46c5b12 6018 }
df7492f9 6019
2f3cbb32
KH		 6020 if (null_byte_found || eight_bit_found
6021 || coding->head_ascii < coding->src_bytes
6cb21a4f 6022 || detect_info.found)
d46c5b12 6023 {
ff0dacd7
KH		 6024 enum coding_category category;
6025 struct coding_system *this;
df7492f9 6026
6cb21a4f
KH		 6027 if (coding->head_ascii == coding->src_bytes)
6028 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
6029 for (i = 0; i < coding_category_raw_text; i++)
6030 {
6031 category = coding_priorities[i];
6032 this = coding_categories + category;
6033 if (detect_info.found & (1 << category))
24a73b0a 6034 break;
6cb21a4f
KH		 6035 }
6036 else
2f3cbb32
KH		 6037 {
6038 if (null_byte_found)
ff0dacd7 6039 {
2f3cbb32
KH		 6040 detect_info.checked |= ~CATEGORY_MASK_UTF_16;
6041 detect_info.rejected |= ~CATEGORY_MASK_UTF_16;
ff0dacd7 6042 }
2f3cbb32
KH		 6043 for (i = 0; i < coding_category_raw_text; i++)
6044 {
6045 category = coding_priorities[i];
6046 this = coding_categories + category;
6047 if (this->id < 0)
6048 {
6049 /* No coding system of this category is defined. */
6050 detect_info.rejected |= (1 << category);
6051 }
6052 else if (category >= coding_category_raw_text)
6053 continue;
6054 else if (detect_info.checked & (1 << category))
6055 {
6056 if (detect_info.found & (1 << category))
6057 break;
6058 }
6059 else if ((*(this->detector)) (coding, &detect_info)
6060 && detect_info.found & (1 << category))
6061 {
6062 if (category == coding_category_utf_16_auto)
6063 {
6064 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
6065 category = coding_category_utf_16_le;
6066 else
6067 category = coding_category_utf_16_be;
6068 }
6069 break;
6070 }
6071 }
2f3cbb32 6072 }
c0e16b14
KH		 6073
6074 if (i < coding_category_raw_text)
6075 setup_coding_system (CODING_ID_NAME (this->id), coding);
6076 else if (null_byte_found)
6077 setup_coding_system (Qno_conversion, coding);
6078 else if ((detect_info.rejected & CATEGORY_MASK_ANY)
6079 == CATEGORY_MASK_ANY)
6080 setup_coding_system (Qraw_text, coding);
6081 else if (detect_info.rejected)
6082 for (i = 0; i < coding_category_raw_text; i++)
6083 if (! (detect_info.rejected & (1 << coding_priorities[i])))
6084 {
6085 this = coding_categories + coding_priorities[i];
6086 setup_coding_system (CODING_ID_NAME (this->id), coding);
6087 break;
6088 }
d46c5b12 6089 }
b73bfc1c 6090 }
a470d443
KH		 6091 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
6092 == coding_category_utf_8_auto)
6093 {
6094 Lisp_Object coding_systems;
6095 struct coding_detection_info detect_info;
6096
6097 coding_systems
6098 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_bom);
6099 detect_info.found = detect_info.rejected = 0;
6100 coding->head_ascii = 0;
6101 if (CONSP (coding_systems)
6102 && detect_coding_utf_8 (coding, &detect_info))
6103 {
6104 if (detect_info.found & CATEGORY_MASK_UTF_8_SIG)
6105 setup_coding_system (XCAR (coding_systems), coding);
6106 else
6107 setup_coding_system (XCDR (coding_systems), coding);
6108 }
6109 }
24a73b0a
KH		 6110 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
6111 == coding_category_utf_16_auto)
b49a1807
KH		 6112 {
6113 Lisp_Object coding_systems;
6114 struct coding_detection_info detect_info;
6115
6116 coding_systems
a470d443 6117 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_bom);
b49a1807 6118 detect_info.found = detect_info.rejected = 0;
a470d443 6119 coding->head_ascii = 0;
b49a1807 6120 if (CONSP (coding_systems)
24a73b0a 6121 && detect_coding_utf_16 (coding, &detect_info))
b49a1807
KH		 6122 {
6123 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
6124 setup_coding_system (XCAR (coding_systems), coding);
24a73b0a 6125 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
b49a1807
KH		 6126 setup_coding_system (XCDR (coding_systems), coding);
6127 }
6128 }
73cce38d 6129 coding->mode = saved_mode;
4ed46869 6130}
4ed46869 6131
d46c5b12 6132
aaaf0b1e 6133static void
df7492f9 6134decode_eol (coding)
aaaf0b1e 6135 struct coding_system *coding;
aaaf0b1e 6136{
24a73b0a
KH		 6137 Lisp_Object eol_type;
6138 unsigned char *p, *pbeg, *pend;
3ed051d4 6139
24a73b0a
KH		 6140 eol_type = CODING_ID_EOL_TYPE (coding->id);
6141 if (EQ (eol_type, Qunix))
6142 return;
6143
6144 if (NILP (coding->dst_object))
6145 pbeg = coding->destination;
6146 else
6147 pbeg = BYTE_POS_ADDR (coding->dst_pos_byte);
6148 pend = pbeg + coding->produced;
6149
6150 if (VECTORP (eol_type))
aaaf0b1e 6151 {
df7492f9 6152 int eol_seen = EOL_SEEN_NONE;
4ed46869 6153
24a73b0a 6154 for (p = pbeg; p < pend; p++)
aaaf0b1e 6155 {
df7492f9
KH		 6156 if (*p == '\n')
6157 eol_seen |= EOL_SEEN_LF;
6158 else if (*p == '\r')
aaaf0b1e 6159 {
df7492f9 6160 if (p + 1 < pend && *(p + 1) == '\n')
aaaf0b1e 6161 {
df7492f9
KH		 6162 eol_seen |= EOL_SEEN_CRLF;
6163 p++;
aaaf0b1e 6164 }
aaaf0b1e 6165 else
df7492f9 6166 eol_seen |= EOL_SEEN_CR;
aaaf0b1e 6167 }
aaaf0b1e 6168 }
75f4f1ac
EZ		 6169 /* Handle DOS-style EOLs in a file with stray ^M characters. */
6170 if ((eol_seen & EOL_SEEN_CRLF) != 0
6171 && (eol_seen & EOL_SEEN_CR) != 0
6172 && (eol_seen & EOL_SEEN_LF) == 0)
6173 eol_seen = EOL_SEEN_CRLF;
6174 else if (eol_seen != EOL_SEEN_NONE
24a73b0a
KH		 6175 && eol_seen != EOL_SEEN_LF
6176 && eol_seen != EOL_SEEN_CRLF
6177 && eol_seen != EOL_SEEN_CR)
6178 eol_seen = EOL_SEEN_LF;
df7492f9 6179 if (eol_seen != EOL_SEEN_NONE)
24a73b0a 6180 eol_type = adjust_coding_eol_type (coding, eol_seen);
aaaf0b1e 6181 }
d46c5b12 6182
24a73b0a 6183 if (EQ (eol_type, Qmac))
27901516 6184 {
24a73b0a 6185 for (p = pbeg; p < pend; p++)
df7492f9
KH		 6186 if (*p == '\r')
6187 *p = '\n';
4ed46869 6188 }
24a73b0a 6189 else if (EQ (eol_type, Qdos))
df7492f9 6190 {
24a73b0a 6191 int n = 0;
b73bfc1c 6192
24a73b0a
KH		 6193 if (NILP (coding->dst_object))
6194 {
4347441b
KH		 6195 /* Start deleting '\r' from the tail to minimize the memory
6196 movement. */
24a73b0a
KH		 6197 for (p = pend - 2; p >= pbeg; p--)
6198 if (*p == '\r')
6199 {
6200 safe_bcopy ((char *) (p + 1), (char *) p, pend-- - p - 1);
6201 n++;
6202 }
6203 }
6204 else
6205 {
4347441b
KH		 6206 int pos_byte = coding->dst_pos_byte;
6207 int pos = coding->dst_pos;
6208 int pos_end = pos + coding->produced_char - 1;
6209
6210 while (pos < pos_end)
6211 {
6212 p = BYTE_POS_ADDR (pos_byte);
6213 if (*p == '\r' && p[1] == '\n')
6214 {
6215 del_range_2 (pos, pos_byte, pos + 1, pos_byte + 1, 0);
6216 n++;
6217 pos_end--;
6218 }
6219 pos++;
69b8522d
KH		 6220 if (coding->dst_multibyte)
6221 pos_byte += BYTES_BY_CHAR_HEAD (*p);
6222 else
6223 pos_byte++;
4347441b 6224 }
24a73b0a
KH		 6225 }
6226 coding->produced -= n;
6227 coding->produced_char -= n;
aaaf0b1e 6228 }
4ed46869
KH		 6229}
6230
7d64c6ad 6231
a6f87d34
KH		 6232/* Return a translation table (or list of them) from coding system
6233 attribute vector ATTRS for encoding (ENCODEP is nonzero) or
6234 decoding (ENCODEP is zero). */
7d64c6ad 6235
e6a54062 6236static Lisp_Object
09ee6fdd
KH		 6237get_translation_table (attrs, encodep, max_lookup)
6238 Lisp_Object attrs;
6239 int encodep, *max_lookup;
7d64c6ad
KH		 6240{
6241 Lisp_Object standard, translation_table;
09ee6fdd 6242 Lisp_Object val;
7d64c6ad
KH		 6243
6244 if (encodep)
6245 translation_table = CODING_ATTR_ENCODE_TBL (attrs),
6246 standard = Vstandard_translation_table_for_encode;
6247 else
6248 translation_table = CODING_ATTR_DECODE_TBL (attrs),
6249 standard = Vstandard_translation_table_for_decode;
7d64c6ad 6250 if (NILP (translation_table))
09ee6fdd
KH		 6251 translation_table = standard;
6252 else
a6f87d34 6253 {
09ee6fdd
KH		 6254 if (SYMBOLP (translation_table))
6255 translation_table = Fget (translation_table, Qtranslation_table);
6256 else if (CONSP (translation_table))
6257 {
6258 translation_table = Fcopy_sequence (translation_table);
6259 for (val = translation_table; CONSP (val); val = XCDR (val))
6260 if (SYMBOLP (XCAR (val)))
6261 XSETCAR (val, Fget (XCAR (val), Qtranslation_table));
6262 }
6263 if (CHAR_TABLE_P (standard))
6264 {
6265 if (CONSP (translation_table))
6266 translation_table = nconc2 (translation_table,
6267 Fcons (standard, Qnil));
6268 else
6269 translation_table = Fcons (translation_table,
6270 Fcons (standard, Qnil));
6271 }
a6f87d34 6272 }
2170c8f0
KH		 6273
6274 if (max_lookup)
09ee6fdd 6275 {
2170c8f0
KH		 6276 *max_lookup = 1;
6277 if (CHAR_TABLE_P (translation_table)
6278 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (translation_table)) > 1)
6279 {
6280 val = XCHAR_TABLE (translation_table)->extras[1];
6281 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
6282 *max_lookup = XFASTINT (val);
6283 }
6284 else if (CONSP (translation_table))
6285 {
6286 Lisp_Object tail, val;
09ee6fdd 6287
2170c8f0
KH		 6288 for (tail = translation_table; CONSP (tail); tail = XCDR (tail))
6289 if (CHAR_TABLE_P (XCAR (tail))
6290 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (XCAR (tail))) > 1)
6291 {
6292 val = XCHAR_TABLE (XCAR (tail))->extras[1];
6293 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
6294 *max_lookup = XFASTINT (val);
6295 }
6296 }
a6f87d34 6297 }
7d64c6ad
KH		 6298 return translation_table;
6299}
6300
09ee6fdd
KH		 6301#define LOOKUP_TRANSLATION_TABLE(table, c, trans) \
6302 do { \
6303 trans = Qnil; \
6304 if (CHAR_TABLE_P (table)) \
6305 { \
6306 trans = CHAR_TABLE_REF (table, c); \
6307 if (CHARACTERP (trans)) \
6308 c = XFASTINT (trans), trans = Qnil; \
6309 } \
6310 else if (CONSP (table)) \
6311 { \
6312 Lisp_Object tail; \
6313 \
6314 for (tail = table; CONSP (tail); tail = XCDR (tail)) \
6315 if (CHAR_TABLE_P (XCAR (tail))) \
6316 { \
6317 trans = CHAR_TABLE_REF (XCAR (tail), c); \
6318 if (CHARACTERP (trans)) \
6319 c = XFASTINT (trans), trans = Qnil; \
6320 else if (! NILP (trans)) \
6321 break; \
6322 } \
6323 } \
e6a54062
KH		 6324 } while (0)
6325
7d64c6ad 6326
69a80ea3
KH		 6327static Lisp_Object
6328get_translation (val, buf, buf_end, last_block, from_nchars, to_nchars)
6329 Lisp_Object val;
6330 int *buf, *buf_end;
6331 int last_block;
6332 int *from_nchars, *to_nchars;
6333{
433f7f87
KH		 6334 /* VAL is TO or (([FROM-CHAR ...] . TO) ...) where TO is TO-CHAR or
6335 [TO-CHAR ...]. */
69a80ea3
KH		 6336 if (CONSP (val))
6337 {
433f7f87 6338 Lisp_Object from, tail;
69a80ea3
KH		 6339 int i, len;
6340
433f7f87 6341 for (tail = val; CONSP (tail); tail = XCDR (tail))
69a80ea3 6342 {
433f7f87
KH		 6343 val = XCAR (tail);
6344 from = XCAR (val);
6345 len = ASIZE (from);
6346 for (i = 0; i < len; i++)
6347 {
6348 if (buf + i == buf_end)
6349 {
6350 if (! last_block)
6351 return Qt;
6352 break;
6353 }
6354 if (XINT (AREF (from, i)) != buf[i])
6355 break;
6356 }
6357 if (i == len)
6358 {
6359 val = XCDR (val);
6360 *from_nchars = len;
6361 break;
6362 }
69a80ea3 6363 }
433f7f87
KH		 6364 if (! CONSP (tail))
6365 return Qnil;
69a80ea3
KH		 6366 }
6367 if (VECTORP (val))
6368 *buf = XINT (AREF (val, 0)), *to_nchars = ASIZE (val);
6369 else
6370 *buf = XINT (val);
6371 return val;
6372}
6373
6374
d46c5b12 6375static int
69a80ea3 6376produce_chars (coding, translation_table, last_block)
df7492f9 6377 struct coding_system *coding;
69a80ea3
KH		 6378 Lisp_Object translation_table;
6379 int last_block;
4ed46869 6380{
df7492f9
KH		 6381 unsigned char *dst = coding->destination + coding->produced;
6382 unsigned char *dst_end = coding->destination + coding->dst_bytes;
119852e7
KH		 6383 EMACS_INT produced;
6384 EMACS_INT produced_chars = 0;
69a80ea3 6385 int carryover = 0;
4ed46869 6386
df7492f9 6387 if (! coding->chars_at_source)
4ed46869 6388 {
119852e7 6389 /* Source characters are in coding->charbuf. */
fba4576f
AS		 6390 int *buf = coding->charbuf;
6391 int *buf_end = buf + coding->charbuf_used;
4ed46869 6392
db274c7a
KH		 6393 if (EQ (coding->src_object, coding->dst_object))
6394 {
6395 coding_set_source (coding);
6396 dst_end = ((unsigned char *) coding->source) + coding->consumed;
6397 }
4ed46869 6398
df7492f9 6399 while (buf < buf_end)
4ed46869 6400 {
69a80ea3 6401 int c = *buf, i;
bc4bc72a 6402
df7492f9
KH		 6403 if (c >= 0)
6404 {
69a80ea3
KH		 6405 int from_nchars = 1, to_nchars = 1;
6406 Lisp_Object trans = Qnil;
6407
09ee6fdd 6408 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 6409 if (! NILP (trans))
69a80ea3
KH		 6410 {
6411 trans = get_translation (trans, buf, buf_end, last_block,
6412 &from_nchars, &to_nchars);
6413 if (EQ (trans, Qt))
6414 break;
6415 c = *buf;
6416 }
6417
6418 if (dst + MAX_MULTIBYTE_LENGTH * to_nchars > dst_end)
6419 {
6420 dst = alloc_destination (coding,
6421 buf_end - buf
6422 + MAX_MULTIBYTE_LENGTH * to_nchars,
6423 dst);
db274c7a
KH		 6424 if (EQ (coding->src_object, coding->dst_object))
6425 {
6426 coding_set_source (coding);
6427 dst_end = ((unsigned char *) coding->source) + coding->consumed;
6428 }
6429 else
6430 dst_end = coding->destination + coding->dst_bytes;
69a80ea3
KH		 6431 }
6432
433f7f87 6433 for (i = 0; i < to_nchars; i++)
69a80ea3 6434 {
433f7f87
KH		 6435 if (i > 0)
6436 c = XINT (AREF (trans, i));
69a80ea3
KH		 6437 if (coding->dst_multibyte
6438 || ! CHAR_BYTE8_P (c))
db274c7a 6439 CHAR_STRING_ADVANCE_NO_UNIFY (c, dst);
69a80ea3
KH		 6440 else
6441 *dst++ = CHAR_TO_BYTE8 (c);
6442 }
6443 produced_chars += to_nchars;
6444 *buf++ = to_nchars;
6445 while (--from_nchars > 0)
6446 *buf++ = 0;
d46c5b12 6447 }
df7492f9 6448 else
69a80ea3
KH		 6449 /* This is an annotation datum. (-C) is the length. */
6450 buf += -c;
4ed46869 6451 }
69a80ea3 6452 carryover = buf_end - buf;
4ed46869 6453 }
fa42c37f 6454 else
fa42c37f 6455 {
119852e7 6456 /* Source characters are at coding->source. */
8f924df7 6457 const unsigned char *src = coding->source;
119852e7 6458 const unsigned char *src_end = src + coding->consumed;
4ed46869 6459
db274c7a
KH		 6460 if (EQ (coding->dst_object, coding->src_object))
6461 dst_end = (unsigned char *) src;
df7492f9 6462 if (coding->src_multibyte != coding->dst_multibyte)
fa42c37f 6463 {
df7492f9 6464 if (coding->src_multibyte)
fa42c37f 6465 {
71c81426 6466 int multibytep = 1;
4533845d 6467 EMACS_INT consumed_chars = 0;
d46c5b12 6468
df7492f9
KH		 6469 while (1)
6470 {
8f924df7 6471 const unsigned char *src_base = src;
df7492f9 6472 int c;
b73bfc1c 6473
df7492f9 6474 ONE_MORE_BYTE (c);
119852e7 6475 if (dst == dst_end)
df7492f9 6476 {
119852e7
KH		 6477 if (EQ (coding->src_object, coding->dst_object))
6478 dst_end = (unsigned char *) src;
6479 if (dst == dst_end)
df7492f9 6480 {
119852e7
KH		 6481 EMACS_INT offset = src - coding->source;
6482
6483 dst = alloc_destination (coding, src_end - src + 1,
6484 dst);
6485 dst_end = coding->destination + coding->dst_bytes;
6486 coding_set_source (coding);
6487 src = coding->source + offset;
6488 src_end = coding->source + coding->src_bytes;
db274c7a
KH		 6489 if (EQ (coding->src_object, coding->dst_object))
6490 dst_end = (unsigned char *) src;
df7492f9 6491 }
df7492f9
KH		 6492 }
6493 *dst++ = c;
6494 produced_chars++;
6495 }
6496 no_more_source:
6497 ;
fa42c37f
KH		 6498 }
6499 else
df7492f9
KH		 6500 while (src < src_end)
6501 {
71c81426 6502 int multibytep = 1;
df7492f9 6503 int c = *src++;
b73bfc1c 6504
df7492f9
KH		 6505 if (dst >= dst_end - 1)
6506 {
2c78b7e1 6507 if (EQ (coding->src_object, coding->dst_object))
8f924df7 6508 dst_end = (unsigned char *) src;
2c78b7e1
KH		 6509 if (dst >= dst_end - 1)
6510 {
119852e7 6511 EMACS_INT offset = src - coding->source;
db274c7a 6512 EMACS_INT more_bytes;
119852e7 6513
db274c7a
KH		 6514 if (EQ (coding->src_object, coding->dst_object))
6515 more_bytes = ((src_end - src) / 2) + 2;
6516 else
6517 more_bytes = src_end - src + 2;
6518 dst = alloc_destination (coding, more_bytes, dst);
2c78b7e1
KH		 6519 dst_end = coding->destination + coding->dst_bytes;
6520 coding_set_source (coding);
119852e7 6521 src = coding->source + offset;
2c78b7e1 6522 src_end = coding->source + coding->src_bytes;
db274c7a
KH		 6523 if (EQ (coding->src_object, coding->dst_object))
6524 dst_end = (unsigned char *) src;
2c78b7e1 6525 }
df7492f9
KH		 6526 }
6527 EMIT_ONE_BYTE (c);
6528 }
d46c5b12 6529 }
df7492f9
KH		 6530 else
6531 {
6532 if (!EQ (coding->src_object, coding->dst_object))
fa42c37f 6533 {
119852e7 6534 EMACS_INT require = coding->src_bytes - coding->dst_bytes;
4ed46869 6535
df7492f9 6536 if (require > 0)
fa42c37f 6537 {
df7492f9
KH		 6538 EMACS_INT offset = src - coding->source;
6539
6540 dst = alloc_destination (coding, require, dst);
6541 coding_set_source (coding);
6542 src = coding->source + offset;
6543 src_end = coding->source + coding->src_bytes;
fa42c37f
KH		 6544 }
6545 }
119852e7 6546 produced_chars = coding->consumed_char;
df7492f9 6547 while (src < src_end)
14daee73 6548 *dst++ = *src++;
fa42c37f
KH		 6549 }
6550 }
6551
df7492f9 6552 produced = dst - (coding->destination + coding->produced);
284201e4 6553 if (BUFFERP (coding->dst_object) && produced_chars > 0)
df7492f9
KH		 6554 insert_from_gap (produced_chars, produced);
6555 coding->produced += produced;
6556 coding->produced_char += produced_chars;
69a80ea3 6557 return carryover;
fa42c37f
KH		 6558}
6559
ff0dacd7
KH		 6560/* Compose text in CODING->object according to the annotation data at
6561 CHARBUF. CHARBUF is an array:
6562 [ -LENGTH ANNOTATION_MASK FROM TO METHOD COMP_LEN [ COMPONENTS... ] ]
df7492f9 6563 */
4ed46869 6564
df7492f9 6565static INLINE void
69a80ea3 6566produce_composition (coding, charbuf, pos)
4ed46869 6567 struct coding_system *coding;
df7492f9 6568 int *charbuf;
69a80ea3 6569 EMACS_INT pos;
4ed46869 6570{
df7492f9 6571 int len;
69a80ea3 6572 EMACS_INT to;
df7492f9 6573 enum composition_method method;
df7492f9 6574 Lisp_Object components;
fa42c37f 6575
df7492f9 6576 len = -charbuf[0];
69a80ea3 6577 to = pos + charbuf[2];
9ffd559c
KH		 6578 if (to <= pos)
6579 return;
69a80ea3 6580 method = (enum composition_method) (charbuf[3]);
d46c5b12 6581
df7492f9
KH		 6582 if (method == COMPOSITION_RELATIVE)
6583 components = Qnil;
9ffd559c
KH		 6584 else if (method >= COMPOSITION_WITH_RULE
6585 && method <= COMPOSITION_WITH_RULE_ALTCHARS)
d46c5b12 6586 {
df7492f9
KH		 6587 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
6588 int i;
b73bfc1c 6589
69a80ea3
KH		 6590 len -= 4;
6591 charbuf += 4;
df7492f9 6592 for (i = 0; i < len; i++)
9ffd559c
KH		 6593 {
6594 args[i] = make_number (charbuf[i]);
f75c90a9 6595 if (charbuf[i] < 0)
9ffd559c
KH		 6596 return;
6597 }
df7492f9
KH		 6598 components = (method == COMPOSITION_WITH_ALTCHARS
6599 ? Fstring (len, args) : Fvector (len, args));
d46c5b12 6600 }
9ffd559c
KH		 6601 else
6602 return;
69a80ea3 6603 compose_text (pos, to, components, Qnil, coding->dst_object);
d46c5b12
KH		 6604}
6605
d46c5b12 6606
ff0dacd7
KH		 6607/* Put `charset' property on text in CODING->object according to
6608 the annotation data at CHARBUF. CHARBUF is an array:
69a80ea3 6609 [ -LENGTH ANNOTATION_MASK NCHARS CHARSET-ID ]
ff0dacd7 6610 */
d46c5b12 6611
ff0dacd7 6612static INLINE void
69a80ea3 6613produce_charset (coding, charbuf, pos)
d46c5b12 6614 struct coding_system *coding;
ff0dacd7 6615 int *charbuf;
69a80ea3 6616 EMACS_INT pos;
d46c5b12 6617{
69a80ea3
KH		 6618 EMACS_INT from = pos - charbuf[2];
6619 struct charset *charset = CHARSET_FROM_ID (charbuf[3]);
b73bfc1c 6620
69a80ea3 6621 Fput_text_property (make_number (from), make_number (pos),
ff0dacd7
KH		 6622 Qcharset, CHARSET_NAME (charset),
6623 coding->dst_object);
d46c5b12
KH		 6624}
6625
d46c5b12 6626
df7492f9
KH		 6627#define CHARBUF_SIZE 0x4000
6628
6629#define ALLOC_CONVERSION_WORK_AREA(coding) \
6630 do { \
8510724d 6631 int size = CHARBUF_SIZE; \
df7492f9
KH		 6632 \
6633 coding->charbuf = NULL; \
6634 while (size > 1024) \
6635 { \
6636 coding->charbuf = (int *) alloca (sizeof (int) * size); \
6637 if (coding->charbuf) \
6638 break; \
6639 size >>= 1; \
6640 } \
6641 if (! coding->charbuf) \
6642 { \
065e3595 6643 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_MEM); \
df7492f9
KH		 6644 return coding->result; \
6645 } \
6646 coding->charbuf_size = size; \
6647 } while (0)
4ed46869 6648
d46c5b12
KH		 6649
6650static void
69a80ea3 6651produce_annotation (coding, pos)
d46c5b12 6652 struct coding_system *coding;
69a80ea3 6653 EMACS_INT pos;
d46c5b12 6654{
df7492f9
KH		 6655 int *charbuf = coding->charbuf;
6656 int *charbuf_end = charbuf + coding->charbuf_used;
d46c5b12 6657
ff0dacd7
KH		 6658 if (NILP (coding->dst_object))
6659 return;
d46c5b12 6660
df7492f9 6661 while (charbuf < charbuf_end)
a84f1519 6662 {
df7492f9 6663 if (*charbuf >= 0)
69a80ea3 6664 pos += *charbuf++;
d46c5b12 6665 else
d46c5b12 6666 {
df7492f9 6667 int len = -*charbuf;
ff0dacd7 6668 switch (charbuf[1])
df7492f9
KH		 6669 {
6670 case CODING_ANNOTATE_COMPOSITION_MASK:
69a80ea3 6671 produce_composition (coding, charbuf, pos);
df7492f9 6672 break;
ff0dacd7 6673 case CODING_ANNOTATE_CHARSET_MASK:
69a80ea3 6674 produce_charset (coding, charbuf, pos);
ff0dacd7 6675 break;
df7492f9
KH		 6676 default:
6677 abort ();
6678 }
6679 charbuf += len;
d46c5b12 6680 }
a84f1519 6681 }
d46c5b12
KH		 6682}
6683
df7492f9
KH		 6684/* Decode the data at CODING->src_object into CODING->dst_object.
6685 CODING->src_object is a buffer, a string, or nil.
6686 CODING->dst_object is a buffer.
d46c5b12 6687
df7492f9
KH		 6688 If CODING->src_object is a buffer, it must be the current buffer.
6689 In this case, if CODING->src_pos is positive, it is a position of
6690 the source text in the buffer, otherwise, the source text is in the
6691 gap area of the buffer, and CODING->src_pos specifies the offset of
6692 the text from GPT (which must be the same as PT). If this is the
6693 same buffer as CODING->dst_object, CODING->src_pos must be
6694 negative.
d46c5b12 6695
b6828792 6696 If CODING->src_object is a string, CODING->src_pos is an index to
df7492f9 6697 that string.
d46c5b12 6698
df7492f9
KH		 6699 If CODING->src_object is nil, CODING->source must already point to
6700 the non-relocatable memory area. In this case, CODING->src_pos is
6701 an offset from CODING->source.
73be902c 6702
df7492f9
KH		 6703 The decoded data is inserted at the current point of the buffer
6704 CODING->dst_object.
6705*/
d46c5b12 6706
df7492f9
KH		 6707static int
6708decode_coding (coding)
d46c5b12 6709 struct coding_system *coding;
d46c5b12 6710{
df7492f9 6711 Lisp_Object attrs;
24a73b0a 6712 Lisp_Object undo_list;
7d64c6ad 6713 Lisp_Object translation_table;
69a80ea3
KH		 6714 int carryover;
6715 int i;
d46c5b12 6716
df7492f9
KH		 6717 if (BUFFERP (coding->src_object)
6718 && coding->src_pos > 0
6719 && coding->src_pos < GPT
6720 && coding->src_pos + coding->src_chars > GPT)
6721 move_gap_both (coding->src_pos, coding->src_pos_byte);
d46c5b12 6722
24a73b0a 6723 undo_list = Qt;
df7492f9 6724 if (BUFFERP (coding->dst_object))
1c3478b0 6725 {
df7492f9
KH		 6726 if (current_buffer != XBUFFER (coding->dst_object))
6727 set_buffer_internal (XBUFFER (coding->dst_object));
6728 if (GPT != PT)
6729 move_gap_both (PT, PT_BYTE);
24a73b0a
KH		 6730 undo_list = current_buffer->undo_list;
6731 current_buffer->undo_list = Qt;
1c3478b0
KH		 6732 }
6733
df7492f9
KH		 6734 coding->consumed = coding->consumed_char = 0;
6735 coding->produced = coding->produced_char = 0;
6736 coding->chars_at_source = 0;
065e3595 6737 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 6738 coding->errors = 0;
1c3478b0 6739
df7492f9
KH		 6740 ALLOC_CONVERSION_WORK_AREA (coding);
6741
6742 attrs = CODING_ID_ATTRS (coding->id);
2170c8f0 6743 translation_table = get_translation_table (attrs, 0, NULL);
df7492f9 6744
69a80ea3 6745 carryover = 0;
df7492f9 6746 do
b73bfc1c 6747 {
69a80ea3
KH		 6748 EMACS_INT pos = coding->dst_pos + coding->produced_char;
6749
df7492f9
KH		 6750 coding_set_source (coding);
6751 coding->annotated = 0;
69a80ea3 6752 coding->charbuf_used = carryover;
df7492f9 6753 (*(coding->decoder)) (coding);
df7492f9 6754 coding_set_destination (coding);
69a80ea3 6755 carryover = produce_chars (coding, translation_table, 0);
df7492f9 6756 if (coding->annotated)
69a80ea3
KH		 6757 produce_annotation (coding, pos);
6758 for (i = 0; i < carryover; i++)
6759 coding->charbuf[i]
6760 = coding->charbuf[coding->charbuf_used - carryover + i];
d46c5b12 6761 }
df7492f9 6762 while (coding->consumed < coding->src_bytes
54b367bb
KH		 6763 && (coding->result == CODING_RESULT_SUCCESS
6764 || coding->result == CODING_RESULT_INVALID_SRC));
d46c5b12 6765
69a80ea3
KH		 6766 if (carryover > 0)
6767 {
6768 coding_set_destination (coding);
6769 coding->charbuf_used = carryover;
6770 produce_chars (coding, translation_table, 1);
6771 }
6772
df7492f9
KH		 6773 coding->carryover_bytes = 0;
6774 if (coding->consumed < coding->src_bytes)
d46c5b12 6775 {
df7492f9 6776 int nbytes = coding->src_bytes - coding->consumed;
8f924df7 6777 const unsigned char *src;
df7492f9
KH		 6778
6779 coding_set_source (coding);
6780 coding_set_destination (coding);
6781 src = coding->source + coding->consumed;
6782
6783 if (coding->mode & CODING_MODE_LAST_BLOCK)
1c3478b0 6784 {
df7492f9
KH		 6785 /* Flush out unprocessed data as binary chars. We are sure
6786 that the number of data is less than the size of
6787 coding->charbuf. */
065e3595 6788 coding->charbuf_used = 0;
b2dab6c8
JR		 6789 coding->chars_at_source = 0;
6790
df7492f9 6791 while (nbytes-- > 0)
1c3478b0 6792 {
df7492f9 6793 int c = *src++;
98725083 6794
1c91457d
KH		 6795 if (c & 0x80)
6796 c = BYTE8_TO_CHAR (c);
6797 coding->charbuf[coding->charbuf_used++] = c;
1c3478b0 6798 }
f6cbaf43 6799 produce_chars (coding, Qnil, 1);
d46c5b12 6800 }
d46c5b12 6801 else
df7492f9
KH		 6802 {
6803 /* Record unprocessed bytes in coding->carryover. We are
6804 sure that the number of data is less than the size of
6805 coding->carryover. */
6806 unsigned char *p = coding->carryover;
6807
6808 coding->carryover_bytes = nbytes;
6809 while (nbytes-- > 0)
6810 *p++ = *src++;
1c3478b0 6811 }
df7492f9 6812 coding->consumed = coding->src_bytes;
b73bfc1c 6813 }
69f76525 6814
4347441b
KH		 6815 if (! EQ (CODING_ID_EOL_TYPE (coding->id), Qunix))
6816 decode_eol (coding);
24a73b0a
KH		 6817 if (BUFFERP (coding->dst_object))
6818 {
6819 current_buffer->undo_list = undo_list;
6820 record_insert (coding->dst_pos, coding->produced_char);
6821 }
73be902c 6822 return coding->result;
4ed46869
KH		 6823}
6824
aaaf0b1e 6825
e1c23804 6826/* Extract an annotation datum from a composition starting at POS and
ff0dacd7
KH		 6827 ending before LIMIT of CODING->src_object (buffer or string), store
6828 the data in BUF, set *STOP to a starting position of the next
6829 composition (if any) or to LIMIT, and return the address of the
6830 next element of BUF.
6831
6832 If such an annotation is not found, set *STOP to a starting
6833 position of a composition after POS (if any) or to LIMIT, and
6834 return BUF. */
6835
6836static INLINE int *
6837handle_composition_annotation (pos, limit, coding, buf, stop)
6838 EMACS_INT pos, limit;
aaaf0b1e 6839 struct coding_system *coding;
ff0dacd7
KH		 6840 int *buf;
6841 EMACS_INT *stop;
aaaf0b1e 6842{
ff0dacd7
KH		 6843 EMACS_INT start, end;
6844 Lisp_Object prop;
aaaf0b1e 6845
ff0dacd7
KH		 6846 if (! find_composition (pos, limit, &start, &end, &prop, coding->src_object)
6847 || end > limit)
6848 *stop = limit;
6849 else if (start > pos)
6850 *stop = start;
6851 else
aaaf0b1e 6852 {
ff0dacd7 6853 if (start == pos)
aaaf0b1e 6854 {
ff0dacd7
KH		 6855 /* We found a composition. Store the corresponding
6856 annotation data in BUF. */
6857 int *head = buf;
6858 enum composition_method method = COMPOSITION_METHOD (prop);
6859 int nchars = COMPOSITION_LENGTH (prop);
6860
69a80ea3 6861 ADD_COMPOSITION_DATA (buf, nchars, method);
ff0dacd7 6862 if (method != COMPOSITION_RELATIVE)
aaaf0b1e 6863 {
ff0dacd7
KH		 6864 Lisp_Object components;
6865 int len, i, i_byte;
6866
6867 components = COMPOSITION_COMPONENTS (prop);
6868 if (VECTORP (components))
aaaf0b1e 6869 {
ff0dacd7
KH		 6870 len = XVECTOR (components)->size;
6871 for (i = 0; i < len; i++)
6872 *buf++ = XINT (AREF (components, i));
aaaf0b1e 6873 }
ff0dacd7 6874 else if (STRINGP (components))
aaaf0b1e 6875 {
8f924df7 6876 len = SCHARS (components);
ff0dacd7
KH		 6877 i = i_byte = 0;
6878 while (i < len)
6879 {
6880 FETCH_STRING_CHAR_ADVANCE (*buf, components, i, i_byte);
6881 buf++;
6882 }
6883 }
6884 else if (INTEGERP (components))
6885 {
6886 len = 1;
6887 *buf++ = XINT (components);
6888 }
6889 else if (CONSP (components))
6890 {
6891 for (len = 0; CONSP (components);
6892 len++, components = XCDR (components))
6893 *buf++ = XINT (XCAR (components));
aaaf0b1e 6894 }
aaaf0b1e 6895 else
ff0dacd7
KH		 6896 abort ();
6897 *head -= len;
aaaf0b1e 6898 }
aaaf0b1e 6899 }
ff0dacd7
KH		 6900
6901 if (find_composition (end, limit, &start, &end, &prop,
6902 coding->src_object)
6903 && end <= limit)
6904 *stop = start;
6905 else
6906 *stop = limit;
aaaf0b1e 6907 }
ff0dacd7
KH		 6908 return buf;
6909}
6910
6911
e1c23804 6912/* Extract an annotation datum from a text property `charset' at POS of
ff0dacd7
KH		 6913 CODING->src_object (buffer of string), store the data in BUF, set
6914 *STOP to the position where the value of `charset' property changes
6915 (limiting by LIMIT), and return the address of the next element of
6916 BUF.
6917
6918 If the property value is nil, set *STOP to the position where the
6919 property value is non-nil (limiting by LIMIT), and return BUF. */
6920
6921static INLINE int *
6922handle_charset_annotation (pos, limit, coding, buf, stop)
6923 EMACS_INT pos, limit;
6924 struct coding_system *coding;
6925 int *buf;
6926 EMACS_INT *stop;
6927{
6928 Lisp_Object val, next;
6929 int id;
6930
6931 val = Fget_text_property (make_number (pos), Qcharset, coding->src_object);
6932 if (! NILP (val) && CHARSETP (val))
6933 id = XINT (CHARSET_SYMBOL_ID (val));
6934 else
6935 id = -1;
69a80ea3 6936 ADD_CHARSET_DATA (buf, 0, id);
ff0dacd7
KH		 6937 next = Fnext_single_property_change (make_number (pos), Qcharset,
6938 coding->src_object,
6939 make_number (limit));
6940 *stop = XINT (next);
6941 return buf;
6942}
6943
6944
df7492f9 6945static void
09ee6fdd 6946consume_chars (coding, translation_table, max_lookup)
df7492f9 6947 struct coding_system *coding;
433f7f87 6948 Lisp_Object translation_table;
09ee6fdd 6949 int max_lookup;
df7492f9
KH		 6950{
6951 int *buf = coding->charbuf;
ff0dacd7 6952 int *buf_end = coding->charbuf + coding->charbuf_size;
7c78e542 6953 const unsigned char *src = coding->source + coding->consumed;
4776e638 6954 const unsigned char *src_end = coding->source + coding->src_bytes;
ff0dacd7
KH		 6955 EMACS_INT pos = coding->src_pos + coding->consumed_char;
6956 EMACS_INT end_pos = coding->src_pos + coding->src_chars;
df7492f9
KH		 6957 int multibytep = coding->src_multibyte;
6958 Lisp_Object eol_type;
6959 int c;
ff0dacd7 6960 EMACS_INT stop, stop_composition, stop_charset;
09ee6fdd 6961 int *lookup_buf = NULL;
433f7f87
KH		 6962
6963 if (! NILP (translation_table))
09ee6fdd 6964 lookup_buf = alloca (sizeof (int) * max_lookup);
88993dfd 6965
df7492f9
KH		 6966 eol_type = CODING_ID_EOL_TYPE (coding->id);
6967 if (VECTORP (eol_type))
6968 eol_type = Qunix;
88993dfd 6969
df7492f9
KH		 6970 /* Note: composition handling is not yet implemented. */
6971 coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
ec6d2bb8 6972
0b5670c9
KH		 6973 if (NILP (coding->src_object))
6974 stop = stop_composition = stop_charset = end_pos;
ff0dacd7 6975 else
0b5670c9
KH		 6976 {
6977 if (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK)
6978 stop = stop_composition = pos;
6979 else
6980 stop = stop_composition = end_pos;
6981 if (coding->common_flags & CODING_ANNOTATE_CHARSET_MASK)
6982 stop = stop_charset = pos;
6983 else
6984 stop_charset = end_pos;
6985 }
ec6d2bb8 6986
24a73b0a 6987 /* Compensate for CRLF and conversion. */
ff0dacd7 6988 buf_end -= 1 + MAX_ANNOTATION_LENGTH;
df7492f9 6989 while (buf < buf_end)
aaaf0b1e 6990 {
433f7f87
KH		 6991 Lisp_Object trans;
6992
df7492f9 6993 if (pos == stop)
ec6d2bb8 6994 {
df7492f9
KH		 6995 if (pos == end_pos)
6996 break;
ff0dacd7
KH		 6997 if (pos == stop_composition)
6998 buf = handle_composition_annotation (pos, end_pos, coding,
6999 buf, &stop_composition);
7000 if (pos == stop_charset)
7001 buf = handle_charset_annotation (pos, end_pos, coding,
7002 buf, &stop_charset);
7003 stop = (stop_composition < stop_charset
7004 ? stop_composition : stop_charset);
df7492f9
KH		 7005 }
7006
7007 if (! multibytep)
4776e638 7008 {
d3e4cb56 7009 EMACS_INT bytes;
aaaf0b1e 7010
ea29edf2
KH		 7011 if (coding->encoder == encode_coding_raw_text)
7012 c = *src++, pos++;
7013 else if ((bytes = MULTIBYTE_LENGTH (src, src_end)) > 0)
db274c7a 7014 c = STRING_CHAR_ADVANCE_NO_UNIFY (src), pos += bytes;
4776e638 7015 else
f03caae0 7016 c = BYTE8_TO_CHAR (*src), src++, pos++;
4776e638 7017 }
df7492f9 7018 else
db274c7a 7019 c = STRING_CHAR_ADVANCE_NO_UNIFY (src), pos++;
df7492f9
KH		 7020 if ((c == '\r') && (coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
7021 c = '\n';
7022 if (! EQ (eol_type, Qunix))
aaaf0b1e 7023 {
df7492f9 7024 if (c == '\n')
aaaf0b1e 7025 {
df7492f9
KH		 7026 if (EQ (eol_type, Qdos))
7027 *buf++ = '\r';
7028 else
7029 c = '\r';
aaaf0b1e
KH		 7030 }
7031 }
433f7f87 7032
e6a54062 7033 trans = Qnil;
09ee6fdd 7034 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 7035 if (NILP (trans))
433f7f87
KH		 7036 *buf++ = c;
7037 else
7038 {
7039 int from_nchars = 1, to_nchars = 1;
7040 int *lookup_buf_end;
7041 const unsigned char *p = src;
7042 int i;
7043
7044 lookup_buf[0] = c;
7045 for (i = 1; i < max_lookup && p < src_end; i++)
7046 lookup_buf[i] = STRING_CHAR_ADVANCE (p);
7047 lookup_buf_end = lookup_buf + i;
7048 trans = get_translation (trans, lookup_buf, lookup_buf_end, 1,
7049 &from_nchars, &to_nchars);
7050 if (EQ (trans, Qt)
7051 || buf + to_nchars > buf_end)
7052 break;
7053 *buf++ = *lookup_buf;
7054 for (i = 1; i < to_nchars; i++)
7055 *buf++ = XINT (AREF (trans, i));
7056 for (i = 1; i < from_nchars; i++, pos++)
7057 src += MULTIBYTE_LENGTH_NO_CHECK (src);
7058 }
aaaf0b1e 7059 }
ec6d2bb8 7060
df7492f9
KH		 7061 coding->consumed = src - coding->source;
7062 coding->consumed_char = pos - coding->src_pos;
7063 coding->charbuf_used = buf - coding->charbuf;
7064 coding->chars_at_source = 0;
aaaf0b1e
KH		 7065}
7066
4ed46869 7067
df7492f9
KH		 7068/* Encode the text at CODING->src_object into CODING->dst_object.
7069 CODING->src_object is a buffer or a string.
7070 CODING->dst_object is a buffer or nil.
7071
7072 If CODING->src_object is a buffer, it must be the current buffer.
7073 In this case, if CODING->src_pos is positive, it is a position of
7074 the source text in the buffer, otherwise. the source text is in the
7075 gap area of the buffer, and coding->src_pos specifies the offset of
7076 the text from GPT (which must be the same as PT). If this is the
7077 same buffer as CODING->dst_object, CODING->src_pos must be
7078 negative and CODING should not have `pre-write-conversion'.
7079
7080 If CODING->src_object is a string, CODING should not have
7081 `pre-write-conversion'.
7082
7083 If CODING->dst_object is a buffer, the encoded data is inserted at
7084 the current point of that buffer.
7085
7086 If CODING->dst_object is nil, the encoded data is placed at the
7087 memory area specified by CODING->destination. */
7088
7089static int
7090encode_coding (coding)
4ed46869 7091 struct coding_system *coding;
4ed46869 7092{
df7492f9 7093 Lisp_Object attrs;
7d64c6ad 7094 Lisp_Object translation_table;
09ee6fdd 7095 int max_lookup;
9861e777 7096
df7492f9 7097 attrs = CODING_ID_ATTRS (coding->id);
ea29edf2
KH		 7098 if (coding->encoder == encode_coding_raw_text)
7099 translation_table = Qnil, max_lookup = 0;
7100 else
7101 translation_table = get_translation_table (attrs, 1, &max_lookup);
4ed46869 7102
df7492f9 7103 if (BUFFERP (coding->dst_object))
8844fa83 7104 {
df7492f9
KH		 7105 set_buffer_internal (XBUFFER (coding->dst_object));
7106 coding->dst_multibyte
7107 = ! NILP (current_buffer->enable_multibyte_characters);
8844fa83 7108 }
4ed46869 7109
b73bfc1c 7110 coding->consumed = coding->consumed_char = 0;
df7492f9 7111 coding->produced = coding->produced_char = 0;
065e3595 7112 record_conversion_result (coding, CODING_RESULT_SUCCESS);
b73bfc1c 7113 coding->errors = 0;
b73bfc1c 7114
df7492f9 7115 ALLOC_CONVERSION_WORK_AREA (coding);
4ed46869 7116
df7492f9
KH		 7117 do {
7118 coding_set_source (coding);
09ee6fdd 7119 consume_chars (coding, translation_table, max_lookup);
df7492f9
KH		 7120 coding_set_destination (coding);
7121 (*(coding->encoder)) (coding);
7122 } while (coding->consumed_char < coding->src_chars);
7123
284201e4 7124 if (BUFFERP (coding->dst_object) && coding->produced_char > 0)
df7492f9
KH		 7125 insert_from_gap (coding->produced_char, coding->produced);
7126
7127 return (coding->result);
ec6d2bb8
KH		 7128}
7129
fb88bf2d 7130
24a73b0a
KH		 7131/* Name (or base name) of work buffer for code conversion. */
7132static Lisp_Object Vcode_conversion_workbuf_name;
d46c5b12 7133
24a73b0a
KH		 7134/* A working buffer used by the top level conversion. Once it is
7135 created, it is never destroyed. It has the name
7136 Vcode_conversion_workbuf_name. The other working buffers are
7137 destroyed after the use is finished, and their names are modified
7138 versions of Vcode_conversion_workbuf_name. */
7139static Lisp_Object Vcode_conversion_reused_workbuf;
b73bfc1c 7140
24a73b0a
KH		 7141/* 1 iff Vcode_conversion_reused_workbuf is already in use. */
7142static int reused_workbuf_in_use;
4ed46869 7143
24a73b0a
KH		 7144
7145/* Return a working buffer of code convesion. MULTIBYTE specifies the
7146 multibyteness of returning buffer. */
b73bfc1c 7147
f6cbaf43 7148static Lisp_Object
24a73b0a 7149make_conversion_work_buffer (multibyte)
f6cbaf43 7150 int multibyte;
df7492f9 7151{
24a73b0a
KH		 7152 Lisp_Object name, workbuf;
7153 struct buffer *current;
4ed46869 7154
24a73b0a 7155 if (reused_workbuf_in_use++)
065e3595
KH		 7156 {
7157 name = Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name, Qnil);
7158 workbuf = Fget_buffer_create (name);
7159 }
df7492f9 7160 else
065e3595 7161 {
159bd5a2 7162 if (NILP (Fbuffer_live_p (Vcode_conversion_reused_workbuf)))
a993c7a1
KH		 7163 Vcode_conversion_reused_workbuf
7164 = Fget_buffer_create (Vcode_conversion_workbuf_name);
7165 workbuf = Vcode_conversion_reused_workbuf;
065e3595 7166 }
24a73b0a
KH		 7167 current = current_buffer;
7168 set_buffer_internal (XBUFFER (workbuf));
df36ff1f
CY		 7169 /* We can't allow modification hooks to run in the work buffer. For
7170 instance, directory_files_internal assumes that file decoding
7171 doesn't compile new regexps. */
7172 Fset (Fmake_local_variable (Qinhibit_modification_hooks), Qt);
3ed051d4 7173 Ferase_buffer ();
df7492f9 7174 current_buffer->undo_list = Qt;
24a73b0a 7175 current_buffer->enable_multibyte_characters = multibyte ? Qt : Qnil;
df7492f9 7176 set_buffer_internal (current);
24a73b0a 7177 return workbuf;
df7492f9 7178}
d46c5b12 7179
24a73b0a 7180
4776e638 7181static Lisp_Object
24a73b0a
KH		 7182code_conversion_restore (arg)
7183 Lisp_Object arg;
4776e638 7184{
24a73b0a 7185 Lisp_Object current, workbuf;
948bdcf3 7186 struct gcpro gcpro1;
24a73b0a 7187
948bdcf3 7188 GCPRO1 (arg);
24a73b0a
KH		 7189 current = XCAR (arg);
7190 workbuf = XCDR (arg);
7191 if (! NILP (workbuf))
7192 {
7193 if (EQ (workbuf, Vcode_conversion_reused_workbuf))
7194 reused_workbuf_in_use = 0;
7195 else if (! NILP (Fbuffer_live_p (workbuf)))
7196 Fkill_buffer (workbuf);
7197 }
7198 set_buffer_internal (XBUFFER (current));
948bdcf3 7199 UNGCPRO;
4776e638
KH		 7200 return Qnil;
7201}
b73bfc1c 7202
24a73b0a
KH		 7203Lisp_Object
7204code_conversion_save (with_work_buf, multibyte)
4776e638 7205 int with_work_buf, multibyte;
df7492f9 7206{
24a73b0a 7207 Lisp_Object workbuf = Qnil;
b73bfc1c 7208
4776e638 7209 if (with_work_buf)
24a73b0a
KH		 7210 workbuf = make_conversion_work_buffer (multibyte);
7211 record_unwind_protect (code_conversion_restore,
7212 Fcons (Fcurrent_buffer (), workbuf));
4776e638 7213 return workbuf;
df7492f9 7214}
d46c5b12 7215
df7492f9
KH		 7216int
7217decode_coding_gap (coding, chars, bytes)
7218 struct coding_system *coding;
7219 EMACS_INT chars, bytes;
7220{
7221 int count = specpdl_ptr - specpdl;
5e5c78be 7222 Lisp_Object attrs;
fb88bf2d 7223
24a73b0a 7224 code_conversion_save (0, 0);
ec6d2bb8 7225
24a73b0a 7226 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7227 coding->src_chars = chars;
7228 coding->src_bytes = bytes;
7229 coding->src_pos = -chars;
7230 coding->src_pos_byte = -bytes;
7231 coding->src_multibyte = chars < bytes;
24a73b0a 7232 coding->dst_object = coding->src_object;
df7492f9
KH		 7233 coding->dst_pos = PT;
7234 coding->dst_pos_byte = PT_BYTE;
71c81426 7235 coding->dst_multibyte = ! NILP (current_buffer->enable_multibyte_characters);
4ed46869 7236
df7492f9
KH		 7237 if (CODING_REQUIRE_DETECTION (coding))
7238 detect_coding (coding);
8f924df7 7239
9286b333 7240 coding->mode |= CODING_MODE_LAST_BLOCK;
287c57d7 7241 current_buffer->text->inhibit_shrinking = 1;
df7492f9 7242 decode_coding (coding);
287c57d7 7243 current_buffer->text->inhibit_shrinking = 0;
d46c5b12 7244
5e5c78be
KH		 7245 attrs = CODING_ID_ATTRS (coding->id);
7246 if (! NILP (CODING_ATTR_POST_READ (attrs)))
b73bfc1c 7247 {
5e5c78be
KH		 7248 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
7249 Lisp_Object val;
7250
7251 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
5e5c78be
KH		 7252 val = call1 (CODING_ATTR_POST_READ (attrs),
7253 make_number (coding->produced_char));
5e5c78be
KH		 7254 CHECK_NATNUM (val);
7255 coding->produced_char += Z - prev_Z;
7256 coding->produced += Z_BYTE - prev_Z_BYTE;
b73bfc1c 7257 }
4ed46869 7258
df7492f9 7259 unbind_to (count, Qnil);
b73bfc1c
KH		 7260 return coding->result;
7261}
52d41803 7262
4ed46869 7263int
df7492f9 7264encode_coding_gap (coding, chars, bytes)
4ed46869 7265 struct coding_system *coding;
df7492f9 7266 EMACS_INT chars, bytes;
4ed46869 7267{
df7492f9 7268 int count = specpdl_ptr - specpdl;
4ed46869 7269
24a73b0a 7270 code_conversion_save (0, 0);
4ed46869 7271
24a73b0a 7272 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7273 coding->src_chars = chars;
7274 coding->src_bytes = bytes;
7275 coding->src_pos = -chars;
7276 coding->src_pos_byte = -bytes;
7277 coding->src_multibyte = chars < bytes;
7278 coding->dst_object = coding->src_object;
7279 coding->dst_pos = PT;
7280 coding->dst_pos_byte = PT_BYTE;
4ed46869 7281
df7492f9 7282 encode_coding (coding);
b73bfc1c 7283
df7492f9
KH		 7284 unbind_to (count, Qnil);
7285 return coding->result;
7286}
4ed46869 7287
d46c5b12 7288
df7492f9
KH		 7289/* Decode the text in the range FROM/FROM_BYTE and TO/TO_BYTE in
7290 SRC_OBJECT into DST_OBJECT by coding context CODING.
b73bfc1c 7291
df7492f9 7292 SRC_OBJECT is a buffer, a string, or Qnil.
b73bfc1c 7293
df7492f9
KH		 7294 If it is a buffer, the text is at point of the buffer. FROM and TO
7295 are positions in the buffer.
b73bfc1c 7296
df7492f9
KH		 7297 If it is a string, the text is at the beginning of the string.
7298 FROM and TO are indices to the string.
4ed46869 7299
df7492f9
KH		 7300 If it is nil, the text is at coding->source. FROM and TO are
7301 indices to coding->source.
bb10be8b 7302
df7492f9 7303 DST_OBJECT is a buffer, Qt, or Qnil.
4ed46869 7304
df7492f9
KH		 7305 If it is a buffer, the decoded text is inserted at point of the
7306 buffer. If the buffer is the same as SRC_OBJECT, the source text
7307 is deleted.
4ed46869 7308
df7492f9
KH		 7309 If it is Qt, a string is made from the decoded text, and
7310 set in CODING->dst_object.
d46c5b12 7311
df7492f9 7312 If it is Qnil, the decoded text is stored at CODING->destination.
2cb26057 7313 The caller must allocate CODING->dst_bytes bytes at
df7492f9
KH		 7314 CODING->destination by xmalloc. If the decoded text is longer than
7315 CODING->dst_bytes, CODING->destination is relocated by xrealloc.
7316 */
d46c5b12 7317
df7492f9
KH		 7318void
7319decode_coding_object (coding, src_object, from, from_byte, to, to_byte,
7320 dst_object)
d46c5b12 7321 struct coding_system *coding;
df7492f9
KH		 7322 Lisp_Object src_object;
7323 EMACS_INT from, from_byte, to, to_byte;
7324 Lisp_Object dst_object;
d46c5b12 7325{
df7492f9
KH		 7326 int count = specpdl_ptr - specpdl;
7327 unsigned char *destination;
7328 EMACS_INT dst_bytes;
7329 EMACS_INT chars = to - from;
7330 EMACS_INT bytes = to_byte - from_byte;
7331 Lisp_Object attrs;
4776e638 7332 int saved_pt = -1, saved_pt_byte;
64cedb0c 7333 int need_marker_adjustment = 0;
b3bfad50 7334 Lisp_Object old_deactivate_mark;
d46c5b12 7335
b3bfad50 7336 old_deactivate_mark = Vdeactivate_mark;
93dec019 7337
df7492f9 7338 if (NILP (dst_object))
d46c5b12 7339 {
df7492f9
KH		 7340 destination = coding->destination;
7341 dst_bytes = coding->dst_bytes;
d46c5b12 7342 }
93dec019 7343
df7492f9
KH		 7344 coding->src_object = src_object;
7345 coding->src_chars = chars;
7346 coding->src_bytes = bytes;
7347 coding->src_multibyte = chars < bytes;
70ad9fc4 7348
df7492f9 7349 if (STRINGP (src_object))
d46c5b12 7350 {
df7492f9
KH		 7351 coding->src_pos = from;
7352 coding->src_pos_byte = from_byte;
d46c5b12 7353 }
df7492f9 7354 else if (BUFFERP (src_object))
88993dfd 7355 {
df7492f9
KH		 7356 set_buffer_internal (XBUFFER (src_object));
7357 if (from != GPT)
7358 move_gap_both (from, from_byte);
7359 if (EQ (src_object, dst_object))
fb88bf2d 7360 {
64cedb0c
KH		 7361 struct Lisp_Marker *tail;
7362
7363 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7364 {
7365 tail->need_adjustment
7366 = tail->charpos == (tail->insertion_type ? from : to);
7367 need_marker_adjustment |= tail->need_adjustment;
7368 }
4776e638 7369 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9 7370 TEMP_SET_PT_BOTH (from, from_byte);
f4a3cc44 7371 current_buffer->text->inhibit_shrinking = 1;
df7492f9
KH		 7372 del_range_both (from, from_byte, to, to_byte, 1);
7373 coding->src_pos = -chars;
7374 coding->src_pos_byte = -bytes;
fb88bf2d 7375 }
df7492f9 7376 else
fb88bf2d 7377 {
df7492f9
KH		 7378 coding->src_pos = from;
7379 coding->src_pos_byte = from_byte;
fb88bf2d 7380 }
88993dfd
KH		 7381 }
7382
df7492f9
KH		 7383 if (CODING_REQUIRE_DETECTION (coding))
7384 detect_coding (coding);
7385 attrs = CODING_ID_ATTRS (coding->id);
d46c5b12 7386
2cb26057
KH		 7387 if (EQ (dst_object, Qt)
7388 || (! NILP (CODING_ATTR_POST_READ (attrs))
7389 && NILP (dst_object)))
b73bfc1c 7390 {
a1567c45
SM		 7391 coding->dst_multibyte = !CODING_FOR_UNIBYTE (coding);
7392 coding->dst_object = code_conversion_save (1, coding->dst_multibyte);
df7492f9
KH		 7393 coding->dst_pos = BEG;
7394 coding->dst_pos_byte = BEG_BYTE;
b73bfc1c 7395 }
df7492f9 7396 else if (BUFFERP (dst_object))
d46c5b12 7397 {
24a73b0a 7398 code_conversion_save (0, 0);
df7492f9
KH		 7399 coding->dst_object = dst_object;
7400 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
7401 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
7402 coding->dst_multibyte
7403 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
d46c5b12
KH		 7404 }
7405 else
7406 {
24a73b0a 7407 code_conversion_save (0, 0);
df7492f9 7408 coding->dst_object = Qnil;
0154725e
SM		 7409 /* Most callers presume this will return a multibyte result, and they
7410 won't use `binary' or `raw-text' anyway, so let's not worry about
7411 CODING_FOR_UNIBYTE. */
bb555731 7412 coding->dst_multibyte = 1;
d46c5b12
KH		 7413 }
7414
df7492f9 7415 decode_coding (coding);
fa46990e 7416
df7492f9
KH		 7417 if (BUFFERP (coding->dst_object))
7418 set_buffer_internal (XBUFFER (coding->dst_object));
d46c5b12 7419
df7492f9 7420 if (! NILP (CODING_ATTR_POST_READ (attrs)))
d46c5b12 7421 {
b3bfad50 7422 struct gcpro gcpro1, gcpro2, gcpro3, gcpro4, gcpro5;
df7492f9 7423 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
2b4f9037 7424 Lisp_Object val;
d46c5b12 7425
c0cc7f7f 7426 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
b3bfad50
KH		 7427 GCPRO5 (coding->src_object, coding->dst_object, src_object, dst_object,
7428 old_deactivate_mark);
d4850d67
KH		 7429 val = safe_call1 (CODING_ATTR_POST_READ (attrs),
7430 make_number (coding->produced_char));
df7492f9
KH		 7431 UNGCPRO;
7432 CHECK_NATNUM (val);
7433 coding->produced_char += Z - prev_Z;
7434 coding->produced += Z_BYTE - prev_Z_BYTE;
d46c5b12 7435 }
de79a6a5 7436
df7492f9 7437 if (EQ (dst_object, Qt))
ec6d2bb8 7438 {
df7492f9
KH		 7439 coding->dst_object = Fbuffer_string ();
7440 }
7441 else if (NILP (dst_object) && BUFFERP (coding->dst_object))
7442 {
7443 set_buffer_internal (XBUFFER (coding->dst_object));
7444 if (dst_bytes < coding->produced)
7445 {
b3bfad50 7446 destination = xrealloc (destination, coding->produced);
df7492f9
KH		 7447 if (! destination)
7448 {
065e3595
KH		 7449 record_conversion_result (coding,
7450 CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 7451 unbind_to (count, Qnil);
7452 return;
7453 }
7454 if (BEGV < GPT && GPT < BEGV + coding->produced_char)
7455 move_gap_both (BEGV, BEGV_BYTE);
7456 bcopy (BEGV_ADDR, destination, coding->produced);
7457 coding->destination = destination;
d46c5b12 7458 }
ec6d2bb8 7459 }
b73bfc1c 7460
4776e638
KH		 7461 if (saved_pt >= 0)
7462 {
7463 /* This is the case of:
7464 (BUFFERP (src_object) && EQ (src_object, dst_object))
7465 As we have moved PT while replacing the original buffer
7466 contents, we must recover it now. */
7467 set_buffer_internal (XBUFFER (src_object));
f4a3cc44 7468 current_buffer->text->inhibit_shrinking = 0;
4776e638
KH		 7469 if (saved_pt < from)
7470 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7471 else if (saved_pt < from + chars)
7472 TEMP_SET_PT_BOTH (from, from_byte);
7473 else if (! NILP (current_buffer->enable_multibyte_characters))
7474 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7475 saved_pt_byte + (coding->produced - bytes));
7476 else
7477 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7478 saved_pt_byte + (coding->produced - bytes));
64cedb0c
KH		 7479
7480 if (need_marker_adjustment)
7481 {
7482 struct Lisp_Marker *tail;
7483
7484 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7485 if (tail->need_adjustment)
7486 {
7487 tail->need_adjustment = 0;
7488 if (tail->insertion_type)
7489 {
7490 tail->bytepos = from_byte;
7491 tail->charpos = from;
7492 }
7493 else
7494 {
7495 tail->bytepos = from_byte + coding->produced;
7496 tail->charpos
7497 = (NILP (current_buffer->enable_multibyte_characters)
7498 ? tail->bytepos : from + coding->produced_char);
7499 }
7500 }
7501 }
d46c5b12 7502 }
4776e638 7503
b3bfad50 7504 Vdeactivate_mark = old_deactivate_mark;
065e3595 7505 unbind_to (count, coding->dst_object);
d46c5b12
KH		 7506}
7507
d46c5b12 7508
df7492f9
KH		 7509void
7510encode_coding_object (coding, src_object, from, from_byte, to, to_byte,
7511 dst_object)
d46c5b12 7512 struct coding_system *coding;
df7492f9
KH		 7513 Lisp_Object src_object;
7514 EMACS_INT from, from_byte, to, to_byte;
7515 Lisp_Object dst_object;
d46c5b12 7516{
b73bfc1c 7517 int count = specpdl_ptr - specpdl;
df7492f9
KH		 7518 EMACS_INT chars = to - from;
7519 EMACS_INT bytes = to_byte - from_byte;
7520 Lisp_Object attrs;
4776e638 7521 int saved_pt = -1, saved_pt_byte;
64cedb0c 7522 int need_marker_adjustment = 0;
c02d943b 7523 int kill_src_buffer = 0;
b3bfad50 7524 Lisp_Object old_deactivate_mark;
df7492f9 7525
b3bfad50 7526 old_deactivate_mark = Vdeactivate_mark;
df7492f9
KH		 7527
7528 coding->src_object = src_object;
7529 coding->src_chars = chars;
7530 coding->src_bytes = bytes;
7531 coding->src_multibyte = chars < bytes;
7532
7533 attrs = CODING_ID_ATTRS (coding->id);
7534
64cedb0c
KH		 7535 if (EQ (src_object, dst_object))
7536 {
7537 struct Lisp_Marker *tail;
7538
7539 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7540 {
7541 tail->need_adjustment
7542 = tail->charpos == (tail->insertion_type ? from : to);
7543 need_marker_adjustment |= tail->need_adjustment;
7544 }
7545 }
7546
df7492f9 7547 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
6bac5b12 7548 {
24a73b0a 7549 coding->src_object = code_conversion_save (1, coding->src_multibyte);
df7492f9
KH		 7550 set_buffer_internal (XBUFFER (coding->src_object));
7551 if (STRINGP (src_object))
7552 insert_from_string (src_object, from, from_byte, chars, bytes, 0);
7553 else if (BUFFERP (src_object))
7554 insert_from_buffer (XBUFFER (src_object), from, chars, 0);
7555 else
7556 insert_1_both (coding->source + from, chars, bytes, 0, 0, 0);
d46c5b12 7557
df7492f9
KH		 7558 if (EQ (src_object, dst_object))
7559 {
7560 set_buffer_internal (XBUFFER (src_object));
4776e638 7561 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9
KH		 7562 del_range_both (from, from_byte, to, to_byte, 1);
7563 set_buffer_internal (XBUFFER (coding->src_object));
7564 }
7565
d4850d67
KH		 7566 {
7567 Lisp_Object args[3];
b3bfad50 7568 struct gcpro gcpro1, gcpro2, gcpro3, gcpro4, gcpro5;
d4850d67 7569
b3bfad50
KH		 7570 GCPRO5 (coding->src_object, coding->dst_object, src_object, dst_object,
7571 old_deactivate_mark);
d4850d67
KH		 7572 args[0] = CODING_ATTR_PRE_WRITE (attrs);
7573 args[1] = make_number (BEG);
7574 args[2] = make_number (Z);
7575 safe_call (3, args);
b3bfad50 7576 UNGCPRO;
d4850d67 7577 }
c02d943b
KH		 7578 if (XBUFFER (coding->src_object) != current_buffer)
7579 kill_src_buffer = 1;
ac87bbef 7580 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7581 if (BEG != GPT)
7582 move_gap_both (BEG, BEG_BYTE);
7583 coding->src_chars = Z - BEG;
7584 coding->src_bytes = Z_BYTE - BEG_BYTE;
7585 coding->src_pos = BEG;
7586 coding->src_pos_byte = BEG_BYTE;
7587 coding->src_multibyte = Z < Z_BYTE;
7588 }
7589 else if (STRINGP (src_object))
d46c5b12 7590 {
24a73b0a 7591 code_conversion_save (0, 0);
df7492f9
KH		 7592 coding->src_pos = from;
7593 coding->src_pos_byte = from_byte;
b73bfc1c 7594 }
df7492f9 7595 else if (BUFFERP (src_object))
b73bfc1c 7596 {
24a73b0a 7597 code_conversion_save (0, 0);
df7492f9 7598 set_buffer_internal (XBUFFER (src_object));
df7492f9 7599 if (EQ (src_object, dst_object))
d46c5b12 7600 {
4776e638 7601 saved_pt = PT, saved_pt_byte = PT_BYTE;
ff0dacd7
KH		 7602 coding->src_object = del_range_1 (from, to, 1, 1);
7603 coding->src_pos = 0;
7604 coding->src_pos_byte = 0;
d46c5b12 7605 }
df7492f9 7606 else
d46c5b12 7607 {
ff0dacd7
KH		 7608 if (from < GPT && to >= GPT)
7609 move_gap_both (from, from_byte);
df7492f9
KH		 7610 coding->src_pos = from;
7611 coding->src_pos_byte = from_byte;
d46c5b12 7612 }
d46c5b12 7613 }
4776e638 7614 else
24a73b0a 7615 code_conversion_save (0, 0);
d46c5b12 7616
df7492f9 7617 if (BUFFERP (dst_object))
88993dfd 7618 {
df7492f9 7619 coding->dst_object = dst_object;
28f67a95
KH		 7620 if (EQ (src_object, dst_object))
7621 {
7622 coding->dst_pos = from;
7623 coding->dst_pos_byte = from_byte;
7624 }
7625 else
7626 {
319a3947
KH		 7627 struct buffer *current = current_buffer;
7628
7629 set_buffer_temp (XBUFFER (dst_object));
7630 coding->dst_pos = PT;
7631 coding->dst_pos_byte = PT_BYTE;
7632 move_gap_both (coding->dst_pos, coding->dst_pos_byte);
7633 set_buffer_temp (current);
28f67a95 7634 }
df7492f9
KH		 7635 coding->dst_multibyte
7636 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
88993dfd 7637 }
df7492f9 7638 else if (EQ (dst_object, Qt))
d46c5b12 7639 {
df7492f9 7640 coding->dst_object = Qnil;
df7492f9 7641 coding->dst_bytes = coding->src_chars;
ac87bbef
KH		 7642 if (coding->dst_bytes == 0)
7643 coding->dst_bytes = 1;
7644 coding->destination = (unsigned char *) xmalloc (coding->dst_bytes);
df7492f9 7645 coding->dst_multibyte = 0;
d46c5b12
KH		 7646 }
7647 else
7648 {
df7492f9
KH		 7649 coding->dst_object = Qnil;
7650 coding->dst_multibyte = 0;
d46c5b12
KH		 7651 }
7652
df7492f9 7653 encode_coding (coding);
d46c5b12 7654
df7492f9 7655 if (EQ (dst_object, Qt))
d46c5b12 7656 {
df7492f9
KH		 7657 if (BUFFERP (coding->dst_object))
7658 coding->dst_object = Fbuffer_string ();
7659 else
d46c5b12 7660 {
df7492f9
KH		 7661 coding->dst_object
7662 = make_unibyte_string ((char *) coding->destination,
7663 coding->produced);
7664 xfree (coding->destination);
d46c5b12 7665 }
4ed46869 7666 }
d46c5b12 7667
4776e638
KH		 7668 if (saved_pt >= 0)
7669 {
7670 /* This is the case of:
7671 (BUFFERP (src_object) && EQ (src_object, dst_object))
7672 As we have moved PT while replacing the original buffer
7673 contents, we must recover it now. */
7674 set_buffer_internal (XBUFFER (src_object));
7675 if (saved_pt < from)
7676 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7677 else if (saved_pt < from + chars)
7678 TEMP_SET_PT_BOTH (from, from_byte);
7679 else if (! NILP (current_buffer->enable_multibyte_characters))
7680 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7681 saved_pt_byte + (coding->produced - bytes));
d46c5b12 7682 else
4776e638
KH		 7683 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7684 saved_pt_byte + (coding->produced - bytes));
64cedb0c
KH		 7685
7686 if (need_marker_adjustment)
7687 {
7688 struct Lisp_Marker *tail;
7689
7690 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7691 if (tail->need_adjustment)
7692 {
7693 tail->need_adjustment = 0;
7694 if (tail->insertion_type)
7695 {
7696 tail->bytepos = from_byte;
7697 tail->charpos = from;
7698 }
7699 else
7700 {
7701 tail->bytepos = from_byte + coding->produced;
7702 tail->charpos
7703 = (NILP (current_buffer->enable_multibyte_characters)
7704 ? tail->bytepos : from + coding->produced_char);
7705 }
7706 }
7707 }
4776e638
KH		 7708 }
7709
c02d943b
KH		 7710 if (kill_src_buffer)
7711 Fkill_buffer (coding->src_object);
b3bfad50
KH		 7712
7713 Vdeactivate_mark = old_deactivate_mark;
df7492f9 7714 unbind_to (count, Qnil);
b73bfc1c
KH		 7715}
7716
df7492f9 7717
b73bfc1c 7718Lisp_Object
df7492f9 7719preferred_coding_system ()
b73bfc1c 7720{
df7492f9 7721 int id = coding_categories[coding_priorities[0]].id;
2391eaa4 7722
df7492f9 7723 return CODING_ID_NAME (id);
4ed46869
KH		 7724}
7725
7726\f
7727#ifdef emacs
1397dc18 7728/*** 8. Emacs Lisp library functions ***/
4ed46869 7729
4ed46869 7730DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0,
48b0f3ae 7731 doc: /* Return t if OBJECT is nil or a coding-system.
df7492f9 7732See the documentation of `define-coding-system' for information
48b0f3ae 7733about coding-system objects. */)
d4a1d553
JB		 7734 (object)
7735 Lisp_Object object;
4ed46869 7736{
d4a1d553
JB		 7737 if (NILP (object)
7738 || CODING_SYSTEM_ID (object) >= 0)
44e8490d 7739 return Qt;
d4a1d553
JB		 7740 if (! SYMBOLP (object)
7741 || NILP (Fget (object, Qcoding_system_define_form)))
44e8490d
KH		 7742 return Qnil;
7743 return Qt;
4ed46869
KH		 7744}
7745
9d991de8
RS		 7746DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system,
7747 Sread_non_nil_coding_system, 1, 1, 0,
48b0f3ae
PJ		 7748 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
7749 (prompt)
4ed46869
KH		 7750 Lisp_Object prompt;
7751{
e0e989f6 7752 Lisp_Object val;
9d991de8
RS		 7753 do
7754 {
4608c386
KH		 7755 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
7756 Qt, Qnil, Qcoding_system_history, Qnil, Qnil);
9d991de8 7757 }
8f924df7 7758 while (SCHARS (val) == 0);
e0e989f6 7759 return (Fintern (val, Qnil));
4ed46869
KH		 7760}
7761
9b787f3e 7762DEFUN ("read-coding-system", Fread_coding_system, Sread_coding_system, 1, 2, 0,
48b0f3ae 7763 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT.
c7183fb8
GM		 7764If the user enters null input, return second argument DEFAULT-CODING-SYSTEM.
7765Ignores case when completing coding systems (all Emacs coding systems
7766are lower-case). */)
48b0f3ae 7767 (prompt, default_coding_system)
9b787f3e 7768 Lisp_Object prompt, default_coding_system;
4ed46869 7769{
f44d27ce 7770 Lisp_Object val;
c7183fb8
GM		 7771 int count = SPECPDL_INDEX ();
7772
9b787f3e 7773 if (SYMBOLP (default_coding_system))
57d25e6f 7774 default_coding_system = SYMBOL_NAME (default_coding_system);
c7183fb8 7775 specbind (Qcompletion_ignore_case, Qt);
4608c386 7776 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
9b787f3e
RS		 7777 Qt, Qnil, Qcoding_system_history,
7778 default_coding_system, Qnil);
c7183fb8 7779 unbind_to (count, Qnil);
8f924df7 7780 return (SCHARS (val) == 0 ? Qnil : Fintern (val, Qnil));
4ed46869
KH		 7781}
7782
7783DEFUN ("check-coding-system", Fcheck_coding_system, Scheck_coding_system,
7784 1, 1, 0,
48b0f3ae 7785 doc: /* Check validity of CODING-SYSTEM.
9ffd559c
KH		 7786If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
7787It is valid if it is nil or a symbol defined as a coding system by the
7788function `define-coding-system'. */)
df7492f9 7789 (coding_system)
4ed46869
KH		 7790 Lisp_Object coding_system;
7791{
44e8490d
KH		 7792 Lisp_Object define_form;
7793
7794 define_form = Fget (coding_system, Qcoding_system_define_form);
7795 if (! NILP (define_form))
7796 {
7797 Fput (coding_system, Qcoding_system_define_form, Qnil);
7798 safe_eval (define_form);
7799 }
4ed46869
KH		 7800 if (!NILP (Fcoding_system_p (coding_system)))
7801 return coding_system;
fcad4ec4 7802 xsignal1 (Qcoding_system_error, coding_system);
4ed46869 7803}
df7492f9 7804
3a73fa5d 7805\f
89528eb3
KH		 7806/* Detect how the bytes at SRC of length SRC_BYTES are encoded. If
7807 HIGHEST is nonzero, return the coding system of the highest
7808 priority among the detected coding systems. Otherwize return a
7809 list of detected coding systems sorted by their priorities. If
7810 MULTIBYTEP is nonzero, it is assumed that the bytes are in correct
7811 multibyte form but contains only ASCII and eight-bit chars.
7812 Otherwise, the bytes are raw bytes.
7813
7814 CODING-SYSTEM controls the detection as below:
7815
7816 If it is nil, detect both text-format and eol-format. If the
7817 text-format part of CODING-SYSTEM is already specified
7818 (e.g. `iso-latin-1'), detect only eol-format. If the eol-format
7819 part of CODING-SYSTEM is already specified (e.g. `undecided-unix'),
7820 detect only text-format. */
7821
d46c5b12 7822Lisp_Object
24a73b0a
KH		 7823detect_coding_system (src, src_chars, src_bytes, highest, multibytep,
7824 coding_system)
8f924df7 7825 const unsigned char *src;
13818c30
SM		 7826 EMACS_INT src_chars, src_bytes;
7827 int highest;
0a28aafb 7828 int multibytep;
df7492f9 7829 Lisp_Object coding_system;
4ed46869 7830{
8f924df7 7831 const unsigned char *src_end = src + src_bytes;
df7492f9 7832 Lisp_Object attrs, eol_type;
4533845d 7833 Lisp_Object val = Qnil;
df7492f9 7834 struct coding_system coding;
89528eb3 7835 int id;
ff0dacd7 7836 struct coding_detection_info detect_info;
24a73b0a 7837 enum coding_category base_category;
2f3cbb32 7838 int null_byte_found = 0, eight_bit_found = 0;
b73bfc1c 7839
df7492f9
KH		 7840 if (NILP (coding_system))
7841 coding_system = Qundecided;
7842 setup_coding_system (coding_system, &coding);
7843 attrs = CODING_ID_ATTRS (coding.id);
7844 eol_type = CODING_ID_EOL_TYPE (coding.id);
89528eb3 7845 coding_system = CODING_ATTR_BASE_NAME (attrs);
4ed46869 7846
df7492f9 7847 coding.source = src;
24a73b0a 7848 coding.src_chars = src_chars;
df7492f9
KH		 7849 coding.src_bytes = src_bytes;
7850 coding.src_multibyte = multibytep;
7851 coding.consumed = 0;
89528eb3 7852 coding.mode |= CODING_MODE_LAST_BLOCK;
c0e16b14 7853 coding.head_ascii = 0;
d46c5b12 7854
ff0dacd7 7855 detect_info.checked = detect_info.found = detect_info.rejected = 0;
d46c5b12 7856
89528eb3 7857 /* At first, detect text-format if necessary. */
24a73b0a
KH		 7858 base_category = XINT (CODING_ATTR_CATEGORY (attrs));
7859 if (base_category == coding_category_undecided)
4ed46869 7860 {
ff0dacd7
KH		 7861 enum coding_category category;
7862 struct coding_system *this;
7863 int c, i;
88993dfd 7864
24a73b0a 7865 /* Skip all ASCII bytes except for a few ISO2022 controls. */
2f3cbb32 7866 for (; src < src_end; src++)
4ed46869 7867 {
df7492f9 7868 c = *src;
6cb21a4f 7869 if (c & 0x80)
6cb21a4f 7870 {
2f3cbb32 7871 eight_bit_found = 1;
2f3cbb32
KH		 7872 if (null_byte_found)
7873 break;
7874 }
c0e16b14 7875 else if (c < 0x20)
2f3cbb32
KH		 7876 {
7877 if ((c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
7878 && ! inhibit_iso_escape_detection
7879 && ! detect_info.checked)
6cb21a4f 7880 {
2f3cbb32
KH		 7881 if (detect_coding_iso_2022 (&coding, &detect_info))
7882 {
7883 /* We have scanned the whole data. */
7884 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
c0e16b14
KH		 7885 {
7886 /* We didn't find an 8-bit code. We may
7887 have found a null-byte, but it's very
7888 rare that a binary file confirm to
7889 ISO-2022. */
7890 src = src_end;
7891 coding.head_ascii = src - coding.source;
7892 }
7893 detect_info.rejected |= ~CATEGORY_MASK_ISO_ESCAPE;
2f3cbb32
KH		 7894 break;
7895 }
7896 }
97b1b294 7897 else if (! c && !inhibit_null_byte_detection)
2f3cbb32
KH		 7898 {
7899 null_byte_found = 1;
7900 if (eight_bit_found)
7901 break;
6cb21a4f 7902 }
c006c0c8
KH		 7903 if (! eight_bit_found)
7904 coding.head_ascii++;
6cb21a4f 7905 }
c006c0c8 7906 else if (! eight_bit_found)
c0e16b14 7907 coding.head_ascii++;
4ed46869 7908 }
88993dfd 7909
2f3cbb32
KH		 7910 if (null_byte_found || eight_bit_found
7911 || coding.head_ascii < coding.src_bytes
6cb21a4f
KH		 7912 || detect_info.found)
7913 {
2f3cbb32 7914 if (coding.head_ascii == coding.src_bytes)
6cb21a4f
KH		 7915 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
7916 for (i = 0; i < coding_category_raw_text; i++)
ff0dacd7 7917 {
6cb21a4f 7918 category = coding_priorities[i];
c7266f4a 7919 this = coding_categories + category;
6cb21a4f 7920 if (detect_info.found & (1 << category))
ff0dacd7
KH		 7921 break;
7922 }
6cb21a4f 7923 else
2f3cbb32
KH		 7924 {
7925 if (null_byte_found)
7926 {
7927 detect_info.checked |= ~CATEGORY_MASK_UTF_16;
7928 detect_info.rejected |= ~CATEGORY_MASK_UTF_16;
7929 }
7930 for (i = 0; i < coding_category_raw_text; i++)
7931 {
7932 category = coding_priorities[i];
7933 this = coding_categories + category;
6cb21a4f 7934
2f3cbb32
KH		 7935 if (this->id < 0)
7936 {
7937 /* No coding system of this category is defined. */
7938 detect_info.rejected |= (1 << category);
7939 }
7940 else if (category >= coding_category_raw_text)
7941 continue;
7942 else if (detect_info.checked & (1 << category))
7943 {
7944 if (highest
7945 && (detect_info.found & (1 << category)))
6cb21a4f 7946 break;
2f3cbb32
KH		 7947 }
7948 else if ((*(this->detector)) (&coding, &detect_info)
7949 && highest
7950 && (detect_info.found & (1 << category)))
7951 {
7952 if (category == coding_category_utf_16_auto)
7953 {
7954 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
7955 category = coding_category_utf_16_le;
7956 else
7957 category = coding_category_utf_16_be;
7958 }
7959 break;
7960 }
7961 }
7962 }
6cb21a4f 7963 }
ec6d2bb8 7964
4cddb209
KH		 7965 if ((detect_info.rejected & CATEGORY_MASK_ANY) == CATEGORY_MASK_ANY
7966 || null_byte_found)
ec6d2bb8 7967 {
ff0dacd7 7968 detect_info.found = CATEGORY_MASK_RAW_TEXT;
4cddb209 7969 id = CODING_SYSTEM_ID (Qno_conversion);
89528eb3
KH		 7970 val = Fcons (make_number (id), Qnil);
7971 }
ff0dacd7 7972 else if (! detect_info.rejected && ! detect_info.found)
89528eb3 7973 {
ff0dacd7 7974 detect_info.found = CATEGORY_MASK_ANY;
89528eb3
KH		 7975 id = coding_categories[coding_category_undecided].id;
7976 val = Fcons (make_number (id), Qnil);
7977 }
7978 else if (highest)
7979 {
ff0dacd7 7980 if (detect_info.found)
ec6d2bb8 7981 {
ff0dacd7
KH		 7982 detect_info.found = 1 << category;
7983 val = Fcons (make_number (this->id), Qnil);
7984 }
7985 else
7986 for (i = 0; i < coding_category_raw_text; i++)
7987 if (! (detect_info.rejected & (1 << coding_priorities[i])))
7988 {
7989 detect_info.found = 1 << coding_priorities[i];
7990 id = coding_categories[coding_priorities[i]].id;
7991 val = Fcons (make_number (id), Qnil);
7992 break;
7993 }
7994 }
89528eb3
KH		 7995 else
7996 {
ff0dacd7
KH		 7997 int mask = detect_info.rejected | detect_info.found;
7998 int found = 0;
ec6d2bb8 7999
89528eb3 8000 for (i = coding_category_raw_text - 1; i >= 0; i--)
ff0dacd7
KH		 8001 {
8002 category = coding_priorities[i];
8003 if (! (mask & (1 << category)))
ec6d2bb8 8004 {
ff0dacd7
KH		 8005 found |= 1 << category;
8006 id = coding_categories[category].id;
c7266f4a
KH		 8007 if (id >= 0)
8008 val = Fcons (make_number (id), val);
ff0dacd7
KH		 8009 }
8010 }
8011 for (i = coding_category_raw_text - 1; i >= 0; i--)
8012 {
8013 category = coding_priorities[i];
8014 if (detect_info.found & (1 << category))
8015 {
8016 id = coding_categories[category].id;
8017 val = Fcons (make_number (id), val);
ec6d2bb8 8018 }
ec6d2bb8 8019 }
ff0dacd7 8020 detect_info.found |= found;
ec6d2bb8 8021 }
ec6d2bb8 8022 }
a470d443
KH		 8023 else if (base_category == coding_category_utf_8_auto)
8024 {
8025 if (detect_coding_utf_8 (&coding, &detect_info))
8026 {
8027 struct coding_system *this;
8028
8029 if (detect_info.found & CATEGORY_MASK_UTF_8_SIG)
8030 this = coding_categories + coding_category_utf_8_sig;
8031 else
8032 this = coding_categories + coding_category_utf_8_nosig;
8033 val = Fcons (make_number (this->id), Qnil);
8034 }
8035 }
24a73b0a
KH		 8036 else if (base_category == coding_category_utf_16_auto)
8037 {
8038 if (detect_coding_utf_16 (&coding, &detect_info))
8039 {
24a73b0a
KH		 8040 struct coding_system *this;
8041
8042 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
8043 this = coding_categories + coding_category_utf_16_le;
8044 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
8045 this = coding_categories + coding_category_utf_16_be;
8046 else if (detect_info.rejected & CATEGORY_MASK_UTF_16_LE_NOSIG)
8047 this = coding_categories + coding_category_utf_16_be_nosig;
8048 else
8049 this = coding_categories + coding_category_utf_16_le_nosig;
8050 val = Fcons (make_number (this->id), Qnil);
8051 }
8052 }
df7492f9
KH		 8053 else
8054 {
ff0dacd7 8055 detect_info.found = 1 << XINT (CODING_ATTR_CATEGORY (attrs));
89528eb3 8056 val = Fcons (make_number (coding.id), Qnil);
4ed46869 8057 }
df7492f9 8058
89528eb3 8059 /* Then, detect eol-format if necessary. */
df7492f9 8060 {
4533845d 8061 int normal_eol = -1, utf_16_be_eol = -1, utf_16_le_eol = -1;
df7492f9
KH		 8062 Lisp_Object tail;
8063
89528eb3
KH		 8064 if (VECTORP (eol_type))
8065 {
ff0dacd7 8066 if (detect_info.found & ~CATEGORY_MASK_UTF_16)
2f3cbb32
KH		 8067 {
8068 if (null_byte_found)
8069 normal_eol = EOL_SEEN_LF;
8070 else
8071 normal_eol = detect_eol (coding.source, src_bytes,
8072 coding_category_raw_text);
8073 }
ff0dacd7
KH		 8074 if (detect_info.found & (CATEGORY_MASK_UTF_16_BE
8075 | CATEGORY_MASK_UTF_16_BE_NOSIG))
89528eb3
KH		 8076 utf_16_be_eol = detect_eol (coding.source, src_bytes,
8077 coding_category_utf_16_be);
ff0dacd7
KH		 8078 if (detect_info.found & (CATEGORY_MASK_UTF_16_LE
8079 | CATEGORY_MASK_UTF_16_LE_NOSIG))
89528eb3
KH		 8080 utf_16_le_eol = detect_eol (coding.source, src_bytes,
8081 coding_category_utf_16_le);
8082 }
8083 else
8084 {
8085 if (EQ (eol_type, Qunix))
8086 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_LF;
8087 else if (EQ (eol_type, Qdos))
8088 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CRLF;
8089 else
8090 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CR;
8091 }
8092
df7492f9
KH		 8093 for (tail = val; CONSP (tail); tail = XCDR (tail))
8094 {
89528eb3 8095 enum coding_category category;
df7492f9 8096 int this_eol;
89528eb3
KH		 8097
8098 id = XINT (XCAR (tail));
8099 attrs = CODING_ID_ATTRS (id);
8100 category = XINT (CODING_ATTR_CATEGORY (attrs));
8101 eol_type = CODING_ID_EOL_TYPE (id);
df7492f9
KH		 8102 if (VECTORP (eol_type))
8103 {
89528eb3
KH		 8104 if (category == coding_category_utf_16_be
8105 || category == coding_category_utf_16_be_nosig)
8106 this_eol = utf_16_be_eol;
8107 else if (category == coding_category_utf_16_le
8108 || category == coding_category_utf_16_le_nosig)
8109 this_eol = utf_16_le_eol;
df7492f9 8110 else
89528eb3
KH		 8111 this_eol = normal_eol;
8112
df7492f9
KH		 8113 if (this_eol == EOL_SEEN_LF)
8114 XSETCAR (tail, AREF (eol_type, 0));
8115 else if (this_eol == EOL_SEEN_CRLF)
8116 XSETCAR (tail, AREF (eol_type, 1));
8117 else if (this_eol == EOL_SEEN_CR)
8118 XSETCAR (tail, AREF (eol_type, 2));
89528eb3
KH		 8119 else
8120 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9 8121 }
89528eb3
KH		 8122 else
8123 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9
KH		 8124 }
8125 }
ec6d2bb8 8126
4533845d 8127 return (highest ? (CONSP (val) ? XCAR (val) : Qnil) : val);
ec6d2bb8
KH		 8128}
8129
ec6d2bb8 8130
d46c5b12
KH		 8131DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region,
8132 2, 3, 0,
48b0f3ae
PJ		 8133 doc: /* Detect coding system of the text in the region between START and END.
8134Return a list of possible coding systems ordered by priority.
ec6d2bb8 8135
12e0131a 8136If only ASCII characters are found (except for such ISO-2022 control
d4a1d553
JB		 8137characters as ESC), it returns a list of single element `undecided'
8138or its subsidiary coding system according to a detected end-of-line
8139format.
ec6d2bb8 8140
48b0f3ae
PJ		 8141If optional argument HIGHEST is non-nil, return the coding system of
8142highest priority. */)
8143 (start, end, highest)
d46c5b12
KH		 8144 Lisp_Object start, end, highest;
8145{
8146 int from, to;
8147 int from_byte, to_byte;
ec6d2bb8 8148
b7826503
PJ		 8149 CHECK_NUMBER_COERCE_MARKER (start);
8150 CHECK_NUMBER_COERCE_MARKER (end);
ec6d2bb8 8151
d46c5b12
KH		 8152 validate_region (&start, &end);
8153 from = XINT (start), to = XINT (end);
8154 from_byte = CHAR_TO_BYTE (from);
8155 to_byte = CHAR_TO_BYTE (to);
ec6d2bb8 8156
d46c5b12
KH		 8157 if (from < GPT && to >= GPT)
8158 move_gap_both (to, to_byte);
c210f766 8159
d46c5b12 8160 return detect_coding_system (BYTE_POS_ADDR (from_byte),
24a73b0a 8161 to - from, to_byte - from_byte,
0a28aafb
KH		 8162 !NILP (highest),
8163 !NILP (current_buffer
df7492f9
KH		 8164 ->enable_multibyte_characters),
8165 Qnil);
ec6d2bb8
KH		 8166}
8167
d46c5b12
KH		 8168DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string,
8169 1, 2, 0,
48b0f3ae
PJ		 8170 doc: /* Detect coding system of the text in STRING.
8171Return a list of possible coding systems ordered by priority.
fb88bf2d 8172
12e0131a 8173If only ASCII characters are found (except for such ISO-2022 control
d4a1d553
JB		 8174characters as ESC), it returns a list of single element `undecided'
8175or its subsidiary coding system according to a detected end-of-line
8176format.
d46c5b12 8177
48b0f3ae
PJ		 8178If optional argument HIGHEST is non-nil, return the coding system of
8179highest priority. */)
8180 (string, highest)
d46c5b12
KH		 8181 Lisp_Object string, highest;
8182{
b7826503 8183 CHECK_STRING (string);
b73bfc1c 8184
24a73b0a
KH		 8185 return detect_coding_system (SDATA (string),
8186 SCHARS (string), SBYTES (string),
8f924df7 8187 !NILP (highest), STRING_MULTIBYTE (string),
df7492f9 8188 Qnil);
4ed46869 8189}
4ed46869 8190
b73bfc1c 8191
df7492f9
KH		 8192static INLINE int
8193char_encodable_p (c, attrs)
8194 int c;
8195 Lisp_Object attrs;
05e6f5dc 8196{
df7492f9 8197 Lisp_Object tail;
df7492f9 8198 struct charset *charset;
7d64c6ad 8199 Lisp_Object translation_table;
d46c5b12 8200
7d64c6ad 8201 translation_table = CODING_ATTR_TRANS_TBL (attrs);
a6f87d34 8202 if (! NILP (translation_table))
7d64c6ad 8203 c = translate_char (translation_table, c);
df7492f9
KH		 8204 for (tail = CODING_ATTR_CHARSET_LIST (attrs);
8205 CONSP (tail); tail = XCDR (tail))
e133c8fa 8206 {
df7492f9
KH		 8207 charset = CHARSET_FROM_ID (XINT (XCAR (tail)));
8208 if (CHAR_CHARSET_P (c, charset))
8209 break;
e133c8fa 8210 }
df7492f9 8211 return (! NILP (tail));
05e6f5dc 8212}
83fa074f 8213
fb88bf2d 8214
df7492f9
KH		 8215/* Return a list of coding systems that safely encode the text between
8216 START and END. If EXCLUDE is non-nil, it is a list of coding
8217 systems not to check. The returned list doesn't contain any such
48468dac 8218 coding systems. In any case, if the text contains only ASCII or is
df7492f9 8219 unibyte, return t. */
e077cc80 8220
df7492f9
KH		 8221DEFUN ("find-coding-systems-region-internal",
8222 Ffind_coding_systems_region_internal,
8223 Sfind_coding_systems_region_internal, 2, 3, 0,
8224 doc: /* Internal use only. */)
8225 (start, end, exclude)
8226 Lisp_Object start, end, exclude;
8227{
8228 Lisp_Object coding_attrs_list, safe_codings;
8229 EMACS_INT start_byte, end_byte;
7c78e542 8230 const unsigned char *p, *pbeg, *pend;
df7492f9
KH		 8231 int c;
8232 Lisp_Object tail, elt;
d46c5b12 8233
df7492f9
KH		 8234 if (STRINGP (start))
8235 {
8236 if (!STRING_MULTIBYTE (start)
8f924df7 8237 || SCHARS (start) == SBYTES (start))
df7492f9
KH		 8238 return Qt;
8239 start_byte = 0;
8f924df7 8240 end_byte = SBYTES (start);
df7492f9
KH		 8241 }
8242 else
d46c5b12 8243 {
df7492f9
KH		 8244 CHECK_NUMBER_COERCE_MARKER (start);
8245 CHECK_NUMBER_COERCE_MARKER (end);
8246 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
8247 args_out_of_range (start, end);
8248 if (NILP (current_buffer->enable_multibyte_characters))
8249 return Qt;
8250 start_byte = CHAR_TO_BYTE (XINT (start));
8251 end_byte = CHAR_TO_BYTE (XINT (end));
8252 if (XINT (end) - XINT (start) == end_byte - start_byte)
8253 return Qt;
d46c5b12 8254
e1c23804 8255 if (XINT (start) < GPT && XINT (end) > GPT)
d46c5b12 8256 {
e1c23804
DL		 8257 if ((GPT - XINT (start)) < (XINT (end) - GPT))
8258 move_gap_both (XINT (start), start_byte);
df7492f9 8259 else
e1c23804 8260 move_gap_both (XINT (end), end_byte);
d46c5b12
KH		 8261 }
8262 }
8263
df7492f9
KH		 8264 coding_attrs_list = Qnil;
8265 for (tail = Vcoding_system_list; CONSP (tail); tail = XCDR (tail))
8266 if (NILP (exclude)
8267 || NILP (Fmemq (XCAR (tail), exclude)))
8268 {
8269 Lisp_Object attrs;
d46c5b12 8270
df7492f9
KH		 8271 attrs = AREF (CODING_SYSTEM_SPEC (XCAR (tail)), 0);
8272 if (EQ (XCAR (tail), CODING_ATTR_BASE_NAME (attrs))
8273 && ! EQ (CODING_ATTR_TYPE (attrs), Qundecided))
7d64c6ad
KH		 8274 {
8275 ASET (attrs, coding_attr_trans_tbl,
2170c8f0 8276 get_translation_table (attrs, 1, NULL));
7d64c6ad
KH		 8277 coding_attrs_list = Fcons (attrs, coding_attrs_list);
8278 }
df7492f9 8279 }
d46c5b12 8280
df7492f9 8281 if (STRINGP (start))
8f924df7 8282 p = pbeg = SDATA (start);
df7492f9
KH		 8283 else
8284 p = pbeg = BYTE_POS_ADDR (start_byte);
8285 pend = p + (end_byte - start_byte);
b843d1ae 8286
df7492f9
KH		 8287 while (p < pend && ASCII_BYTE_P (*p)) p++;
8288 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
d46c5b12 8289
05e6f5dc 8290 while (p < pend)
72d1a715 8291 {
df7492f9
KH		 8292 if (ASCII_BYTE_P (*p))
8293 p++;
72d1a715
RS		 8294 else
8295 {
df7492f9 8296 c = STRING_CHAR_ADVANCE (p);
12410ef1 8297
df7492f9
KH		 8298 charset_map_loaded = 0;
8299 for (tail = coding_attrs_list; CONSP (tail);)
8300 {
8301 elt = XCAR (tail);
8302 if (NILP (elt))
8303 tail = XCDR (tail);
8304 else if (char_encodable_p (c, elt))
8305 tail = XCDR (tail);
8306 else if (CONSP (XCDR (tail)))
8307 {
8308 XSETCAR (tail, XCAR (XCDR (tail)));
8309 XSETCDR (tail, XCDR (XCDR (tail)));
8310 }
8311 else
8312 {
8313 XSETCAR (tail, Qnil);
8314 tail = XCDR (tail);
8315 }
8316 }
8317 if (charset_map_loaded)
8318 {
8319 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
05e6f5dc 8320
df7492f9 8321 if (STRINGP (start))
8f924df7 8322 pbeg = SDATA (start);
df7492f9
KH		 8323 else
8324 pbeg = BYTE_POS_ADDR (start_byte);
8325 p = pbeg + p_offset;
8326 pend = pbeg + pend_offset;
8327 }
8328 }
ec6d2bb8 8329 }
fb88bf2d 8330
988b3759 8331 safe_codings = list2 (Qraw_text, Qno_conversion);
df7492f9
KH		 8332 for (tail = coding_attrs_list; CONSP (tail); tail = XCDR (tail))
8333 if (! NILP (XCAR (tail)))
8334 safe_codings = Fcons (CODING_ATTR_BASE_NAME (XCAR (tail)), safe_codings);
ec6d2bb8 8335
05e6f5dc
KH		 8336 return safe_codings;
8337}
4956c225 8338
d46c5b12 8339
8f924df7
KH		 8340DEFUN ("unencodable-char-position", Funencodable_char_position,
8341 Sunencodable_char_position, 3, 5, 0,
8342 doc: /*
8343Return position of first un-encodable character in a region.
d4a1d553 8344START and END specify the region and CODING-SYSTEM specifies the
8f924df7 8345encoding to check. Return nil if CODING-SYSTEM does encode the region.
d46c5b12 8346
8f924df7
KH		 8347If optional 4th argument COUNT is non-nil, it specifies at most how
8348many un-encodable characters to search. In this case, the value is a
8349list of positions.
d46c5b12 8350
8f924df7
KH		 8351If optional 5th argument STRING is non-nil, it is a string to search
8352for un-encodable characters. In that case, START and END are indexes
8353to the string. */)
8354 (start, end, coding_system, count, string)
8355 Lisp_Object start, end, coding_system, count, string;
8356{
8357 int n;
8358 struct coding_system coding;
7d64c6ad 8359 Lisp_Object attrs, charset_list, translation_table;
8f924df7
KH		 8360 Lisp_Object positions;
8361 int from, to;
8362 const unsigned char *p, *stop, *pend;
8363 int ascii_compatible;
fb88bf2d 8364
8f924df7
KH		 8365 setup_coding_system (Fcheck_coding_system (coding_system), &coding);
8366 attrs = CODING_ID_ATTRS (coding.id);
8367 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
8368 return Qnil;
8369 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
8370 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2170c8f0 8371 translation_table = get_translation_table (attrs, 1, NULL);
fb88bf2d 8372
8f924df7
KH		 8373 if (NILP (string))
8374 {
8375 validate_region (&start, &end);
8376 from = XINT (start);
8377 to = XINT (end);
8378 if (NILP (current_buffer->enable_multibyte_characters)
8379 || (ascii_compatible
8380 && (to - from) == (CHAR_TO_BYTE (to) - (CHAR_TO_BYTE (from)))))
8381 return Qnil;
8382 p = CHAR_POS_ADDR (from);
8383 pend = CHAR_POS_ADDR (to);
8384 if (from < GPT && to >= GPT)
8385 stop = GPT_ADDR;
8386 else
8387 stop = pend;
8388 }
8389 else
8390 {
8391 CHECK_STRING (string);
8392 CHECK_NATNUM (start);
8393 CHECK_NATNUM (end);
8394 from = XINT (start);
8395 to = XINT (end);
8396 if (from > to
8397 || to > SCHARS (string))
8398 args_out_of_range_3 (string, start, end);
8399 if (! STRING_MULTIBYTE (string))
8400 return Qnil;
8401 p = SDATA (string) + string_char_to_byte (string, from);
8402 stop = pend = SDATA (string) + string_char_to_byte (string, to);
8403 if (ascii_compatible && (to - from) == (pend - p))
8404 return Qnil;
8405 }
f2558efd 8406
8f924df7
KH		 8407 if (NILP (count))
8408 n = 1;
8409 else
b73bfc1c 8410 {
8f924df7
KH		 8411 CHECK_NATNUM (count);
8412 n = XINT (count);
b73bfc1c
KH		 8413 }
8414
8f924df7
KH		 8415 positions = Qnil;
8416 while (1)
d46c5b12 8417 {
8f924df7 8418 int c;
ec6d2bb8 8419
8f924df7
KH		 8420 if (ascii_compatible)
8421 while (p < stop && ASCII_BYTE_P (*p))
8422 p++, from++;
8423 if (p >= stop)
0e79d667 8424 {
8f924df7
KH		 8425 if (p >= pend)
8426 break;
8427 stop = pend;
8428 p = GAP_END_ADDR;
0e79d667 8429 }
ec6d2bb8 8430
8f924df7
KH		 8431 c = STRING_CHAR_ADVANCE (p);
8432 if (! (ASCII_CHAR_P (c) && ascii_compatible)
7d64c6ad
KH		 8433 && ! char_charset (translate_char (translation_table, c),
8434 charset_list, NULL))
ec6d2bb8 8435 {
8f924df7
KH		 8436 positions = Fcons (make_number (from), positions);
8437 n--;
8438 if (n == 0)
8439 break;
ec6d2bb8
KH		 8440 }
8441
8f924df7
KH		 8442 from++;
8443 }
d46c5b12 8444
8f924df7
KH		 8445 return (NILP (count) ? Fcar (positions) : Fnreverse (positions));
8446}
d46c5b12 8447
d46c5b12 8448
df7492f9
KH		 8449DEFUN ("check-coding-systems-region", Fcheck_coding_systems_region,
8450 Scheck_coding_systems_region, 3, 3, 0,
8451 doc: /* Check if the region is encodable by coding systems.
d46c5b12 8452
df7492f9
KH		 8453START and END are buffer positions specifying the region.
8454CODING-SYSTEM-LIST is a list of coding systems to check.
d46c5b12 8455
df7492f9 8456The value is an alist ((CODING-SYSTEM POS0 POS1 ...) ...), where
d4a1d553 8457CODING-SYSTEM is a member of CODING-SYSTEM-LIST and can't encode the
df7492f9
KH		 8458whole region, POS0, POS1, ... are buffer positions where non-encodable
8459characters are found.
93dec019 8460
df7492f9
KH		 8461If all coding systems in CODING-SYSTEM-LIST can encode the region, the
8462value is nil.
93dec019 8463
df7492f9
KH		 8464START may be a string. In that case, check if the string is
8465encodable, and the value contains indices to the string instead of
5704f39a
KH		 8466buffer positions. END is ignored.
8467
4c1958f4 8468If the current buffer (or START if it is a string) is unibyte, the value
5704f39a 8469is nil. */)
df7492f9
KH		 8470 (start, end, coding_system_list)
8471 Lisp_Object start, end, coding_system_list;
05e6f5dc 8472{
df7492f9
KH		 8473 Lisp_Object list;
8474 EMACS_INT start_byte, end_byte;
8475 int pos;
7c78e542 8476 const unsigned char *p, *pbeg, *pend;
df7492f9 8477 int c;
7d64c6ad 8478 Lisp_Object tail, elt, attrs;
70ad9fc4 8479
05e6f5dc
KH		 8480 if (STRINGP (start))
8481 {
df7492f9 8482 if (!STRING_MULTIBYTE (start)
4c1958f4 8483 || SCHARS (start) == SBYTES (start))
df7492f9
KH		 8484 return Qnil;
8485 start_byte = 0;
8f924df7 8486 end_byte = SBYTES (start);
df7492f9 8487 pos = 0;
d46c5b12 8488 }
05e6f5dc 8489 else
b73bfc1c 8490 {
b7826503
PJ		 8491 CHECK_NUMBER_COERCE_MARKER (start);
8492 CHECK_NUMBER_COERCE_MARKER (end);
05e6f5dc
KH		 8493 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
8494 args_out_of_range (start, end);
8495 if (NILP (current_buffer->enable_multibyte_characters))
df7492f9
KH		 8496 return Qnil;
8497 start_byte = CHAR_TO_BYTE (XINT (start));
8498 end_byte = CHAR_TO_BYTE (XINT (end));
8499 if (XINT (end) - XINT (start) == end_byte - start_byte)
5704f39a 8500 return Qnil;
df7492f9 8501
e1c23804 8502 if (XINT (start) < GPT && XINT (end) > GPT)
b73bfc1c 8503 {
e1c23804
DL		 8504 if ((GPT - XINT (start)) < (XINT (end) - GPT))
8505 move_gap_both (XINT (start), start_byte);
df7492f9 8506 else
e1c23804 8507 move_gap_both (XINT (end), end_byte);
b73bfc1c 8508 }
e1c23804 8509 pos = XINT (start);
b73bfc1c 8510 }
7553d0e1 8511
df7492f9
KH		 8512 list = Qnil;
8513 for (tail = coding_system_list; CONSP (tail); tail = XCDR (tail))
12410ef1 8514 {
df7492f9 8515 elt = XCAR (tail);
7d64c6ad 8516 attrs = AREF (CODING_SYSTEM_SPEC (elt), 0);
2170c8f0
KH		 8517 ASET (attrs, coding_attr_trans_tbl,
8518 get_translation_table (attrs, 1, NULL));
7d64c6ad 8519 list = Fcons (Fcons (elt, Fcons (attrs, Qnil)), list);
12410ef1
KH		 8520 }
8521
df7492f9 8522 if (STRINGP (start))
8f924df7 8523 p = pbeg = SDATA (start);
72d1a715 8524 else
df7492f9
KH		 8525 p = pbeg = BYTE_POS_ADDR (start_byte);
8526 pend = p + (end_byte - start_byte);
4ed46869 8527
df7492f9
KH		 8528 while (p < pend && ASCII_BYTE_P (*p)) p++, pos++;
8529 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
ec6d2bb8 8530
df7492f9 8531 while (p < pend)
d46c5b12 8532 {
df7492f9
KH		 8533 if (ASCII_BYTE_P (*p))
8534 p++;
e133c8fa 8535 else
05e6f5dc 8536 {
df7492f9
KH		 8537 c = STRING_CHAR_ADVANCE (p);
8538
8539 charset_map_loaded = 0;
8540 for (tail = list; CONSP (tail); tail = XCDR (tail))
8541 {
8542 elt = XCDR (XCAR (tail));
8543 if (! char_encodable_p (c, XCAR (elt)))
8544 XSETCDR (elt, Fcons (make_number (pos), XCDR (elt)));
8545 }
8546 if (charset_map_loaded)
8547 {
8548 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
8549
8550 if (STRINGP (start))
8f924df7 8551 pbeg = SDATA (start);
df7492f9
KH		 8552 else
8553 pbeg = BYTE_POS_ADDR (start_byte);
8554 p = pbeg + p_offset;
8555 pend = pbeg + pend_offset;
8556 }
05e6f5dc 8557 }
df7492f9 8558 pos++;
d46c5b12 8559 }
4ed46869 8560
df7492f9
KH		 8561 tail = list;
8562 list = Qnil;
8563 for (; CONSP (tail); tail = XCDR (tail))
ec6d2bb8 8564 {
df7492f9
KH		 8565 elt = XCAR (tail);
8566 if (CONSP (XCDR (XCDR (elt))))
8567 list = Fcons (Fcons (XCAR (elt), Fnreverse (XCDR (XCDR (elt)))),
8568 list);
ec6d2bb8 8569 }
2b4f9037 8570
df7492f9 8571 return list;
d46c5b12
KH		 8572}
8573
3fd9494b 8574
b73bfc1c 8575Lisp_Object
df7492f9
KH		 8576code_convert_region (start, end, coding_system, dst_object, encodep, norecord)
8577 Lisp_Object start, end, coding_system, dst_object;
8578 int encodep, norecord;
4ed46869 8579{
3a73fa5d 8580 struct coding_system coding;
df7492f9
KH		 8581 EMACS_INT from, from_byte, to, to_byte;
8582 Lisp_Object src_object;
4ed46869 8583
b7826503
PJ		 8584 CHECK_NUMBER_COERCE_MARKER (start);
8585 CHECK_NUMBER_COERCE_MARKER (end);
df7492f9
KH		 8586 if (NILP (coding_system))
8587 coding_system = Qno_conversion;
8588 else
8589 CHECK_CODING_SYSTEM (coding_system);
8590 src_object = Fcurrent_buffer ();
8591 if (NILP (dst_object))
8592 dst_object = src_object;
8593 else if (! EQ (dst_object, Qt))
8594 CHECK_BUFFER (dst_object);
3a73fa5d 8595
d46c5b12
KH		 8596 validate_region (&start, &end);
8597 from = XFASTINT (start);
df7492f9 8598 from_byte = CHAR_TO_BYTE (from);
d46c5b12 8599 to = XFASTINT (end);
df7492f9 8600 to_byte = CHAR_TO_BYTE (to);
764ca8da 8601
df7492f9
KH		 8602 setup_coding_system (coding_system, &coding);
8603 coding.mode |= CODING_MODE_LAST_BLOCK;
ec6d2bb8 8604
df7492f9
KH		 8605 if (encodep)
8606 encode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
8607 dst_object);
8608 else
8609 decode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
8610 dst_object);
8611 if (! norecord)
8612 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
ec6d2bb8 8613
df7492f9
KH		 8614 return (BUFFERP (dst_object)
8615 ? make_number (coding.produced_char)
8616 : coding.dst_object);
4031e2bf 8617}
78108bcd 8618
4ed46869 8619
4031e2bf 8620DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region,
df7492f9 8621 3, 4, "r\nzCoding system: ",
48b0f3ae 8622 doc: /* Decode the current region from the specified coding system.
df7492f9
KH		 8623When called from a program, takes four arguments:
8624 START, END, CODING-SYSTEM, and DESTINATION.
8625START and END are buffer positions.
8844fa83 8626
df7492f9 8627Optional 4th arguments DESTINATION specifies where the decoded text goes.
2354b80b 8628If nil, the region between START and END is replaced by the decoded text.
1560f91a
EZ		 8629If buffer, the decoded text is inserted in that buffer after point (point
8630does not move).
446dcd75 8631In those cases, the length of the decoded text is returned.
319a3947 8632If DESTINATION is t, the decoded text is returned.
8844fa83 8633
48b0f3ae
PJ		 8634This function sets `last-coding-system-used' to the precise coding system
8635used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
319a3947 8636not fully specified.) */)
df7492f9
KH		 8637 (start, end, coding_system, destination)
8638 Lisp_Object start, end, coding_system, destination;
4031e2bf 8639{
df7492f9 8640 return code_convert_region (start, end, coding_system, destination, 0, 0);
3a73fa5d 8641}
8844fa83 8642
3a73fa5d 8643DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region,
df7492f9
KH		 8644 3, 4, "r\nzCoding system: ",
8645 doc: /* Encode the current region by specified coding system.
d4a1d553
JB		 8646When called from a program, takes four arguments:
8647 START, END, CODING-SYSTEM and DESTINATION.
8648START and END are buffer positions.
d46c5b12 8649
df7492f9
KH		 8650Optional 4th arguments DESTINATION specifies where the encoded text goes.
8651If nil, the region between START and END is replace by the encoded text.
1560f91a
EZ		 8652If buffer, the encoded text is inserted in that buffer after point (point
8653does not move).
446dcd75 8654In those cases, the length of the encoded text is returned.
319a3947 8655If DESTINATION is t, the encoded text is returned.
2391eaa4 8656
48b0f3ae
PJ		 8657This function sets `last-coding-system-used' to the precise coding system
8658used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
319a3947 8659not fully specified.) */)
df7492f9
KH		 8660 (start, end, coding_system, destination)
8661 Lisp_Object start, end, coding_system, destination;
3a73fa5d 8662{
df7492f9 8663 return code_convert_region (start, end, coding_system, destination, 1, 0);
b73bfc1c
KH		 8664}
8665
8666Lisp_Object
df7492f9
KH		 8667code_convert_string (string, coding_system, dst_object,
8668 encodep, nocopy, norecord)
8669 Lisp_Object string, coding_system, dst_object;
8670 int encodep, nocopy, norecord;
b73bfc1c 8671{
4031e2bf 8672 struct coding_system coding;
df7492f9 8673 EMACS_INT chars, bytes;
ec6d2bb8 8674
b7826503 8675 CHECK_STRING (string);
d46c5b12 8676 if (NILP (coding_system))
4956c225 8677 {
df7492f9
KH		 8678 if (! norecord)
8679 Vlast_coding_system_used = Qno_conversion;
8680 if (NILP (dst_object))
8681 return (nocopy ? Fcopy_sequence (string) : string);
4956c225 8682 }
b73bfc1c 8683
df7492f9
KH		 8684 if (NILP (coding_system))
8685 coding_system = Qno_conversion;
8686 else
8687 CHECK_CODING_SYSTEM (coding_system);
8688 if (NILP (dst_object))
8689 dst_object = Qt;
8690 else if (! EQ (dst_object, Qt))
8691 CHECK_BUFFER (dst_object);
73be902c 8692
df7492f9 8693 setup_coding_system (coding_system, &coding);
d46c5b12 8694 coding.mode |= CODING_MODE_LAST_BLOCK;
8f924df7
KH		 8695 chars = SCHARS (string);
8696 bytes = SBYTES (string);
df7492f9
KH		 8697 if (encodep)
8698 encode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8699 else
8700 decode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8701 if (! norecord)
8702 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
73be902c 8703
df7492f9
KH		 8704 return (BUFFERP (dst_object)
8705 ? make_number (coding.produced_char)
8706 : coding.dst_object);
4ed46869 8707}
73be902c 8708
b73bfc1c 8709
ecec61c1 8710/* Encode or decode STRING according to CODING_SYSTEM.
ec6d2bb8 8711 Do not set Vlast_coding_system_used.
4ed46869 8712
ec6d2bb8
KH		 8713 This function is called only from macros DECODE_FILE and
8714 ENCODE_FILE, thus we ignore character composition. */
4ed46869 8715
ecec61c1
KH		 8716Lisp_Object
8717code_convert_string_norecord (string, coding_system, encodep)
8718 Lisp_Object string, coding_system;
8719 int encodep;
4ed46869 8720{
0be8721c 8721 return code_convert_string (string, coding_system, Qt, encodep, 0, 1);
4ed46869
KH		 8722}
8723
4ed46869 8724
df7492f9
KH		 8725DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string,
8726 2, 4, 0,
8727 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
8728
8729Optional third arg NOCOPY non-nil means it is OK to return STRING itself
8730if the decoding operation is trivial.
ecec61c1 8731
d4a1d553 8732Optional fourth arg BUFFER non-nil means that the decoded text is
1560f91a
EZ		 8733inserted in that buffer after point (point does not move). In this
8734case, the return value is the length of the decoded text.
ecec61c1 8735
df7492f9
KH		 8736This function sets `last-coding-system-used' to the precise coding system
8737used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
d4a1d553 8738not fully specified.) */)
df7492f9
KH		 8739 (string, coding_system, nocopy, buffer)
8740 Lisp_Object string, coding_system, nocopy, buffer;
4ed46869 8741{
df7492f9
KH		 8742 return code_convert_string (string, coding_system, buffer,
8743 0, ! NILP (nocopy), 0);
4ed46869
KH		 8744}
8745
df7492f9
KH		 8746DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
8747 2, 4, 0,
8748 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
8749
8750Optional third arg NOCOPY non-nil means it is OK to return STRING
8751itself if the encoding operation is trivial.
8752
d4a1d553 8753Optional fourth arg BUFFER non-nil means that the encoded text is
1560f91a
EZ		 8754inserted in that buffer after point (point does not move). In this
8755case, the return value is the length of the encoded text.
df7492f9
KH		 8756
8757This function sets `last-coding-system-used' to the precise coding system
8758used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
8759not fully specified.) */)
8760 (string, coding_system, nocopy, buffer)
8761 Lisp_Object string, coding_system, nocopy, buffer;
4ed46869 8762{
df7492f9 8763 return code_convert_string (string, coding_system, buffer,
c197f191 8764 1, ! NILP (nocopy), 1);
4ed46869 8765}
df7492f9 8766
3a73fa5d 8767\f
4ed46869 8768DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0,
48b0f3ae
PJ		 8769 doc: /* Decode a Japanese character which has CODE in shift_jis encoding.
8770Return the corresponding character. */)
8771 (code)
4ed46869 8772 Lisp_Object code;
4ed46869 8773{
df7492f9
KH		 8774 Lisp_Object spec, attrs, val;
8775 struct charset *charset_roman, *charset_kanji, *charset_kana, *charset;
8776 int c;
4ed46869 8777
df7492f9
KH		 8778 CHECK_NATNUM (code);
8779 c = XFASTINT (code);
8780 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8781 attrs = AREF (spec, 0);
4ed46869 8782
df7492f9
KH		 8783 if (ASCII_BYTE_P (c)
8784 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8785 return code;
4ed46869 8786
df7492f9
KH		 8787 val = CODING_ATTR_CHARSET_LIST (attrs);
8788 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
004068e4
KH		 8789 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8790 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 8791
df7492f9
KH		 8792 if (c <= 0x7F)
8793 charset = charset_roman;
8794 else if (c >= 0xA0 && c < 0xDF)
55ab7be3 8795 {
df7492f9
KH		 8796 charset = charset_kana;
8797 c -= 0x80;
4ed46869 8798 }
55ab7be3 8799 else
4ed46869 8800 {
004068e4 8801 int s1 = c >> 8, s2 = c & 0xFF;
df7492f9
KH		 8802
8803 if (s1 < 0x81 || (s1 > 0x9F && s1 < 0xE0) || s1 > 0xEF
8804 || s2 < 0x40 || s2 == 0x7F || s2 > 0xFC)
8805 error ("Invalid code: %d", code);
8806 SJIS_TO_JIS (c);
8807 charset = charset_kanji;
4ed46869 8808 }
df7492f9
KH		 8809 c = DECODE_CHAR (charset, c);
8810 if (c < 0)
8811 error ("Invalid code: %d", code);
8812 return make_number (c);
93dec019 8813}
4ed46869 8814
48b0f3ae 8815
4ed46869 8816DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0,
8acf0c0e 8817 doc: /* Encode a Japanese character CH to shift_jis encoding.
48b0f3ae
PJ		 8818Return the corresponding code in SJIS. */)
8819 (ch)
df7492f9 8820 Lisp_Object ch;
4ed46869 8821{
df7492f9
KH		 8822 Lisp_Object spec, attrs, charset_list;
8823 int c;
8824 struct charset *charset;
8825 unsigned code;
48b0f3ae 8826
df7492f9
KH		 8827 CHECK_CHARACTER (ch);
8828 c = XFASTINT (ch);
8829 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8830 attrs = AREF (spec, 0);
8831
8832 if (ASCII_CHAR_P (c)
8833 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8834 return ch;
8835
8836 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8837 charset = char_charset (c, charset_list, &code);
8838 if (code == CHARSET_INVALID_CODE (charset))
8839 error ("Can't encode by shift_jis encoding: %d", c);
8840 JIS_TO_SJIS (code);
8841
8842 return make_number (code);
4ed46869
KH		 8843}
8844
8845DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0,
48b0f3ae
PJ		 8846 doc: /* Decode a Big5 character which has CODE in BIG5 coding system.
8847Return the corresponding character. */)
8848 (code)
4ed46869 8849 Lisp_Object code;
d46c5b12 8850{
df7492f9
KH		 8851 Lisp_Object spec, attrs, val;
8852 struct charset *charset_roman, *charset_big5, *charset;
8853 int c;
6289dd10 8854
df7492f9
KH		 8855 CHECK_NATNUM (code);
8856 c = XFASTINT (code);
8857 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8858 attrs = AREF (spec, 0);
4ed46869 8859
df7492f9
KH		 8860 if (ASCII_BYTE_P (c)
8861 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8862 return code;
6289dd10 8863
df7492f9
KH		 8864 val = CODING_ATTR_CHARSET_LIST (attrs);
8865 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8866 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
c210f766 8867
df7492f9
KH		 8868 if (c <= 0x7F)
8869 charset = charset_roman;
c28a9453
KH		 8870 else
8871 {
df7492f9
KH		 8872 int b1 = c >> 8, b2 = c & 0x7F;
8873 if (b1 < 0xA1 || b1 > 0xFE
8874 || b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE)
8875 error ("Invalid code: %d", code);
8876 charset = charset_big5;
c28a9453 8877 }
df7492f9
KH		 8878 c = DECODE_CHAR (charset, (unsigned )c);
8879 if (c < 0)
8880 error ("Invalid code: %d", code);
8881 return make_number (c);
d46c5b12 8882}
6289dd10 8883
4ed46869 8884DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0,
8acf0c0e 8885 doc: /* Encode the Big5 character CH to BIG5 coding system.
48b0f3ae
PJ		 8886Return the corresponding character code in Big5. */)
8887 (ch)
4ed46869
KH		 8888 Lisp_Object ch;
8889{
df7492f9
KH		 8890 Lisp_Object spec, attrs, charset_list;
8891 struct charset *charset;
8892 int c;
8893 unsigned code;
8894
8895 CHECK_CHARACTER (ch);
8896 c = XFASTINT (ch);
8897 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8898 attrs = AREF (spec, 0);
8899 if (ASCII_CHAR_P (c)
8900 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8901 return ch;
8902
8903 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8904 charset = char_charset (c, charset_list, &code);
8905 if (code == CHARSET_INVALID_CODE (charset))
8906 error ("Can't encode by Big5 encoding: %d", c);
8907
8908 return make_number (code);
4ed46869 8909}
48b0f3ae 8910
3a73fa5d 8911\f
002fdb44 8912DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal,
68bba4e4 8913 Sset_terminal_coding_system_internal, 1, 2, 0,
48b0f3ae 8914 doc: /* Internal use only. */)
6ed8eeff 8915 (coding_system, terminal)
b74e4686 8916 Lisp_Object coding_system;
6ed8eeff 8917 Lisp_Object terminal;
4ed46869 8918{
6ed8eeff 8919 struct coding_system *terminal_coding = TERMINAL_TERMINAL_CODING (get_terminal (terminal, 1));
b7826503 8920 CHECK_SYMBOL (coding_system);
b8299c66 8921 setup_coding_system (Fcheck_coding_system (coding_system), terminal_coding);
70c22245 8922 /* We had better not send unsafe characters to terminal. */
c73bd236 8923 terminal_coding->mode |= CODING_MODE_SAFE_ENCODING;
df7492f9 8924 /* Characer composition should be disabled. */
c73bd236 8925 terminal_coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b8299c66
KL		 8926 terminal_coding->src_multibyte = 1;
8927 terminal_coding->dst_multibyte = 0;
4ed46869
KH		 8928 return Qnil;
8929}
8930
c4825358
KH		 8931DEFUN ("set-safe-terminal-coding-system-internal",
8932 Fset_safe_terminal_coding_system_internal,
48b0f3ae 8933 Sset_safe_terminal_coding_system_internal, 1, 1, 0,
ddb67bdc 8934 doc: /* Internal use only. */)
48b0f3ae 8935 (coding_system)
b74e4686 8936 Lisp_Object coding_system;
d46c5b12 8937{
b7826503 8938 CHECK_SYMBOL (coding_system);
c4825358
KH		 8939 setup_coding_system (Fcheck_coding_system (coding_system),
8940 &safe_terminal_coding);
df7492f9
KH		 8941 /* Characer composition should be disabled. */
8942 safe_terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b73bfc1c
KH		 8943 safe_terminal_coding.src_multibyte = 1;
8944 safe_terminal_coding.dst_multibyte = 0;
c4825358
KH		 8945 return Qnil;
8946}
4ed46869 8947
002fdb44 8948DEFUN ("terminal-coding-system", Fterminal_coding_system,
68bba4e4 8949 Sterminal_coding_system, 0, 1, 0,
6ed8eeff
KL		 8950 doc: /* Return coding system specified for terminal output on the given terminal.
8951TERMINAL may be a terminal id, a frame, or nil for the selected
8952frame's terminal device. */)
8953 (terminal)
8954 Lisp_Object terminal;
4ed46869 8955{
985773c9
MB		 8956 struct coding_system *terminal_coding
8957 = TERMINAL_TERMINAL_CODING (get_terminal (terminal, 1));
8958 Lisp_Object coding_system = CODING_ID_NAME (terminal_coding->id);
ae6f73fa 8959
ae6f73fa 8960 /* For backward compatibility, return nil if it is `undecided'. */
f75c90a9 8961 return (! EQ (coding_system, Qundecided) ? coding_system : Qnil);
4ed46869
KH		 8962}
8963
002fdb44 8964DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal,
68bba4e4 8965 Sset_keyboard_coding_system_internal, 1, 2, 0,
48b0f3ae 8966 doc: /* Internal use only. */)
6ed8eeff 8967 (coding_system, terminal)
4ed46869 8968 Lisp_Object coding_system;
6ed8eeff 8969 Lisp_Object terminal;
4ed46869 8970{
6ed8eeff 8971 struct terminal *t = get_terminal (terminal, 1);
b7826503 8972 CHECK_SYMBOL (coding_system);
df7492f9 8973 setup_coding_system (Fcheck_coding_system (coding_system),
c73bd236 8974 TERMINAL_KEYBOARD_CODING (t));
df7492f9 8975 /* Characer composition should be disabled. */
c73bd236
MB		 8976 TERMINAL_KEYBOARD_CODING (t)->common_flags
8977 &= ~CODING_ANNOTATE_COMPOSITION_MASK;
4ed46869
KH		 8978 return Qnil;
8979}
8980
8981DEFUN ("keyboard-coding-system",
985773c9 8982 Fkeyboard_coding_system, Skeyboard_coding_system, 0, 1, 0,
48b0f3ae 8983 doc: /* Return coding system specified for decoding keyboard input. */)
985773c9
MB		 8984 (terminal)
8985 Lisp_Object terminal;
4ed46869 8986{
985773c9
MB		 8987 return CODING_ID_NAME (TERMINAL_KEYBOARD_CODING
8988 (get_terminal (terminal, 1))->id);
4ed46869
KH		 8989}
8990
4ed46869 8991\f
a5d301df
KH		 8992DEFUN ("find-operation-coding-system", Ffind_operation_coding_system,
8993 Sfind_operation_coding_system, 1, MANY, 0,
48b0f3ae
PJ		 8994 doc: /* Choose a coding system for an operation based on the target name.
8995The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
8996DECODING-SYSTEM is the coding system to use for decoding
8997\(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
8998for encoding (in case OPERATION does encoding).
05e6f5dc 8999
48b0f3ae
PJ		 9000The first argument OPERATION specifies an I/O primitive:
9001 For file I/O, `insert-file-contents' or `write-region'.
9002 For process I/O, `call-process', `call-process-region', or `start-process'.
9003 For network I/O, `open-network-stream'.
05e6f5dc 9004
48b0f3ae
PJ		 9005The remaining arguments should be the same arguments that were passed
9006to the primitive. Depending on which primitive, one of those arguments
9007is selected as the TARGET. For example, if OPERATION does file I/O,
9008whichever argument specifies the file name is TARGET.
05e6f5dc 9009
48b0f3ae 9010TARGET has a meaning which depends on OPERATION:
b883cdb2 9011 For file I/O, TARGET is a file name (except for the special case below).
48b0f3ae 9012 For process I/O, TARGET is a process name.
d4a1d553 9013 For network I/O, TARGET is a service name or a port number.
05e6f5dc 9014
d4a1d553 9015This function looks up what is specified for TARGET in
48b0f3ae
PJ		 9016`file-coding-system-alist', `process-coding-system-alist',
9017or `network-coding-system-alist' depending on OPERATION.
9018They may specify a coding system, a cons of coding systems,
9019or a function symbol to call.
9020In the last case, we call the function with one argument,
9021which is a list of all the arguments given to this function.
1011c487
MB		 9022If the function can't decide a coding system, it can return
9023`undecided' so that the normal code-detection is performed.
48b0f3ae 9024
b883cdb2
MB		 9025If OPERATION is `insert-file-contents', the argument corresponding to
9026TARGET may be a cons (FILENAME . BUFFER). In that case, FILENAME is a
9027file name to look up, and BUFFER is a buffer that contains the file's
9028contents (not yet decoded). If `file-coding-system-alist' specifies a
9029function to call for FILENAME, that function should examine the
9030contents of BUFFER instead of reading the file.
9031
d918f936 9032usage: (find-operation-coding-system OPERATION ARGUMENTS...) */)
48b0f3ae 9033 (nargs, args)
4ed46869
KH		 9034 int nargs;
9035 Lisp_Object *args;
6b89e3aa 9036{
4ed46869
KH		 9037 Lisp_Object operation, target_idx, target, val;
9038 register Lisp_Object chain;
177c0ea7 9039
4ed46869
KH		 9040 if (nargs < 2)
9041 error ("Too few arguments");
9042 operation = args[0];
9043 if (!SYMBOLP (operation)
9044 || !INTEGERP (target_idx = Fget (operation, Qtarget_idx)))
3ed051d4 9045 error ("Invalid first argument");
4ed46869
KH		 9046 if (nargs < 1 + XINT (target_idx))
9047 error ("Too few arguments for operation: %s",
8f924df7 9048 SDATA (SYMBOL_NAME (operation)));
4ed46869
KH		 9049 target = args[XINT (target_idx) + 1];
9050 if (!(STRINGP (target)
091a0ff0
KH		 9051 || (EQ (operation, Qinsert_file_contents) && CONSP (target)
9052 && STRINGP (XCAR (target)) && BUFFERP (XCDR (target)))
4ed46869 9053 || (EQ (operation, Qopen_network_stream) && INTEGERP (target))))
df7492f9 9054 error ("Invalid %dth argument", XINT (target_idx) + 1);
091a0ff0
KH		 9055 if (CONSP (target))
9056 target = XCAR (target);
4ed46869 9057
2e34157c
RS		 9058 chain = ((EQ (operation, Qinsert_file_contents)
9059 || EQ (operation, Qwrite_region))
02ba4723 9060 ? Vfile_coding_system_alist
2e34157c 9061 : (EQ (operation, Qopen_network_stream)
02ba4723
KH		 9062 ? Vnetwork_coding_system_alist
9063 : Vprocess_coding_system_alist));
4ed46869
KH		 9064 if (NILP (chain))
9065 return Qnil;
9066
03699b14 9067 for (; CONSP (chain); chain = XCDR (chain))
6b89e3aa 9068 {
f44d27ce 9069 Lisp_Object elt;
6b89e3aa 9070
df7492f9 9071 elt = XCAR (chain);
4ed46869
KH		 9072 if (CONSP (elt)
9073 && ((STRINGP (target)
03699b14
KR		 9074 && STRINGP (XCAR (elt))
9075 && fast_string_match (XCAR (elt), target) >= 0)
9076 || (INTEGERP (target) && EQ (target, XCAR (elt)))))
6b89e3aa 9077 {
03699b14 9078 val = XCDR (elt);
b19fd4c5
KH		 9079 /* Here, if VAL is both a valid coding system and a valid
9080 function symbol, we return VAL as a coding system. */
02ba4723
KH		 9081 if (CONSP (val))
9082 return val;
9083 if (! SYMBOLP (val))
9084 return Qnil;
9085 if (! NILP (Fcoding_system_p (val)))
9086 return Fcons (val, val);
b19fd4c5 9087 if (! NILP (Ffboundp (val)))
6b89e3aa 9088 {
e2b97060
MB		 9089 /* We use call1 rather than safe_call1
9090 so as to get bug reports about functions called here
9091 which don't handle the current interface. */
9092 val = call1 (val, Flist (nargs, args));
b19fd4c5
KH		 9093 if (CONSP (val))
9094 return val;
9095 if (SYMBOLP (val) && ! NILP (Fcoding_system_p (val)))
9096 return Fcons (val, val);
6b89e3aa 9097 }
02ba4723 9098 return Qnil;
6b89e3aa
KH		 9099 }
9100 }
4ed46869 9101 return Qnil;
6b89e3aa
KH		 9102}
9103
df7492f9 9104DEFUN ("set-coding-system-priority", Fset_coding_system_priority,
a3f6ee6d 9105 Sset_coding_system_priority, 0, MANY, 0,
da7db224 9106 doc: /* Assign higher priority to the coding systems given as arguments.
d4a1d553 9107If multiple coding systems belong to the same category,
a3181084
DL		 9108all but the first one are ignored.
9109
d4a1d553 9110usage: (set-coding-system-priority &rest coding-systems) */)
df7492f9
KH		 9111 (nargs, args)
9112 int nargs;
9113 Lisp_Object *args;
9114{
9115 int i, j;
9116 int changed[coding_category_max];
9117 enum coding_category priorities[coding_category_max];
9118
9119 bzero (changed, sizeof changed);
6b89e3aa 9120
df7492f9 9121 for (i = j = 0; i < nargs; i++)
6b89e3aa 9122 {
df7492f9
KH		 9123 enum coding_category category;
9124 Lisp_Object spec, attrs;
6b89e3aa 9125
df7492f9
KH		 9126 CHECK_CODING_SYSTEM_GET_SPEC (args[i], spec);
9127 attrs = AREF (spec, 0);
9128 category = XINT (CODING_ATTR_CATEGORY (attrs));
9129 if (changed[category])
9130 /* Ignore this coding system because a coding system of the
9131 same category already had a higher priority. */
9132 continue;
9133 changed[category] = 1;
9134 priorities[j++] = category;
9135 if (coding_categories[category].id >= 0
9136 && ! EQ (args[i], CODING_ID_NAME (coding_categories[category].id)))
9137 setup_coding_system (args[i], &coding_categories[category]);
ff563fce 9138 Fset (AREF (Vcoding_category_table, category), args[i]);
df7492f9 9139 }
6b89e3aa 9140
df7492f9
KH		 9141 /* Now we have decided top J priorities. Reflect the order of the
9142 original priorities to the remaining priorities. */
6b89e3aa 9143
df7492f9 9144 for (i = j, j = 0; i < coding_category_max; i++, j++)
6b89e3aa 9145 {
df7492f9
KH		 9146 while (j < coding_category_max
9147 && changed[coding_priorities[j]])
9148 j++;
9149 if (j == coding_category_max)
9150 abort ();
9151 priorities[i] = coding_priorities[j];
9152 }
6b89e3aa 9153
df7492f9 9154 bcopy (priorities, coding_priorities, sizeof priorities);
177c0ea7 9155
ff563fce
KH		 9156 /* Update `coding-category-list'. */
9157 Vcoding_category_list = Qnil;
9158 for (i = coding_category_max - 1; i >= 0; i--)
9159 Vcoding_category_list
9160 = Fcons (AREF (Vcoding_category_table, priorities[i]),
9161 Vcoding_category_list);
6b89e3aa 9162
df7492f9 9163 return Qnil;
6b89e3aa
KH		 9164}
9165
df7492f9
KH		 9166DEFUN ("coding-system-priority-list", Fcoding_system_priority_list,
9167 Scoding_system_priority_list, 0, 1, 0,
da7db224
DL		 9168 doc: /* Return a list of coding systems ordered by their priorities.
9169HIGHESTP non-nil means just return the highest priority one. */)
df7492f9
KH		 9170 (highestp)
9171 Lisp_Object highestp;
d46c5b12
KH		 9172{
9173 int i;
df7492f9 9174 Lisp_Object val;
6b89e3aa 9175
df7492f9 9176 for (i = 0, val = Qnil; i < coding_category_max; i++)
d46c5b12 9177 {
df7492f9
KH		 9178 enum coding_category category = coding_priorities[i];
9179 int id = coding_categories[category].id;
9180 Lisp_Object attrs;
068a9dbd 9181
df7492f9
KH		 9182 if (id < 0)
9183 continue;
9184 attrs = CODING_ID_ATTRS (id);
9185 if (! NILP (highestp))
9186 return CODING_ATTR_BASE_NAME (attrs);
9187 val = Fcons (CODING_ATTR_BASE_NAME (attrs), val);
9188 }
9189 return Fnreverse (val);
9190}
068a9dbd 9191
f0064e1f 9192static char *suffixes[] = { "-unix", "-dos", "-mac" };
068a9dbd
KH		 9193
9194static Lisp_Object
df7492f9
KH		 9195make_subsidiaries (base)
9196 Lisp_Object base;
068a9dbd 9197{
df7492f9 9198 Lisp_Object subsidiaries;
8f924df7 9199 int base_name_len = SBYTES (SYMBOL_NAME (base));
df7492f9
KH		 9200 char *buf = (char *) alloca (base_name_len + 6);
9201 int i;
068a9dbd 9202
8f924df7 9203 bcopy (SDATA (SYMBOL_NAME (base)), buf, base_name_len);
df7492f9
KH		 9204 subsidiaries = Fmake_vector (make_number (3), Qnil);
9205 for (i = 0; i < 3; i++)
068a9dbd 9206 {
df7492f9
KH		 9207 bcopy (suffixes[i], buf + base_name_len, strlen (suffixes[i]) + 1);
9208 ASET (subsidiaries, i, intern (buf));
068a9dbd 9209 }
df7492f9 9210 return subsidiaries;
068a9dbd
KH		 9211}
9212
9213
df7492f9
KH		 9214DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal,
9215 Sdefine_coding_system_internal, coding_arg_max, MANY, 0,
1fcd6c8b
DL		 9216 doc: /* For internal use only.
9217usage: (define-coding-system-internal ...) */)
df7492f9
KH		 9218 (nargs, args)
9219 int nargs;
9220 Lisp_Object *args;
068a9dbd 9221{
df7492f9
KH		 9222 Lisp_Object name;
9223 Lisp_Object spec_vec; /* [ ATTRS ALIASE EOL_TYPE ] */
9224 Lisp_Object attrs; /* Vector of attributes. */
9225 Lisp_Object eol_type;
9226 Lisp_Object aliases;
9227 Lisp_Object coding_type, charset_list, safe_charsets;
9228 enum coding_category category;
9229 Lisp_Object tail, val;
9230 int max_charset_id = 0;
9231 int i;
068a9dbd 9232
df7492f9
KH		 9233 if (nargs < coding_arg_max)
9234 goto short_args;
068a9dbd 9235
df7492f9 9236 attrs = Fmake_vector (make_number (coding_attr_last_index), Qnil);
068a9dbd 9237
df7492f9
KH		 9238 name = args[coding_arg_name];
9239 CHECK_SYMBOL (name);
9240 CODING_ATTR_BASE_NAME (attrs) = name;
068a9dbd 9241
df7492f9
KH		 9242 val = args[coding_arg_mnemonic];
9243 if (! STRINGP (val))
9244 CHECK_CHARACTER (val);
9245 CODING_ATTR_MNEMONIC (attrs) = val;
068a9dbd 9246
df7492f9
KH		 9247 coding_type = args[coding_arg_coding_type];
9248 CHECK_SYMBOL (coding_type);
9249 CODING_ATTR_TYPE (attrs) = coding_type;
068a9dbd 9250
df7492f9
KH		 9251 charset_list = args[coding_arg_charset_list];
9252 if (SYMBOLP (charset_list))
9253 {
9254 if (EQ (charset_list, Qiso_2022))
9255 {
9256 if (! EQ (coding_type, Qiso_2022))
9257 error ("Invalid charset-list");
9258 charset_list = Viso_2022_charset_list;
9259 }
9260 else if (EQ (charset_list, Qemacs_mule))
9261 {
9262 if (! EQ (coding_type, Qemacs_mule))
9263 error ("Invalid charset-list");
9264 charset_list = Vemacs_mule_charset_list;
9265 }
9266 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
9267 if (max_charset_id < XFASTINT (XCAR (tail)))
9268 max_charset_id = XFASTINT (XCAR (tail));
9269 }
068a9dbd
KH		 9270 else
9271 {
df7492f9 9272 charset_list = Fcopy_sequence (charset_list);
985773c9 9273 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
068a9dbd 9274 {
df7492f9
KH		 9275 struct charset *charset;
9276
985773c9 9277 val = XCAR (tail);
df7492f9
KH		 9278 CHECK_CHARSET_GET_CHARSET (val, charset);
9279 if (EQ (coding_type, Qiso_2022)
9280 ? CHARSET_ISO_FINAL (charset) < 0
9281 : EQ (coding_type, Qemacs_mule)
9282 ? CHARSET_EMACS_MULE_ID (charset) < 0
9283 : 0)
9284 error ("Can't handle charset `%s'",
8f924df7 9285 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9 9286
8f924df7 9287 XSETCAR (tail, make_number (charset->id));
df7492f9
KH		 9288 if (max_charset_id < charset->id)
9289 max_charset_id = charset->id;
068a9dbd
KH		 9290 }
9291 }
df7492f9 9292 CODING_ATTR_CHARSET_LIST (attrs) = charset_list;
068a9dbd 9293
df7492f9
KH		 9294 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
9295 make_number (255));
9296 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8f924df7 9297 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 9298 CODING_ATTR_SAFE_CHARSETS (attrs) = safe_charsets;
068a9dbd 9299
584948ac 9300 CODING_ATTR_ASCII_COMPAT (attrs) = args[coding_arg_ascii_compatible_p];
3a73fa5d 9301
df7492f9 9302 val = args[coding_arg_decode_translation_table];
a6f87d34 9303 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 9304 CHECK_SYMBOL (val);
df7492f9 9305 CODING_ATTR_DECODE_TBL (attrs) = val;
3a73fa5d 9306
df7492f9 9307 val = args[coding_arg_encode_translation_table];
a6f87d34 9308 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 9309 CHECK_SYMBOL (val);
df7492f9 9310 CODING_ATTR_ENCODE_TBL (attrs) = val;
d46c5b12 9311
df7492f9
KH		 9312 val = args[coding_arg_post_read_conversion];
9313 CHECK_SYMBOL (val);
9314 CODING_ATTR_POST_READ (attrs) = val;
d46c5b12 9315
df7492f9
KH		 9316 val = args[coding_arg_pre_write_conversion];
9317 CHECK_SYMBOL (val);
9318 CODING_ATTR_PRE_WRITE (attrs) = val;
3a73fa5d 9319
df7492f9
KH		 9320 val = args[coding_arg_default_char];
9321 if (NILP (val))
9322 CODING_ATTR_DEFAULT_CHAR (attrs) = make_number (' ');
9323 else
9324 {
8f924df7 9325 CHECK_CHARACTER (val);
df7492f9
KH		 9326 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
9327 }
4031e2bf 9328
8f924df7
KH		 9329 val = args[coding_arg_for_unibyte];
9330 CODING_ATTR_FOR_UNIBYTE (attrs) = NILP (val) ? Qnil : Qt;
3a73fa5d 9331
df7492f9
KH		 9332 val = args[coding_arg_plist];
9333 CHECK_LIST (val);
9334 CODING_ATTR_PLIST (attrs) = val;
3a73fa5d 9335
df7492f9
KH		 9336 if (EQ (coding_type, Qcharset))
9337 {
c7c66a95
KH		 9338 /* Generate a lisp vector of 256 elements. Each element is nil,
9339 integer, or a list of charset IDs.
3a73fa5d 9340
c7c66a95
KH		 9341 If Nth element is nil, the byte code N is invalid in this
9342 coding system.
4ed46869 9343
c7c66a95
KH		 9344 If Nth element is a number NUM, N is the first byte of a
9345 charset whose ID is NUM.
4ed46869 9346
c7c66a95
KH		 9347 If Nth element is a list of charset IDs, N is the first byte
9348 of one of them. The list is sorted by dimensions of the
2bc515e4 9349 charsets. A charset of smaller dimension comes firtst. */
df7492f9 9350 val = Fmake_vector (make_number (256), Qnil);
4ed46869 9351
5c99c2e6 9352 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
df7492f9 9353 {
c7c66a95
KH		 9354 struct charset *charset = CHARSET_FROM_ID (XFASTINT (XCAR (tail)));
9355 int dim = CHARSET_DIMENSION (charset);
9356 int idx = (dim - 1) * 4;
4ed46869 9357
5c99c2e6 9358 if (CHARSET_ASCII_COMPATIBLE_P (charset))
584948ac 9359 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
4031e2bf 9360
15d143f7
KH		 9361 for (i = charset->code_space[idx];
9362 i <= charset->code_space[idx + 1]; i++)
9363 {
c7c66a95
KH		 9364 Lisp_Object tmp, tmp2;
9365 int dim2;
ec6d2bb8 9366
c7c66a95
KH		 9367 tmp = AREF (val, i);
9368 if (NILP (tmp))
9369 tmp = XCAR (tail);
9370 else if (NUMBERP (tmp))
9371 {
9372 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (tmp)));
9373 if (dim < dim2)
c7c66a95 9374 tmp = Fcons (XCAR (tail), Fcons (tmp, Qnil));
f9d71dcd
KH		 9375 else
9376 tmp = Fcons (tmp, Fcons (XCAR (tail), Qnil));
c7c66a95 9377 }
15d143f7 9378 else
c7c66a95
KH		 9379 {
9380 for (tmp2 = tmp; CONSP (tmp2); tmp2 = XCDR (tmp2))
9381 {
9382 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (XCAR (tmp2))));
9383 if (dim < dim2)
9384 break;
9385 }
9386 if (NILP (tmp2))
9387 tmp = nconc2 (tmp, Fcons (XCAR (tail), Qnil));
9388 else
9389 {
9390 XSETCDR (tmp2, Fcons (XCAR (tmp2), XCDR (tmp2)));
9391 XSETCAR (tmp2, XCAR (tail));
9392 }
9393 }
9394 ASET (val, i, tmp);
15d143f7 9395 }
df7492f9
KH		 9396 }
9397 ASET (attrs, coding_attr_charset_valids, val);
9398 category = coding_category_charset;
9399 }
9400 else if (EQ (coding_type, Qccl))
9401 {
9402 Lisp_Object valids;
ecec61c1 9403
df7492f9
KH		 9404 if (nargs < coding_arg_ccl_max)
9405 goto short_args;
ecec61c1 9406
df7492f9
KH		 9407 val = args[coding_arg_ccl_decoder];
9408 CHECK_CCL_PROGRAM (val);
9409 if (VECTORP (val))
9410 val = Fcopy_sequence (val);
9411 ASET (attrs, coding_attr_ccl_decoder, val);
ecec61c1 9412
df7492f9
KH		 9413 val = args[coding_arg_ccl_encoder];
9414 CHECK_CCL_PROGRAM (val);
9415 if (VECTORP (val))
9416 val = Fcopy_sequence (val);
9417 ASET (attrs, coding_attr_ccl_encoder, val);
ecec61c1 9418
df7492f9
KH		 9419 val = args[coding_arg_ccl_valids];
9420 valids = Fmake_string (make_number (256), make_number (0));
9421 for (tail = val; !NILP (tail); tail = Fcdr (tail))
9422 {
8dcbea82 9423 int from, to;
ecec61c1 9424
df7492f9
KH		 9425 val = Fcar (tail);
9426 if (INTEGERP (val))
8dcbea82
KH		 9427 {
9428 from = to = XINT (val);
9429 if (from < 0 || from > 255)
9430 args_out_of_range_3 (val, make_number (0), make_number (255));
9431 }
df7492f9
KH		 9432 else
9433 {
df7492f9 9434 CHECK_CONS (val);
8f924df7
KH		 9435 CHECK_NATNUM_CAR (val);
9436 CHECK_NATNUM_CDR (val);
df7492f9 9437 from = XINT (XCAR (val));
8f924df7 9438 if (from > 255)
8dcbea82
KH		 9439 args_out_of_range_3 (XCAR (val),
9440 make_number (0), make_number (255));
df7492f9 9441 to = XINT (XCDR (val));
8dcbea82
KH		 9442 if (to < from || to > 255)
9443 args_out_of_range_3 (XCDR (val),
9444 XCAR (val), make_number (255));
df7492f9 9445 }
8dcbea82 9446 for (i = from; i <= to; i++)
8f924df7 9447 SSET (valids, i, 1);
df7492f9
KH		 9448 }
9449 ASET (attrs, coding_attr_ccl_valids, valids);
4ed46869 9450
df7492f9 9451 category = coding_category_ccl;
55ab7be3 9452 }
df7492f9 9453 else if (EQ (coding_type, Qutf_16))
55ab7be3 9454 {
df7492f9 9455 Lisp_Object bom, endian;
4ed46869 9456
584948ac 9457 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
4ed46869 9458
df7492f9
KH		 9459 if (nargs < coding_arg_utf16_max)
9460 goto short_args;
4ed46869 9461
df7492f9
KH		 9462 bom = args[coding_arg_utf16_bom];
9463 if (! NILP (bom) && ! EQ (bom, Qt))
9464 {
9465 CHECK_CONS (bom);
8f924df7
KH		 9466 val = XCAR (bom);
9467 CHECK_CODING_SYSTEM (val);
9468 val = XCDR (bom);
9469 CHECK_CODING_SYSTEM (val);
df7492f9 9470 }
a470d443 9471 ASET (attrs, coding_attr_utf_bom, bom);
df7492f9
KH		 9472
9473 endian = args[coding_arg_utf16_endian];
b49a1807
KH		 9474 CHECK_SYMBOL (endian);
9475 if (NILP (endian))
9476 endian = Qbig;
9477 else if (! EQ (endian, Qbig) && ! EQ (endian, Qlittle))
8f924df7 9478 error ("Invalid endian: %s", SDATA (SYMBOL_NAME (endian)));
df7492f9
KH		 9479 ASET (attrs, coding_attr_utf_16_endian, endian);
9480
9481 category = (CONSP (bom)
9482 ? coding_category_utf_16_auto
9483 : NILP (bom)
b49a1807 9484 ? (EQ (endian, Qbig)
df7492f9
KH		 9485 ? coding_category_utf_16_be_nosig
9486 : coding_category_utf_16_le_nosig)
b49a1807 9487 : (EQ (endian, Qbig)
df7492f9
KH		 9488 ? coding_category_utf_16_be
9489 : coding_category_utf_16_le));
9490 }
9491 else if (EQ (coding_type, Qiso_2022))
9492 {
9493 Lisp_Object initial, reg_usage, request, flags;
4776e638 9494 int i;
1397dc18 9495
df7492f9
KH		 9496 if (nargs < coding_arg_iso2022_max)
9497 goto short_args;
9498
9499 initial = Fcopy_sequence (args[coding_arg_iso2022_initial]);
9500 CHECK_VECTOR (initial);
9501 for (i = 0; i < 4; i++)
9502 {
9503 val = Faref (initial, make_number (i));
9504 if (! NILP (val))
9505 {
584948ac
KH		 9506 struct charset *charset;
9507
9508 CHECK_CHARSET_GET_CHARSET (val, charset);
9509 ASET (initial, i, make_number (CHARSET_ID (charset)));
9510 if (i == 0 && CHARSET_ASCII_COMPATIBLE_P (charset))
9511 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9512 }
9513 else
9514 ASET (initial, i, make_number (-1));
9515 }
9516
9517 reg_usage = args[coding_arg_iso2022_reg_usage];
9518 CHECK_CONS (reg_usage);
8f924df7
KH		 9519 CHECK_NUMBER_CAR (reg_usage);
9520 CHECK_NUMBER_CDR (reg_usage);
df7492f9
KH		 9521
9522 request = Fcopy_sequence (args[coding_arg_iso2022_request]);
9523 for (tail = request; ! NILP (tail); tail = Fcdr (tail))
1397dc18 9524 {
df7492f9 9525 int id;
8f924df7 9526 Lisp_Object tmp;
df7492f9
KH		 9527
9528 val = Fcar (tail);
9529 CHECK_CONS (val);
8f924df7
KH		 9530 tmp = XCAR (val);
9531 CHECK_CHARSET_GET_ID (tmp, id);
9532 CHECK_NATNUM_CDR (val);
df7492f9
KH		 9533 if (XINT (XCDR (val)) >= 4)
9534 error ("Invalid graphic register number: %d", XINT (XCDR (val)));
8f924df7 9535 XSETCAR (val, make_number (id));
1397dc18 9536 }
4ed46869 9537
df7492f9
KH		 9538 flags = args[coding_arg_iso2022_flags];
9539 CHECK_NATNUM (flags);
9540 i = XINT (flags);
9541 if (EQ (args[coding_arg_charset_list], Qiso_2022))
9542 flags = make_number (i | CODING_ISO_FLAG_FULL_SUPPORT);
9543
9544 ASET (attrs, coding_attr_iso_initial, initial);
9545 ASET (attrs, coding_attr_iso_usage, reg_usage);
9546 ASET (attrs, coding_attr_iso_request, request);
9547 ASET (attrs, coding_attr_iso_flags, flags);
9548 setup_iso_safe_charsets (attrs);
9549
9550 if (i & CODING_ISO_FLAG_SEVEN_BITS)
9551 category = ((i & (CODING_ISO_FLAG_LOCKING_SHIFT
9552 | CODING_ISO_FLAG_SINGLE_SHIFT))
9553 ? coding_category_iso_7_else
9554 : EQ (args[coding_arg_charset_list], Qiso_2022)
9555 ? coding_category_iso_7
9556 : coding_category_iso_7_tight);
9557 else
9558 {
9559 int id = XINT (AREF (initial, 1));
9560
c6fb6e98 9561 category = (((i & CODING_ISO_FLAG_LOCKING_SHIFT)
df7492f9
KH		 9562 || EQ (args[coding_arg_charset_list], Qiso_2022)
9563 || id < 0)
9564 ? coding_category_iso_8_else
9565 : (CHARSET_DIMENSION (CHARSET_FROM_ID (id)) == 1)
9566 ? coding_category_iso_8_1
9567 : coding_category_iso_8_2);
9568 }
0ce7886f
KH		 9569 if (category != coding_category_iso_8_1
9570 && category != coding_category_iso_8_2)
9571 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
df7492f9
KH		 9572 }
9573 else if (EQ (coding_type, Qemacs_mule))
c28a9453 9574 {
df7492f9
KH		 9575 if (EQ (args[coding_arg_charset_list], Qemacs_mule))
9576 ASET (attrs, coding_attr_emacs_mule_full, Qt);
584948ac 9577 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9 9578 category = coding_category_emacs_mule;
c28a9453 9579 }
df7492f9 9580 else if (EQ (coding_type, Qshift_jis))
c28a9453 9581 {
df7492f9
KH		 9582
9583 struct charset *charset;
9584
7d64c6ad 9585 if (XINT (Flength (charset_list)) != 3
6e07c25f 9586 && XINT (Flength (charset_list)) != 4)
7d64c6ad 9587 error ("There should be three or four charsets");
df7492f9
KH		 9588
9589 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9590 if (CHARSET_DIMENSION (charset) != 1)
9591 error ("Dimension of charset %s is not one",
8f924df7 9592 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 9593 if (CHARSET_ASCII_COMPATIBLE_P (charset))
9594 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9595
9596 charset_list = XCDR (charset_list);
9597 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9598 if (CHARSET_DIMENSION (charset) != 1)
9599 error ("Dimension of charset %s is not one",
8f924df7 9600 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9
KH		 9601
9602 charset_list = XCDR (charset_list);
9603 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9604 if (CHARSET_DIMENSION (charset) != 2)
7d64c6ad
KH		 9605 error ("Dimension of charset %s is not two",
9606 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9607
9608 charset_list = XCDR (charset_list);
2b917a06
KH		 9609 if (! NILP (charset_list))
9610 {
9611 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9612 if (CHARSET_DIMENSION (charset) != 2)
9613 error ("Dimension of charset %s is not two",
9614 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9615 }
df7492f9
KH		 9616
9617 category = coding_category_sjis;
9618 Vsjis_coding_system = name;
c28a9453 9619 }
df7492f9
KH		 9620 else if (EQ (coding_type, Qbig5))
9621 {
9622 struct charset *charset;
4ed46869 9623
df7492f9
KH		 9624 if (XINT (Flength (charset_list)) != 2)
9625 error ("There should be just two charsets");
9626
9627 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9628 if (CHARSET_DIMENSION (charset) != 1)
9629 error ("Dimension of charset %s is not one",
8f924df7 9630 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 9631 if (CHARSET_ASCII_COMPATIBLE_P (charset))
9632 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9633
9634 charset_list = XCDR (charset_list);
9635 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9636 if (CHARSET_DIMENSION (charset) != 2)
9637 error ("Dimension of charset %s is not two",
8f924df7 9638 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
4ed46869 9639
df7492f9
KH		 9640 category = coding_category_big5;
9641 Vbig5_coding_system = name;
9642 }
9643 else if (EQ (coding_type, Qraw_text))
c28a9453 9644 {
584948ac
KH		 9645 category = coding_category_raw_text;
9646 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
c28a9453 9647 }
df7492f9 9648 else if (EQ (coding_type, Qutf_8))
4ed46869 9649 {
a470d443
KH		 9650 Lisp_Object bom;
9651
584948ac 9652 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
a470d443
KH		 9653
9654 if (nargs < coding_arg_utf8_max)
9655 goto short_args;
9656
9657 bom = args[coding_arg_utf8_bom];
9658 if (! NILP (bom) && ! EQ (bom, Qt))
9659 {
9660 CHECK_CONS (bom);
9661 val = XCAR (bom);
9662 CHECK_CODING_SYSTEM (val);
9663 val = XCDR (bom);
9664 CHECK_CODING_SYSTEM (val);
9665 }
9666 ASET (attrs, coding_attr_utf_bom, bom);
9667
9668 category = (CONSP (bom) ? coding_category_utf_8_auto
9669 : NILP (bom) ? coding_category_utf_8_nosig
9670 : coding_category_utf_8_sig);
4ed46869 9671 }
df7492f9
KH		 9672 else if (EQ (coding_type, Qundecided))
9673 category = coding_category_undecided;
4ed46869 9674 else
df7492f9 9675 error ("Invalid coding system type: %s",
8f924df7 9676 SDATA (SYMBOL_NAME (coding_type)));
4ed46869 9677
df7492f9 9678 CODING_ATTR_CATEGORY (attrs) = make_number (category);
01378f49
KH		 9679 CODING_ATTR_PLIST (attrs)
9680 = Fcons (QCcategory, Fcons (AREF (Vcoding_category_table, category),
9681 CODING_ATTR_PLIST (attrs)));
35befdaa 9682 CODING_ATTR_PLIST (attrs)
3ed051d4 9683 = Fcons (QCascii_compatible_p,
35befdaa
KH		 9684 Fcons (CODING_ATTR_ASCII_COMPAT (attrs),
9685 CODING_ATTR_PLIST (attrs)));
c4825358 9686
df7492f9
KH		 9687 eol_type = args[coding_arg_eol_type];
9688 if (! NILP (eol_type)
9689 && ! EQ (eol_type, Qunix)
9690 && ! EQ (eol_type, Qdos)
9691 && ! EQ (eol_type, Qmac))
9692 error ("Invalid eol-type");
4ed46869 9693
df7492f9 9694 aliases = Fcons (name, Qnil);
4ed46869 9695
df7492f9
KH		 9696 if (NILP (eol_type))
9697 {
9698 eol_type = make_subsidiaries (name);
9699 for (i = 0; i < 3; i++)
1397dc18 9700 {
df7492f9
KH		 9701 Lisp_Object this_spec, this_name, this_aliases, this_eol_type;
9702
9703 this_name = AREF (eol_type, i);
9704 this_aliases = Fcons (this_name, Qnil);
9705 this_eol_type = (i == 0 ? Qunix : i == 1 ? Qdos : Qmac);
9706 this_spec = Fmake_vector (make_number (3), attrs);
9707 ASET (this_spec, 1, this_aliases);
9708 ASET (this_spec, 2, this_eol_type);
9709 Fputhash (this_name, this_spec, Vcoding_system_hash_table);
9710 Vcoding_system_list = Fcons (this_name, Vcoding_system_list);
583f71ca
KH		 9711 val = Fassoc (Fsymbol_name (this_name), Vcoding_system_alist);
9712 if (NILP (val))
9713 Vcoding_system_alist
9714 = Fcons (Fcons (Fsymbol_name (this_name), Qnil),
9715 Vcoding_system_alist);
1397dc18 9716 }
d46c5b12 9717 }
4ed46869 9718
df7492f9
KH		 9719 spec_vec = Fmake_vector (make_number (3), attrs);
9720 ASET (spec_vec, 1, aliases);
9721 ASET (spec_vec, 2, eol_type);
48b0f3ae 9722
df7492f9
KH		 9723 Fputhash (name, spec_vec, Vcoding_system_hash_table);
9724 Vcoding_system_list = Fcons (name, Vcoding_system_list);
583f71ca
KH		 9725 val = Fassoc (Fsymbol_name (name), Vcoding_system_alist);
9726 if (NILP (val))
9727 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (name), Qnil),
9728 Vcoding_system_alist);
48b0f3ae 9729
df7492f9
KH		 9730 {
9731 int id = coding_categories[category].id;
48b0f3ae 9732
df7492f9
KH		 9733 if (id < 0 || EQ (name, CODING_ID_NAME (id)))
9734 setup_coding_system (name, &coding_categories[category]);
9735 }
48b0f3ae 9736
d46c5b12 9737 return Qnil;
48b0f3ae 9738
df7492f9
KH		 9739 short_args:
9740 return Fsignal (Qwrong_number_of_arguments,
9741 Fcons (intern ("define-coding-system-internal"),
9742 make_number (nargs)));
d46c5b12 9743}
4ed46869 9744
d6925f38 9745
a6f87d34
KH		 9746DEFUN ("coding-system-put", Fcoding_system_put, Scoding_system_put,
9747 3, 3, 0,
9748 doc: /* Change value in CODING-SYSTEM's property list PROP to VAL. */)
9749 (coding_system, prop, val)
9750 Lisp_Object coding_system, prop, val;
9751{
3dbe7859 9752 Lisp_Object spec, attrs;
a6f87d34
KH		 9753
9754 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9755 attrs = AREF (spec, 0);
9756 if (EQ (prop, QCmnemonic))
9757 {
9758 if (! STRINGP (val))
9759 CHECK_CHARACTER (val);
9760 CODING_ATTR_MNEMONIC (attrs) = val;
9761 }
2133e2d1 9762 else if (EQ (prop, QCdefault_char))
a6f87d34
KH		 9763 {
9764 if (NILP (val))
9765 val = make_number (' ');
9766 else
9767 CHECK_CHARACTER (val);
9768 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
9769 }
9770 else if (EQ (prop, QCdecode_translation_table))
9771 {
9772 if (! CHAR_TABLE_P (val) && ! CONSP (val))
9773 CHECK_SYMBOL (val);
9774 CODING_ATTR_DECODE_TBL (attrs) = val;
9775 }
9776 else if (EQ (prop, QCencode_translation_table))
9777 {
9778 if (! CHAR_TABLE_P (val) && ! CONSP (val))
9779 CHECK_SYMBOL (val);
9780 CODING_ATTR_ENCODE_TBL (attrs) = val;
9781 }
9782 else if (EQ (prop, QCpost_read_conversion))
9783 {
9784 CHECK_SYMBOL (val);
9785 CODING_ATTR_POST_READ (attrs) = val;
9786 }
9787 else if (EQ (prop, QCpre_write_conversion))
9788 {
9789 CHECK_SYMBOL (val);
9790 CODING_ATTR_PRE_WRITE (attrs) = val;
9791 }
35befdaa
KH		 9792 else if (EQ (prop, QCascii_compatible_p))
9793 {
9794 CODING_ATTR_ASCII_COMPAT (attrs) = val;
9795 }
a6f87d34
KH		 9796
9797 CODING_ATTR_PLIST (attrs)
9798 = Fplist_put (CODING_ATTR_PLIST (attrs), prop, val);
9799 return val;
9800}
9801
9802
df7492f9
KH		 9803DEFUN ("define-coding-system-alias", Fdefine_coding_system_alias,
9804 Sdefine_coding_system_alias, 2, 2, 0,
9805 doc: /* Define ALIAS as an alias for CODING-SYSTEM. */)
9806 (alias, coding_system)
9807 Lisp_Object alias, coding_system;
66cfb530 9808{
583f71ca 9809 Lisp_Object spec, aliases, eol_type, val;
4ed46869 9810
df7492f9
KH		 9811 CHECK_SYMBOL (alias);
9812 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9813 aliases = AREF (spec, 1);
d4a1d553 9814 /* ALIASES should be a list of length more than zero, and the first
d6925f38
KH		 9815 element is a base coding system. Append ALIAS at the tail of the
9816 list. */
df7492f9
KH		 9817 while (!NILP (XCDR (aliases)))
9818 aliases = XCDR (aliases);
8f924df7 9819 XSETCDR (aliases, Fcons (alias, Qnil));
4ed46869 9820
df7492f9
KH		 9821 eol_type = AREF (spec, 2);
9822 if (VECTORP (eol_type))
4ed46869 9823 {
df7492f9
KH		 9824 Lisp_Object subsidiaries;
9825 int i;
4ed46869 9826
df7492f9
KH		 9827 subsidiaries = make_subsidiaries (alias);
9828 for (i = 0; i < 3; i++)
9829 Fdefine_coding_system_alias (AREF (subsidiaries, i),
9830 AREF (eol_type, i));
4ed46869 9831 }
df7492f9
KH		 9832
9833 Fputhash (alias, spec, Vcoding_system_hash_table);
d6925f38 9834 Vcoding_system_list = Fcons (alias, Vcoding_system_list);
583f71ca
KH		 9835 val = Fassoc (Fsymbol_name (alias), Vcoding_system_alist);
9836 if (NILP (val))
9837 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (alias), Qnil),
9838 Vcoding_system_alist);
66cfb530 9839
4ed46869
KH		 9840 return Qnil;
9841}
9842
df7492f9
KH		 9843DEFUN ("coding-system-base", Fcoding_system_base, Scoding_system_base,
9844 1, 1, 0,
9845 doc: /* Return the base of CODING-SYSTEM.
da7db224 9846Any alias or subsidiary coding system is not a base coding system. */)
df7492f9
KH		 9847 (coding_system)
9848 Lisp_Object coding_system;
d46c5b12 9849{
df7492f9 9850 Lisp_Object spec, attrs;
d46c5b12 9851
df7492f9
KH		 9852 if (NILP (coding_system))
9853 return (Qno_conversion);
9854 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9855 attrs = AREF (spec, 0);
9856 return CODING_ATTR_BASE_NAME (attrs);
9857}
1397dc18 9858
df7492f9
KH		 9859DEFUN ("coding-system-plist", Fcoding_system_plist, Scoding_system_plist,
9860 1, 1, 0,
9861 doc: "Return the property list of CODING-SYSTEM.")
9862 (coding_system)
9863 Lisp_Object coding_system;
9864{
9865 Lisp_Object spec, attrs;
1397dc18 9866
df7492f9
KH		 9867 if (NILP (coding_system))
9868 coding_system = Qno_conversion;
9869 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9870 attrs = AREF (spec, 0);
9871 return CODING_ATTR_PLIST (attrs);
d46c5b12
KH		 9872}
9873
df7492f9
KH		 9874
9875DEFUN ("coding-system-aliases", Fcoding_system_aliases, Scoding_system_aliases,
9876 1, 1, 0,
da7db224 9877 doc: /* Return the list of aliases of CODING-SYSTEM. */)
df7492f9
KH		 9878 (coding_system)
9879 Lisp_Object coding_system;
66cfb530 9880{
df7492f9 9881 Lisp_Object spec;
84d60297 9882
df7492f9
KH		 9883 if (NILP (coding_system))
9884 coding_system = Qno_conversion;
9885 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
da7db224 9886 return AREF (spec, 1);
df7492f9 9887}
66cfb530 9888
df7492f9
KH		 9889DEFUN ("coding-system-eol-type", Fcoding_system_eol_type,
9890 Scoding_system_eol_type, 1, 1, 0,
9891 doc: /* Return eol-type of CODING-SYSTEM.
d4a1d553 9892An eol-type is an integer 0, 1, 2, or a vector of coding systems.
66cfb530 9893
df7492f9
KH		 9894Integer values 0, 1, and 2 indicate a format of end-of-line; LF, CRLF,
9895and CR respectively.
66cfb530 9896
df7492f9
KH		 9897A vector value indicates that a format of end-of-line should be
9898detected automatically. Nth element of the vector is the subsidiary
9899coding system whose eol-type is N. */)
6b89e3aa
KH		 9900 (coding_system)
9901 Lisp_Object coding_system;
9902{
df7492f9
KH		 9903 Lisp_Object spec, eol_type;
9904 int n;
6b89e3aa 9905
df7492f9
KH		 9906 if (NILP (coding_system))
9907 coding_system = Qno_conversion;
9908 if (! CODING_SYSTEM_P (coding_system))
9909 return Qnil;
9910 spec = CODING_SYSTEM_SPEC (coding_system);
9911 eol_type = AREF (spec, 2);
9912 if (VECTORP (eol_type))
9913 return Fcopy_sequence (eol_type);
9914 n = EQ (eol_type, Qunix) ? 0 : EQ (eol_type, Qdos) ? 1 : 2;
9915 return make_number (n);
6b89e3aa
KH		 9916}
9917
4ed46869
KH		 9918#endif /* emacs */
9919
9920\f
1397dc18 9921/*** 9. Post-amble ***/
4ed46869 9922
dfcf069d 9923void
4ed46869
KH		 9924init_coding_once ()
9925{
9926 int i;
9927
df7492f9
KH		 9928 for (i = 0; i < coding_category_max; i++)
9929 {
9930 coding_categories[i].id = -1;
9931 coding_priorities[i] = i;
9932 }
4ed46869
KH		 9933
9934 /* ISO2022 specific initialize routine. */
9935 for (i = 0; i < 0x20; i++)
b73bfc1c 9936 iso_code_class[i] = ISO_control_0;
4ed46869
KH		 9937 for (i = 0x21; i < 0x7F; i++)
9938 iso_code_class[i] = ISO_graphic_plane_0;
9939 for (i = 0x80; i < 0xA0; i++)
b73bfc1c 9940 iso_code_class[i] = ISO_control_1;
4ed46869
KH		 9941 for (i = 0xA1; i < 0xFF; i++)
9942 iso_code_class[i] = ISO_graphic_plane_1;
9943 iso_code_class[0x20] = iso_code_class[0x7F] = ISO_0x20_or_0x7F;
9944 iso_code_class[0xA0] = iso_code_class[0xFF] = ISO_0xA0_or_0xFF;
4ed46869
KH		 9945 iso_code_class[ISO_CODE_SO] = ISO_shift_out;
9946 iso_code_class[ISO_CODE_SI] = ISO_shift_in;
9947 iso_code_class[ISO_CODE_SS2_7] = ISO_single_shift_2_7;
9948 iso_code_class[ISO_CODE_ESC] = ISO_escape;
9949 iso_code_class[ISO_CODE_SS2] = ISO_single_shift_2;
9950 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3;
9951 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer;
9952
df7492f9
KH		 9953 for (i = 0; i < 256; i++)
9954 {
9955 emacs_mule_bytes[i] = 1;
9956 }
7c78e542
KH		 9957 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_11] = 3;
9958 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_12] = 3;
9959 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_21] = 4;
9960 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_22] = 4;
e0e989f6
KH		 9961}
9962
9963#ifdef emacs
9964
dfcf069d 9965void
e0e989f6
KH		 9966syms_of_coding ()
9967{
df7492f9 9968 staticpro (&Vcoding_system_hash_table);
8f924df7
KH		 9969 {
9970 Lisp_Object args[2];
9971 args[0] = QCtest;
9972 args[1] = Qeq;
9973 Vcoding_system_hash_table = Fmake_hash_table (2, args);
9974 }
df7492f9
KH		 9975
9976 staticpro (&Vsjis_coding_system);
9977 Vsjis_coding_system = Qnil;
e0e989f6 9978
df7492f9
KH		 9979 staticpro (&Vbig5_coding_system);
9980 Vbig5_coding_system = Qnil;
9981
24a73b0a
KH		 9982 staticpro (&Vcode_conversion_reused_workbuf);
9983 Vcode_conversion_reused_workbuf = Qnil;
9984
9985 staticpro (&Vcode_conversion_workbuf_name);
9986 Vcode_conversion_workbuf_name = build_string (" *code-conversion-work*");
e0e989f6 9987
24a73b0a 9988 reused_workbuf_in_use = 0;
df7492f9
KH		 9989
9990 DEFSYM (Qcharset, "charset");
9991 DEFSYM (Qtarget_idx, "target-idx");
9992 DEFSYM (Qcoding_system_history, "coding-system-history");
bb0115a2
RS		 9993 Fset (Qcoding_system_history, Qnil);
9994
9ce27fde 9995 /* Target FILENAME is the first argument. */
e0e989f6 9996 Fput (Qinsert_file_contents, Qtarget_idx, make_number (0));
9ce27fde 9997 /* Target FILENAME is the third argument. */
e0e989f6
KH		 9998 Fput (Qwrite_region, Qtarget_idx, make_number (2));
9999
df7492f9 10000 DEFSYM (Qcall_process, "call-process");
9ce27fde 10001 /* Target PROGRAM is the first argument. */
e0e989f6
KH		 10002 Fput (Qcall_process, Qtarget_idx, make_number (0));
10003
df7492f9 10004 DEFSYM (Qcall_process_region, "call-process-region");
9ce27fde 10005 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 10006 Fput (Qcall_process_region, Qtarget_idx, make_number (2));
10007
df7492f9 10008 DEFSYM (Qstart_process, "start-process");
9ce27fde 10009 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 10010 Fput (Qstart_process, Qtarget_idx, make_number (2));
10011
df7492f9 10012 DEFSYM (Qopen_network_stream, "open-network-stream");
9ce27fde 10013 /* Target SERVICE is the fourth argument. */
e0e989f6
KH		 10014 Fput (Qopen_network_stream, Qtarget_idx, make_number (3));
10015
df7492f9
KH		 10016 DEFSYM (Qcoding_system, "coding-system");
10017 DEFSYM (Qcoding_aliases, "coding-aliases");
4ed46869 10018
df7492f9
KH		 10019 DEFSYM (Qeol_type, "eol-type");
10020 DEFSYM (Qunix, "unix");
10021 DEFSYM (Qdos, "dos");
4ed46869 10022
df7492f9
KH		 10023 DEFSYM (Qbuffer_file_coding_system, "buffer-file-coding-system");
10024 DEFSYM (Qpost_read_conversion, "post-read-conversion");
10025 DEFSYM (Qpre_write_conversion, "pre-write-conversion");
10026 DEFSYM (Qdefault_char, "default-char");
10027 DEFSYM (Qundecided, "undecided");
10028 DEFSYM (Qno_conversion, "no-conversion");
10029 DEFSYM (Qraw_text, "raw-text");
4ed46869 10030
df7492f9 10031 DEFSYM (Qiso_2022, "iso-2022");
4ed46869 10032
df7492f9 10033 DEFSYM (Qutf_8, "utf-8");
8f924df7 10034 DEFSYM (Qutf_8_emacs, "utf-8-emacs");
27901516 10035
df7492f9 10036 DEFSYM (Qutf_16, "utf-16");
df7492f9
KH		 10037 DEFSYM (Qbig, "big");
10038 DEFSYM (Qlittle, "little");
27901516 10039
df7492f9
KH		 10040 DEFSYM (Qshift_jis, "shift-jis");
10041 DEFSYM (Qbig5, "big5");
4ed46869 10042
df7492f9 10043 DEFSYM (Qcoding_system_p, "coding-system-p");
4ed46869 10044
df7492f9 10045 DEFSYM (Qcoding_system_error, "coding-system-error");
4ed46869
KH		 10046 Fput (Qcoding_system_error, Qerror_conditions,
10047 Fcons (Qcoding_system_error, Fcons (Qerror, Qnil)));
10048 Fput (Qcoding_system_error, Qerror_message,
9ce27fde 10049 build_string ("Invalid coding system"));
4ed46869 10050
05e6f5dc
KH		 10051 /* Intern this now in case it isn't already done.
10052 Setting this variable twice is harmless.
10053 But don't staticpro it here--that is done in alloc.c. */
10054 Qchar_table_extra_slots = intern ("char-table-extra-slots");
70c22245 10055
df7492f9 10056 DEFSYM (Qtranslation_table, "translation-table");
433f7f87 10057 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (2));
df7492f9
KH		 10058 DEFSYM (Qtranslation_table_id, "translation-table-id");
10059 DEFSYM (Qtranslation_table_for_decode, "translation-table-for-decode");
10060 DEFSYM (Qtranslation_table_for_encode, "translation-table-for-encode");
1397dc18 10061
df7492f9 10062 DEFSYM (Qvalid_codes, "valid-codes");
9ce27fde 10063
df7492f9 10064 DEFSYM (Qemacs_mule, "emacs-mule");
d46c5b12 10065
01378f49 10066 DEFSYM (QCcategory, ":category");
a6f87d34 10067 DEFSYM (QCmnemonic, ":mnemonic");
2133e2d1 10068 DEFSYM (QCdefault_char, ":default-char");
a6f87d34
KH		 10069 DEFSYM (QCdecode_translation_table, ":decode-translation-table");
10070 DEFSYM (QCencode_translation_table, ":encode-translation-table");
10071 DEFSYM (QCpost_read_conversion, ":post-read-conversion");
10072 DEFSYM (QCpre_write_conversion, ":pre-write-conversion");
35befdaa 10073 DEFSYM (QCascii_compatible_p, ":ascii-compatible-p");
01378f49 10074
df7492f9
KH		 10075 Vcoding_category_table
10076 = Fmake_vector (make_number (coding_category_max), Qnil);
10077 staticpro (&Vcoding_category_table);
10078 /* Followings are target of code detection. */
10079 ASET (Vcoding_category_table, coding_category_iso_7,
10080 intern ("coding-category-iso-7"));
10081 ASET (Vcoding_category_table, coding_category_iso_7_tight,
10082 intern ("coding-category-iso-7-tight"));
10083 ASET (Vcoding_category_table, coding_category_iso_8_1,
10084 intern ("coding-category-iso-8-1"));
10085 ASET (Vcoding_category_table, coding_category_iso_8_2,
10086 intern ("coding-category-iso-8-2"));
10087 ASET (Vcoding_category_table, coding_category_iso_7_else,
10088 intern ("coding-category-iso-7-else"));
10089 ASET (Vcoding_category_table, coding_category_iso_8_else,
10090 intern ("coding-category-iso-8-else"));
a470d443
KH		 10091 ASET (Vcoding_category_table, coding_category_utf_8_auto,
10092 intern ("coding-category-utf-8-auto"));
10093 ASET (Vcoding_category_table, coding_category_utf_8_nosig,
df7492f9 10094 intern ("coding-category-utf-8"));
a470d443
KH		 10095 ASET (Vcoding_category_table, coding_category_utf_8_sig,
10096 intern ("coding-category-utf-8-sig"));
df7492f9
KH		 10097 ASET (Vcoding_category_table, coding_category_utf_16_be,
10098 intern ("coding-category-utf-16-be"));
ff563fce
KH		 10099 ASET (Vcoding_category_table, coding_category_utf_16_auto,
10100 intern ("coding-category-utf-16-auto"));
df7492f9
KH		 10101 ASET (Vcoding_category_table, coding_category_utf_16_le,
10102 intern ("coding-category-utf-16-le"));
10103 ASET (Vcoding_category_table, coding_category_utf_16_be_nosig,
10104 intern ("coding-category-utf-16-be-nosig"));
10105 ASET (Vcoding_category_table, coding_category_utf_16_le_nosig,
10106 intern ("coding-category-utf-16-le-nosig"));
10107 ASET (Vcoding_category_table, coding_category_charset,
10108 intern ("coding-category-charset"));
10109 ASET (Vcoding_category_table, coding_category_sjis,
10110 intern ("coding-category-sjis"));
10111 ASET (Vcoding_category_table, coding_category_big5,
10112 intern ("coding-category-big5"));
10113 ASET (Vcoding_category_table, coding_category_ccl,
10114 intern ("coding-category-ccl"));
10115 ASET (Vcoding_category_table, coding_category_emacs_mule,
10116 intern ("coding-category-emacs-mule"));
10117 /* Followings are NOT target of code detection. */
10118 ASET (Vcoding_category_table, coding_category_raw_text,
10119 intern ("coding-category-raw-text"));
10120 ASET (Vcoding_category_table, coding_category_undecided,
10121 intern ("coding-category-undecided"));
ecf488bc 10122
065e3595
KH		 10123 DEFSYM (Qinsufficient_source, "insufficient-source");
10124 DEFSYM (Qinconsistent_eol, "inconsistent-eol");
10125 DEFSYM (Qinvalid_source, "invalid-source");
10126 DEFSYM (Qinterrupted, "interrupted");
10127 DEFSYM (Qinsufficient_memory, "insufficient-memory");
44e8490d 10128 DEFSYM (Qcoding_system_define_form, "coding-system-define-form");
065e3595 10129
4ed46869
KH		 10130 defsubr (&Scoding_system_p);
10131 defsubr (&Sread_coding_system);
10132 defsubr (&Sread_non_nil_coding_system);
10133 defsubr (&Scheck_coding_system);
10134 defsubr (&Sdetect_coding_region);
d46c5b12 10135 defsubr (&Sdetect_coding_string);
05e6f5dc 10136 defsubr (&Sfind_coding_systems_region_internal);
068a9dbd 10137 defsubr (&Sunencodable_char_position);
df7492f9 10138 defsubr (&Scheck_coding_systems_region);
4ed46869
KH		 10139 defsubr (&Sdecode_coding_region);
10140 defsubr (&Sencode_coding_region);
10141 defsubr (&Sdecode_coding_string);
10142 defsubr (&Sencode_coding_string);
10143 defsubr (&Sdecode_sjis_char);
10144 defsubr (&Sencode_sjis_char);
10145 defsubr (&Sdecode_big5_char);
10146 defsubr (&Sencode_big5_char);
1ba9e4ab 10147 defsubr (&Sset_terminal_coding_system_internal);
c4825358 10148 defsubr (&Sset_safe_terminal_coding_system_internal);
4ed46869 10149 defsubr (&Sterminal_coding_system);
1ba9e4ab 10150 defsubr (&Sset_keyboard_coding_system_internal);
4ed46869 10151 defsubr (&Skeyboard_coding_system);
a5d301df 10152 defsubr (&Sfind_operation_coding_system);
df7492f9 10153 defsubr (&Sset_coding_system_priority);
6b89e3aa 10154 defsubr (&Sdefine_coding_system_internal);
df7492f9 10155 defsubr (&Sdefine_coding_system_alias);
a6f87d34 10156 defsubr (&Scoding_system_put);
df7492f9
KH		 10157 defsubr (&Scoding_system_base);
10158 defsubr (&Scoding_system_plist);
10159 defsubr (&Scoding_system_aliases);
10160 defsubr (&Scoding_system_eol_type);
10161 defsubr (&Scoding_system_priority_list);
4ed46869 10162
4608c386 10163 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list,
48b0f3ae
PJ		 10164 doc: /* List of coding systems.
10165
10166Do not alter the value of this variable manually. This variable should be
df7492f9 10167updated by the functions `define-coding-system' and
48b0f3ae 10168`define-coding-system-alias'. */);
4608c386
KH		 10169 Vcoding_system_list = Qnil;
10170
10171 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist,
48b0f3ae
PJ		 10172 doc: /* Alist of coding system names.
10173Each element is one element list of coding system name.
446dcd75 10174This variable is given to `completing-read' as COLLECTION argument.
48b0f3ae
PJ		 10175
10176Do not alter the value of this variable manually. This variable should be
10177updated by the functions `make-coding-system' and
10178`define-coding-system-alias'. */);
4608c386
KH		 10179 Vcoding_system_alist = Qnil;
10180
4ed46869 10181 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list,
48b0f3ae
PJ		 10182 doc: /* List of coding-categories (symbols) ordered by priority.
10183
10184On detecting a coding system, Emacs tries code detection algorithms
10185associated with each coding-category one by one in this order. When
10186one algorithm agrees with a byte sequence of source text, the coding
0ec31faf
KH		 10187system bound to the corresponding coding-category is selected.
10188
42205607 10189Don't modify this variable directly, but use `set-coding-priority'. */);
4ed46869
KH		 10190 {
10191 int i;
10192
10193 Vcoding_category_list = Qnil;
df7492f9 10194 for (i = coding_category_max - 1; i >= 0; i--)
4ed46869 10195 Vcoding_category_list
d46c5b12
KH		 10196 = Fcons (XVECTOR (Vcoding_category_table)->contents[i],
10197 Vcoding_category_list);
4ed46869
KH		 10198 }
10199
10200 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read,
48b0f3ae
PJ		 10201 doc: /* Specify the coding system for read operations.
10202It is useful to bind this variable with `let', but do not set it globally.
10203If the value is a coding system, it is used for decoding on read operation.
446dcd75
JB		 10204If not, an appropriate element is used from one of the coding system alists.
10205There are three such tables: `file-coding-system-alist',
48b0f3ae 10206`process-coding-system-alist', and `network-coding-system-alist'. */);
4ed46869
KH		 10207 Vcoding_system_for_read = Qnil;
10208
10209 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write,
48b0f3ae
PJ		 10210 doc: /* Specify the coding system for write operations.
10211Programs bind this variable with `let', but you should not set it globally.
10212If the value is a coding system, it is used for encoding of output,
10213when writing it to a file and when sending it to a file or subprocess.
10214
10215If this does not specify a coding system, an appropriate element
446dcd75
JB		 10216is used from one of the coding system alists.
10217There are three such tables: `file-coding-system-alist',
48b0f3ae
PJ		 10218`process-coding-system-alist', and `network-coding-system-alist'.
10219For output to files, if the above procedure does not specify a coding system,
10220the value of `buffer-file-coding-system' is used. */);
4ed46869
KH		 10221 Vcoding_system_for_write = Qnil;
10222
10223 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used,
df7492f9
KH		 10224 doc: /*
10225Coding system used in the latest file or process I/O. */);
4ed46869
KH		 10226 Vlast_coding_system_used = Qnil;
10227
065e3595
KH		 10228 DEFVAR_LISP ("last-code-conversion-error", &Vlast_code_conversion_error,
10229 doc: /*
10230Error status of the last code conversion.
10231
10232When an error was detected in the last code conversion, this variable
10233is set to one of the following symbols.
10234 `insufficient-source'
10235 `inconsistent-eol'
10236 `invalid-source'
10237 `interrupted'
10238 `insufficient-memory'
10239When no error was detected, the value doesn't change. So, to check
10240the error status of a code conversion by this variable, you must
10241explicitly set this variable to nil before performing code
10242conversion. */);
10243 Vlast_code_conversion_error = Qnil;
10244
9ce27fde 10245 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion,
df7492f9
KH		 10246 doc: /*
10247*Non-nil means always inhibit code conversion of end-of-line format.
48b0f3ae
PJ		 10248See info node `Coding Systems' and info node `Text and Binary' concerning
10249such conversion. */);
9ce27fde
KH		 10250 inhibit_eol_conversion = 0;
10251
ed29121d 10252 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system,
df7492f9
KH		 10253 doc: /*
10254Non-nil means process buffer inherits coding system of process output.
48b0f3ae
PJ		 10255Bind it to t if the process output is to be treated as if it were a file
10256read from some filesystem. */);
ed29121d
EZ		 10257 inherit_process_coding_system = 0;
10258
02ba4723 10259 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist,
df7492f9
KH		 10260 doc: /*
10261Alist to decide a coding system to use for a file I/O operation.
48b0f3ae
PJ		 10262The format is ((PATTERN . VAL) ...),
10263where PATTERN is a regular expression matching a file name,
10264VAL is a coding system, a cons of coding systems, or a function symbol.
10265If VAL is a coding system, it is used for both decoding and encoding
10266the file contents.
10267If VAL is a cons of coding systems, the car part is used for decoding,
10268and the cdr part is used for encoding.
10269If VAL is a function symbol, the function must return a coding system
2c53e699
KH		 10270or a cons of coding systems which are used as above. The function is
10271called with an argument that is a list of the arguments with which
5a0bbd9a
KH		 10272`find-operation-coding-system' was called. If the function can't decide
10273a coding system, it can return `undecided' so that the normal
10274code-detection is performed.
48b0f3ae
PJ		 10275
10276See also the function `find-operation-coding-system'
10277and the variable `auto-coding-alist'. */);
02ba4723
KH		 10278 Vfile_coding_system_alist = Qnil;
10279
10280 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist,
df7492f9
KH		 10281 doc: /*
10282Alist to decide a coding system to use for a process I/O operation.
48b0f3ae
PJ		 10283The format is ((PATTERN . VAL) ...),
10284where PATTERN is a regular expression matching a program name,
10285VAL is a coding system, a cons of coding systems, or a function symbol.
10286If VAL is a coding system, it is used for both decoding what received
10287from the program and encoding what sent to the program.
10288If VAL is a cons of coding systems, the car part is used for decoding,
10289and the cdr part is used for encoding.
10290If VAL is a function symbol, the function must return a coding system
10291or a cons of coding systems which are used as above.
10292
10293See also the function `find-operation-coding-system'. */);
02ba4723
KH		 10294 Vprocess_coding_system_alist = Qnil;
10295
10296 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist,
df7492f9
KH		 10297 doc: /*
10298Alist to decide a coding system to use for a network I/O operation.
48b0f3ae
PJ		 10299The format is ((PATTERN . VAL) ...),
10300where PATTERN is a regular expression matching a network service name
10301or is a port number to connect to,
10302VAL is a coding system, a cons of coding systems, or a function symbol.
10303If VAL is a coding system, it is used for both decoding what received
10304from the network stream and encoding what sent to the network stream.
10305If VAL is a cons of coding systems, the car part is used for decoding,
10306and the cdr part is used for encoding.
10307If VAL is a function symbol, the function must return a coding system
10308or a cons of coding systems which are used as above.
10309
10310See also the function `find-operation-coding-system'. */);
02ba4723 10311 Vnetwork_coding_system_alist = Qnil;
4ed46869 10312
68c45bf0 10313 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system,
75205970
RS		 10314 doc: /* Coding system to use with system messages.
10315Also used for decoding keyboard input on X Window system. */);
68c45bf0
PE		 10316 Vlocale_coding_system = Qnil;
10317
005f0d35 10318 /* The eol mnemonics are reset in startup.el system-dependently. */
7722baf9 10319 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix,
df7492f9
KH		 10320 doc: /*
10321*String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7722baf9 10322 eol_mnemonic_unix = build_string (":");
4ed46869 10323
7722baf9 10324 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos,
df7492f9
KH		 10325 doc: /*
10326*String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7722baf9 10327 eol_mnemonic_dos = build_string ("\\");
4ed46869 10328
7722baf9 10329 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac,
df7492f9
KH		 10330 doc: /*
10331*String displayed in mode line for MAC-like (CR) end-of-line format. */);
7722baf9 10332 eol_mnemonic_mac = build_string ("/");
4ed46869 10333
7722baf9 10334 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided,
df7492f9
KH		 10335 doc: /*
10336*String displayed in mode line when end-of-line format is not yet determined. */);
7722baf9 10337 eol_mnemonic_undecided = build_string (":");
4ed46869 10338
84fbb8a0 10339 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation,
df7492f9
KH		 10340 doc: /*
10341*Non-nil enables character translation while encoding and decoding. */);
84fbb8a0 10342 Venable_character_translation = Qt;
bdd9fb48 10343
f967223b 10344 DEFVAR_LISP ("standard-translation-table-for-decode",
48b0f3ae
PJ		 10345 &Vstandard_translation_table_for_decode,
10346 doc: /* Table for translating characters while decoding. */);
f967223b 10347 Vstandard_translation_table_for_decode = Qnil;
bdd9fb48 10348
f967223b 10349 DEFVAR_LISP ("standard-translation-table-for-encode",
48b0f3ae
PJ		 10350 &Vstandard_translation_table_for_encode,
10351 doc: /* Table for translating characters while encoding. */);
f967223b 10352 Vstandard_translation_table_for_encode = Qnil;
4ed46869 10353
df7492f9 10354 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_table,
48b0f3ae
PJ		 10355 doc: /* Alist of charsets vs revision numbers.
10356While encoding, if a charset (car part of an element) is found,
df7492f9
KH		 10357designate it with the escape sequence identifying revision (cdr part
10358of the element). */);
10359 Vcharset_revision_table = Qnil;
02ba4723
KH		 10360
10361 DEFVAR_LISP ("default-process-coding-system",
10362 &Vdefault_process_coding_system,
48b0f3ae
PJ		 10363 doc: /* Cons of coding systems used for process I/O by default.
10364The car part is used for decoding a process output,
10365the cdr part is used for encoding a text to be sent to a process. */);
02ba4723 10366 Vdefault_process_coding_system = Qnil;
c4825358 10367
3f003981 10368 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table,
df7492f9
KH		 10369 doc: /*
10370Table of extra Latin codes in the range 128..159 (inclusive).
48b0f3ae
PJ		 10371This is a vector of length 256.
10372If Nth element is non-nil, the existence of code N in a file
10373\(or output of subprocess) doesn't prevent it to be detected as
10374a coding system of ISO 2022 variant which has a flag
10375`accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
10376or reading output of a subprocess.
446dcd75 10377Only 128th through 159th elements have a meaning. */);
3f003981 10378 Vlatin_extra_code_table = Fmake_vector (make_number (256), Qnil);
d46c5b12
KH		 10379
10380 DEFVAR_LISP ("select-safe-coding-system-function",
10381 &Vselect_safe_coding_system_function,
df7492f9
KH		 10382 doc: /*
10383Function to call to select safe coding system for encoding a text.
48b0f3ae
PJ		 10384
10385If set, this function is called to force a user to select a proper
10386coding system which can encode the text in the case that a default
fdecf907
GM		 10387coding system used in each operation can't encode the text. The
10388function should take care that the buffer is not modified while
10389the coding system is being selected.
48b0f3ae
PJ		 10390
10391The default value is `select-safe-coding-system' (which see). */);
d46c5b12
KH		 10392 Vselect_safe_coding_system_function = Qnil;
10393
5d5bf4d8
KH		 10394 DEFVAR_BOOL ("coding-system-require-warning",
10395 &coding_system_require_warning,
10396 doc: /* Internal use only.
6b89e3aa
KH		 10397If non-nil, on writing a file, `select-safe-coding-system-function' is
10398called even if `coding-system-for-write' is non-nil. The command
10399`universal-coding-system-argument' binds this variable to t temporarily. */);
5d5bf4d8
KH		 10400 coding_system_require_warning = 0;
10401
10402
22ab2303 10403 DEFVAR_BOOL ("inhibit-iso-escape-detection",
74383408 10404 &inhibit_iso_escape_detection,
df7492f9 10405 doc: /*
97b1b294 10406If non-nil, Emacs ignores ISO-2022 escape sequences during code detection.
48b0f3ae 10407
97b1b294
EZ		 10408When Emacs reads text, it tries to detect how the text is encoded.
10409This code detection is sensitive to escape sequences. If Emacs sees
10410a valid ISO-2022 escape sequence, it assumes the text is encoded in one
10411of the ISO2022 encodings, and decodes text by the corresponding coding
10412system (e.g. `iso-2022-7bit').
48b0f3ae
PJ		 10413
10414However, there may be a case that you want to read escape sequences in
10415a file as is. In such a case, you can set this variable to non-nil.
97b1b294
EZ		 10416Then the code detection will ignore any escape sequences, and no text is
10417detected as encoded in some ISO-2022 encoding. The result is that all
48b0f3ae
PJ		 10418escape sequences become visible in a buffer.
10419
10420The default value is nil, and it is strongly recommended not to change
10421it. That is because many Emacs Lisp source files that contain
10422non-ASCII characters are encoded by the coding system `iso-2022-7bit'
10423in Emacs's distribution, and they won't be decoded correctly on
10424reading if you suppress escape sequence detection.
10425
10426The other way to read escape sequences in a file without decoding is
97b1b294 10427to explicitly specify some coding system that doesn't use ISO-2022
48b0f3ae 10428escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
74383408 10429 inhibit_iso_escape_detection = 0;
002fdb44 10430
97b1b294
EZ		 10431 DEFVAR_BOOL ("inhibit-null-byte-detection",
10432 &inhibit_null_byte_detection,
10433 doc: /* If non-nil, Emacs ignores null bytes on code detection.
10434By default, Emacs treats it as binary data, and does not attempt to
10435decode it. The effect is as if you specified `no-conversion' for
10436reading that text.
10437
10438Set this to non-nil when a regular text happens to include null bytes.
10439Examples are Index nodes of Info files and null-byte delimited output
10440from GNU Find and GNU Grep. Emacs will then ignore the null bytes and
10441decode text as usual. */);
10442 inhibit_null_byte_detection = 0;
10443
002fdb44 10444 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input,
15c8f9d1 10445 doc: /* Char table for translating self-inserting characters.
446dcd75 10446This is applied to the result of input methods, not their input.
8434d0b8
EZ		 10447See also `keyboard-translate-table'.
10448
10449Use of this variable for character code unification was rendered
10450obsolete in Emacs 23.1 and later, since Unicode is now the basis of
10451internal character representation. */);
002fdb44 10452 Vtranslation_table_for_input = Qnil;
8f924df7 10453
2c78b7e1
KH		 10454 {
10455 Lisp_Object args[coding_arg_max];
8f924df7 10456 Lisp_Object plist[16];
2c78b7e1
KH		 10457 int i;
10458
10459 for (i = 0; i < coding_arg_max; i++)
10460 args[i] = Qnil;
10461
10462 plist[0] = intern (":name");
10463 plist[1] = args[coding_arg_name] = Qno_conversion;
10464 plist[2] = intern (":mnemonic");
10465 plist[3] = args[coding_arg_mnemonic] = make_number ('=');
10466 plist[4] = intern (":coding-type");
10467 plist[5] = args[coding_arg_coding_type] = Qraw_text;
10468 plist[6] = intern (":ascii-compatible-p");
10469 plist[7] = args[coding_arg_ascii_compatible_p] = Qt;
10470 plist[8] = intern (":default-char");
10471 plist[9] = args[coding_arg_default_char] = make_number (0);
8f924df7
KH		 10472 plist[10] = intern (":for-unibyte");
10473 plist[11] = args[coding_arg_for_unibyte] = Qt;
10474 plist[12] = intern (":docstring");
10475 plist[13] = build_string ("Do no conversion.\n\
2c78b7e1
KH		 10476\n\
10477When you visit a file with this coding, the file is read into a\n\
10478unibyte buffer as is, thus each byte of a file is treated as a\n\
10479character.");
8f924df7
KH		 10480 plist[14] = intern (":eol-type");
10481 plist[15] = args[coding_arg_eol_type] = Qunix;
10482 args[coding_arg_plist] = Flist (16, plist);
2c78b7e1 10483 Fdefine_coding_system_internal (coding_arg_max, args);
ae6f73fa
KH		 10484
10485 plist[1] = args[coding_arg_name] = Qundecided;
10486 plist[3] = args[coding_arg_mnemonic] = make_number ('-');
10487 plist[5] = args[coding_arg_coding_type] = Qundecided;
10488 /* This is already set.
35befdaa 10489 plist[7] = args[coding_arg_ascii_compatible_p] = Qt; */
ae6f73fa
KH		 10490 plist[8] = intern (":charset-list");
10491 plist[9] = args[coding_arg_charset_list] = Fcons (Qascii, Qnil);
10492 plist[11] = args[coding_arg_for_unibyte] = Qnil;
10493 plist[13] = build_string ("No conversion on encoding, automatic conversion on decoding.");
10494 plist[15] = args[coding_arg_eol_type] = Qnil;
10495 args[coding_arg_plist] = Flist (16, plist);
10496 Fdefine_coding_system_internal (coding_arg_max, args);
2c78b7e1
KH		 10497 }
10498
2c78b7e1 10499 setup_coding_system (Qno_conversion, &safe_terminal_coding);
ff563fce
KH		 10500
10501 {
10502 int i;
10503
10504 for (i = 0; i < coding_category_max; i++)
10505 Fset (AREF (Vcoding_category_table, i), Qno_conversion);
10506 }
fcbcfb64
KH		 10507#if defined (MSDOS) || defined (WINDOWSNT)
10508 system_eol_type = Qdos;
10509#else
10510 system_eol_type = Qunix;
10511#endif
10512 staticpro (&system_eol_type);
4ed46869
KH		 10513}
10514
68c45bf0
PE		 10515char *
10516emacs_strerror (error_number)
10517 int error_number;
10518{
10519 char *str;
10520
ca9c0567 10521 synchronize_system_messages_locale ();
68c45bf0
PE		 10522 str = strerror (error_number);
10523
10524 if (! NILP (Vlocale_coding_system))
10525 {
10526 Lisp_Object dec = code_convert_string_norecord (build_string (str),
10527 Vlocale_coding_system,
10528 0);
d5db4077 10529 str = (char *) SDATA (dec);
68c45bf0
PE		 10530 }
10531
10532 return str;
10533}
10534
4ed46869 10535#endif /* emacs */
9ffd559c
KH		 10536
10537/* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
10538 (do not change this comment) */
Unnamed repository; edit this file 'description' to name the repository.