HCoop / bpt / emacs.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
* coding.c: Integer and buffer overflow fixes.
[bpt/emacs.git] / src / coding.c
CommitLineData
9542cb1f 1/* Coding system handler (conversion, detection, etc).
73b0cd50 2 Copyright (C) 2001-2011 Free Software Foundation, Inc.
7976eda0 3 Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
5df4f04c 4 2005, 2006, 2007, 2008, 2009, 2010, 2011
ce03bf76
KH		 5 National Institute of Advanced Industrial Science and Technology (AIST)
6 Registration Number H14PRO021
8f924df7 7 Copyright (C) 2003
df7492f9
KH		 8 National Institute of Advanced Industrial Science and Technology (AIST)
9 Registration Number H13PRO009
4ed46869 10
369314dc
KH		 11This file is part of GNU Emacs.
12
9ec0b715 13GNU Emacs is free software: you can redistribute it and/or modify
369314dc 14it under the terms of the GNU General Public License as published by
9ec0b715
GM		 15the Free Software Foundation, either version 3 of the License, or
16(at your option) any later version.
4ed46869 17
369314dc
KH		 18GNU Emacs is distributed in the hope that it will be useful,
19but WITHOUT ANY WARRANTY; without even the implied warranty of
20MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
21GNU General Public License for more details.
4ed46869 22
369314dc 23You should have received a copy of the GNU General Public License
9ec0b715 24along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>. */
4ed46869
KH		 25
26/*** TABLE OF CONTENTS ***
27
b73bfc1c 28 0. General comments
4ed46869 29 1. Preamble
df7492f9
KH		 30 2. Emacs' internal format (emacs-utf-8) handlers
31 3. UTF-8 handlers
32 4. UTF-16 handlers
33 5. Charset-base coding systems handlers
34 6. emacs-mule (old Emacs' internal format) handlers
35 7. ISO2022 handlers
36 8. Shift-JIS and BIG5 handlers
37 9. CCL handlers
38 10. C library functions
39 11. Emacs Lisp library functions
40 12. Postamble
4ed46869
KH		 41
42*/
43
df7492f9 44/*** 0. General comments ***
b73bfc1c
KH		 45
46
df7492f9 47CODING SYSTEM
4ed46869 48
5bad0796
DL		 49 A coding system is an object for an encoding mechanism that contains
50 information about how to convert byte sequences to character
e19c3639
KH		 51 sequences and vice versa. When we say "decode", it means converting
52 a byte sequence of a specific coding system into a character
53 sequence that is represented by Emacs' internal coding system
54 `emacs-utf-8', and when we say "encode", it means converting a
55 character sequence of emacs-utf-8 to a byte sequence of a specific
0ef69138 56 coding system.
4ed46869 57
34809aa6
EZ		 58 In Emacs Lisp, a coding system is represented by a Lisp symbol. On
59 the C level, a coding system is represented by a vector of attributes
5bad0796 60 stored in the hash table Vcharset_hash_table. The conversion from
e19c3639
KH		 61 coding system symbol to attributes vector is done by looking up
62 Vcharset_hash_table by the symbol.
4ed46869 63
e19c3639 64 Coding systems are classified into the following types depending on
5bad0796 65 the encoding mechanism. Here's a brief description of the types.
4ed46869 66
df7492f9
KH		 67 o UTF-8
68
69 o UTF-16
70
71 o Charset-base coding system
72
73 A coding system defined by one or more (coded) character sets.
5bad0796 74 Decoding and encoding are done by a code converter defined for each
df7492f9
KH		 75 character set.
76
5bad0796 77 o Old Emacs internal format (emacs-mule)
df7492f9 78
5bad0796 79 The coding system adopted by old versions of Emacs (20 and 21).
4ed46869 80
df7492f9 81 o ISO2022-base coding system
4ed46869
KH		 82
83 The most famous coding system for multiple character sets. X's
df7492f9
KH		 84 Compound Text, various EUCs (Extended Unix Code), and coding systems
85 used in the Internet communication such as ISO-2022-JP are all
86 variants of ISO2022.
4ed46869 87
df7492f9 88 o SJIS (or Shift-JIS or MS-Kanji-Code)
93dec019 89
4ed46869
KH		 90 A coding system to encode character sets: ASCII, JISX0201, and
91 JISX0208. Widely used for PC's in Japan. Details are described in
df7492f9 92 section 8.
4ed46869 93
df7492f9 94 o BIG5
4ed46869 95
df7492f9 96 A coding system to encode character sets: ASCII and Big5. Widely
cfb43547 97 used for Chinese (mainly in Taiwan and Hong Kong). Details are
df7492f9
KH		 98 described in section 8. In this file, when we write "big5" (all
99 lowercase), we mean the coding system, and when we write "Big5"
100 (capitalized), we mean the character set.
4ed46869 101
df7492f9 102 o CCL
27901516 103
5bad0796 104 If a user wants to decode/encode text encoded in a coding system
df7492f9
KH		 105 not listed above, he can supply a decoder and an encoder for it in
106 CCL (Code Conversion Language) programs. Emacs executes the CCL
107 program while decoding/encoding.
27901516 108
df7492f9 109 o Raw-text
4ed46869 110
5a936b46 111 A coding system for text containing raw eight-bit data. Emacs
5bad0796 112 treats each byte of source text as a character (except for
df7492f9 113 end-of-line conversion).
4ed46869 114
df7492f9
KH		 115 o No-conversion
116
117 Like raw text, but don't do end-of-line conversion.
4ed46869 118
4ed46869 119
df7492f9 120END-OF-LINE FORMAT
4ed46869 121
5bad0796 122 How text end-of-line is encoded depends on operating system. For
df7492f9 123 instance, Unix's format is just one byte of LF (line-feed) code,
f4dee582 124 whereas DOS's format is two-byte sequence of `carriage-return' and
d46c5b12
KH		 125 `line-feed' codes. MacOS's format is usually one byte of
126 `carriage-return'.
4ed46869 127
cfb43547 128 Since text character encoding and end-of-line encoding are
df7492f9
KH		 129 independent, any coding system described above can take any format
130 of end-of-line (except for no-conversion).
4ed46869 131
e19c3639
KH		 132STRUCT CODING_SYSTEM
133
134 Before using a coding system for code conversion (i.e. decoding and
135 encoding), we setup a structure of type `struct coding_system'.
136 This structure keeps various information about a specific code
5bad0796 137 conversion (e.g. the location of source and destination data).
4ed46869
KH		 138
139*/
140
df7492f9
KH		 141/* COMMON MACROS */
142
143
4ed46869
KH		 144/*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
145
df7492f9 146 These functions check if a byte sequence specified as a source in
ff0dacd7
KH		 147 CODING conforms to the format of XXX, and update the members of
148 DETECT_INFO.
df7492f9 149
ff0dacd7 150 Return 1 if the byte sequence conforms to XXX, otherwise return 0.
df7492f9
KH		 151
152 Below is the template of these functions. */
153
4ed46869 154#if 0
df7492f9 155static int
cf84bb53
JB		 156detect_coding_XXX (struct coding_system *coding,
157 struct coding_detection_info *detect_info)
4ed46869 158{
f1d34bca
MB		 159 const unsigned char *src = coding->source;
160 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 161 int multibytep = coding->src_multibyte;
d311d28c 162 ptrdiff_t consumed_chars = 0;
df7492f9
KH		 163 int found = 0;
164 ...;
165
166 while (1)
167 {
ad1746f5 168 /* Get one byte from the source. If the source is exhausted, jump
df7492f9
KH		 169 to no_more_source:. */
170 ONE_MORE_BYTE (c);
ff0dacd7
KH		 171
172 if (! __C_conforms_to_XXX___ (c))
173 break;
174 if (! __C_strongly_suggests_XXX__ (c))
175 found = CATEGORY_MASK_XXX;
df7492f9 176 }
ff0dacd7
KH		 177 /* The byte sequence is invalid for XXX. */
178 detect_info->rejected |= CATEGORY_MASK_XXX;
df7492f9 179 return 0;
ff0dacd7 180
df7492f9 181 no_more_source:
ad1746f5 182 /* The source exhausted successfully. */
ff0dacd7 183 detect_info->found |= found;
df7492f9 184 return 1;
4ed46869
KH		 185}
186#endif
187
188/*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
189
df7492f9
KH		 190 These functions decode a byte sequence specified as a source by
191 CODING. The resulting multibyte text goes to a place pointed to by
192 CODING->charbuf, the length of which should not exceed
193 CODING->charbuf_size;
d46c5b12 194
df7492f9
KH		 195 These functions set the information of original and decoded texts in
196 CODING->consumed, CODING->consumed_char, and CODING->charbuf_used.
197 They also set CODING->result to one of CODING_RESULT_XXX indicating
198 how the decoding is finished.
d46c5b12 199
df7492f9 200 Below is the template of these functions. */
d46c5b12 201
4ed46869 202#if 0
b73bfc1c 203static void
cf84bb53 204decode_coding_XXXX (struct coding_system *coding)
4ed46869 205{
f1d34bca
MB		 206 const unsigned char *src = coding->source + coding->consumed;
207 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 208 /* SRC_BASE remembers the start position in source in each loop.
209 The loop will be exited when there's not enough source code, or
210 when there's no room in CHARBUF for a decoded character. */
f1d34bca 211 const unsigned char *src_base;
df7492f9 212 /* A buffer to produce decoded characters. */
69a80ea3
KH		 213 int *charbuf = coding->charbuf + coding->charbuf_used;
214 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 215 int multibytep = coding->src_multibyte;
216
217 while (1)
218 {
219 src_base = src;
220 if (charbuf < charbuf_end)
221 /* No more room to produce a decoded character. */
222 break;
223 ONE_MORE_BYTE (c);
224 /* Decode it. */
225 }
226
227 no_more_source:
228 if (src_base < src_end
229 && coding->mode & CODING_MODE_LAST_BLOCK)
230 /* If the source ends by partial bytes to construct a character,
231 treat them as eight-bit raw data. */
232 while (src_base < src_end && charbuf < charbuf_end)
233 *charbuf++ = *src_base++;
234 /* Remember how many bytes and characters we consumed. If the
235 source is multibyte, the bytes and chars are not identical. */
236 coding->consumed = coding->consumed_char = src_base - coding->source;
237 /* Remember how many characters we produced. */
238 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 239}
240#endif
241
242/*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
243
df7492f9
KH		 244 These functions encode SRC_BYTES length text at SOURCE of Emacs'
245 internal multibyte format by CODING. The resulting byte sequence
b73bfc1c
KH		 246 goes to a place pointed to by DESTINATION, the length of which
247 should not exceed DST_BYTES.
d46c5b12 248
df7492f9
KH		 249 These functions set the information of original and encoded texts in
250 the members produced, produced_char, consumed, and consumed_char of
251 the structure *CODING. They also set the member result to one of
252 CODING_RESULT_XXX indicating how the encoding finished.
d46c5b12 253
df7492f9
KH		 254 DST_BYTES zero means that source area and destination area are
255 overlapped, which means that we can produce a encoded text until it
256 reaches at the head of not-yet-encoded source text.
d46c5b12 257
df7492f9 258 Below is a template of these functions. */
4ed46869 259#if 0
b73bfc1c 260static void
cf84bb53 261encode_coding_XXX (struct coding_system *coding)
4ed46869 262{
df7492f9
KH		 263 int multibytep = coding->dst_multibyte;
264 int *charbuf = coding->charbuf;
265 int *charbuf_end = charbuf->charbuf + coding->charbuf_used;
266 unsigned char *dst = coding->destination + coding->produced;
267 unsigned char *dst_end = coding->destination + coding->dst_bytes;
268 unsigned char *adjusted_dst_end = dst_end - _MAX_BYTES_PRODUCED_IN_LOOP_;
d311d28c 269 ptrdiff_t produced_chars = 0;
df7492f9
KH		 270
271 for (; charbuf < charbuf_end && dst < adjusted_dst_end; charbuf++)
272 {
273 int c = *charbuf;
274 /* Encode C into DST, and increment DST. */
275 }
276 label_no_more_destination:
277 /* How many chars and bytes we produced. */
278 coding->produced_char += produced_chars;
279 coding->produced = dst - coding->destination;
4ed46869
KH		 280}
281#endif
282
4ed46869
KH		 283\f
284/*** 1. Preamble ***/
285
68c45bf0 286#include <config.h>
4ed46869 287#include <stdio.h>
d7306fe6 288#include <setjmp.h>
4ed46869 289
4ed46869
KH		 290#include "lisp.h"
291#include "buffer.h"
df7492f9 292#include "character.h"
4ed46869
KH		 293#include "charset.h"
294#include "ccl.h"
df7492f9 295#include "composite.h"
4ed46869
KH		 296#include "coding.h"
297#include "window.h"
b8299c66
KL		 298#include "frame.h"
299#include "termhooks.h"
4ed46869 300
df7492f9 301Lisp_Object Vcoding_system_hash_table;
4ed46869 302
955cbe7b
PE		 303static Lisp_Object Qcoding_system, Qeol_type;
304static Lisp_Object Qcoding_aliases;
1965cb73 305Lisp_Object Qunix, Qdos;
4ed46869 306Lisp_Object Qbuffer_file_coding_system;
955cbe7b
PE		 307static Lisp_Object Qpost_read_conversion, Qpre_write_conversion;
308static Lisp_Object Qdefault_char;
27901516 309Lisp_Object Qno_conversion, Qundecided;
955cbe7b
PE		 310Lisp_Object Qcharset, Qutf_8;
311static Lisp_Object Qiso_2022;
312static Lisp_Object Qutf_16, Qshift_jis, Qbig5;
313static Lisp_Object Qbig, Qlittle;
314static Lisp_Object Qcoding_system_history;
315static Lisp_Object Qvalid_codes;
316static Lisp_Object QCcategory, QCmnemonic, QCdefault_char;
317static Lisp_Object QCdecode_translation_table, QCencode_translation_table;
318static Lisp_Object QCpost_read_conversion, QCpre_write_conversion;
319static Lisp_Object QCascii_compatible_p;
4ed46869 320
387f6ba5 321Lisp_Object Qcall_process, Qcall_process_region;
4ed46869 322Lisp_Object Qstart_process, Qopen_network_stream;
955cbe7b 323static Lisp_Object Qtarget_idx;
4ed46869 324
955cbe7b
PE		 325static Lisp_Object Qinsufficient_source, Qinconsistent_eol, Qinvalid_source;
326static Lisp_Object Qinterrupted, Qinsufficient_memory;
065e3595 327
44e8490d
KH		 328/* If a symbol has this property, evaluate the value to define the
329 symbol as a coding system. */
330static Lisp_Object Qcoding_system_define_form;
331
fcbcfb64
KH		 332/* Format of end-of-line decided by system. This is Qunix on
333 Unix and Mac, Qdos on DOS/Windows.
334 This has an effect only for external encoding (i.e. for output to
335 file and process), not for in-buffer or Lisp string encoding. */
336static Lisp_Object system_eol_type;
337
4ed46869
KH		 338#ifdef emacs
339
4608c386 340Lisp_Object Qcoding_system_p, Qcoding_system_error;
4ed46869 341
d46c5b12
KH		 342/* Coding system emacs-mule and raw-text are for converting only
343 end-of-line format. */
344Lisp_Object Qemacs_mule, Qraw_text;
8f924df7 345Lisp_Object Qutf_8_emacs;
ecf488bc 346
4ed46869
KH		 347/* Coding-systems are handed between Emacs Lisp programs and C internal
348 routines by the following three variables. */
c4825358
KH		 349/* Coding system to be used to encode text for terminal display when
350 terminal coding system is nil. */
351struct coding_system safe_terminal_coding;
352
4ed46869
KH		 353#endif /* emacs */
354
f967223b
KH		 355Lisp_Object Qtranslation_table;
356Lisp_Object Qtranslation_table_id;
955cbe7b
PE		 357static Lisp_Object Qtranslation_table_for_decode;
358static Lisp_Object Qtranslation_table_for_encode;
4ed46869 359
df7492f9 360/* Two special coding systems. */
74ab6df5
PE		 361static Lisp_Object Vsjis_coding_system;
362static Lisp_Object Vbig5_coding_system;
df7492f9 363
df7492f9
KH		 364/* ISO2022 section */
365
366#define CODING_ISO_INITIAL(coding, reg) \
367 (XINT (AREF (AREF (CODING_ID_ATTRS ((coding)->id), \
368 coding_attr_iso_initial), \
369 reg)))
370
371
1b3b981b
AS		 372#define CODING_ISO_REQUEST(coding, charset_id) \
373 (((charset_id) <= (coding)->max_charset_id \
374 ? ((coding)->safe_charsets[charset_id] != 255 \
375 ? (coding)->safe_charsets[charset_id] \
376 : -1) \
df7492f9
KH		 377 : -1))
378
379
380#define CODING_ISO_FLAGS(coding) \
381 ((coding)->spec.iso_2022.flags)
382#define CODING_ISO_DESIGNATION(coding, reg) \
383 ((coding)->spec.iso_2022.current_designation[reg])
384#define CODING_ISO_INVOCATION(coding, plane) \
385 ((coding)->spec.iso_2022.current_invocation[plane])
386#define CODING_ISO_SINGLE_SHIFTING(coding) \
387 ((coding)->spec.iso_2022.single_shifting)
388#define CODING_ISO_BOL(coding) \
389 ((coding)->spec.iso_2022.bol)
390#define CODING_ISO_INVOKED_CHARSET(coding, plane) \
391 CODING_ISO_DESIGNATION ((coding), CODING_ISO_INVOCATION ((coding), (plane)))
e951386e
KH		 392#define CODING_ISO_CMP_STATUS(coding) \
393 (&(coding)->spec.iso_2022.cmp_status)
394#define CODING_ISO_EXTSEGMENT_LEN(coding) \
395 ((coding)->spec.iso_2022.ctext_extended_segment_len)
396#define CODING_ISO_EMBEDDED_UTF_8(coding) \
397 ((coding)->spec.iso_2022.embedded_utf_8)
df7492f9
KH		 398
399/* Control characters of ISO2022. */
400 /* code */ /* function */
df7492f9
KH		 401#define ISO_CODE_SO 0x0E /* shift-out */
402#define ISO_CODE_SI 0x0F /* shift-in */
403#define ISO_CODE_SS2_7 0x19 /* single-shift-2 for 7-bit code */
404#define ISO_CODE_ESC 0x1B /* escape */
405#define ISO_CODE_SS2 0x8E /* single-shift-2 */
406#define ISO_CODE_SS3 0x8F /* single-shift-3 */
407#define ISO_CODE_CSI 0x9B /* control-sequence-introducer */
408
409/* All code (1-byte) of ISO2022 is classified into one of the
410 followings. */
411enum iso_code_class_type
412 {
413 ISO_control_0, /* Control codes in the range
414 0x00..0x1F and 0x7F, except for the
415 following 5 codes. */
df7492f9
KH		 416 ISO_shift_out, /* ISO_CODE_SO (0x0E) */
417 ISO_shift_in, /* ISO_CODE_SI (0x0F) */
418 ISO_single_shift_2_7, /* ISO_CODE_SS2_7 (0x19) */
419 ISO_escape, /* ISO_CODE_SO (0x1B) */
420 ISO_control_1, /* Control codes in the range
421 0x80..0x9F, except for the
422 following 3 codes. */
423 ISO_single_shift_2, /* ISO_CODE_SS2 (0x8E) */
424 ISO_single_shift_3, /* ISO_CODE_SS3 (0x8F) */
425 ISO_control_sequence_introducer, /* ISO_CODE_CSI (0x9B) */
426 ISO_0x20_or_0x7F, /* Codes of the values 0x20 or 0x7F. */
427 ISO_graphic_plane_0, /* Graphic codes in the range 0x21..0x7E. */
428 ISO_0xA0_or_0xFF, /* Codes of the values 0xA0 or 0xFF. */
429 ISO_graphic_plane_1 /* Graphic codes in the range 0xA1..0xFE. */
430 };
05e6f5dc 431
df7492f9
KH		 432/** The macros CODING_ISO_FLAG_XXX defines a flag bit of the
433 `iso-flags' attribute of an iso2022 coding system. */
05e6f5dc 434
df7492f9
KH		 435/* If set, produce long-form designation sequence (e.g. ESC $ ( A)
436 instead of the correct short-form sequence (e.g. ESC $ A). */
437#define CODING_ISO_FLAG_LONG_FORM 0x0001
93dec019 438
df7492f9
KH		 439/* If set, reset graphic planes and registers at end-of-line to the
440 initial state. */
441#define CODING_ISO_FLAG_RESET_AT_EOL 0x0002
05e6f5dc 442
df7492f9
KH		 443/* If set, reset graphic planes and registers before any control
444 characters to the initial state. */
445#define CODING_ISO_FLAG_RESET_AT_CNTL 0x0004
05e6f5dc 446
df7492f9
KH		 447/* If set, encode by 7-bit environment. */
448#define CODING_ISO_FLAG_SEVEN_BITS 0x0008
4ed46869 449
df7492f9
KH		 450/* If set, use locking-shift function. */
451#define CODING_ISO_FLAG_LOCKING_SHIFT 0x0010
b73bfc1c 452
df7492f9
KH		 453/* If set, use single-shift function. Overwrite
454 CODING_ISO_FLAG_LOCKING_SHIFT. */
455#define CODING_ISO_FLAG_SINGLE_SHIFT 0x0020
b73bfc1c 456
df7492f9
KH		 457/* If set, use designation escape sequence. */
458#define CODING_ISO_FLAG_DESIGNATION 0x0040
b73bfc1c 459
df7492f9
KH		 460/* If set, produce revision number sequence. */
461#define CODING_ISO_FLAG_REVISION 0x0080
b73bfc1c 462
df7492f9
KH		 463/* If set, produce ISO6429's direction specifying sequence. */
464#define CODING_ISO_FLAG_DIRECTION 0x0100
f4dee582 465
df7492f9
KH		 466/* If set, assume designation states are reset at beginning of line on
467 output. */
468#define CODING_ISO_FLAG_INIT_AT_BOL 0x0200
4ed46869 469
df7492f9
KH		 470/* If set, designation sequence should be placed at beginning of line
471 on output. */
472#define CODING_ISO_FLAG_DESIGNATE_AT_BOL 0x0400
aa72b389 473
ad1746f5 474/* If set, do not encode unsafe characters on output. */
df7492f9 475#define CODING_ISO_FLAG_SAFE 0x0800
aa72b389 476
df7492f9
KH		 477/* If set, extra latin codes (128..159) are accepted as a valid code
478 on input. */
479#define CODING_ISO_FLAG_LATIN_EXTRA 0x1000
aa72b389 480
df7492f9 481#define CODING_ISO_FLAG_COMPOSITION 0x2000
aa72b389 482
5f58e762 483/* #define CODING_ISO_FLAG_EUC_TW_SHIFT 0x4000 */
aa72b389 484
bf16eb23 485#define CODING_ISO_FLAG_USE_ROMAN 0x8000
aa72b389 486
bf16eb23 487#define CODING_ISO_FLAG_USE_OLDJIS 0x10000
aa72b389 488
bf16eb23 489#define CODING_ISO_FLAG_FULL_SUPPORT 0x100000
aa72b389 490
df7492f9
KH		 491/* A character to be produced on output if encoding of the original
492 character is prohibited by CODING_ISO_FLAG_SAFE. */
493#define CODING_INHIBIT_CHARACTER_SUBSTITUTION '?'
aa72b389 494
a470d443
KH		 495/* UTF-8 section */
496#define CODING_UTF_8_BOM(coding) \
497 ((coding)->spec.utf_8_bom)
4ed46869 498
df7492f9
KH		 499/* UTF-16 section */
500#define CODING_UTF_16_BOM(coding) \
501 ((coding)->spec.utf_16.bom)
4ed46869 502
df7492f9
KH		 503#define CODING_UTF_16_ENDIAN(coding) \
504 ((coding)->spec.utf_16.endian)
4ed46869 505
df7492f9
KH		 506#define CODING_UTF_16_SURROGATE(coding) \
507 ((coding)->spec.utf_16.surrogate)
4ed46869 508
4ed46869 509
df7492f9
KH		 510/* CCL section */
511#define CODING_CCL_DECODER(coding) \
512 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_decoder)
513#define CODING_CCL_ENCODER(coding) \
514 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_encoder)
515#define CODING_CCL_VALIDS(coding) \
8f924df7 516 (SDATA (AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_valids)))
4ed46869 517
5a936b46 518/* Index for each coding category in `coding_categories' */
4ed46869 519
df7492f9
KH		 520enum coding_category
521 {
522 coding_category_iso_7,
523 coding_category_iso_7_tight,
524 coding_category_iso_8_1,
525 coding_category_iso_8_2,
526 coding_category_iso_7_else,
527 coding_category_iso_8_else,
a470d443
KH		 528 coding_category_utf_8_auto,
529 coding_category_utf_8_nosig,
530 coding_category_utf_8_sig,
df7492f9
KH		 531 coding_category_utf_16_auto,
532 coding_category_utf_16_be,
533 coding_category_utf_16_le,
534 coding_category_utf_16_be_nosig,
535 coding_category_utf_16_le_nosig,
536 coding_category_charset,
537 coding_category_sjis,
538 coding_category_big5,
539 coding_category_ccl,
540 coding_category_emacs_mule,
541 /* All above are targets of code detection. */
542 coding_category_raw_text,
543 coding_category_undecided,
544 coding_category_max
545 };
546
547/* Definitions of flag bits used in detect_coding_XXXX. */
548#define CATEGORY_MASK_ISO_7 (1 << coding_category_iso_7)
549#define CATEGORY_MASK_ISO_7_TIGHT (1 << coding_category_iso_7_tight)
550#define CATEGORY_MASK_ISO_8_1 (1 << coding_category_iso_8_1)
551#define CATEGORY_MASK_ISO_8_2 (1 << coding_category_iso_8_2)
552#define CATEGORY_MASK_ISO_7_ELSE (1 << coding_category_iso_7_else)
553#define CATEGORY_MASK_ISO_8_ELSE (1 << coding_category_iso_8_else)
a470d443
KH		 554#define CATEGORY_MASK_UTF_8_AUTO (1 << coding_category_utf_8_auto)
555#define CATEGORY_MASK_UTF_8_NOSIG (1 << coding_category_utf_8_nosig)
556#define CATEGORY_MASK_UTF_8_SIG (1 << coding_category_utf_8_sig)
b49a1807 557#define CATEGORY_MASK_UTF_16_AUTO (1 << coding_category_utf_16_auto)
df7492f9
KH		 558#define CATEGORY_MASK_UTF_16_BE (1 << coding_category_utf_16_be)
559#define CATEGORY_MASK_UTF_16_LE (1 << coding_category_utf_16_le)
560#define CATEGORY_MASK_UTF_16_BE_NOSIG (1 << coding_category_utf_16_be_nosig)
561#define CATEGORY_MASK_UTF_16_LE_NOSIG (1 << coding_category_utf_16_le_nosig)
562#define CATEGORY_MASK_CHARSET (1 << coding_category_charset)
563#define CATEGORY_MASK_SJIS (1 << coding_category_sjis)
564#define CATEGORY_MASK_BIG5 (1 << coding_category_big5)
565#define CATEGORY_MASK_CCL (1 << coding_category_ccl)
566#define CATEGORY_MASK_EMACS_MULE (1 << coding_category_emacs_mule)
ff0dacd7 567#define CATEGORY_MASK_RAW_TEXT (1 << coding_category_raw_text)
df7492f9
KH		 568
569/* This value is returned if detect_coding_mask () find nothing other
570 than ASCII characters. */
571#define CATEGORY_MASK_ANY \
572 (CATEGORY_MASK_ISO_7 \
573 | CATEGORY_MASK_ISO_7_TIGHT \
574 | CATEGORY_MASK_ISO_8_1 \
575 | CATEGORY_MASK_ISO_8_2 \
576 | CATEGORY_MASK_ISO_7_ELSE \
577 | CATEGORY_MASK_ISO_8_ELSE \
a470d443
KH		 578 | CATEGORY_MASK_UTF_8_AUTO \
579 | CATEGORY_MASK_UTF_8_NOSIG \
580 | CATEGORY_MASK_UTF_8_SIG \
2f3cbb32 581 | CATEGORY_MASK_UTF_16_AUTO \
df7492f9
KH		 582 | CATEGORY_MASK_UTF_16_BE \
583 | CATEGORY_MASK_UTF_16_LE \
584 | CATEGORY_MASK_UTF_16_BE_NOSIG \
585 | CATEGORY_MASK_UTF_16_LE_NOSIG \
586 | CATEGORY_MASK_CHARSET \
587 | CATEGORY_MASK_SJIS \
588 | CATEGORY_MASK_BIG5 \
589 | CATEGORY_MASK_CCL \
590 | CATEGORY_MASK_EMACS_MULE)
591
592
593#define CATEGORY_MASK_ISO_7BIT \
594 (CATEGORY_MASK_ISO_7 | CATEGORY_MASK_ISO_7_TIGHT)
595
596#define CATEGORY_MASK_ISO_8BIT \
597 (CATEGORY_MASK_ISO_8_1 | CATEGORY_MASK_ISO_8_2)
598
599#define CATEGORY_MASK_ISO_ELSE \
600 (CATEGORY_MASK_ISO_7_ELSE | CATEGORY_MASK_ISO_8_ELSE)
601
602#define CATEGORY_MASK_ISO_ESCAPE \
603 (CATEGORY_MASK_ISO_7 \
604 | CATEGORY_MASK_ISO_7_TIGHT \
605 | CATEGORY_MASK_ISO_7_ELSE \
606 | CATEGORY_MASK_ISO_8_ELSE)
607
608#define CATEGORY_MASK_ISO \
609 ( CATEGORY_MASK_ISO_7BIT \
610 | CATEGORY_MASK_ISO_8BIT \
611 | CATEGORY_MASK_ISO_ELSE)
612
613#define CATEGORY_MASK_UTF_16 \
2f3cbb32
KH		 614 (CATEGORY_MASK_UTF_16_AUTO \
615 | CATEGORY_MASK_UTF_16_BE \
df7492f9
KH		 616 | CATEGORY_MASK_UTF_16_LE \
617 | CATEGORY_MASK_UTF_16_BE_NOSIG \
618 | CATEGORY_MASK_UTF_16_LE_NOSIG)
619
a470d443
KH		 620#define CATEGORY_MASK_UTF_8 \
621 (CATEGORY_MASK_UTF_8_AUTO \
622 | CATEGORY_MASK_UTF_8_NOSIG \
623 | CATEGORY_MASK_UTF_8_SIG)
df7492f9 624
df7492f9 625/* Table of coding categories (Lisp symbols). This variable is for
ad1746f5 626 internal use only. */
df7492f9
KH		 627static Lisp_Object Vcoding_category_table;
628
629/* Table of coding-categories ordered by priority. */
630static enum coding_category coding_priorities[coding_category_max];
631
632/* Nth element is a coding context for the coding system bound to the
633 Nth coding category. */
634static struct coding_system coding_categories[coding_category_max];
635
df7492f9
KH		 636/*** Commonly used macros and functions ***/
637
638#ifndef min
639#define min(a, b) ((a) < (b) ? (a) : (b))
640#endif
641#ifndef max
642#define max(a, b) ((a) > (b) ? (a) : (b))
643#endif
4ed46869 644
24a73b0a
KH		 645#define CODING_GET_INFO(coding, attrs, charset_list) \
646 do { \
647 (attrs) = CODING_ID_ATTRS ((coding)->id); \
648 (charset_list) = CODING_ATTR_CHARSET_LIST (attrs); \
df7492f9 649 } while (0)
4ed46869 650
4ed46869 651
df7492f9
KH		 652/* Safely get one byte from the source text pointed by SRC which ends
653 at SRC_END, and set C to that byte. If there are not enough bytes
065e3595
KH		 654 in the source, it jumps to `no_more_source'. If multibytep is
655 nonzero, and a multibyte character is found at SRC, set C to the
656 negative value of the character code. The caller should declare
657 and set these variables appropriately in advance:
658 src, src_end, multibytep */
aa72b389 659
065e3595
KH		 660#define ONE_MORE_BYTE(c) \
661 do { \
662 if (src == src_end) \
663 { \
664 if (src_base < src) \
665 record_conversion_result \
666 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
667 goto no_more_source; \
668 } \
669 c = *src++; \
670 if (multibytep && (c & 0x80)) \
671 { \
672 if ((c & 0xFE) == 0xC0) \
673 c = ((c & 1) << 6) | *src++; \
674 else \
675 { \
35befdaa
KH		 676 src--; \
677 c = - string_char (src, &src, NULL); \
065e3595
KH		 678 record_conversion_result \
679 (coding, CODING_RESULT_INVALID_SRC); \
680 } \
681 } \
682 consumed_chars++; \
aa72b389
KH		 683 } while (0)
684
f56a4450 685/* Safely get two bytes from the source text pointed by SRC which ends
220eeac9
KH		 686 at SRC_END, and set C1 and C2 to those bytes while skipping the
687 heading multibyte characters. If there are not enough bytes in the
688 source, it jumps to `no_more_source'. If multibytep is nonzero and
689 a multibyte character is found for C2, set C2 to the negative value
690 of the character code. The caller should declare and set these
691 variables appropriately in advance:
f56a4450
KH		 692 src, src_end, multibytep
693 It is intended that this macro is used in detect_coding_utf_16. */
694
220eeac9
KH		 695#define TWO_MORE_BYTES(c1, c2) \
696 do { \
697 do { \
698 if (src == src_end) \
699 goto no_more_source; \
700 c1 = *src++; \
701 if (multibytep && (c1 & 0x80)) \
702 { \
703 if ((c1 & 0xFE) == 0xC0) \
704 c1 = ((c1 & 1) << 6) | *src++; \
705 else \
706 { \
707 src += BYTES_BY_CHAR_HEAD (c1) - 1; \
708 c1 = -1; \
709 } \
710 } \
711 } while (c1 < 0); \
712 if (src == src_end) \
713 goto no_more_source; \
714 c2 = *src++; \
715 if (multibytep && (c2 & 0x80)) \
716 { \
717 if ((c2 & 0xFE) == 0xC0) \
718 c2 = ((c2 & 1) << 6) | *src++; \
719 else \
720 c2 = -1; \
721 } \
f56a4450
KH		 722 } while (0)
723
aa72b389 724
df7492f9
KH		 725/* Store a byte C in the place pointed by DST and increment DST to the
726 next free point, and increment PRODUCED_CHARS. The caller should
727 assure that C is 0..127, and declare and set the variable `dst'
728 appropriately in advance.
729*/
aa72b389
KH		 730
731
df7492f9
KH		 732#define EMIT_ONE_ASCII_BYTE(c) \
733 do { \
734 produced_chars++; \
735 *dst++ = (c); \
b6871cc7 736 } while (0)
aa72b389
KH		 737
738
ad1746f5 739/* Like EMIT_ONE_ASCII_BYTE but store two bytes; C1 and C2. */
aa72b389 740
df7492f9
KH		 741#define EMIT_TWO_ASCII_BYTES(c1, c2) \
742 do { \
743 produced_chars += 2; \
744 *dst++ = (c1), *dst++ = (c2); \
745 } while (0)
aa72b389
KH		 746
747
df7492f9
KH		 748/* Store a byte C in the place pointed by DST and increment DST to the
749 next free point, and increment PRODUCED_CHARS. If MULTIBYTEP is
750 nonzero, store in an appropriate multibyte from. The caller should
751 declare and set the variables `dst' and `multibytep' appropriately
752 in advance. */
753
754#define EMIT_ONE_BYTE(c) \
755 do { \
756 produced_chars++; \
757 if (multibytep) \
758 { \
b25d760e 759 unsigned ch = (c); \
df7492f9
KH		 760 if (ch >= 0x80) \
761 ch = BYTE8_TO_CHAR (ch); \
762 CHAR_STRING_ADVANCE (ch, dst); \
763 } \
764 else \
765 *dst++ = (c); \
aa72b389 766 } while (0)
aa72b389 767
aa72b389 768
df7492f9 769/* Like EMIT_ONE_BYTE, but emit two bytes; C1 and C2. */
aa72b389 770
e19c3639
KH		 771#define EMIT_TWO_BYTES(c1, c2) \
772 do { \
773 produced_chars += 2; \
774 if (multibytep) \
775 { \
b25d760e 776 unsigned ch; \
e19c3639
KH		 777 \
778 ch = (c1); \
779 if (ch >= 0x80) \
780 ch = BYTE8_TO_CHAR (ch); \
781 CHAR_STRING_ADVANCE (ch, dst); \
782 ch = (c2); \
783 if (ch >= 0x80) \
784 ch = BYTE8_TO_CHAR (ch); \
785 CHAR_STRING_ADVANCE (ch, dst); \
786 } \
787 else \
788 { \
789 *dst++ = (c1); \
790 *dst++ = (c2); \
791 } \
aa72b389
KH		 792 } while (0)
793
794
df7492f9
KH		 795#define EMIT_THREE_BYTES(c1, c2, c3) \
796 do { \
797 EMIT_ONE_BYTE (c1); \
798 EMIT_TWO_BYTES (c2, c3); \
799 } while (0)
aa72b389 800
aa72b389 801
df7492f9
KH		 802#define EMIT_FOUR_BYTES(c1, c2, c3, c4) \
803 do { \
804 EMIT_TWO_BYTES (c1, c2); \
805 EMIT_TWO_BYTES (c3, c4); \
806 } while (0)
aa72b389 807
aa72b389 808
f6cbaf43 809/* Prototypes for static functions. */
f57e2426
J		 810static void record_conversion_result (struct coding_system *coding,
811 enum coding_result_code result);
812static int detect_coding_utf_8 (struct coding_system *,
813 struct coding_detection_info *info);
814static void decode_coding_utf_8 (struct coding_system *);
815static int encode_coding_utf_8 (struct coding_system *);
816
817static int detect_coding_utf_16 (struct coding_system *,
818 struct coding_detection_info *info);
819static void decode_coding_utf_16 (struct coding_system *);
820static int encode_coding_utf_16 (struct coding_system *);
821
822static int detect_coding_iso_2022 (struct coding_system *,
823 struct coding_detection_info *info);
824static void decode_coding_iso_2022 (struct coding_system *);
825static int encode_coding_iso_2022 (struct coding_system *);
826
827static int detect_coding_emacs_mule (struct coding_system *,
828 struct coding_detection_info *info);
829static void decode_coding_emacs_mule (struct coding_system *);
830static int encode_coding_emacs_mule (struct coding_system *);
831
832static int detect_coding_sjis (struct coding_system *,
833 struct coding_detection_info *info);
834static void decode_coding_sjis (struct coding_system *);
835static int encode_coding_sjis (struct coding_system *);
836
837static int detect_coding_big5 (struct coding_system *,
838 struct coding_detection_info *info);
839static void decode_coding_big5 (struct coding_system *);
840static int encode_coding_big5 (struct coding_system *);
841
842static int detect_coding_ccl (struct coding_system *,
843 struct coding_detection_info *info);
844static void decode_coding_ccl (struct coding_system *);
845static int encode_coding_ccl (struct coding_system *);
846
847static void decode_coding_raw_text (struct coding_system *);
848static int encode_coding_raw_text (struct coding_system *);
849
850static void coding_set_source (struct coding_system *);
851static void coding_set_destination (struct coding_system *);
d311d28c 852static void coding_alloc_by_realloc (struct coding_system *, ptrdiff_t);
f57e2426 853static void coding_alloc_by_making_gap (struct coding_system *,
d311d28c 854 ptrdiff_t, ptrdiff_t);
f57e2426 855static unsigned char *alloc_destination (struct coding_system *,
d311d28c 856 ptrdiff_t, unsigned char *);
f57e2426
J		 857static void setup_iso_safe_charsets (Lisp_Object);
858static unsigned char *encode_designation_at_bol (struct coding_system *,
461c2ab9 859 int *, unsigned char *);
f57e2426 860static int detect_eol (const unsigned char *,
d311d28c 861 ptrdiff_t, enum coding_category);
f57e2426
J		 862static Lisp_Object adjust_coding_eol_type (struct coding_system *, int);
863static void decode_eol (struct coding_system *);
864static Lisp_Object get_translation_table (Lisp_Object, int, int *);
865static Lisp_Object get_translation (Lisp_Object, int *, int *);
866static int produce_chars (struct coding_system *, Lisp_Object, int);
55d4c1b2 867static inline void produce_charset (struct coding_system *, int *,
d311d28c
PE		 868 ptrdiff_t);
869static void produce_annotation (struct coding_system *, ptrdiff_t);
f57e2426 870static int decode_coding (struct coding_system *);
d311d28c 871static inline int *handle_composition_annotation (ptrdiff_t, ptrdiff_t,
f57e2426 872 struct coding_system *,
d311d28c
PE		 873 int *, ptrdiff_t *);
874static inline int *handle_charset_annotation (ptrdiff_t, ptrdiff_t,
f57e2426 875 struct coding_system *,
d311d28c 876 int *, ptrdiff_t *);
f57e2426
J		 877static void consume_chars (struct coding_system *, Lisp_Object, int);
878static int encode_coding (struct coding_system *);
879static Lisp_Object make_conversion_work_buffer (int);
880static Lisp_Object code_conversion_restore (Lisp_Object);
55d4c1b2 881static inline int char_encodable_p (int, Lisp_Object);
f57e2426 882static Lisp_Object make_subsidiaries (Lisp_Object);
f6cbaf43 883
065e3595
KH		 884static void
885record_conversion_result (struct coding_system *coding,
886 enum coding_result_code result)
887{
888 coding->result = result;
889 switch (result)
890 {
891 case CODING_RESULT_INSUFFICIENT_SRC:
892 Vlast_code_conversion_error = Qinsufficient_source;
893 break;
894 case CODING_RESULT_INCONSISTENT_EOL:
895 Vlast_code_conversion_error = Qinconsistent_eol;
896 break;
897 case CODING_RESULT_INVALID_SRC:
898 Vlast_code_conversion_error = Qinvalid_source;
899 break;
900 case CODING_RESULT_INTERRUPT:
901 Vlast_code_conversion_error = Qinterrupted;
902 break;
903 case CODING_RESULT_INSUFFICIENT_MEM:
904 Vlast_code_conversion_error = Qinsufficient_memory;
905 break;
ebaf11b6
KH		 906 case CODING_RESULT_INSUFFICIENT_DST:
907 /* Don't record this error in Vlast_code_conversion_error
908 because it happens just temporarily and is resolved when the
909 whole conversion is finished. */
910 break;
409ea3a1
AS		 911 case CODING_RESULT_SUCCESS:
912 break;
35befdaa
KH		 913 default:
914 Vlast_code_conversion_error = intern ("Unknown error");
065e3595
KH		 915 }
916}
917
75f80e63
EZ		 918/* This wrapper macro is used to preserve validity of pointers into
919 buffer text across calls to decode_char, which could cause
920 relocation of buffers if it loads a charset map, because loading a
921 charset map allocates large structures. */
df7492f9
KH		 922#define CODING_DECODE_CHAR(coding, src, src_base, src_end, charset, code, c) \
923 do { \
924 charset_map_loaded = 0; \
925 c = DECODE_CHAR (charset, code); \
926 if (charset_map_loaded) \
927 { \
8f924df7 928 const unsigned char *orig = coding->source; \
d311d28c 929 ptrdiff_t offset; \
df7492f9
KH		 930 \
931 coding_set_source (coding); \
932 offset = coding->source - orig; \
933 src += offset; \
934 src_base += offset; \
935 src_end += offset; \
936 } \
aa72b389
KH		 937 } while (0)
938
939
119852e7
KH		 940/* If there are at least BYTES length of room at dst, allocate memory
941 for coding->destination and update dst and dst_end. We don't have
942 to take care of coding->source which will be relocated. It is
943 handled by calling coding_set_source in encode_coding. */
944
df7492f9
KH		 945#define ASSURE_DESTINATION(bytes) \
946 do { \
947 if (dst + (bytes) >= dst_end) \
948 { \
d311d28c 949 ptrdiff_t more_bytes = charbuf_end - charbuf + (bytes); \
df7492f9
KH		 950 \
951 dst = alloc_destination (coding, more_bytes, dst); \
952 dst_end = coding->destination + coding->dst_bytes; \
953 } \
954 } while (0)
aa72b389 955
aa72b389 956
db274c7a
KH		 957/* Store multibyte form of the character C in P, and advance P to the
958 end of the multibyte form. This is like CHAR_STRING_ADVANCE but it
959 never calls MAYBE_UNIFY_CHAR. */
960
961#define CHAR_STRING_ADVANCE_NO_UNIFY(c, p) \
962 do { \
963 if ((c) <= MAX_1_BYTE_CHAR) \
964 *(p)++ = (c); \
965 else if ((c) <= MAX_2_BYTE_CHAR) \
966 *(p)++ = (0xC0 | ((c) >> 6)), \
967 *(p)++ = (0x80 | ((c) & 0x3F)); \
968 else if ((c) <= MAX_3_BYTE_CHAR) \
969 *(p)++ = (0xE0 | ((c) >> 12)), \
970 *(p)++ = (0x80 | (((c) >> 6) & 0x3F)), \
971 *(p)++ = (0x80 | ((c) & 0x3F)); \
972 else if ((c) <= MAX_4_BYTE_CHAR) \
973 *(p)++ = (0xF0 | (c >> 18)), \
974 *(p)++ = (0x80 | ((c >> 12) & 0x3F)), \
975 *(p)++ = (0x80 | ((c >> 6) & 0x3F)), \
976 *(p)++ = (0x80 | (c & 0x3F)); \
977 else if ((c) <= MAX_5_BYTE_CHAR) \
978 *(p)++ = 0xF8, \
979 *(p)++ = (0x80 | ((c >> 18) & 0x0F)), \
980 *(p)++ = (0x80 | ((c >> 12) & 0x3F)), \
981 *(p)++ = (0x80 | ((c >> 6) & 0x3F)), \
982 *(p)++ = (0x80 | (c & 0x3F)); \
983 else \
984 (p) += BYTE8_STRING ((c) - 0x3FFF80, p); \
985 } while (0)
986
987
988/* Return the character code of character whose multibyte form is at
989 P, and advance P to the end of the multibyte form. This is like
990 STRING_CHAR_ADVANCE, but it never calls MAYBE_UNIFY_CHAR. */
991
992#define STRING_CHAR_ADVANCE_NO_UNIFY(p) \
993 (!((p)[0] & 0x80) \
994 ? *(p)++ \
995 : ! ((p)[0] & 0x20) \
996 ? ((p) += 2, \
997 ((((p)[-2] & 0x1F) << 6) \
998 | ((p)[-1] & 0x3F) \
999 | ((unsigned char) ((p)[-2]) < 0xC2 ? 0x3FFF80 : 0))) \
1000 : ! ((p)[0] & 0x10) \
1001 ? ((p) += 3, \
1002 ((((p)[-3] & 0x0F) << 12) \
1003 | (((p)[-2] & 0x3F) << 6) \
1004 | ((p)[-1] & 0x3F))) \
1005 : ! ((p)[0] & 0x08) \
1006 ? ((p) += 4, \
1007 ((((p)[-4] & 0xF) << 18) \
1008 | (((p)[-3] & 0x3F) << 12) \
1009 | (((p)[-2] & 0x3F) << 6) \
1010 | ((p)[-1] & 0x3F))) \
1011 : ((p) += 5, \
1012 ((((p)[-4] & 0x3F) << 18) \
1013 | (((p)[-3] & 0x3F) << 12) \
1014 | (((p)[-2] & 0x3F) << 6) \
1015 | ((p)[-1] & 0x3F))))
1016
aa72b389 1017
df7492f9 1018static void
971de7fb 1019coding_set_source (struct coding_system *coding)
aa72b389 1020{
df7492f9
KH		 1021 if (BUFFERP (coding->src_object))
1022 {
2cb26057 1023 struct buffer *buf = XBUFFER (coding->src_object);
aa72b389 1024
df7492f9 1025 if (coding->src_pos < 0)
2cb26057 1026 coding->source = BUF_GAP_END_ADDR (buf) + coding->src_pos_byte;
df7492f9 1027 else
2cb26057 1028 coding->source = BUF_BYTE_ADDRESS (buf, coding->src_pos_byte);
aa72b389 1029 }
df7492f9 1030 else if (STRINGP (coding->src_object))
aa72b389 1031 {
8f924df7 1032 coding->source = SDATA (coding->src_object) + coding->src_pos_byte;
aa72b389 1033 }
df7492f9 1034 else
f38b440c
PE		 1035 {
1036 /* Otherwise, the source is C string and is never relocated
1037 automatically. Thus we don't have to update anything. */
1038 }
df7492f9 1039}
aa72b389 1040
df7492f9 1041static void
971de7fb 1042coding_set_destination (struct coding_system *coding)
df7492f9
KH		 1043{
1044 if (BUFFERP (coding->dst_object))
aa72b389 1045 {
df7492f9 1046 if (coding->src_pos < 0)
aa72b389 1047 {
13818c30 1048 coding->destination = BEG_ADDR + coding->dst_pos_byte - BEG_BYTE;
28f67a95
KH		 1049 coding->dst_bytes = (GAP_END_ADDR
1050 - (coding->src_bytes - coding->consumed)
1051 - coding->destination);
aa72b389 1052 }
df7492f9 1053 else
28f67a95
KH		 1054 {
1055 /* We are sure that coding->dst_pos_byte is before the gap
1056 of the buffer. */
1057 coding->destination = (BUF_BEG_ADDR (XBUFFER (coding->dst_object))
13818c30 1058 + coding->dst_pos_byte - BEG_BYTE);
28f67a95
KH		 1059 coding->dst_bytes = (BUF_GAP_END_ADDR (XBUFFER (coding->dst_object))
1060 - coding->destination);
1061 }
df7492f9
KH		 1062 }
1063 else
f38b440c
PE		 1064 {
1065 /* Otherwise, the destination is C string and is never relocated
1066 automatically. Thus we don't have to update anything. */
1067 }
df7492f9
KH		 1068}
1069
1070
1071static void
d311d28c 1072coding_alloc_by_realloc (struct coding_system *coding, ptrdiff_t bytes)
df7492f9 1073{
c9d624c6 1074 if (STRING_BYTES_BOUND - coding->dst_bytes < bytes)
d1f3d2af 1075 string_overflow ();
df7492f9
KH		 1076 coding->destination = (unsigned char *) xrealloc (coding->destination,
1077 coding->dst_bytes + bytes);
1078 coding->dst_bytes += bytes;
1079}
1080
1081static void
cf84bb53 1082coding_alloc_by_making_gap (struct coding_system *coding,
d311d28c 1083 ptrdiff_t gap_head_used, ptrdiff_t bytes)
df7492f9 1084{
db274c7a 1085 if (EQ (coding->src_object, coding->dst_object))
df7492f9 1086 {
db274c7a
KH		 1087 /* The gap may contain the produced data at the head and not-yet
1088 consumed data at the tail. To preserve those data, we at
1089 first make the gap size to zero, then increase the gap
1090 size. */
d311d28c 1091 ptrdiff_t add = GAP_SIZE;
db274c7a
KH		 1092
1093 GPT += gap_head_used, GPT_BYTE += gap_head_used;
1094 GAP_SIZE = 0; ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
df7492f9
KH		 1095 make_gap (bytes);
1096 GAP_SIZE += add; ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
db274c7a 1097 GPT -= gap_head_used, GPT_BYTE -= gap_head_used;
df7492f9 1098 }
730fff51 1099 else
df7492f9 1100 {
2c78b7e1
KH		 1101 Lisp_Object this_buffer;
1102
1103 this_buffer = Fcurrent_buffer ();
df7492f9
KH		 1104 set_buffer_internal (XBUFFER (coding->dst_object));
1105 make_gap (bytes);
1106 set_buffer_internal (XBUFFER (this_buffer));
aa72b389 1107 }
df7492f9 1108}
8f924df7 1109
df7492f9
KH		 1110
1111static unsigned char *
d311d28c 1112alloc_destination (struct coding_system *coding, ptrdiff_t nbytes,
cf84bb53 1113 unsigned char *dst)
df7492f9 1114{
d311d28c 1115 ptrdiff_t offset = dst - coding->destination;
df7492f9
KH		 1116
1117 if (BUFFERP (coding->dst_object))
db274c7a
KH		 1118 {
1119 struct buffer *buf = XBUFFER (coding->dst_object);
1120
1121 coding_alloc_by_making_gap (coding, dst - BUF_GPT_ADDR (buf), nbytes);
1122 }
aa72b389 1123 else
df7492f9 1124 coding_alloc_by_realloc (coding, nbytes);
df7492f9
KH		 1125 coding_set_destination (coding);
1126 dst = coding->destination + offset;
1127 return dst;
1128}
aa72b389 1129
ff0dacd7
KH		 1130/** Macros for annotations. */
1131
ff0dacd7
KH		 1132/* An annotation data is stored in the array coding->charbuf in this
1133 format:
69a80ea3 1134 [ -LENGTH ANNOTATION_MASK NCHARS ... ]
ff0dacd7
KH		 1135 LENGTH is the number of elements in the annotation.
1136 ANNOTATION_MASK is one of CODING_ANNOTATE_XXX_MASK.
69a80ea3 1137 NCHARS is the number of characters in the text annotated.
ff0dacd7
KH		 1138
1139 The format of the following elements depend on ANNOTATION_MASK.
1140
1141 In the case of CODING_ANNOTATE_COMPOSITION_MASK, these elements
1142 follows:
e951386e
KH		 1143 ... NBYTES METHOD [ COMPOSITION-COMPONENTS ... ]
1144
1145 NBYTES is the number of bytes specified in the header part of
1146 old-style emacs-mule encoding, or 0 for the other kind of
1147 composition.
1148
ff0dacd7 1149 METHOD is one of enum composition_method.
e951386e 1150
ad1746f5 1151 Optional COMPOSITION-COMPONENTS are characters and composition
ff0dacd7
KH		 1152 rules.
1153
1154 In the case of CODING_ANNOTATE_CHARSET_MASK, one element CHARSET-ID
e951386e
KH		 1155 follows.
1156
1157 If ANNOTATION_MASK is 0, this annotation is just a space holder to
1158 recover from an invalid annotation, and should be skipped by
1159 produce_annotation. */
1160
1161/* Maximum length of the header of annotation data. */
1162#define MAX_ANNOTATION_LENGTH 5
ff0dacd7 1163
69a80ea3 1164#define ADD_ANNOTATION_DATA(buf, len, mask, nchars) \
ff0dacd7
KH		 1165 do { \
1166 *(buf)++ = -(len); \
1167 *(buf)++ = (mask); \
69a80ea3 1168 *(buf)++ = (nchars); \
ff0dacd7
KH		 1169 coding->annotated = 1; \
1170 } while (0);
1171
e951386e 1172#define ADD_COMPOSITION_DATA(buf, nchars, nbytes, method) \
69a80ea3 1173 do { \
e951386e
KH		 1174 ADD_ANNOTATION_DATA (buf, 5, CODING_ANNOTATE_COMPOSITION_MASK, nchars); \
1175 *buf++ = nbytes; \
69a80ea3 1176 *buf++ = method; \
ff0dacd7
KH		 1177 } while (0)
1178
1179
69a80ea3
KH		 1180#define ADD_CHARSET_DATA(buf, nchars, id) \
1181 do { \
1182 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_CHARSET_MASK, nchars); \
1183 *buf++ = id; \
ff0dacd7
KH		 1184 } while (0)
1185
df7492f9
KH		 1186\f
1187/*** 2. Emacs' internal format (emacs-utf-8) ***/
1188
1189
1190
1191\f
1192/*** 3. UTF-8 ***/
1193
1194/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1195 Check if a text is encoded in UTF-8. If it is, return 1, else
1196 return 0. */
df7492f9
KH		 1197
1198#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
1199#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
1200#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
1201#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
1202#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
1203#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
1204
a470d443
KH		 1205#define UTF_8_BOM_1 0xEF
1206#define UTF_8_BOM_2 0xBB
1207#define UTF_8_BOM_3 0xBF
1208
df7492f9 1209static int
cf84bb53
JB		 1210detect_coding_utf_8 (struct coding_system *coding,
1211 struct coding_detection_info *detect_info)
df7492f9 1212{
065e3595 1213 const unsigned char *src = coding->source, *src_base;
8f924df7 1214 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 1215 int multibytep = coding->src_multibyte;
d311d28c 1216 ptrdiff_t consumed_chars = 0;
a470d443 1217 int bom_found = 0;
df7492f9
KH		 1218 int found = 0;
1219
ff0dacd7 1220 detect_info->checked |= CATEGORY_MASK_UTF_8;
df7492f9
KH		 1221 /* A coding system of this category is always ASCII compatible. */
1222 src += coding->head_ascii;
1223
1224 while (1)
aa72b389 1225 {
df7492f9 1226 int c, c1, c2, c3, c4;
aa72b389 1227
065e3595 1228 src_base = src;
df7492f9 1229 ONE_MORE_BYTE (c);
065e3595 1230 if (c < 0 || UTF_8_1_OCTET_P (c))
df7492f9
KH		 1231 continue;
1232 ONE_MORE_BYTE (c1);
065e3595 1233 if (c1 < 0 || ! UTF_8_EXTRA_OCTET_P (c1))
df7492f9
KH		 1234 break;
1235 if (UTF_8_2_OCTET_LEADING_P (c))
aa72b389 1236 {
a470d443 1237 found = 1;
df7492f9 1238 continue;
aa72b389 1239 }
df7492f9 1240 ONE_MORE_BYTE (c2);
065e3595 1241 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1242 break;
1243 if (UTF_8_3_OCTET_LEADING_P (c))
aa72b389 1244 {
a470d443
KH		 1245 found = 1;
1246 if (src_base == coding->source
1247 && c == UTF_8_BOM_1 && c1 == UTF_8_BOM_2 && c2 == UTF_8_BOM_3)
1248 bom_found = 1;
df7492f9 1249 continue;
aa72b389 1250 }
df7492f9 1251 ONE_MORE_BYTE (c3);
065e3595 1252 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1253 break;
1254 if (UTF_8_4_OCTET_LEADING_P (c))
aa72b389 1255 {
a470d443 1256 found = 1;
df7492f9
KH		 1257 continue;
1258 }
1259 ONE_MORE_BYTE (c4);
065e3595 1260 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1261 break;
1262 if (UTF_8_5_OCTET_LEADING_P (c))
1263 {
a470d443 1264 found = 1;
df7492f9
KH		 1265 continue;
1266 }
1267 break;
aa72b389 1268 }
ff0dacd7 1269 detect_info->rejected |= CATEGORY_MASK_UTF_8;
df7492f9 1270 return 0;
aa72b389 1271
df7492f9 1272 no_more_source:
065e3595 1273 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
aa72b389 1274 {
ff0dacd7 1275 detect_info->rejected |= CATEGORY_MASK_UTF_8;
89528eb3 1276 return 0;
aa72b389 1277 }
a470d443
KH		 1278 if (bom_found)
1279 {
1280 /* The first character 0xFFFE doesn't necessarily mean a BOM. */
1281 detect_info->found |= CATEGORY_MASK_UTF_8_SIG | CATEGORY_MASK_UTF_8_NOSIG;
1282 }
1283 else
1284 {
1285 detect_info->rejected |= CATEGORY_MASK_UTF_8_SIG;
0e17387a
KH		 1286 if (found)
1287 detect_info->found |= CATEGORY_MASK_UTF_8_NOSIG;
a470d443 1288 }
ff0dacd7 1289 return 1;
aa72b389
KH		 1290}
1291
4ed46869 1292
b73bfc1c 1293static void
971de7fb 1294decode_coding_utf_8 (struct coding_system *coding)
b73bfc1c 1295{
8f924df7
KH		 1296 const unsigned char *src = coding->source + coding->consumed;
1297 const unsigned char *src_end = coding->source + coding->src_bytes;
1298 const unsigned char *src_base;
69a80ea3
KH		 1299 int *charbuf = coding->charbuf + coding->charbuf_used;
1300 int *charbuf_end = coding->charbuf + coding->charbuf_size;
d311d28c 1301 ptrdiff_t consumed_chars = 0, consumed_chars_base = 0;
df7492f9 1302 int multibytep = coding->src_multibyte;
a470d443 1303 enum utf_bom_type bom = CODING_UTF_8_BOM (coding);
2735d060 1304 int eol_dos =
0a9564cb 1305 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 1306 int byte_after_cr = -1;
4ed46869 1307
a470d443
KH		 1308 if (bom != utf_without_bom)
1309 {
1310 int c1, c2, c3;
1311
1312 src_base = src;
1313 ONE_MORE_BYTE (c1);
1314 if (! UTF_8_3_OCTET_LEADING_P (c1))
1315 src = src_base;
1316 else
1317 {
159bd5a2 1318 ONE_MORE_BYTE (c2);
a470d443
KH		 1319 if (! UTF_8_EXTRA_OCTET_P (c2))
1320 src = src_base;
1321 else
1322 {
159bd5a2 1323 ONE_MORE_BYTE (c3);
a470d443
KH		 1324 if (! UTF_8_EXTRA_OCTET_P (c3))
1325 src = src_base;
1326 else
1327 {
1328 if ((c1 != UTF_8_BOM_1)
1329 || (c2 != UTF_8_BOM_2) || (c3 != UTF_8_BOM_3))
1330 src = src_base;
1331 else
1332 CODING_UTF_8_BOM (coding) = utf_without_bom;
1333 }
1334 }
1335 }
1336 }
1337 CODING_UTF_8_BOM (coding) = utf_without_bom;
1338
df7492f9 1339 while (1)
b73bfc1c 1340 {
df7492f9 1341 int c, c1, c2, c3, c4, c5;
ec6d2bb8 1342
df7492f9
KH		 1343 src_base = src;
1344 consumed_chars_base = consumed_chars;
4af310db 1345
df7492f9 1346 if (charbuf >= charbuf_end)
b71f6f73
KH		 1347 {
1348 if (byte_after_cr >= 0)
1349 src_base--;
1350 break;
1351 }
df7492f9 1352
119852e7
KH		 1353 if (byte_after_cr >= 0)
1354 c1 = byte_after_cr, byte_after_cr = -1;
1355 else
1356 ONE_MORE_BYTE (c1);
065e3595
KH		 1357 if (c1 < 0)
1358 {
1359 c = - c1;
1360 }
1a4990fb 1361 else if (UTF_8_1_OCTET_P (c1))
df7492f9 1362 {
2735d060 1363 if (eol_dos && c1 == '\r')
119852e7 1364 ONE_MORE_BYTE (byte_after_cr);
df7492f9 1365 c = c1;
4af310db 1366 }
df7492f9 1367 else
4af310db 1368 {
df7492f9 1369 ONE_MORE_BYTE (c2);
065e3595 1370 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1371 goto invalid_code;
1372 if (UTF_8_2_OCTET_LEADING_P (c1))
4af310db 1373 {
b0edb2c5
DL		 1374 c = ((c1 & 0x1F) << 6) | (c2 & 0x3F);
1375 /* Reject overlong sequences here and below. Encoders
1376 producing them are incorrect, they can be misleading,
1377 and they mess up read/write invariance. */
1378 if (c < 128)
1379 goto invalid_code;
4af310db 1380 }
df7492f9 1381 else
aa72b389 1382 {
df7492f9 1383 ONE_MORE_BYTE (c3);
065e3595 1384 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1385 goto invalid_code;
1386 if (UTF_8_3_OCTET_LEADING_P (c1))
b0edb2c5
DL		 1387 {
1388 c = (((c1 & 0xF) << 12)
1389 | ((c2 & 0x3F) << 6) | (c3 & 0x3F));
72fe1301
DL		 1390 if (c < 0x800
1391 || (c >= 0xd800 && c < 0xe000)) /* surrogates (invalid) */
b0edb2c5
DL		 1392 goto invalid_code;
1393 }
df7492f9
KH		 1394 else
1395 {
1396 ONE_MORE_BYTE (c4);
065e3595 1397 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1398 goto invalid_code;
1399 if (UTF_8_4_OCTET_LEADING_P (c1))
b0edb2c5 1400 {
df7492f9
KH		 1401 c = (((c1 & 0x7) << 18) | ((c2 & 0x3F) << 12)
1402 | ((c3 & 0x3F) << 6) | (c4 & 0x3F));
b0edb2c5
DL		 1403 if (c < 0x10000)
1404 goto invalid_code;
1405 }
df7492f9
KH		 1406 else
1407 {
1408 ONE_MORE_BYTE (c5);
065e3595 1409 if (c5 < 0 || ! UTF_8_EXTRA_OCTET_P (c5))
df7492f9
KH		 1410 goto invalid_code;
1411 if (UTF_8_5_OCTET_LEADING_P (c1))
1412 {
1413 c = (((c1 & 0x3) << 24) | ((c2 & 0x3F) << 18)
1414 | ((c3 & 0x3F) << 12) | ((c4 & 0x3F) << 6)
1415 | (c5 & 0x3F));
b0edb2c5 1416 if ((c > MAX_CHAR) || (c < 0x200000))
df7492f9
KH		 1417 goto invalid_code;
1418 }
1419 else
1420 goto invalid_code;
1421 }
1422 }
aa72b389 1423 }
b73bfc1c 1424 }
df7492f9
KH		 1425
1426 *charbuf++ = c;
1427 continue;
1428
1429 invalid_code:
1430 src = src_base;
1431 consumed_chars = consumed_chars_base;
1432 ONE_MORE_BYTE (c);
1433 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
1434 coding->errors++;
aa72b389
KH		 1435 }
1436
df7492f9
KH		 1437 no_more_source:
1438 coding->consumed_char += consumed_chars_base;
1439 coding->consumed = src_base - coding->source;
1440 coding->charbuf_used = charbuf - coding->charbuf;
1441}
1442
1443
1444static int
971de7fb 1445encode_coding_utf_8 (struct coding_system *coding)
df7492f9
KH		 1446{
1447 int multibytep = coding->dst_multibyte;
1448 int *charbuf = coding->charbuf;
1449 int *charbuf_end = charbuf + coding->charbuf_used;
1450 unsigned char *dst = coding->destination + coding->produced;
1451 unsigned char *dst_end = coding->destination + coding->dst_bytes;
d311d28c 1452 ptrdiff_t produced_chars = 0;
df7492f9
KH		 1453 int c;
1454
a470d443
KH		 1455 if (CODING_UTF_8_BOM (coding) == utf_with_bom)
1456 {
1457 ASSURE_DESTINATION (3);
1458 EMIT_THREE_BYTES (UTF_8_BOM_1, UTF_8_BOM_2, UTF_8_BOM_3);
1459 CODING_UTF_8_BOM (coding) = utf_without_bom;
1460 }
1461
df7492f9 1462 if (multibytep)
aa72b389 1463 {
df7492f9
KH		 1464 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
1465
1466 while (charbuf < charbuf_end)
b73bfc1c 1467 {
df7492f9 1468 unsigned char str[MAX_MULTIBYTE_LENGTH], *p, *pend = str;
8f924df7 1469
df7492f9
KH		 1470 ASSURE_DESTINATION (safe_room);
1471 c = *charbuf++;
28f67a95
KH		 1472 if (CHAR_BYTE8_P (c))
1473 {
1474 c = CHAR_TO_BYTE8 (c);
1475 EMIT_ONE_BYTE (c);
1476 }
1477 else
1478 {
db274c7a 1479 CHAR_STRING_ADVANCE_NO_UNIFY (c, pend);
28f67a95
KH		 1480 for (p = str; p < pend; p++)
1481 EMIT_ONE_BYTE (*p);
1482 }
b73bfc1c 1483 }
aa72b389 1484 }
df7492f9
KH		 1485 else
1486 {
1487 int safe_room = MAX_MULTIBYTE_LENGTH;
1488
1489 while (charbuf < charbuf_end)
b73bfc1c 1490 {
df7492f9
KH		 1491 ASSURE_DESTINATION (safe_room);
1492 c = *charbuf++;
f03caae0
KH		 1493 if (CHAR_BYTE8_P (c))
1494 *dst++ = CHAR_TO_BYTE8 (c);
1495 else
db274c7a 1496 CHAR_STRING_ADVANCE_NO_UNIFY (c, dst);
df7492f9 1497 produced_chars++;
4ed46869
KH		 1498 }
1499 }
065e3595 1500 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1501 coding->produced_char += produced_chars;
1502 coding->produced = dst - coding->destination;
1503 return 0;
4ed46869
KH		 1504}
1505
b73bfc1c 1506
df7492f9 1507/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1508 Check if a text is encoded in one of UTF-16 based coding systems.
1509 If it is, return 1, else return 0. */
aa72b389 1510
df7492f9
KH		 1511#define UTF_16_HIGH_SURROGATE_P(val) \
1512 (((val) & 0xFC00) == 0xD800)
1513
1514#define UTF_16_LOW_SURROGATE_P(val) \
1515 (((val) & 0xFC00) == 0xDC00)
93dec019 1516
aa72b389 1517
df7492f9 1518static int
cf84bb53
JB		 1519detect_coding_utf_16 (struct coding_system *coding,
1520 struct coding_detection_info *detect_info)
aa72b389 1521{
ef1b0ba7 1522 const unsigned char *src = coding->source;
8f924df7 1523 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 1524 int multibytep = coding->src_multibyte;
df7492f9 1525 int c1, c2;
aa72b389 1526
ff0dacd7 1527 detect_info->checked |= CATEGORY_MASK_UTF_16;
ff0dacd7 1528 if (coding->mode & CODING_MODE_LAST_BLOCK
24a73b0a 1529 && (coding->src_chars & 1))
ff0dacd7
KH		 1530 {
1531 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1532 return 0;
1533 }
24a73b0a 1534
f56a4450 1535 TWO_MORE_BYTES (c1, c2);
df7492f9 1536 if ((c1 == 0xFF) && (c2 == 0xFE))
aa72b389 1537 {
b49a1807
KH		 1538 detect_info->found |= (CATEGORY_MASK_UTF_16_LE
1539 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1540 detect_info->rejected |= (CATEGORY_MASK_UTF_16_BE
1541 | CATEGORY_MASK_UTF_16_BE_NOSIG
1542 | CATEGORY_MASK_UTF_16_LE_NOSIG);
aa72b389 1543 }
df7492f9 1544 else if ((c1 == 0xFE) && (c2 == 0xFF))
ff0dacd7 1545 {
b49a1807
KH		 1546 detect_info->found |= (CATEGORY_MASK_UTF_16_BE
1547 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1548 detect_info->rejected |= (CATEGORY_MASK_UTF_16_LE
1549 | CATEGORY_MASK_UTF_16_BE_NOSIG
1550 | CATEGORY_MASK_UTF_16_LE_NOSIG);
1551 }
220eeac9 1552 else if (c2 < 0)
f56a4450
KH		 1553 {
1554 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1555 return 0;
1556 }
2f3cbb32 1557 else
24a73b0a 1558 {
2f3cbb32
KH		 1559 /* We check the dispersion of Eth and Oth bytes where E is even and
1560 O is odd. If both are high, we assume binary data.*/
1561 unsigned char e[256], o[256];
1562 unsigned e_num = 1, o_num = 1;
1563
1564 memset (e, 0, 256);
1565 memset (o, 0, 256);
1566 e[c1] = 1;
1567 o[c2] = 1;
1568
cc13543e
KH		 1569 detect_info->rejected |= (CATEGORY_MASK_UTF_16_AUTO
1570 |CATEGORY_MASK_UTF_16_BE
1571 | CATEGORY_MASK_UTF_16_LE);
2f3cbb32 1572
7f1faf1c
KH		 1573 while ((detect_info->rejected & CATEGORY_MASK_UTF_16)
1574 != CATEGORY_MASK_UTF_16)
2f3cbb32 1575 {
f56a4450 1576 TWO_MORE_BYTES (c1, c2);
220eeac9 1577 if (c2 < 0)
f56a4450 1578 break;
2f3cbb32
KH		 1579 if (! e[c1])
1580 {
1581 e[c1] = 1;
1582 e_num++;
cc13543e
KH		 1583 if (e_num >= 128)
1584 detect_info->rejected |= CATEGORY_MASK_UTF_16_BE_NOSIG;
2f3cbb32
KH		 1585 }
1586 if (! o[c2])
1587 {
977b85f4 1588 o[c2] = 1;
2f3cbb32 1589 o_num++;
cc13543e
KH		 1590 if (o_num >= 128)
1591 detect_info->rejected |= CATEGORY_MASK_UTF_16_LE_NOSIG;
2f3cbb32
KH		 1592 }
1593 }
2f3cbb32 1594 return 0;
ff0dacd7 1595 }
2f3cbb32 1596
df7492f9 1597 no_more_source:
ff0dacd7 1598 return 1;
df7492f9 1599}
aa72b389 1600
df7492f9 1601static void
971de7fb 1602decode_coding_utf_16 (struct coding_system *coding)
df7492f9 1603{
8f924df7
KH		 1604 const unsigned char *src = coding->source + coding->consumed;
1605 const unsigned char *src_end = coding->source + coding->src_bytes;
1606 const unsigned char *src_base;
69a80ea3 1607 int *charbuf = coding->charbuf + coding->charbuf_used;
df80c7f0
KH		 1608 /* We may produces at most 3 chars in one loop. */
1609 int *charbuf_end = coding->charbuf + coding->charbuf_size - 2;
d311d28c 1610 ptrdiff_t consumed_chars = 0, consumed_chars_base = 0;
df7492f9 1611 int multibytep = coding->src_multibyte;
a470d443 1612 enum utf_bom_type bom = CODING_UTF_16_BOM (coding);
df7492f9
KH		 1613 enum utf_16_endian_type endian = CODING_UTF_16_ENDIAN (coding);
1614 int surrogate = CODING_UTF_16_SURROGATE (coding);
2735d060 1615 int eol_dos =
0a9564cb 1616 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 1617 int byte_after_cr1 = -1, byte_after_cr2 = -1;
df7492f9 1618
a470d443 1619 if (bom == utf_with_bom)
aa72b389 1620 {
df7492f9 1621 int c, c1, c2;
4af310db 1622
aa72b389 1623 src_base = src;
df7492f9
KH		 1624 ONE_MORE_BYTE (c1);
1625 ONE_MORE_BYTE (c2);
e19c3639 1626 c = (c1 << 8) | c2;
aa72b389 1627
b49a1807
KH		 1628 if (endian == utf_16_big_endian
1629 ? c != 0xFEFF : c != 0xFFFE)
aa72b389 1630 {
b49a1807
KH		 1631 /* The first two bytes are not BOM. Treat them as bytes
1632 for a normal character. */
1633 src = src_base;
1634 coding->errors++;
aa72b389 1635 }
a470d443 1636 CODING_UTF_16_BOM (coding) = utf_without_bom;
b49a1807 1637 }
a470d443 1638 else if (bom == utf_detect_bom)
b49a1807
KH		 1639 {
1640 /* We have already tried to detect BOM and failed in
1641 detect_coding. */
a470d443 1642 CODING_UTF_16_BOM (coding) = utf_without_bom;
df7492f9 1643 }
aa72b389 1644
df7492f9
KH		 1645 while (1)
1646 {
1647 int c, c1, c2;
1648
1649 src_base = src;
1650 consumed_chars_base = consumed_chars;
1651
df80c7f0 1652 if (charbuf >= charbuf_end)
b71f6f73
KH		 1653 {
1654 if (byte_after_cr1 >= 0)
1655 src_base -= 2;
1656 break;
1657 }
df7492f9 1658
119852e7
KH		 1659 if (byte_after_cr1 >= 0)
1660 c1 = byte_after_cr1, byte_after_cr1 = -1;
1661 else
1662 ONE_MORE_BYTE (c1);
065e3595
KH		 1663 if (c1 < 0)
1664 {
1665 *charbuf++ = -c1;
1666 continue;
1667 }
119852e7
KH		 1668 if (byte_after_cr2 >= 0)
1669 c2 = byte_after_cr2, byte_after_cr2 = -1;
1670 else
1671 ONE_MORE_BYTE (c2);
065e3595
KH		 1672 if (c2 < 0)
1673 {
1674 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
1675 *charbuf++ = -c2;
1676 continue;
1677 }
df7492f9 1678 c = (endian == utf_16_big_endian
e19c3639 1679 ? ((c1 << 8) | c2) : ((c2 << 8) | c1));
119852e7 1680
df7492f9 1681 if (surrogate)
fd3ae0b9 1682 {
df7492f9 1683 if (! UTF_16_LOW_SURROGATE_P (c))
fd3ae0b9 1684 {
df7492f9
KH		 1685 if (endian == utf_16_big_endian)
1686 c1 = surrogate >> 8, c2 = surrogate & 0xFF;
1687 else
1688 c1 = surrogate & 0xFF, c2 = surrogate >> 8;
1689 *charbuf++ = c1;
1690 *charbuf++ = c2;
1691 coding->errors++;
1692 if (UTF_16_HIGH_SURROGATE_P (c))
1693 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
fd3ae0b9 1694 else
df7492f9 1695 *charbuf++ = c;
fd3ae0b9
KH		 1696 }
1697 else
df7492f9
KH		 1698 {
1699 c = ((surrogate - 0xD800) << 10) | (c - 0xDC00);
1700 CODING_UTF_16_SURROGATE (coding) = surrogate = 0;
29f7ffd0 1701 *charbuf++ = 0x10000 + c;
df7492f9 1702 }
fd3ae0b9 1703 }
aa72b389 1704 else
df7492f9
KH		 1705 {
1706 if (UTF_16_HIGH_SURROGATE_P (c))
1707 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1708 else
119852e7 1709 {
2735d060 1710 if (eol_dos && c == '\r')
119852e7
KH		 1711 {
1712 ONE_MORE_BYTE (byte_after_cr1);
1713 ONE_MORE_BYTE (byte_after_cr2);
1714 }
1715 *charbuf++ = c;
1716 }
8f924df7 1717 }
aa72b389 1718 }
df7492f9
KH		 1719
1720 no_more_source:
1721 coding->consumed_char += consumed_chars_base;
1722 coding->consumed = src_base - coding->source;
1723 coding->charbuf_used = charbuf - coding->charbuf;
aa72b389 1724}
b73bfc1c 1725
df7492f9 1726static int
971de7fb 1727encode_coding_utf_16 (struct coding_system *coding)
df7492f9
KH		 1728{
1729 int multibytep = coding->dst_multibyte;
1730 int *charbuf = coding->charbuf;
1731 int *charbuf_end = charbuf + coding->charbuf_used;
1732 unsigned char *dst = coding->destination + coding->produced;
1733 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1734 int safe_room = 8;
a470d443 1735 enum utf_bom_type bom = CODING_UTF_16_BOM (coding);
df7492f9 1736 int big_endian = CODING_UTF_16_ENDIAN (coding) == utf_16_big_endian;
d311d28c 1737 ptrdiff_t produced_chars = 0;
df7492f9 1738 int c;
4ed46869 1739
a470d443 1740 if (bom != utf_without_bom)
df7492f9
KH		 1741 {
1742 ASSURE_DESTINATION (safe_room);
1743 if (big_endian)
df7492f9 1744 EMIT_TWO_BYTES (0xFE, 0xFF);
880cf180
KH		 1745 else
1746 EMIT_TWO_BYTES (0xFF, 0xFE);
a470d443 1747 CODING_UTF_16_BOM (coding) = utf_without_bom;
df7492f9
KH		 1748 }
1749
1750 while (charbuf < charbuf_end)
1751 {
1752 ASSURE_DESTINATION (safe_room);
1753 c = *charbuf++;
60afa08d 1754 if (c > MAX_UNICODE_CHAR)
e19c3639 1755 c = coding->default_char;
df7492f9
KH		 1756
1757 if (c < 0x10000)
1758 {
1759 if (big_endian)
1760 EMIT_TWO_BYTES (c >> 8, c & 0xFF);
1761 else
1762 EMIT_TWO_BYTES (c & 0xFF, c >> 8);
1763 }
1764 else
1765 {
1766 int c1, c2;
1767
1768 c -= 0x10000;
1769 c1 = (c >> 10) + 0xD800;
1770 c2 = (c & 0x3FF) + 0xDC00;
1771 if (big_endian)
1772 EMIT_FOUR_BYTES (c1 >> 8, c1 & 0xFF, c2 >> 8, c2 & 0xFF);
1773 else
1774 EMIT_FOUR_BYTES (c1 & 0xFF, c1 >> 8, c2 & 0xFF, c2 >> 8);
1775 }
1776 }
065e3595 1777 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1778 coding->produced = dst - coding->destination;
1779 coding->produced_char += produced_chars;
1780 return 0;
1781}
1782
1783\f
1784/*** 6. Old Emacs' internal format (emacs-mule) ***/
1785
1786/* Emacs' internal format for representation of multiple character
1787 sets is a kind of multi-byte encoding, i.e. characters are
1788 represented by variable-length sequences of one-byte codes.
1789
1790 ASCII characters and control characters (e.g. `tab', `newline') are
1791 represented by one-byte sequences which are their ASCII codes, in
1792 the range 0x00 through 0x7F.
1793
1794 8-bit characters of the range 0x80..0x9F are represented by
1795 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
1796 code + 0x20).
1797
1798 8-bit characters of the range 0xA0..0xFF are represented by
1799 one-byte sequences which are their 8-bit code.
1800
1801 The other characters are represented by a sequence of `base
1802 leading-code', optional `extended leading-code', and one or two
1803 `position-code's. The length of the sequence is determined by the
1804 base leading-code. Leading-code takes the range 0x81 through 0x9D,
1805 whereas extended leading-code and position-code take the range 0xA0
1806 through 0xFF. See `charset.h' for more details about leading-code
1807 and position-code.
1808
1809 --- CODE RANGE of Emacs' internal format ---
1810 character set range
1811 ------------- -----
1812 ascii 0x00..0x7F
1813 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
1814 eight-bit-graphic 0xA0..0xBF
1815 ELSE 0x81..0x9D + [0xA0..0xFF]+
1816 ---------------------------------------------
1817
1818 As this is the internal character representation, the format is
1819 usually not used externally (i.e. in a file or in a data sent to a
1820 process). But, it is possible to have a text externally in this
1821 format (i.e. by encoding by the coding system `emacs-mule').
1822
1823 In that case, a sequence of one-byte codes has a slightly different
1824 form.
1825
1826 At first, all characters in eight-bit-control are represented by
1827 one-byte sequences which are their 8-bit code.
1828
1829 Next, character composition data are represented by the byte
1830 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
1831 where,
e951386e 1832 METHOD is 0xF2 plus one of composition method (enum
df7492f9
KH		 1833 composition_method),
1834
1835 BYTES is 0xA0 plus a byte length of this composition data,
1836
e951386e 1837 CHARS is 0xA0 plus a number of characters composed by this
df7492f9
KH		 1838 data,
1839
ad1746f5 1840 COMPONENTs are characters of multibyte form or composition
df7492f9
KH		 1841 rules encoded by two-byte of ASCII codes.
1842
1843 In addition, for backward compatibility, the following formats are
1844 also recognized as composition data on decoding.
1845
1846 0x80 MSEQ ...
1847 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
1848
1849 Here,
1850 MSEQ is a multibyte form but in these special format:
1851 ASCII: 0xA0 ASCII_CODE+0x80,
1852 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
1853 RULE is a one byte code of the range 0xA0..0xF0 that
1854 represents a composition rule.
1855 */
1856
1857char emacs_mule_bytes[256];
1858
e951386e
KH		 1859
1860/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1861 Check if a text is encoded in `emacs-mule'. If it is, return 1,
1862 else return 0. */
1863
1864static int
cf84bb53
JB		 1865detect_coding_emacs_mule (struct coding_system *coding,
1866 struct coding_detection_info *detect_info)
e951386e
KH		 1867{
1868 const unsigned char *src = coding->source, *src_base;
1869 const unsigned char *src_end = coding->source + coding->src_bytes;
1870 int multibytep = coding->src_multibyte;
d311d28c 1871 ptrdiff_t consumed_chars = 0;
e951386e
KH		 1872 int c;
1873 int found = 0;
1874
1875 detect_info->checked |= CATEGORY_MASK_EMACS_MULE;
1876 /* A coding system of this category is always ASCII compatible. */
1877 src += coding->head_ascii;
1878
1879 while (1)
1880 {
1881 src_base = src;
1882 ONE_MORE_BYTE (c);
1883 if (c < 0)
1884 continue;
1885 if (c == 0x80)
1886 {
1887 /* Perhaps the start of composite character. We simply skip
1888 it because analyzing it is too heavy for detecting. But,
1889 at least, we check that the composite character
1890 constitutes of more than 4 bytes. */
2735d060 1891 const unsigned char *src_start;
e951386e
KH		 1892
1893 repeat:
2735d060 1894 src_start = src;
e951386e
KH		 1895 do
1896 {
1897 ONE_MORE_BYTE (c);
1898 }
1899 while (c >= 0xA0);
1900
2735d060 1901 if (src - src_start <= 4)
e951386e
KH		 1902 break;
1903 found = CATEGORY_MASK_EMACS_MULE;
1904 if (c == 0x80)
1905 goto repeat;
1906 }
1907
1908 if (c < 0x80)
1909 {
1910 if (c < 0x20
1911 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO))
1912 break;
1913 }
1914 else
1915 {
396475b7 1916 int more_bytes = emacs_mule_bytes[c] - 1;
e951386e
KH		 1917
1918 while (more_bytes > 0)
1919 {
1920 ONE_MORE_BYTE (c);
1921 if (c < 0xA0)
1922 {
1923 src--; /* Unread the last byte. */
1924 break;
1925 }
1926 more_bytes--;
1927 }
1928 if (more_bytes != 0)
1929 break;
1930 found = CATEGORY_MASK_EMACS_MULE;
1931 }
1932 }
1933 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
1934 return 0;
1935
1936 no_more_source:
1937 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
1938 {
1939 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
1940 return 0;
1941 }
1942 detect_info->found |= found;
1943 return 1;
1944}
1945
1946
1947/* Parse emacs-mule multibyte sequence at SRC and return the decoded
1948 character. If CMP_STATUS indicates that we must expect MSEQ or
1949 RULE described above, decode it and return the negative value of
685ebdc8 1950 the decoded character or rule. If an invalid byte is found, return
e951386e
KH		 1951 -1. If SRC is too short, return -2. */
1952
e2f1bab9 1953static int
cf84bb53
JB		 1954emacs_mule_char (struct coding_system *coding, const unsigned char *src,
1955 int *nbytes, int *nchars, int *id,
1956 struct composition_status *cmp_status)
df7492f9 1957{
8f924df7
KH		 1958 const unsigned char *src_end = coding->source + coding->src_bytes;
1959 const unsigned char *src_base = src;
df7492f9 1960 int multibytep = coding->src_multibyte;
2735d060 1961 int charset_ID;
df7492f9
KH		 1962 unsigned code;
1963 int c;
1964 int consumed_chars = 0;
e951386e 1965 int mseq_found = 0;
df7492f9
KH		 1966
1967 ONE_MORE_BYTE (c);
065e3595 1968 if (c < 0)
df7492f9 1969 {
065e3595 1970 c = -c;
2735d060 1971 charset_ID = emacs_mule_charset[0];
065e3595
KH		 1972 }
1973 else
1974 {
4d41e8b7
KH		 1975 if (c >= 0xA0)
1976 {
e951386e
KH		 1977 if (cmp_status->state != COMPOSING_NO
1978 && cmp_status->old_form)
4d41e8b7 1979 {
e951386e
KH		 1980 if (cmp_status->state == COMPOSING_CHAR)
1981 {
1982 if (c == 0xA0)
1983 {
1984 ONE_MORE_BYTE (c);
1985 c -= 0x80;
1986 if (c < 0)
1987 goto invalid_code;
1988 }
1989 else
1990 c -= 0x20;
1991 mseq_found = 1;
1992 }
1993 else
1994 {
1995 *nbytes = src - src_base;
1996 *nchars = consumed_chars;
1997 return -c;
1998 }
4d41e8b7
KH		 1999 }
2000 else
e951386e 2001 goto invalid_code;
4d41e8b7
KH		 2002 }
2003
065e3595 2004 switch (emacs_mule_bytes[c])
b73bfc1c 2005 {
065e3595 2006 case 2:
2735d060 2007 if ((charset_ID = emacs_mule_charset[c]) < 0)
df7492f9
KH		 2008 goto invalid_code;
2009 ONE_MORE_BYTE (c);
9ffd559c 2010 if (c < 0xA0)
065e3595 2011 goto invalid_code;
df7492f9 2012 code = c & 0x7F;
065e3595
KH		 2013 break;
2014
2015 case 3:
2016 if (c == EMACS_MULE_LEADING_CODE_PRIVATE_11
2017 || c == EMACS_MULE_LEADING_CODE_PRIVATE_12)
2018 {
2019 ONE_MORE_BYTE (c);
2735d060 2020 if (c < 0xA0 || (charset_ID = emacs_mule_charset[c]) < 0)
065e3595
KH		 2021 goto invalid_code;
2022 ONE_MORE_BYTE (c);
9ffd559c 2023 if (c < 0xA0)
065e3595
KH		 2024 goto invalid_code;
2025 code = c & 0x7F;
2026 }
2027 else
2028 {
2735d060 2029 if ((charset_ID = emacs_mule_charset[c]) < 0)
065e3595
KH		 2030 goto invalid_code;
2031 ONE_MORE_BYTE (c);
9ffd559c 2032 if (c < 0xA0)
065e3595
KH		 2033 goto invalid_code;
2034 code = (c & 0x7F) << 8;
2035 ONE_MORE_BYTE (c);
9ffd559c 2036 if (c < 0xA0)
065e3595
KH		 2037 goto invalid_code;
2038 code |= c & 0x7F;
2039 }
2040 break;
2041
2042 case 4:
2043 ONE_MORE_BYTE (c);
2735d060 2044 if (c < 0 || (charset_ID = emacs_mule_charset[c]) < 0)
df7492f9
KH		 2045 goto invalid_code;
2046 ONE_MORE_BYTE (c);
9ffd559c 2047 if (c < 0xA0)
065e3595 2048 goto invalid_code;
781d7a48 2049 code = (c & 0x7F) << 8;
df7492f9 2050 ONE_MORE_BYTE (c);
9ffd559c 2051 if (c < 0xA0)
065e3595 2052 goto invalid_code;
df7492f9 2053 code |= c & 0x7F;
065e3595 2054 break;
df7492f9 2055
065e3595
KH		 2056 case 1:
2057 code = c;
2735d060 2058 charset_ID = ASCII_BYTE_P (code) ? charset_ascii : charset_eight_bit;
065e3595 2059 break;
df7492f9 2060
065e3595
KH		 2061 default:
2062 abort ();
2063 }
b84ae584 2064 CODING_DECODE_CHAR (coding, src, src_base, src_end,
2735d060 2065 CHARSET_FROM_ID (charset_ID), code, c);
065e3595
KH		 2066 if (c < 0)
2067 goto invalid_code;
df7492f9 2068 }
df7492f9
KH		 2069 *nbytes = src - src_base;
2070 *nchars = consumed_chars;
ff0dacd7 2071 if (id)
2735d060 2072 *id = charset_ID;
e951386e 2073 return (mseq_found ? -c : c);
df7492f9
KH		 2074
2075 no_more_source:
2076 return -2;
2077
2078 invalid_code:
2079 return -1;
2080}
2081
2082
e951386e 2083/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
df7492f9 2084
e951386e
KH		 2085/* Handle these composition sequence ('|': the end of header elements,
2086 BYTES and CHARS >= 0xA0):
df7492f9 2087
e951386e
KH		 2088 (1) relative composition: 0x80 0xF2 BYTES CHARS | CHAR ...
2089 (2) altchar composition: 0x80 0xF4 BYTES CHARS | ALT ... ALT CHAR ...
2090 (3) alt&rule composition: 0x80 0xF5 BYTES CHARS | ALT RULE ... ALT CHAR ...
df7492f9 2091
e951386e 2092 and these old form:
1a4990fb 2093
e951386e
KH		 2094 (4) relative composition: 0x80 | MSEQ ... MSEQ
2095 (5) rulebase composition: 0x80 0xFF | MSEQ MRULE ... MSEQ
df7492f9 2096
e951386e
KH		 2097 When the starter 0x80 and the following header elements are found,
2098 this annotation header is produced.
df7492f9 2099
e951386e 2100 [ -LENGTH(==-5) CODING_ANNOTATE_COMPOSITION_MASK NCHARS NBYTES METHOD ]
df7492f9 2101
e951386e
KH		 2102 NCHARS is CHARS - 0xA0 for (1), (2), (3), and 0 for (4), (5).
2103 NBYTES is BYTES - 0xA0 for (1), (2), (3), and 0 for (4), (5).
df7492f9 2104
e951386e
KH		 2105 Then, upon reading the following elements, these codes are produced
2106 until the composition end is found:
df7492f9 2107
e951386e
KH		 2108 (1) CHAR ... CHAR
2109 (2) ALT ... ALT CHAR ... CHAR
2110 (3) ALT -2 DECODED-RULE ALT -2 DECODED-RULE ... ALT CHAR ... CHAR
2111 (4) CHAR ... CHAR
2112 (5) CHAR -2 DECODED-RULE CHAR -2 DECODED-RULE ... CHAR
4ed46869 2113
e951386e
KH		 2114 When the composition end is found, LENGTH and NCHARS in the
2115 annotation header is updated as below:
b73bfc1c 2116
e951386e
KH		 2117 (1) LENGTH: unchanged, NCHARS: unchanged
2118 (2) LENGTH: length of the whole sequence minus NCHARS, NCHARS: unchanged
2119 (3) LENGTH: length of the whole sequence minus NCHARS, NCHARS: unchanged
2120 (4) LENGTH: unchanged, NCHARS: number of CHARs
2121 (5) LENGTH: unchanged, NCHARS: number of CHARs
df7492f9 2122
e951386e
KH		 2123 If an error is found while composing, the annotation header is
2124 changed to the original composition header (plus filler -1s) as
2125 below:
2126
2127 (1),(2),(3) [ 0x80 0xF2+METHOD BYTES CHARS -1 ]
2128 (5) [ 0x80 0xFF -1 -1- -1 ]
2129
2130 and the sequence [ -2 DECODED-RULE ] is changed to the original
2131 byte sequence as below:
2132 o the original byte sequence is B: [ B -1 ]
2133 o the original byte sequence is B1 B2: [ B1 B2 ]
2134
2135 Most of the routines are implemented by macros because many
2136 variables and labels in the caller decode_coding_emacs_mule must be
2137 accessible, and they are usually called just once (thus doesn't
2138 increase the size of compiled object). */
2139
2140/* Decode a composition rule represented by C as a component of
2141 composition sequence of Emacs 20 style. Set RULE to the decoded
2142 rule. */
2143
2144#define DECODE_EMACS_MULE_COMPOSITION_RULE_20(c, rule) \
df7492f9 2145 do { \
e951386e
KH		 2146 int gref, nref; \
2147 \
4d41e8b7 2148 c -= 0xA0; \
df7492f9
KH		 2149 if (c < 0 || c >= 81) \
2150 goto invalid_code; \
df7492f9 2151 gref = c / 9, nref = c % 9; \
e951386e
KH		 2152 if (gref == 4) gref = 10; \
2153 if (nref == 4) nref = 10; \
2154 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
df7492f9
KH		 2155 } while (0)
2156
2157
e951386e
KH		 2158/* Decode a composition rule represented by C and the following byte
2159 at SRC as a component of composition sequence of Emacs 21 style.
2160 Set RULE to the decoded rule. */
781d7a48 2161
e951386e 2162#define DECODE_EMACS_MULE_COMPOSITION_RULE_21(c, rule) \
781d7a48
KH		 2163 do { \
2164 int gref, nref; \
e951386e
KH		 2165 \
2166 gref = c - 0x20; \
2167 if (gref < 0 || gref >= 81) \
781d7a48 2168 goto invalid_code; \
e951386e
KH		 2169 ONE_MORE_BYTE (c); \
2170 nref = c - 0x20; \
2171 if (nref < 0 || nref >= 81) \
781d7a48 2172 goto invalid_code; \
e951386e 2173 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
781d7a48
KH		 2174 } while (0)
2175
2176
e951386e
KH		 2177/* Start of Emacs 21 style format. The first three bytes at SRC are
2178 (METHOD - 0xF2), (BYTES - 0xA0), (CHARS - 0xA0), where BYTES is the
2179 byte length of this composition information, CHARS is the number of
2180 characters composed by this composition. */
2181
2182#define DECODE_EMACS_MULE_21_COMPOSITION() \
aa72b389 2183 do { \
781d7a48 2184 enum composition_method method = c - 0xF2; \
df7492f9 2185 int nbytes, nchars; \
e951386e 2186 \
df7492f9 2187 ONE_MORE_BYTE (c); \
065e3595
KH		 2188 if (c < 0) \
2189 goto invalid_code; \
df7492f9 2190 nbytes = c - 0xA0; \
e951386e 2191 if (nbytes < 3 || (method == COMPOSITION_RELATIVE && nbytes != 4)) \
df7492f9
KH		 2192 goto invalid_code; \
2193 ONE_MORE_BYTE (c); \
2194 nchars = c - 0xA0; \
e951386e
KH		 2195 if (nchars <= 0 || nchars >= MAX_COMPOSITION_COMPONENTS) \
2196 goto invalid_code; \
2197 cmp_status->old_form = 0; \
2198 cmp_status->method = method; \
2199 if (method == COMPOSITION_RELATIVE) \
2200 cmp_status->state = COMPOSING_CHAR; \
2201 else \
2202 cmp_status->state = COMPOSING_COMPONENT_CHAR; \
2203 cmp_status->length = MAX_ANNOTATION_LENGTH; \
2204 cmp_status->nchars = nchars; \
2205 cmp_status->ncomps = nbytes - 4; \
2206 ADD_COMPOSITION_DATA (charbuf, nchars, nbytes, method); \
aa72b389 2207 } while (0)
93dec019 2208
aa72b389 2209
e951386e
KH		 2210/* Start of Emacs 20 style format for relative composition. */
2211
2212#define DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION() \
2213 do { \
2214 cmp_status->old_form = 1; \
2215 cmp_status->method = COMPOSITION_RELATIVE; \
2216 cmp_status->state = COMPOSING_CHAR; \
2217 cmp_status->length = MAX_ANNOTATION_LENGTH; \
2218 cmp_status->nchars = cmp_status->ncomps = 0; \
2219 ADD_COMPOSITION_DATA (charbuf, 0, 0, cmp_status->method); \
2220 } while (0)
2221
2222
2223/* Start of Emacs 20 style format for rule-base composition. */
2224
2225#define DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION() \
2226 do { \
2227 cmp_status->old_form = 1; \
2228 cmp_status->method = COMPOSITION_WITH_RULE; \
2229 cmp_status->state = COMPOSING_CHAR; \
2230 cmp_status->length = MAX_ANNOTATION_LENGTH; \
2231 cmp_status->nchars = cmp_status->ncomps = 0; \
2232 ADD_COMPOSITION_DATA (charbuf, 0, 0, cmp_status->method); \
df7492f9
KH		 2233 } while (0)
2234
2235
e951386e
KH		 2236#define DECODE_EMACS_MULE_COMPOSITION_START() \
2237 do { \
2238 const unsigned char *current_src = src; \
2239 \
2240 ONE_MORE_BYTE (c); \
2241 if (c < 0) \
2242 goto invalid_code; \
2243 if (c - 0xF2 >= COMPOSITION_RELATIVE \
2244 && c - 0xF2 <= COMPOSITION_WITH_RULE_ALTCHARS) \
2245 DECODE_EMACS_MULE_21_COMPOSITION (); \
2246 else if (c < 0xA0) \
2247 goto invalid_code; \
2248 else if (c < 0xC0) \
2249 { \
2250 DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION (); \
2251 /* Re-read C as a composition component. */ \
2252 src = current_src; \
2253 } \
2254 else if (c == 0xFF) \
2255 DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION (); \
2256 else \
2257 goto invalid_code; \
2258 } while (0)
2259
2260#define EMACS_MULE_COMPOSITION_END() \
df7492f9 2261 do { \
e951386e 2262 int idx = - cmp_status->length; \
4d41e8b7 2263 \
e951386e
KH		 2264 if (cmp_status->old_form) \
2265 charbuf[idx + 2] = cmp_status->nchars; \
2266 else if (cmp_status->method > COMPOSITION_RELATIVE) \
2267 charbuf[idx] = charbuf[idx + 2] - cmp_status->length; \
2268 cmp_status->state = COMPOSING_NO; \
2269 } while (0)
2270
2271
2272static int
cf84bb53
JB		 2273emacs_mule_finish_composition (int *charbuf,
2274 struct composition_status *cmp_status)
e951386e
KH		 2275{
2276 int idx = - cmp_status->length;
2277 int new_chars;
2278
2279 if (cmp_status->old_form && cmp_status->nchars > 0)
2280 {
2281 charbuf[idx + 2] = cmp_status->nchars;
2282 new_chars = 0;
2283 if (cmp_status->method == COMPOSITION_WITH_RULE
2284 && cmp_status->state == COMPOSING_CHAR)
2285 {
2286 /* The last rule was invalid. */
2287 int rule = charbuf[-1] + 0xA0;
2288
2289 charbuf[-2] = BYTE8_TO_CHAR (rule);
2290 charbuf[-1] = -1;
2291 new_chars = 1;
2292 }
2293 }
2294 else
2295 {
2296 charbuf[idx++] = BYTE8_TO_CHAR (0x80);
2297
2298 if (cmp_status->method == COMPOSITION_WITH_RULE)
2299 {
2300 charbuf[idx++] = BYTE8_TO_CHAR (0xFF);
2301 charbuf[idx++] = -3;
2302 charbuf[idx++] = 0;
2303 new_chars = 1;
2304 }
2305 else
2306 {
2307 int nchars = charbuf[idx + 1] + 0xA0;
2308 int nbytes = charbuf[idx + 2] + 0xA0;
2309
2310 charbuf[idx++] = BYTE8_TO_CHAR (0xF2 + cmp_status->method);
2311 charbuf[idx++] = BYTE8_TO_CHAR (nbytes);
2312 charbuf[idx++] = BYTE8_TO_CHAR (nchars);
2313 charbuf[idx++] = -1;
2314 new_chars = 4;
2315 }
2316 }
2317 cmp_status->state = COMPOSING_NO;
2318 return new_chars;
2319}
2320
2321#define EMACS_MULE_MAYBE_FINISH_COMPOSITION() \
2322 do { \
2323 if (cmp_status->state != COMPOSING_NO) \
2324 char_offset += emacs_mule_finish_composition (charbuf, cmp_status); \
df7492f9
KH		 2325 } while (0)
2326
aa72b389
KH		 2327
2328static void
971de7fb 2329decode_coding_emacs_mule (struct coding_system *coding)
aa72b389 2330{
8f924df7
KH		 2331 const unsigned char *src = coding->source + coding->consumed;
2332 const unsigned char *src_end = coding->source + coding->src_bytes;
2333 const unsigned char *src_base;
69a80ea3 2334 int *charbuf = coding->charbuf + coding->charbuf_used;
ad1746f5
KH		 2335 /* We may produce two annotations (charset and composition) in one
2336 loop and one more charset annotation at the end. */
69a80ea3 2337 int *charbuf_end
15cbd324
EZ		 2338 = coding->charbuf + coding->charbuf_size - (MAX_ANNOTATION_LENGTH * 3)
2339 /* We can produce up to 2 characters in a loop. */
2340 - 1;
d311d28c 2341 ptrdiff_t consumed_chars = 0, consumed_chars_base;
df7492f9 2342 int multibytep = coding->src_multibyte;
d311d28c
PE		 2343 ptrdiff_t char_offset = coding->produced_char;
2344 ptrdiff_t last_offset = char_offset;
ff0dacd7 2345 int last_id = charset_ascii;
2735d060 2346 int eol_dos =
0a9564cb 2347 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 2348 int byte_after_cr = -1;
e951386e 2349 struct composition_status *cmp_status = &coding->spec.emacs_mule.cmp_status;
aa72b389 2350
e951386e
KH		 2351 if (cmp_status->state != COMPOSING_NO)
2352 {
2353 int i;
2354
15cbd324
EZ		 2355 if (charbuf_end - charbuf < cmp_status->length)
2356 abort ();
e951386e
KH		 2357 for (i = 0; i < cmp_status->length; i++)
2358 *charbuf++ = cmp_status->carryover[i];
2359 coding->annotated = 1;
2360 }
2361
aa72b389
KH		 2362 while (1)
2363 {
ee05f961 2364 int c, id IF_LINT (= 0);
df7492f9 2365
aa72b389 2366 src_base = src;
df7492f9
KH		 2367 consumed_chars_base = consumed_chars;
2368
2369 if (charbuf >= charbuf_end)
b71f6f73
KH		 2370 {
2371 if (byte_after_cr >= 0)
2372 src_base--;
2373 break;
2374 }
aa72b389 2375
119852e7
KH		 2376 if (byte_after_cr >= 0)
2377 c = byte_after_cr, byte_after_cr = -1;
2378 else
2379 ONE_MORE_BYTE (c);
e951386e
KH		 2380
2381 if (c < 0 || c == 0x80)
065e3595 2382 {
e951386e
KH		 2383 EMACS_MULE_MAYBE_FINISH_COMPOSITION ();
2384 if (c < 0)
2385 {
2386 *charbuf++ = -c;
2387 char_offset++;
2388 }
2389 else
2390 DECODE_EMACS_MULE_COMPOSITION_START ();
2391 continue;
065e3595 2392 }
e951386e
KH		 2393
2394 if (c < 0x80)
aa72b389 2395 {
2735d060 2396 if (eol_dos && c == '\r')
119852e7 2397 ONE_MORE_BYTE (byte_after_cr);
e951386e
KH		 2398 id = charset_ascii;
2399 if (cmp_status->state != COMPOSING_NO)
2400 {
2401 if (cmp_status->old_form)
2402 EMACS_MULE_MAYBE_FINISH_COMPOSITION ();
2403 else if (cmp_status->state >= COMPOSING_COMPONENT_CHAR)
2404 cmp_status->ncomps--;
2405 }
2406 }
2407 else
2408 {
ee05f961 2409 int nchars IF_LINT (= 0), nbytes IF_LINT (= 0);
75f80e63
EZ		 2410 /* emacs_mule_char can load a charset map from a file, which
2411 allocates a large structure and might cause buffer text
2412 to be relocated as result. Thus, we need to remember the
ad1746f5 2413 original pointer to buffer text, and fix up all related
75f80e63
EZ		 2414 pointers after the call. */
2415 const unsigned char *orig = coding->source;
d311d28c 2416 ptrdiff_t offset;
e951386e
KH		 2417
2418 c = emacs_mule_char (coding, src_base, &nbytes, &nchars, &id,
2419 cmp_status);
75f80e63
EZ		 2420 offset = coding->source - orig;
2421 if (offset)
2422 {
2423 src += offset;
2424 src_base += offset;
2425 src_end += offset;
2426 }
e951386e
KH		 2427 if (c < 0)
2428 {
2429 if (c == -1)
2430 goto invalid_code;
2431 if (c == -2)
2432 break;
2433 }
2434 src = src_base + nbytes;
2435 consumed_chars = consumed_chars_base + nchars;
2436 if (cmp_status->state >= COMPOSING_COMPONENT_CHAR)
2437 cmp_status->ncomps -= nchars;
2438 }
2439
ad1746f5 2440 /* Now if C >= 0, we found a normally encoded character, if C <
e951386e
KH		 2441 0, we found an old-style composition component character or
2442 rule. */
2443
2444 if (cmp_status->state == COMPOSING_NO)
2445 {
2446 if (last_id != id)
2447 {
2448 if (last_id != charset_ascii)
2449 ADD_CHARSET_DATA (charbuf, char_offset - last_offset,
2450 last_id);
2451 last_id = id;
2452 last_offset = char_offset;
2453 }
df7492f9
KH		 2454 *charbuf++ = c;
2455 char_offset++;
aa72b389 2456 }
e951386e 2457 else if (cmp_status->state == COMPOSING_CHAR)
df7492f9 2458 {
e951386e
KH		 2459 if (cmp_status->old_form)
2460 {
2461 if (c >= 0)
2462 {
2463 EMACS_MULE_MAYBE_FINISH_COMPOSITION ();
2464 *charbuf++ = c;
2465 char_offset++;
2466 }
2467 else
2468 {
2469 *charbuf++ = -c;
2470 cmp_status->nchars++;
2471 cmp_status->length++;
2472 if (cmp_status->nchars == MAX_COMPOSITION_COMPONENTS)
2473 EMACS_MULE_COMPOSITION_END ();
2474 else if (cmp_status->method == COMPOSITION_WITH_RULE)
2475 cmp_status->state = COMPOSING_RULE;
2476 }
2477 }
df7492f9 2478 else
e951386e
KH		 2479 {
2480 *charbuf++ = c;
2481 cmp_status->length++;
2482 cmp_status->nchars--;
2483 if (cmp_status->nchars == 0)
2484 EMACS_MULE_COMPOSITION_END ();
2485 }
df7492f9 2486 }
e951386e 2487 else if (cmp_status->state == COMPOSING_RULE)
df7492f9 2488 {
e951386e 2489 int rule;
ff0dacd7 2490
e951386e 2491 if (c >= 0)
df7492f9 2492 {
e951386e
KH		 2493 EMACS_MULE_COMPOSITION_END ();
2494 *charbuf++ = c;
2495 char_offset++;
df7492f9 2496 }
e951386e 2497 else
ff0dacd7 2498 {
e951386e
KH		 2499 c = -c;
2500 DECODE_EMACS_MULE_COMPOSITION_RULE_20 (c, rule);
2501 if (rule < 0)
2502 goto invalid_code;
2503 *charbuf++ = -2;
2504 *charbuf++ = rule;
2505 cmp_status->length += 2;
2506 cmp_status->state = COMPOSING_CHAR;
ff0dacd7 2507 }
e951386e
KH		 2508 }
2509 else if (cmp_status->state == COMPOSING_COMPONENT_CHAR)
2510 {
df7492f9 2511 *charbuf++ = c;
e951386e
KH		 2512 cmp_status->length++;
2513 if (cmp_status->ncomps == 0)
2514 cmp_status->state = COMPOSING_CHAR;
2515 else if (cmp_status->ncomps > 0)
2516 {
2517 if (cmp_status->method == COMPOSITION_WITH_RULE_ALTCHARS)
2518 cmp_status->state = COMPOSING_COMPONENT_RULE;
2519 }
2520 else
2521 EMACS_MULE_MAYBE_FINISH_COMPOSITION ();
df7492f9 2522 }
e951386e
KH		 2523 else /* COMPOSING_COMPONENT_RULE */
2524 {
2525 int rule;
2526
2527 DECODE_EMACS_MULE_COMPOSITION_RULE_21 (c, rule);
2528 if (rule < 0)
2529 goto invalid_code;
2530 *charbuf++ = -2;
2531 *charbuf++ = rule;
2532 cmp_status->length += 2;
2533 cmp_status->ncomps--;
2534 if (cmp_status->ncomps > 0)
2535 cmp_status->state = COMPOSING_COMPONENT_CHAR;
2536 else
2537 EMACS_MULE_MAYBE_FINISH_COMPOSITION ();
2538 }
2539 continue;
2540
df7492f9 2541 invalid_code:
e951386e 2542 EMACS_MULE_MAYBE_FINISH_COMPOSITION ();
df7492f9
KH		 2543 src = src_base;
2544 consumed_chars = consumed_chars_base;
2545 ONE_MORE_BYTE (c);
2546 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 2547 char_offset++;
df7492f9
KH		 2548 coding->errors++;
2549 }
2550
2551 no_more_source:
e951386e
KH		 2552 if (cmp_status->state != COMPOSING_NO)
2553 {
2554 if (coding->mode & CODING_MODE_LAST_BLOCK)
2555 EMACS_MULE_MAYBE_FINISH_COMPOSITION ();
2556 else
2557 {
2558 int i;
2559
2560 charbuf -= cmp_status->length;
2561 for (i = 0; i < cmp_status->length; i++)
2562 cmp_status->carryover[i] = charbuf[i];
2563 }
2564 }
ff0dacd7 2565 if (last_id != charset_ascii)
69a80ea3 2566 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 2567 coding->consumed_char += consumed_chars_base;
2568 coding->consumed = src_base - coding->source;
2569 coding->charbuf_used = charbuf - coding->charbuf;
2570}
2571
2572
2573#define EMACS_MULE_LEADING_CODES(id, codes) \
2574 do { \
2575 if (id < 0xA0) \
2576 codes[0] = id, codes[1] = 0; \
2577 else if (id < 0xE0) \
2578 codes[0] = 0x9A, codes[1] = id; \
2579 else if (id < 0xF0) \
2580 codes[0] = 0x9B, codes[1] = id; \
2581 else if (id < 0xF5) \
2582 codes[0] = 0x9C, codes[1] = id; \
2583 else \
2584 codes[0] = 0x9D, codes[1] = id; \
2585 } while (0);
2586
aa72b389 2587
df7492f9 2588static int
971de7fb 2589encode_coding_emacs_mule (struct coding_system *coding)
df7492f9
KH		 2590{
2591 int multibytep = coding->dst_multibyte;
2592 int *charbuf = coding->charbuf;
2593 int *charbuf_end = charbuf + coding->charbuf_used;
2594 unsigned char *dst = coding->destination + coding->produced;
2595 unsigned char *dst_end = coding->destination + coding->dst_bytes;
2596 int safe_room = 8;
d311d28c 2597 ptrdiff_t produced_chars = 0;
24a73b0a 2598 Lisp_Object attrs, charset_list;
df7492f9 2599 int c;
ff0dacd7 2600 int preferred_charset_id = -1;
df7492f9 2601
24a73b0a 2602 CODING_GET_INFO (coding, attrs, charset_list);
eccb6815
KH		 2603 if (! EQ (charset_list, Vemacs_mule_charset_list))
2604 {
2605 CODING_ATTR_CHARSET_LIST (attrs)
2606 = charset_list = Vemacs_mule_charset_list;
2607 }
df7492f9
KH		 2608
2609 while (charbuf < charbuf_end)
2610 {
2611 ASSURE_DESTINATION (safe_room);
2612 c = *charbuf++;
ff0dacd7
KH		 2613
2614 if (c < 0)
2615 {
2616 /* Handle an annotation. */
2617 switch (*charbuf)
2618 {
2619 case CODING_ANNOTATE_COMPOSITION_MASK:
2620 /* Not yet implemented. */
2621 break;
2622 case CODING_ANNOTATE_CHARSET_MASK:
2623 preferred_charset_id = charbuf[3];
2624 if (preferred_charset_id >= 0
2625 && NILP (Fmemq (make_number (preferred_charset_id),
2626 charset_list)))
2627 preferred_charset_id = -1;
2628 break;
2629 default:
2630 abort ();
2631 }
2632 charbuf += -c - 1;
2633 continue;
2634 }
2635
df7492f9
KH		 2636 if (ASCII_CHAR_P (c))
2637 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 2638 else if (CHAR_BYTE8_P (c))
2639 {
2640 c = CHAR_TO_BYTE8 (c);
2641 EMIT_ONE_BYTE (c);
2642 }
df7492f9 2643 else
aa72b389 2644 {
df7492f9
KH		 2645 struct charset *charset;
2646 unsigned code;
2647 int dimension;
2648 int emacs_mule_id;
2649 unsigned char leading_codes[2];
2650
ff0dacd7
KH		 2651 if (preferred_charset_id >= 0)
2652 {
2653 charset = CHARSET_FROM_ID (preferred_charset_id);
905ca9d2
KH		 2654 if (CHAR_CHARSET_P (c, charset))
2655 code = ENCODE_CHAR (charset, c);
2656 else
2657 charset = char_charset (c, charset_list, &code);
ff0dacd7
KH		 2658 }
2659 else
2660 charset = char_charset (c, charset_list, &code);
df7492f9
KH		 2661 if (! charset)
2662 {
2663 c = coding->default_char;
2664 if (ASCII_CHAR_P (c))
2665 {
2666 EMIT_ONE_ASCII_BYTE (c);
2667 continue;
2668 }
2669 charset = char_charset (c, charset_list, &code);
2670 }
2671 dimension = CHARSET_DIMENSION (charset);
2672 emacs_mule_id = CHARSET_EMACS_MULE_ID (charset);
2673 EMACS_MULE_LEADING_CODES (emacs_mule_id, leading_codes);
2674 EMIT_ONE_BYTE (leading_codes[0]);
2675 if (leading_codes[1])
2676 EMIT_ONE_BYTE (leading_codes[1]);
2677 if (dimension == 1)
1fa663f9 2678 EMIT_ONE_BYTE (code | 0x80);
aa72b389 2679 else
df7492f9 2680 {
1fa663f9 2681 code |= 0x8080;
df7492f9
KH		 2682 EMIT_ONE_BYTE (code >> 8);
2683 EMIT_ONE_BYTE (code & 0xFF);
2684 }
aa72b389 2685 }
aa72b389 2686 }
065e3595 2687 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 2688 coding->produced_char += produced_chars;
2689 coding->produced = dst - coding->destination;
2690 return 0;
aa72b389 2691}
b73bfc1c 2692
4ed46869 2693\f
df7492f9 2694/*** 7. ISO2022 handlers ***/
4ed46869
KH		 2695
2696/* The following note describes the coding system ISO2022 briefly.
39787efd 2697 Since the intention of this note is to help understand the
5a936b46 2698 functions in this file, some parts are NOT ACCURATE or are OVERLY
39787efd 2699 SIMPLIFIED. For thorough understanding, please refer to the
5a936b46 2700 original document of ISO2022. This is equivalent to the standard
cfb43547 2701 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
4ed46869
KH		 2702
2703 ISO2022 provides many mechanisms to encode several character sets
cfb43547 2704 in 7-bit and 8-bit environments. For 7-bit environments, all text
39787efd
KH		 2705 is encoded using bytes less than 128. This may make the encoded
2706 text a little bit longer, but the text passes more easily through
cfb43547 2707 several types of gateway, some of which strip off the MSB (Most
8ca3766a 2708 Significant Bit).
b73bfc1c 2709
cfb43547
DL		 2710 There are two kinds of character sets: control character sets and
2711 graphic character sets. The former contain control characters such
4ed46869 2712 as `newline' and `escape' to provide control functions (control
39787efd 2713 functions are also provided by escape sequences). The latter
cfb43547 2714 contain graphic characters such as 'A' and '-'. Emacs recognizes
4ed46869
KH		 2715 two control character sets and many graphic character sets.
2716
2717 Graphic character sets are classified into one of the following
39787efd
KH		 2718 four classes, according to the number of bytes (DIMENSION) and
2719 number of characters in one dimension (CHARS) of the set:
2720 - DIMENSION1_CHARS94
2721 - DIMENSION1_CHARS96
2722 - DIMENSION2_CHARS94
2723 - DIMENSION2_CHARS96
2724
2725 In addition, each character set is assigned an identification tag,
cfb43547 2726 unique for each set, called the "final character" (denoted as <F>
39787efd
KH		 2727 hereafter). The <F> of each character set is decided by ECMA(*)
2728 when it is registered in ISO. The code range of <F> is 0x30..0x7F
2729 (0x30..0x3F are for private use only).
4ed46869
KH		 2730
2731 Note (*): ECMA = European Computer Manufacturers Association
2732
cfb43547 2733 Here are examples of graphic character sets [NAME(<F>)]:
4ed46869
KH		 2734 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
2735 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
2736 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
2737 o DIMENSION2_CHARS96 -- none for the moment
2738
39787efd 2739 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
4ed46869
KH		 2740 C0 [0x00..0x1F] -- control character plane 0
2741 GL [0x20..0x7F] -- graphic character plane 0
2742 C1 [0x80..0x9F] -- control character plane 1
2743 GR [0xA0..0xFF] -- graphic character plane 1
2744
2745 A control character set is directly designated and invoked to C0 or
39787efd
KH		 2746 C1 by an escape sequence. The most common case is that:
2747 - ISO646's control character set is designated/invoked to C0, and
2748 - ISO6429's control character set is designated/invoked to C1,
2749 and usually these designations/invocations are omitted in encoded
2750 text. In a 7-bit environment, only C0 can be used, and a control
2751 character for C1 is encoded by an appropriate escape sequence to
2752 fit into the environment. All control characters for C1 are
2753 defined to have corresponding escape sequences.
4ed46869
KH		 2754
2755 A graphic character set is at first designated to one of four
2756 graphic registers (G0 through G3), then these graphic registers are
2757 invoked to GL or GR. These designations and invocations can be
2758 done independently. The most common case is that G0 is invoked to
39787efd
KH		 2759 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
2760 these invocations and designations are omitted in encoded text.
2761 In a 7-bit environment, only GL can be used.
4ed46869 2762
39787efd
KH		 2763 When a graphic character set of CHARS94 is invoked to GL, codes
2764 0x20 and 0x7F of the GL area work as control characters SPACE and
2765 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
2766 be used.
4ed46869
KH		 2767
2768 There are two ways of invocation: locking-shift and single-shift.
2769 With locking-shift, the invocation lasts until the next different
39787efd
KH		 2770 invocation, whereas with single-shift, the invocation affects the
2771 following character only and doesn't affect the locking-shift
2772 state. Invocations are done by the following control characters or
2773 escape sequences:
4ed46869
KH		 2774
2775 ----------------------------------------------------------------------
39787efd 2776 abbrev function cntrl escape seq description
4ed46869 2777 ----------------------------------------------------------------------
39787efd
KH		 2778 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
2779 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
2780 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
2781 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
2782 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
2783 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
2784 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
2785 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
2786 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
4ed46869 2787 ----------------------------------------------------------------------
39787efd
KH		 2788 (*) These are not used by any known coding system.
2789
2790 Control characters for these functions are defined by macros
2791 ISO_CODE_XXX in `coding.h'.
4ed46869 2792
39787efd 2793 Designations are done by the following escape sequences:
4ed46869
KH		 2794 ----------------------------------------------------------------------
2795 escape sequence description
2796 ----------------------------------------------------------------------
2797 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
2798 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
2799 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
2800 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
2801 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
2802 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
2803 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
2804 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
2805 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
2806 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
2807 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
2808 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
2809 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
2810 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
2811 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
2812 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
2813 ----------------------------------------------------------------------
2814
2815 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
39787efd 2816 of dimension 1, chars 94, and final character <F>, etc...
4ed46869
KH		 2817
2818 Note (*): Although these designations are not allowed in ISO2022,
2819 Emacs accepts them on decoding, and produces them on encoding
39787efd 2820 CHARS96 character sets in a coding system which is characterized as
4ed46869
KH		 2821 7-bit environment, non-locking-shift, and non-single-shift.
2822
2823 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
df7492f9 2824 '(' must be omitted. We refer to this as "short-form" hereafter.
4ed46869 2825
cfb43547 2826 Now you may notice that there are a lot of ways of encoding the
39787efd
KH		 2827 same multilingual text in ISO2022. Actually, there exist many
2828 coding systems such as Compound Text (used in X11's inter client
8ca3766a
DL		 2829 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
2830 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
4ed46869
KH		 2831 localized platforms), and all of these are variants of ISO2022.
2832
2833 In addition to the above, Emacs handles two more kinds of escape
2834 sequences: ISO6429's direction specification and Emacs' private
2835 sequence for specifying character composition.
2836
39787efd 2837 ISO6429's direction specification takes the following form:
4ed46869
KH		 2838 o CSI ']' -- end of the current direction
2839 o CSI '0' ']' -- end of the current direction
2840 o CSI '1' ']' -- start of left-to-right text
2841 o CSI '2' ']' -- start of right-to-left text
2842 The control character CSI (0x9B: control sequence introducer) is
39787efd
KH		 2843 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
2844
2845 Character composition specification takes the following form:
ec6d2bb8
KH		 2846 o ESC '0' -- start relative composition
2847 o ESC '1' -- end composition
2848 o ESC '2' -- start rule-base composition (*)
2849 o ESC '3' -- start relative composition with alternate chars (**)
2850 o ESC '4' -- start rule-base composition with alternate chars (**)
b73bfc1c 2851 Since these are not standard escape sequences of any ISO standard,
cfb43547 2852 the use of them with these meanings is restricted to Emacs only.
ec6d2bb8 2853
5a936b46
DL		 2854 (*) This form is used only in Emacs 20.7 and older versions,
2855 but newer versions can safely decode it.
cfb43547 2856 (**) This form is used only in Emacs 21.1 and newer versions,
5a936b46 2857 and older versions can't decode it.
ec6d2bb8 2858
cfb43547 2859 Here's a list of example usages of these composition escape
b73bfc1c 2860 sequences (categorized by `enum composition_method').
ec6d2bb8 2861
b73bfc1c 2862 COMPOSITION_RELATIVE:
ec6d2bb8 2863 ESC 0 CHAR [ CHAR ] ESC 1
8ca3766a 2864 COMPOSITION_WITH_RULE:
ec6d2bb8 2865 ESC 2 CHAR [ RULE CHAR ] ESC 1
b73bfc1c 2866 COMPOSITION_WITH_ALTCHARS:
ec6d2bb8 2867 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
b73bfc1c 2868 COMPOSITION_WITH_RULE_ALTCHARS:
ec6d2bb8 2869 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
4ed46869 2870
74ab6df5 2871static enum iso_code_class_type iso_code_class[256];
4ed46869 2872
df7492f9
KH		 2873#define SAFE_CHARSET_P(coding, id) \
2874 ((id) <= (coding)->max_charset_id \
1b3b981b 2875 && (coding)->safe_charsets[id] != 255)
df7492f9 2876
df7492f9 2877static void
971de7fb 2878setup_iso_safe_charsets (Lisp_Object attrs)
df7492f9
KH		 2879{
2880 Lisp_Object charset_list, safe_charsets;
2881 Lisp_Object request;
2882 Lisp_Object reg_usage;
2883 Lisp_Object tail;
d311d28c 2884 EMACS_INT reg94, reg96;
df7492f9
KH		 2885 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
2886 int max_charset_id;
2887
2888 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2889 if ((flags & CODING_ISO_FLAG_FULL_SUPPORT)
2890 && ! EQ (charset_list, Viso_2022_charset_list))
2891 {
2892 CODING_ATTR_CHARSET_LIST (attrs)
2893 = charset_list = Viso_2022_charset_list;
2894 ASET (attrs, coding_attr_safe_charsets, Qnil);
2895 }
2896
2897 if (STRINGP (AREF (attrs, coding_attr_safe_charsets)))
2898 return;
2899
2900 max_charset_id = 0;
2901 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2902 {
2903 int id = XINT (XCAR (tail));
2904 if (max_charset_id < id)
2905 max_charset_id = id;
2906 }
d46c5b12 2907
1b3b981b
AS		 2908 safe_charsets = make_uninit_string (max_charset_id + 1);
2909 memset (SDATA (safe_charsets), 255, max_charset_id + 1);
df7492f9
KH		 2910 request = AREF (attrs, coding_attr_iso_request);
2911 reg_usage = AREF (attrs, coding_attr_iso_usage);
2912 reg94 = XINT (XCAR (reg_usage));
2913 reg96 = XINT (XCDR (reg_usage));
2914
2915 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2916 {
2917 Lisp_Object id;
2918 Lisp_Object reg;
2919 struct charset *charset;
2920
2921 id = XCAR (tail);
2922 charset = CHARSET_FROM_ID (XINT (id));
bf16eb23 2923 reg = Fcdr (Fassq (id, request));
df7492f9 2924 if (! NILP (reg))
8f924df7 2925 SSET (safe_charsets, XINT (id), XINT (reg));
df7492f9
KH		 2926 else if (charset->iso_chars_96)
2927 {
2928 if (reg96 < 4)
8f924df7 2929 SSET (safe_charsets, XINT (id), reg96);
df7492f9
KH		 2930 }
2931 else
2932 {
2933 if (reg94 < 4)
8f924df7 2934 SSET (safe_charsets, XINT (id), reg94);
df7492f9
KH		 2935 }
2936 }
2937 ASET (attrs, coding_attr_safe_charsets, safe_charsets);
2938}
d46c5b12 2939
b6871cc7 2940
4ed46869 2941/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ad1746f5 2942 Check if a text is encoded in one of ISO-2022 based coding systems.
ff0dacd7 2943 If it is, return 1, else return 0. */
4ed46869 2944
0a28aafb 2945static int
cf84bb53
JB		 2946detect_coding_iso_2022 (struct coding_system *coding,
2947 struct coding_detection_info *detect_info)
4ed46869 2948{
8f924df7
KH		 2949 const unsigned char *src = coding->source, *src_base = src;
2950 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 2951 int multibytep = coding->src_multibyte;
ff0dacd7 2952 int single_shifting = 0;
0e48bb22 2953 int id;
df7492f9 2954 int c, c1;
d311d28c 2955 ptrdiff_t consumed_chars = 0;
df7492f9 2956 int i;
ff0dacd7
KH		 2957 int rejected = 0;
2958 int found = 0;
cee53ed4 2959 int composition_count = -1;
ff0dacd7
KH		 2960
2961 detect_info->checked |= CATEGORY_MASK_ISO;
df7492f9
KH		 2962
2963 for (i = coding_category_iso_7; i <= coding_category_iso_8_else; i++)
2964 {
2965 struct coding_system *this = &(coding_categories[i]);
2966 Lisp_Object attrs, val;
2967
c6b278e7
KH		 2968 if (this->id < 0)
2969 continue;
df7492f9
KH		 2970 attrs = CODING_ID_ATTRS (this->id);
2971 if (CODING_ISO_FLAGS (this) & CODING_ISO_FLAG_FULL_SUPPORT
1b3b981b 2972 && ! EQ (CODING_ATTR_CHARSET_LIST (attrs), Viso_2022_charset_list))
df7492f9
KH		 2973 setup_iso_safe_charsets (attrs);
2974 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7 2975 this->max_charset_id = SCHARS (val) - 1;
1b3b981b 2976 this->safe_charsets = SDATA (val);
df7492f9
KH		 2977 }
2978
2979 /* A coding system of this category is always ASCII compatible. */
2980 src += coding->head_ascii;
3f003981 2981
ff0dacd7 2982 while (rejected != CATEGORY_MASK_ISO)
4ed46869 2983 {
065e3595 2984 src_base = src;
df7492f9 2985 ONE_MORE_BYTE (c);
4ed46869
KH		 2986 switch (c)
2987 {
2988 case ISO_CODE_ESC:
74383408
KH		 2989 if (inhibit_iso_escape_detection)
2990 break;
f46869e4 2991 single_shifting = 0;
df7492f9 2992 ONE_MORE_BYTE (c);
0e48bb22 2993 if (c == 'N' || c == 'O')
d46c5b12 2994 {
ae9ff118 2995 /* ESC <Fe> for SS2 or SS3. */
ff0dacd7
KH		 2996 single_shifting = 1;
2997 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
4ed46869 2998 }
cee53ed4
KH		 2999 else if (c == '1')
3000 {
3001 /* End of composition. */
3002 if (composition_count < 0
3003 || composition_count > MAX_COMPOSITION_COMPONENTS)
3004 /* Invalid */
3005 break;
3006 composition_count = -1;
3007 found |= CATEGORY_MASK_ISO;
3008 }
ec6d2bb8
KH		 3009 else if (c >= '0' && c <= '4')
3010 {
3011 /* ESC <Fp> for start/end composition. */
cee53ed4 3012 composition_count = 0;
ec6d2bb8 3013 }
bf9cdd4e 3014 else
df7492f9 3015 {
0e48bb22
AS		 3016 if (c >= '(' && c <= '/')
3017 {
3018 /* Designation sequence for a charset of dimension 1. */
3019 ONE_MORE_BYTE (c1);
3020 if (c1 < ' ' || c1 >= 0x80
3021 || (id = iso_charset_table[0][c >= ','][c1]) < 0)
3022 /* Invalid designation sequence. Just ignore. */
3023 break;
3024 }
3025 else if (c == '$')
3026 {
3027 /* Designation sequence for a charset of dimension 2. */
3028 ONE_MORE_BYTE (c);
3029 if (c >= '@' && c <= 'B')
3030 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
3031 id = iso_charset_table[1][0][c];
3032 else if (c >= '(' && c <= '/')
3033 {
3034 ONE_MORE_BYTE (c1);
3035 if (c1 < ' ' || c1 >= 0x80
3036 || (id = iso_charset_table[1][c >= ','][c1]) < 0)
3037 /* Invalid designation sequence. Just ignore. */
3038 break;
3039 }
3040 else
3041 /* Invalid designation sequence. Just ignore it. */
3042 break;
3043 }
3044 else
3045 {
3046 /* Invalid escape sequence. Just ignore it. */
3047 break;
3048 }
d46c5b12 3049
0e48bb22
AS		 3050 /* We found a valid designation sequence for CHARSET. */
3051 rejected |= CATEGORY_MASK_ISO_8BIT;
3052 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7],
3053 id))
3054 found |= CATEGORY_MASK_ISO_7;
3055 else
3056 rejected |= CATEGORY_MASK_ISO_7;
3057 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_tight],
3058 id))
3059 found |= CATEGORY_MASK_ISO_7_TIGHT;
3060 else
3061 rejected |= CATEGORY_MASK_ISO_7_TIGHT;
3062 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_else],
3063 id))
3064 found |= CATEGORY_MASK_ISO_7_ELSE;
3065 else
3066 rejected |= CATEGORY_MASK_ISO_7_ELSE;
3067 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_8_else],
3068 id))
3069 found |= CATEGORY_MASK_ISO_8_ELSE;
3070 else
3071 rejected |= CATEGORY_MASK_ISO_8_ELSE;
3072 }
4ed46869
KH		 3073 break;
3074
4ed46869 3075 case ISO_CODE_SO:
d46c5b12 3076 case ISO_CODE_SI:
ff0dacd7 3077 /* Locking shift out/in. */
74383408
KH		 3078 if (inhibit_iso_escape_detection)
3079 break;
f46869e4 3080 single_shifting = 0;
ff0dacd7 3081 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
d46c5b12
KH		 3082 break;
3083
4ed46869 3084 case ISO_CODE_CSI:
ff0dacd7 3085 /* Control sequence introducer. */
f46869e4 3086 single_shifting = 0;
ff0dacd7
KH		 3087 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
3088 found |= CATEGORY_MASK_ISO_8_ELSE;
3089 goto check_extra_latin;
3090
4ed46869
KH		 3091 case ISO_CODE_SS2:
3092 case ISO_CODE_SS3:
ff0dacd7
KH		 3093 /* Single shift. */
3094 if (inhibit_iso_escape_detection)
3095 break;
75e2a253 3096 single_shifting = 0;
ff0dacd7
KH		 3097 rejected |= CATEGORY_MASK_ISO_7BIT;
3098 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
3099 & CODING_ISO_FLAG_SINGLE_SHIFT)
0e48bb22
AS		 3100 {
3101 found |= CATEGORY_MASK_ISO_8_1;
3102 single_shifting = 1;
3103 }
ff0dacd7
KH		 3104 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
3105 & CODING_ISO_FLAG_SINGLE_SHIFT)
0e48bb22
AS		 3106 {
3107 found |= CATEGORY_MASK_ISO_8_2;
3108 single_shifting = 1;
3109 }
75e2a253
KH		 3110 if (single_shifting)
3111 break;
0e48bb22
AS		 3112 check_extra_latin:
3113 if (! VECTORP (Vlatin_extra_code_table)
3114 || NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
3115 {
3116 rejected = CATEGORY_MASK_ISO;
3117 break;
3118 }
3119 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
3120 & CODING_ISO_FLAG_LATIN_EXTRA)
3121 found |= CATEGORY_MASK_ISO_8_1;
3122 else
3123 rejected |= CATEGORY_MASK_ISO_8_1;
3124 rejected |= CATEGORY_MASK_ISO_8_2;
3125 break;
4ed46869
KH		 3126
3127 default:
065e3595
KH		 3128 if (c < 0)
3129 continue;
4ed46869 3130 if (c < 0x80)
f46869e4 3131 {
cee53ed4
KH		 3132 if (composition_count >= 0)
3133 composition_count++;
f46869e4
KH		 3134 single_shifting = 0;
3135 break;
3136 }
ff0dacd7 3137 if (c >= 0xA0)
c4825358 3138 {
ff0dacd7
KH		 3139 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
3140 found |= CATEGORY_MASK_ISO_8_1;
f46869e4 3141 /* Check the length of succeeding codes of the range
ff0dacd7
KH		 3142 0xA0..0FF. If the byte length is even, we include
3143 CATEGORY_MASK_ISO_8_2 in `found'. We can check this
3144 only when we are not single shifting. */
3145 if (! single_shifting
3146 && ! (rejected & CATEGORY_MASK_ISO_8_2))
f46869e4 3147 {
2735d060 3148 int len = 1;
b73bfc1c
KH		 3149 while (src < src_end)
3150 {
d12bd917 3151 src_base = src;
df7492f9 3152 ONE_MORE_BYTE (c);
b73bfc1c 3153 if (c < 0xA0)
d12bd917
KH		 3154 {
3155 src = src_base;
3156 break;
3157 }
2735d060 3158 len++;
b73bfc1c
KH		 3159 }
3160
2735d060 3161 if (len & 1 && src < src_end)
cee53ed4
KH		 3162 {
3163 rejected |= CATEGORY_MASK_ISO_8_2;
3164 if (composition_count >= 0)
2735d060 3165 composition_count += len;
cee53ed4 3166 }
f46869e4 3167 else
cee53ed4
KH		 3168 {
3169 found |= CATEGORY_MASK_ISO_8_2;
3170 if (composition_count >= 0)
2735d060 3171 composition_count += len / 2;
cee53ed4 3172 }
f46869e4 3173 }
ff0dacd7 3174 break;
4ed46869 3175 }
4ed46869
KH		 3176 }
3177 }
ff0dacd7
KH		 3178 detect_info->rejected |= CATEGORY_MASK_ISO;
3179 return 0;
4ed46869 3180
df7492f9 3181 no_more_source:
ff0dacd7
KH		 3182 detect_info->rejected |= rejected;
3183 detect_info->found |= (found & ~rejected);
df7492f9 3184 return 1;
4ed46869 3185}
ec6d2bb8 3186
4ed46869 3187
134b9549
KH		 3188/* Set designation state into CODING. Set CHARS_96 to -1 if the
3189 escape sequence should be kept. */
df7492f9
KH		 3190#define DECODE_DESIGNATION(reg, dim, chars_96, final) \
3191 do { \
3192 int id, prev; \
3193 \
3194 if (final < '0' || final >= 128 \
3195 || ((id = ISO_CHARSET_TABLE (dim, chars_96, final)) < 0) \
3196 || !SAFE_CHARSET_P (coding, id)) \
3197 { \
3198 CODING_ISO_DESIGNATION (coding, reg) = -2; \
134b9549
KH		 3199 chars_96 = -1; \
3200 break; \
df7492f9
KH		 3201 } \
3202 prev = CODING_ISO_DESIGNATION (coding, reg); \
bf16eb23
KH		 3203 if (id == charset_jisx0201_roman) \
3204 { \
3205 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
3206 id = charset_ascii; \
3207 } \
3208 else if (id == charset_jisx0208_1978) \
3209 { \
3210 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
3211 id = charset_jisx0208; \
3212 } \
df7492f9
KH		 3213 CODING_ISO_DESIGNATION (coding, reg) = id; \
3214 /* If there was an invalid designation to REG previously, and this \
3215 designation is ASCII to REG, we should keep this designation \
3216 sequence. */ \
3217 if (prev == -2 && id == charset_ascii) \
134b9549 3218 chars_96 = -1; \
4ed46869
KH		 3219 } while (0)
3220
d46c5b12 3221
e951386e
KH		 3222/* Handle these composition sequence (ALT: alternate char):
3223
3224 (1) relative composition: ESC 0 CHAR ... ESC 1
3225 (2) rulebase composition: ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
3226 (3) altchar composition: ESC 3 ALT ... ALT ESC 0 CHAR ... ESC 1
3227 (4) alt&rule composition: ESC 4 ALT RULE ... ALT ESC 0 CHAR ... ESC 1
3228
3229 When the start sequence (ESC 0/2/3/4) is found, this annotation
3230 header is produced.
3231
3232 [ -LENGTH(==-5) CODING_ANNOTATE_COMPOSITION_MASK NCHARS(==0) 0 METHOD ]
3233
3234 Then, upon reading CHAR or RULE (one or two bytes), these codes are
3235 produced until the end sequence (ESC 1) is found:
3236
3237 (1) CHAR ... CHAR
3238 (2) CHAR -2 DECODED-RULE CHAR -2 DECODED-RULE ... CHAR
3239 (3) ALT ... ALT -1 -1 CHAR ... CHAR
3240 (4) ALT -2 DECODED-RULE ALT -2 DECODED-RULE ... ALT -1 -1 CHAR ... CHAR
3241
3242 When the end sequence (ESC 1) is found, LENGTH and NCHARS in the
3243 annotation header is updated as below:
3244
3245 (1) LENGTH: unchanged, NCHARS: number of CHARs
3246 (2) LENGTH: unchanged, NCHARS: number of CHARs
3247 (3) LENGTH: += number of ALTs + 2, NCHARS: number of CHARs
3248 (4) LENGTH: += number of ALTs * 3, NCHARS: number of CHARs
3249
3250 If an error is found while composing, the annotation header is
3251 changed to:
3252
3253 [ ESC '0'/'2'/'3'/'4' -2 0 ]
3254
3255 and the sequence [ -2 DECODED-RULE ] is changed to the original
3256 byte sequence as below:
3257 o the original byte sequence is B: [ B -1 ]
3258 o the original byte sequence is B1 B2: [ B1 B2 ]
3259 and the sequence [ -1 -1 ] is changed to the original byte
3260 sequence:
3261 [ ESC '0' ]
3262*/
3263
3264/* Decode a composition rule C1 and maybe one more byte from the
66ebf983 3265 source, and set RULE to the encoded composition rule. If the rule
d5efd1d1 3266 is invalid, goto invalid_code. */
e951386e 3267
66ebf983 3268#define DECODE_COMPOSITION_RULE(rule) \
e951386e
KH		 3269 do { \
3270 rule = c1 - 32; \
3271 if (rule < 0) \
d5efd1d1 3272 goto invalid_code; \
e951386e
KH		 3273 if (rule < 81) /* old format (before ver.21) */ \
3274 { \
3275 int gref = (rule) / 9; \
3276 int nref = (rule) % 9; \
3277 if (gref == 4) gref = 10; \
3278 if (nref == 4) nref = 10; \
3279 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
e951386e
KH		 3280 } \
3281 else /* new format (after ver.21) */ \
3282 { \
2735d060 3283 int b; \
e951386e 3284 \
2735d060 3285 ONE_MORE_BYTE (b); \
d5efd1d1
PE		 3286 if (! COMPOSITION_ENCODE_RULE_VALID (rule - 81, b - 32)) \
3287 goto invalid_code; \
2735d060 3288 rule = COMPOSITION_ENCODE_RULE (rule - 81, b - 32); \
d5efd1d1 3289 rule += 0x100; /* Distinguish it from the old format. */ \
e951386e
KH		 3290 } \
3291 } while (0)
3292
3293#define ENCODE_COMPOSITION_RULE(rule) \
df7492f9 3294 do { \
e951386e
KH		 3295 int gref = (rule % 0x100) / 12, nref = (rule % 0x100) % 12; \
3296 \
3297 if (rule < 0x100) /* old format */ \
df7492f9 3298 { \
e951386e
KH		 3299 if (gref == 10) gref = 4; \
3300 if (nref == 10) nref = 4; \
3301 charbuf[idx] = 32 + gref * 9 + nref; \
3302 charbuf[idx + 1] = -1; \
3303 new_chars++; \
df7492f9 3304 } \
e951386e 3305 else /* new format */ \
df7492f9 3306 { \
e951386e
KH		 3307 charbuf[idx] = 32 + 81 + gref; \
3308 charbuf[idx + 1] = 32 + nref; \
3309 new_chars += 2; \
df7492f9
KH		 3310 } \
3311 } while (0)
3312
e951386e
KH		 3313/* Finish the current composition as invalid. */
3314
f57e2426 3315static int finish_composition (int *, struct composition_status *);
e951386e
KH		 3316
3317static int
971de7fb 3318finish_composition (int *charbuf, struct composition_status *cmp_status)
e951386e
KH		 3319{
3320 int idx = - cmp_status->length;
3321 int new_chars;
3322
3323 /* Recover the original ESC sequence */
3324 charbuf[idx++] = ISO_CODE_ESC;
3325 charbuf[idx++] = (cmp_status->method == COMPOSITION_RELATIVE ? '0'
3326 : cmp_status->method == COMPOSITION_WITH_RULE ? '2'
3327 : cmp_status->method == COMPOSITION_WITH_ALTCHARS ? '3'
3328 /* cmp_status->method == COMPOSITION_WITH_RULE_ALTCHARS */
3329 : '4');
3330 charbuf[idx++] = -2;
3331 charbuf[idx++] = 0;
3332 charbuf[idx++] = -1;
3333 new_chars = cmp_status->nchars;
3334 if (cmp_status->method >= COMPOSITION_WITH_RULE)
3335 for (; idx < 0; idx++)
3336 {
3337 int elt = charbuf[idx];
3338
3339 if (elt == -2)
3340 {
3341 ENCODE_COMPOSITION_RULE (charbuf[idx + 1]);
3342 idx++;
3343 }
3344 else if (elt == -1)
3345 {
3346 charbuf[idx++] = ISO_CODE_ESC;
3347 charbuf[idx] = '0';
3348 new_chars += 2;
3349 }
3350 }
3351 cmp_status->state = COMPOSING_NO;
3352 return new_chars;
3353}
3354
ad1746f5 3355/* If characters are under composition, finish the composition. */
e951386e
KH		 3356#define MAYBE_FINISH_COMPOSITION() \
3357 do { \
3358 if (cmp_status->state != COMPOSING_NO) \
3359 char_offset += finish_composition (charbuf, cmp_status); \
3360 } while (0)
d46c5b12 3361
aa72b389 3362/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
e951386e 3363
aa72b389
KH		 3364 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
3365 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
df7492f9
KH		 3366 ESC 3 : altchar composition : ESC 3 CHAR ... ESC 0 CHAR ... ESC 1
3367 ESC 4 : alt&rule composition : ESC 4 CHAR RULE ... CHAR ESC 0 CHAR ... ESC 1
ec6d2bb8 3368
e951386e
KH		 3369 Produce this annotation sequence now:
3370
3371 [ -LENGTH(==-4) CODING_ANNOTATE_COMPOSITION_MASK NCHARS(==0) METHOD ]
3372*/
3373
3374#define DECODE_COMPOSITION_START(c1) \
3375 do { \
3376 if (c1 == '0' \
3377 && ((cmp_status->state == COMPOSING_COMPONENT_CHAR \
3378 && cmp_status->method == COMPOSITION_WITH_ALTCHARS) \
3379 || (cmp_status->state == COMPOSING_COMPONENT_RULE \
3380 && cmp_status->method == COMPOSITION_WITH_RULE_ALTCHARS))) \
3381 { \
3382 *charbuf++ = -1; \
3383 *charbuf++= -1; \
3384 cmp_status->state = COMPOSING_CHAR; \
3385 cmp_status->length += 2; \
3386 } \
3387 else \
3388 { \
3389 MAYBE_FINISH_COMPOSITION (); \
3390 cmp_status->method = (c1 == '0' ? COMPOSITION_RELATIVE \
3391 : c1 == '2' ? COMPOSITION_WITH_RULE \
3392 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
3393 : COMPOSITION_WITH_RULE_ALTCHARS); \
3394 cmp_status->state \
3395 = (c1 <= '2' ? COMPOSING_CHAR : COMPOSING_COMPONENT_CHAR); \
3396 ADD_COMPOSITION_DATA (charbuf, 0, 0, cmp_status->method); \
3397 cmp_status->length = MAX_ANNOTATION_LENGTH; \
3398 cmp_status->nchars = cmp_status->ncomps = 0; \
3399 coding->annotated = 1; \
3400 } \
ec6d2bb8
KH		 3401 } while (0)
3402
ec6d2bb8 3403
e951386e 3404/* Handle composition end sequence ESC 1. */
df7492f9
KH		 3405
3406#define DECODE_COMPOSITION_END() \
ec6d2bb8 3407 do { \
e951386e
KH		 3408 if (cmp_status->nchars == 0 \
3409 || ((cmp_status->state == COMPOSING_CHAR) \
3410 == (cmp_status->method == COMPOSITION_WITH_RULE))) \
ec6d2bb8 3411 { \
e951386e
KH		 3412 MAYBE_FINISH_COMPOSITION (); \
3413 goto invalid_code; \
ec6d2bb8 3414 } \
e951386e
KH		 3415 if (cmp_status->method == COMPOSITION_WITH_ALTCHARS) \
3416 charbuf[- cmp_status->length] -= cmp_status->ncomps + 2; \
3417 else if (cmp_status->method == COMPOSITION_WITH_RULE_ALTCHARS) \
3418 charbuf[- cmp_status->length] -= cmp_status->ncomps * 3; \
3419 charbuf[- cmp_status->length + 2] = cmp_status->nchars; \
3420 char_offset += cmp_status->nchars; \
3421 cmp_status->state = COMPOSING_NO; \
ec6d2bb8
KH		 3422 } while (0)
3423
e951386e 3424/* Store a composition rule RULE in charbuf, and update cmp_status. */
df7492f9 3425
e951386e
KH		 3426#define STORE_COMPOSITION_RULE(rule) \
3427 do { \
3428 *charbuf++ = -2; \
3429 *charbuf++ = rule; \
3430 cmp_status->length += 2; \
3431 cmp_status->state--; \
3432 } while (0)
ec6d2bb8 3433
e951386e
KH		 3434/* Store a composed char or a component char C in charbuf, and update
3435 cmp_status. */
3436
3437#define STORE_COMPOSITION_CHAR(c) \
ec6d2bb8 3438 do { \
e951386e
KH		 3439 *charbuf++ = (c); \
3440 cmp_status->length++; \
3441 if (cmp_status->state == COMPOSING_CHAR) \
3442 cmp_status->nchars++; \
df7492f9 3443 else \
e951386e
KH		 3444 cmp_status->ncomps++; \
3445 if (cmp_status->method == COMPOSITION_WITH_RULE \
3446 || (cmp_status->method == COMPOSITION_WITH_RULE_ALTCHARS \
3447 && cmp_status->state == COMPOSING_COMPONENT_CHAR)) \
3448 cmp_status->state++; \
ec6d2bb8 3449 } while (0)
88993dfd 3450
d46c5b12 3451
4ed46869
KH		 3452/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3453
b73bfc1c 3454static void
971de7fb 3455decode_coding_iso_2022 (struct coding_system *coding)
4ed46869 3456{
8f924df7
KH		 3457 const unsigned char *src = coding->source + coding->consumed;
3458 const unsigned char *src_end = coding->source + coding->src_bytes;
3459 const unsigned char *src_base;
69a80ea3 3460 int *charbuf = coding->charbuf + coding->charbuf_used;
ad1746f5
KH		 3461 /* We may produce two annotations (charset and composition) in one
3462 loop and one more charset annotation at the end. */
ff0dacd7 3463 int *charbuf_end
df80c7f0 3464 = coding->charbuf + coding->charbuf_size - (MAX_ANNOTATION_LENGTH * 3);
d311d28c 3465 ptrdiff_t consumed_chars = 0, consumed_chars_base;
df7492f9 3466 int multibytep = coding->src_multibyte;
4ed46869 3467 /* Charsets invoked to graphic plane 0 and 1 respectively. */
df7492f9
KH		 3468 int charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3469 int charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
134b9549 3470 int charset_id_2, charset_id_3;
df7492f9
KH		 3471 struct charset *charset;
3472 int c;
e951386e 3473 struct composition_status *cmp_status = CODING_ISO_CMP_STATUS (coding);
66ebf983 3474 Lisp_Object attrs = CODING_ID_ATTRS (coding->id);
d311d28c
PE		 3475 ptrdiff_t char_offset = coding->produced_char;
3476 ptrdiff_t last_offset = char_offset;
ff0dacd7 3477 int last_id = charset_ascii;
2735d060 3478 int eol_dos =
0a9564cb 3479 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 3480 int byte_after_cr = -1;
e951386e 3481 int i;
df7492f9 3482
df7492f9 3483 setup_iso_safe_charsets (attrs);
1b3b981b 3484 coding->safe_charsets = SDATA (CODING_ATTR_SAFE_CHARSETS (attrs));
b73bfc1c 3485
e951386e
KH		 3486 if (cmp_status->state != COMPOSING_NO)
3487 {
15cbd324
EZ		 3488 if (charbuf_end - charbuf < cmp_status->length)
3489 abort ();
e951386e
KH		 3490 for (i = 0; i < cmp_status->length; i++)
3491 *charbuf++ = cmp_status->carryover[i];
3492 coding->annotated = 1;
3493 }
3494
b73bfc1c 3495 while (1)
4ed46869 3496 {
cf299835 3497 int c1, c2, c3;
b73bfc1c
KH		 3498
3499 src_base = src;
df7492f9
KH		 3500 consumed_chars_base = consumed_chars;
3501
3502 if (charbuf >= charbuf_end)
b71f6f73
KH		 3503 {
3504 if (byte_after_cr >= 0)
3505 src_base--;
3506 break;
3507 }
df7492f9 3508
119852e7
KH		 3509 if (byte_after_cr >= 0)
3510 c1 = byte_after_cr, byte_after_cr = -1;
3511 else
3512 ONE_MORE_BYTE (c1);
065e3595
KH		 3513 if (c1 < 0)
3514 goto invalid_code;
4ed46869 3515
e951386e 3516 if (CODING_ISO_EXTSEGMENT_LEN (coding) > 0)
4ed46869 3517 {
e951386e
KH		 3518 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
3519 char_offset++;
3520 CODING_ISO_EXTSEGMENT_LEN (coding)--;
3521 continue;
3522 }
3523
3524 if (CODING_ISO_EMBEDDED_UTF_8 (coding))
3525 {
3526 if (c1 == ISO_CODE_ESC)
ec6d2bb8 3527 {
e951386e
KH		 3528 if (src + 1 >= src_end)
3529 goto no_more_source;
3530 *charbuf++ = ISO_CODE_ESC;
3531 char_offset++;
3532 if (src[0] == '%' && src[1] == '@')
df7492f9 3533 {
e951386e
KH		 3534 src += 2;
3535 consumed_chars += 2;
3536 char_offset += 2;
3537 /* We are sure charbuf can contain two more chars. */
3538 *charbuf++ = '%';
3539 *charbuf++ = '@';
3540 CODING_ISO_EMBEDDED_UTF_8 (coding) = 0;
df7492f9 3541 }
4ed46869 3542 }
e951386e
KH		 3543 else
3544 {
3545 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
3546 char_offset++;
3547 }
3548 continue;
3549 }
3550
3551 if ((cmp_status->state == COMPOSING_RULE
3552 || cmp_status->state == COMPOSING_COMPONENT_RULE)
3553 && c1 != ISO_CODE_ESC)
3554 {
66ebf983 3555 int rule;
e951386e 3556
66ebf983 3557 DECODE_COMPOSITION_RULE (rule);
e951386e
KH		 3558 STORE_COMPOSITION_RULE (rule);
3559 continue;
3560 }
3561
3562 /* We produce at most one character. */
3563 switch (iso_code_class [c1])
3564 {
3565 case ISO_0x20_or_0x7F:
df7492f9
KH		 3566 if (charset_id_0 < 0
3567 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_0)))
781d7a48
KH		 3568 /* This is SPACE or DEL. */
3569 charset = CHARSET_FROM_ID (charset_ascii);
3570 else
3571 charset = CHARSET_FROM_ID (charset_id_0);
3572 break;
4ed46869
KH		 3573
3574 case ISO_graphic_plane_0:
134b9549
KH		 3575 if (charset_id_0 < 0)
3576 charset = CHARSET_FROM_ID (charset_ascii);
3577 else
3578 charset = CHARSET_FROM_ID (charset_id_0);
4ed46869
KH		 3579 break;
3580
3581 case ISO_0xA0_or_0xFF:
df7492f9
KH		 3582 if (charset_id_1 < 0
3583 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_1))
3584 || CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS)
3585 goto invalid_code;
4ed46869
KH		 3586 /* This is a graphic character, we fall down ... */
3587
3588 case ISO_graphic_plane_1:
df7492f9
KH		 3589 if (charset_id_1 < 0)
3590 goto invalid_code;
3591 charset = CHARSET_FROM_ID (charset_id_1);
4ed46869
KH		 3592 break;
3593
df7492f9 3594 case ISO_control_0:
2735d060 3595 if (eol_dos && c1 == '\r')
119852e7 3596 ONE_MORE_BYTE (byte_after_cr);
df7492f9
KH		 3597 MAYBE_FINISH_COMPOSITION ();
3598 charset = CHARSET_FROM_ID (charset_ascii);
4ed46869
KH		 3599 break;
3600
df7492f9 3601 case ISO_control_1:
df7492f9
KH		 3602 goto invalid_code;
3603
4ed46869 3604 case ISO_shift_out:
df7492f9
KH		 3605 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3606 || CODING_ISO_DESIGNATION (coding, 1) < 0)
3607 goto invalid_code;
3608 CODING_ISO_INVOCATION (coding, 0) = 1;
3609 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3610 continue;
4ed46869
KH		 3611
3612 case ISO_shift_in:
df7492f9
KH		 3613 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT))
3614 goto invalid_code;
3615 CODING_ISO_INVOCATION (coding, 0) = 0;
3616 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3617 continue;
4ed46869
KH		 3618
3619 case ISO_single_shift_2_7:
a63dba42
KH		 3620 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS))
3621 goto invalid_code;
4ed46869 3622 case ISO_single_shift_2:
df7492f9
KH		 3623 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3624 goto invalid_code;
4ed46869
KH		 3625 /* SS2 is handled as an escape sequence of ESC 'N' */
3626 c1 = 'N';
3627 goto label_escape_sequence;
3628
3629 case ISO_single_shift_3:
df7492f9
KH		 3630 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3631 goto invalid_code;
4ed46869
KH		 3632 /* SS2 is handled as an escape sequence of ESC 'O' */
3633 c1 = 'O';
3634 goto label_escape_sequence;
3635
3636 case ISO_control_sequence_introducer:
3637 /* CSI is handled as an escape sequence of ESC '[' ... */
3638 c1 = '[';
3639 goto label_escape_sequence;
3640
3641 case ISO_escape:
3642 ONE_MORE_BYTE (c1);
3643 label_escape_sequence:
df7492f9 3644 /* Escape sequences handled here are invocation,
4ed46869
KH		 3645 designation, direction specification, and character
3646 composition specification. */
3647 switch (c1)
3648 {
3649 case '&': /* revision of following character set */
3650 ONE_MORE_BYTE (c1);
3651 if (!(c1 >= '@' && c1 <= '~'))
df7492f9 3652 goto invalid_code;
4ed46869
KH		 3653 ONE_MORE_BYTE (c1);
3654 if (c1 != ISO_CODE_ESC)
df7492f9 3655 goto invalid_code;
4ed46869
KH		 3656 ONE_MORE_BYTE (c1);
3657 goto label_escape_sequence;
3658
3659 case '$': /* designation of 2-byte character set */
df7492f9
KH		 3660 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3661 goto invalid_code;
134b9549
KH		 3662 {
3663 int reg, chars96;
3664
3665 ONE_MORE_BYTE (c1);
3666 if (c1 >= '@' && c1 <= 'B')
3667 { /* designation of JISX0208.1978, GB2312.1980,
88993dfd 3668 or JISX0208.1980 */
134b9549
KH		 3669 reg = 0, chars96 = 0;
3670 }
3671 else if (c1 >= 0x28 && c1 <= 0x2B)
3672 { /* designation of DIMENSION2_CHARS94 character set */
3673 reg = c1 - 0x28, chars96 = 0;
3674 ONE_MORE_BYTE (c1);
3675 }
3676 else if (c1 >= 0x2C && c1 <= 0x2F)
3677 { /* designation of DIMENSION2_CHARS96 character set */
3678 reg = c1 - 0x2C, chars96 = 1;
3679 ONE_MORE_BYTE (c1);
3680 }
3681 else
3682 goto invalid_code;
3683 DECODE_DESIGNATION (reg, 2, chars96, c1);
3684 /* We must update these variables now. */
3685 if (reg == 0)
3686 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3687 else if (reg == 1)
3688 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3689 if (chars96 < 0)
3690 goto invalid_code;
3691 }
b73bfc1c 3692 continue;
4ed46869
KH		 3693
3694 case 'n': /* invocation of locking-shift-2 */
df7492f9
KH		 3695 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3696 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3697 goto invalid_code;
3698 CODING_ISO_INVOCATION (coding, 0) = 2;
3699 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3700 continue;
4ed46869
KH		 3701
3702 case 'o': /* invocation of locking-shift-3 */
df7492f9
KH		 3703 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3704 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3705 goto invalid_code;
3706 CODING_ISO_INVOCATION (coding, 0) = 3;
3707 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3708 continue;
4ed46869
KH		 3709
3710 case 'N': /* invocation of single-shift-2 */
df7492f9
KH		 3711 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3712 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3713 goto invalid_code;
134b9549
KH		 3714 charset_id_2 = CODING_ISO_DESIGNATION (coding, 2);
3715 if (charset_id_2 < 0)
3716 charset = CHARSET_FROM_ID (charset_ascii);
3717 else
3718 charset = CHARSET_FROM_ID (charset_id_2);
b73bfc1c 3719 ONE_MORE_BYTE (c1);
e7046a18 3720 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3721 goto invalid_code;
4ed46869
KH		 3722 break;
3723
3724 case 'O': /* invocation of single-shift-3 */
df7492f9
KH		 3725 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3726 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3727 goto invalid_code;
134b9549
KH		 3728 charset_id_3 = CODING_ISO_DESIGNATION (coding, 3);
3729 if (charset_id_3 < 0)
3730 charset = CHARSET_FROM_ID (charset_ascii);
3731 else
3732 charset = CHARSET_FROM_ID (charset_id_3);
b73bfc1c 3733 ONE_MORE_BYTE (c1);
e7046a18 3734 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3735 goto invalid_code;
4ed46869
KH		 3736 break;
3737
ec6d2bb8 3738 case '0': case '2': case '3': case '4': /* start composition */
df7492f9
KH		 3739 if (! (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK))
3740 goto invalid_code;
e951386e
KH		 3741 if (last_id != charset_ascii)
3742 {
3743 ADD_CHARSET_DATA (charbuf, char_offset- last_offset, last_id);
3744 last_id = charset_ascii;
3745 last_offset = char_offset;
3746 }
ec6d2bb8 3747 DECODE_COMPOSITION_START (c1);
b73bfc1c 3748 continue;
4ed46869 3749
ec6d2bb8 3750 case '1': /* end composition */
e951386e 3751 if (cmp_status->state == COMPOSING_NO)
df7492f9
KH		 3752 goto invalid_code;
3753 DECODE_COMPOSITION_END ();
b73bfc1c 3754 continue;
4ed46869
KH		 3755
3756 case '[': /* specification of direction */
de59072a 3757 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DIRECTION))
df7492f9 3758 goto invalid_code;
4ed46869 3759 /* For the moment, nested direction is not supported.
d46c5b12 3760 So, `coding->mode & CODING_MODE_DIRECTION' zero means
ad1746f5 3761 left-to-right, and nonzero means right-to-left. */
4ed46869
KH		 3762 ONE_MORE_BYTE (c1);
3763 switch (c1)
3764 {
3765 case ']': /* end of the current direction */
d46c5b12 3766 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869
KH		 3767
3768 case '0': /* end of the current direction */
3769 case '1': /* start of left-to-right direction */
3770 ONE_MORE_BYTE (c1);
3771 if (c1 == ']')
d46c5b12 3772 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869 3773 else
df7492f9 3774 goto invalid_code;
4ed46869
KH		 3775 break;
3776
3777 case '2': /* start of right-to-left direction */
3778 ONE_MORE_BYTE (c1);
3779 if (c1 == ']')
d46c5b12 3780 coding->mode |= CODING_MODE_DIRECTION;
4ed46869 3781 else
df7492f9 3782 goto invalid_code;
4ed46869
KH		 3783 break;
3784
3785 default:
df7492f9 3786 goto invalid_code;
4ed46869 3787 }
b73bfc1c 3788 continue;
4ed46869 3789
103e0180 3790 case '%':
103e0180
KH		 3791 ONE_MORE_BYTE (c1);
3792 if (c1 == '/')
3793 {
3794 /* CTEXT extended segment:
3795 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
3796 We keep these bytes as is for the moment.
3797 They may be decoded by post-read-conversion. */
3798 int dim, M, L;
4776e638 3799 int size;
8f924df7 3800
103e0180 3801 ONE_MORE_BYTE (dim);
7a84eee5 3802 if (dim < '0' || dim > '4')
e951386e 3803 goto invalid_code;
103e0180 3804 ONE_MORE_BYTE (M);
e951386e
KH		 3805 if (M < 128)
3806 goto invalid_code;
103e0180 3807 ONE_MORE_BYTE (L);
e951386e
KH		 3808 if (L < 128)
3809 goto invalid_code;
103e0180 3810 size = ((M - 128) * 128) + (L - 128);
e951386e 3811 if (charbuf + 6 > charbuf_end)
4776e638
KH		 3812 goto break_loop;
3813 *charbuf++ = ISO_CODE_ESC;
3814 *charbuf++ = '%';
3815 *charbuf++ = '/';
3816 *charbuf++ = dim;
3817 *charbuf++ = BYTE8_TO_CHAR (M);
3818 *charbuf++ = BYTE8_TO_CHAR (L);
e951386e 3819 CODING_ISO_EXTSEGMENT_LEN (coding) = size;
103e0180
KH		 3820 }
3821 else if (c1 == 'G')
3822 {
103e0180
KH		 3823 /* XFree86 extension for embedding UTF-8 in CTEXT:
3824 ESC % G --UTF-8-BYTES-- ESC % @
3825 We keep these bytes as is for the moment.
3826 They may be decoded by post-read-conversion. */
e951386e 3827 if (charbuf + 3 > charbuf_end)
4776e638 3828 goto break_loop;
e951386e
KH		 3829 *charbuf++ = ISO_CODE_ESC;
3830 *charbuf++ = '%';
3831 *charbuf++ = 'G';
3832 CODING_ISO_EMBEDDED_UTF_8 (coding) = 1;
103e0180
KH		 3833 }
3834 else
4776e638 3835 goto invalid_code;
103e0180 3836 continue;
4776e638 3837 break;
103e0180 3838
4ed46869 3839 default:
df7492f9
KH		 3840 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3841 goto invalid_code;
134b9549
KH		 3842 {
3843 int reg, chars96;
3844
3845 if (c1 >= 0x28 && c1 <= 0x2B)
3846 { /* designation of DIMENSION1_CHARS94 character set */
3847 reg = c1 - 0x28, chars96 = 0;
3848 ONE_MORE_BYTE (c1);
3849 }
3850 else if (c1 >= 0x2C && c1 <= 0x2F)
3851 { /* designation of DIMENSION1_CHARS96 character set */
3852 reg = c1 - 0x2C, chars96 = 1;
3853 ONE_MORE_BYTE (c1);
3854 }
3855 else
3856 goto invalid_code;
3857 DECODE_DESIGNATION (reg, 1, chars96, c1);
3858 /* We must update these variables now. */
3859 if (reg == 0)
3860 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3861 else if (reg == 1)
3862 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3863 if (chars96 < 0)
3864 goto invalid_code;
3865 }
b73bfc1c 3866 continue;
4ed46869 3867 }
413bb2db
PE		 3868 break;
3869
3870 default:
3871 abort ();
b73bfc1c 3872 }
4ed46869 3873
e951386e
KH		 3874 if (cmp_status->state == COMPOSING_NO
3875 && charset->id != charset_ascii
ff0dacd7
KH		 3876 && last_id != charset->id)
3877 {
3878 if (last_id != charset_ascii)
69a80ea3 3879 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
ff0dacd7
KH		 3880 last_id = charset->id;
3881 last_offset = char_offset;
3882 }
3883
b73bfc1c 3884 /* Now we know CHARSET and 1st position code C1 of a character.
cf299835
KH		 3885 Produce a decoded character while getting 2nd and 3rd
3886 position codes C2, C3 if necessary. */
df7492f9 3887 if (CHARSET_DIMENSION (charset) > 1)
b73bfc1c
KH		 3888 {
3889 ONE_MORE_BYTE (c2);
cf299835
KH		 3890 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0)
3891 || ((c1 & 0x80) != (c2 & 0x80)))
b73bfc1c 3892 /* C2 is not in a valid range. */
df7492f9 3893 goto invalid_code;
cf299835
KH		 3894 if (CHARSET_DIMENSION (charset) == 2)
3895 c1 = (c1 << 8) | c2;
3896 else
df7492f9 3897 {
cf299835
KH		 3898 ONE_MORE_BYTE (c3);
3899 if (c3 < 0x20 || (c3 >= 0x80 && c3 < 0xA0)
3900 || ((c1 & 0x80) != (c3 & 0x80)))
3901 /* C3 is not in a valid range. */
df7492f9 3902 goto invalid_code;
cf299835 3903 c1 = (c1 << 16) | (c2 << 8) | c2;
df7492f9
KH		 3904 }
3905 }
cf299835 3906 c1 &= 0x7F7F7F;
df7492f9
KH		 3907 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c1, c);
3908 if (c < 0)
3909 {
3910 MAYBE_FINISH_COMPOSITION ();
3911 for (; src_base < src; src_base++, char_offset++)
3912 {
3913 if (ASCII_BYTE_P (*src_base))
3914 *charbuf++ = *src_base;
3915 else
3916 *charbuf++ = BYTE8_TO_CHAR (*src_base);
3917 }
3918 }
e951386e 3919 else if (cmp_status->state == COMPOSING_NO)
df7492f9
KH		 3920 {
3921 *charbuf++ = c;
3922 char_offset++;
4ed46869 3923 }
e951386e
KH		 3924 else if ((cmp_status->state == COMPOSING_CHAR
3925 ? cmp_status->nchars
3926 : cmp_status->ncomps)
3927 >= MAX_COMPOSITION_COMPONENTS)
781d7a48 3928 {
e951386e
KH		 3929 /* Too long composition. */
3930 MAYBE_FINISH_COMPOSITION ();
3931 *charbuf++ = c;
3932 char_offset++;
4ed46869 3933 }
e951386e
KH		 3934 else
3935 STORE_COMPOSITION_CHAR (c);
4ed46869
KH		 3936 continue;
3937
df7492f9
KH		 3938 invalid_code:
3939 MAYBE_FINISH_COMPOSITION ();
4ed46869 3940 src = src_base;
df7492f9
KH		 3941 consumed_chars = consumed_chars_base;
3942 ONE_MORE_BYTE (c);
065e3595 3943 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 3944 char_offset++;
df7492f9 3945 coding->errors++;
4776e638
KH		 3946 continue;
3947
3948 break_loop:
3949 break;
4ed46869 3950 }
fb88bf2d 3951
df7492f9 3952 no_more_source:
e951386e
KH		 3953 if (cmp_status->state != COMPOSING_NO)
3954 {
3955 if (coding->mode & CODING_MODE_LAST_BLOCK)
3956 MAYBE_FINISH_COMPOSITION ();
3957 else
3958 {
3959 charbuf -= cmp_status->length;
3960 for (i = 0; i < cmp_status->length; i++)
3961 cmp_status->carryover[i] = charbuf[i];
3962 }
3963 }
3964 else if (last_id != charset_ascii)
69a80ea3 3965 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 3966 coding->consumed_char += consumed_chars_base;
3967 coding->consumed = src_base - coding->source;
3968 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 3969}
3970
b73bfc1c 3971
f4dee582 3972/* ISO2022 encoding stuff. */
4ed46869
KH		 3973
3974/*
f4dee582 3975 It is not enough to say just "ISO2022" on encoding, we have to
df7492f9 3976 specify more details. In Emacs, each coding system of ISO2022
4ed46869 3977 variant has the following specifications:
df7492f9 3978 1. Initial designation to G0 thru G3.
4ed46869
KH		 3979 2. Allows short-form designation?
3980 3. ASCII should be designated to G0 before control characters?
3981 4. ASCII should be designated to G0 at end of line?
3982 5. 7-bit environment or 8-bit environment?
3983 6. Use locking-shift?
3984 7. Use Single-shift?
3985 And the following two are only for Japanese:
3986 8. Use ASCII in place of JIS0201-1976-Roman?
3987 9. Use JISX0208-1983 in place of JISX0208-1978?
df7492f9
KH		 3988 These specifications are encoded in CODING_ISO_FLAGS (coding) as flag bits
3989 defined by macros CODING_ISO_FLAG_XXX. See `coding.h' for more
f4dee582 3990 details.
4ed46869
KH		 3991*/
3992
3993/* Produce codes (escape sequence) for designating CHARSET to graphic
b73bfc1c
KH		 3994 register REG at DST, and increment DST. If <final-char> of CHARSET is
3995 '@', 'A', or 'B' and the coding system CODING allows, produce
3996 designation sequence of short-form. */
4ed46869
KH		 3997
3998#define ENCODE_DESIGNATION(charset, reg, coding) \
3999 do { \
df7492f9 4000 unsigned char final_char = CHARSET_ISO_FINAL (charset); \
675e2c69
DN		 4001 const char *intermediate_char_94 = "()*+"; \
4002 const char *intermediate_char_96 = ",-./"; \
df7492f9 4003 int revision = -1; \
df7492f9
KH		 4004 \
4005 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_REVISION) \
c197f191 4006 revision = CHARSET_ISO_REVISION (charset); \
df7492f9
KH		 4007 \
4008 if (revision >= 0) \
70c22245 4009 { \
df7492f9
KH		 4010 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '&'); \
4011 EMIT_ONE_BYTE ('@' + revision); \
4ed46869 4012 } \
df7492f9 4013 EMIT_ONE_ASCII_BYTE (ISO_CODE_ESC); \
4ed46869
KH		 4014 if (CHARSET_DIMENSION (charset) == 1) \
4015 { \
2735d060 4016 int b; \
df7492f9 4017 if (! CHARSET_ISO_CHARS_96 (charset)) \
2735d060 4018 b = intermediate_char_94[reg]; \
4ed46869 4019 else \
2735d060
PE		 4020 b = intermediate_char_96[reg]; \
4021 EMIT_ONE_ASCII_BYTE (b); \
4ed46869
KH		 4022 } \
4023 else \
4024 { \
df7492f9
KH		 4025 EMIT_ONE_ASCII_BYTE ('$'); \
4026 if (! CHARSET_ISO_CHARS_96 (charset)) \
4ed46869 4027 { \
df7492f9 4028 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LONG_FORM \
b73bfc1c
KH		 4029 || reg != 0 \
4030 || final_char < '@' || final_char > 'B') \
df7492f9 4031 EMIT_ONE_ASCII_BYTE (intermediate_char_94[reg]); \
4ed46869
KH		 4032 } \
4033 else \
df7492f9 4034 EMIT_ONE_ASCII_BYTE (intermediate_char_96[reg]); \
4ed46869 4035 } \
df7492f9
KH		 4036 EMIT_ONE_ASCII_BYTE (final_char); \
4037 \
4038 CODING_ISO_DESIGNATION (coding, reg) = CHARSET_ID (charset); \
4ed46869
KH		 4039 } while (0)
4040
df7492f9 4041
4ed46869
KH		 4042/* The following two macros produce codes (control character or escape
4043 sequence) for ISO2022 single-shift functions (single-shift-2 and
4044 single-shift-3). */
4045
df7492f9
KH		 4046#define ENCODE_SINGLE_SHIFT_2 \
4047 do { \
4048 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
4049 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'N'); \
4050 else \
4051 EMIT_ONE_BYTE (ISO_CODE_SS2); \
4052 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 4053 } while (0)
4054
df7492f9
KH		 4055
4056#define ENCODE_SINGLE_SHIFT_3 \
4057 do { \
4058 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
4059 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'O'); \
4060 else \
4061 EMIT_ONE_BYTE (ISO_CODE_SS3); \
4062 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 4063 } while (0)
4064
df7492f9 4065
4ed46869
KH		 4066/* The following four macros produce codes (control character or
4067 escape sequence) for ISO2022 locking-shift functions (shift-in,
4068 shift-out, locking-shift-2, and locking-shift-3). */
4069
df7492f9
KH		 4070#define ENCODE_SHIFT_IN \
4071 do { \
4072 EMIT_ONE_ASCII_BYTE (ISO_CODE_SI); \
4073 CODING_ISO_INVOCATION (coding, 0) = 0; \
4ed46869
KH		 4074 } while (0)
4075
df7492f9
KH		 4076
4077#define ENCODE_SHIFT_OUT \
4078 do { \
4079 EMIT_ONE_ASCII_BYTE (ISO_CODE_SO); \
4080 CODING_ISO_INVOCATION (coding, 0) = 1; \
4ed46869
KH		 4081 } while (0)
4082
df7492f9
KH		 4083
4084#define ENCODE_LOCKING_SHIFT_2 \
4085 do { \
4086 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
4087 CODING_ISO_INVOCATION (coding, 0) = 2; \
4ed46869
KH		 4088 } while (0)
4089
df7492f9
KH		 4090
4091#define ENCODE_LOCKING_SHIFT_3 \
4092 do { \
4093 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
4094 CODING_ISO_INVOCATION (coding, 0) = 3; \
4ed46869
KH		 4095 } while (0)
4096
df7492f9 4097
f4dee582
RS		 4098/* Produce codes for a DIMENSION1 character whose character set is
4099 CHARSET and whose position-code is C1. Designation and invocation
4ed46869
KH		 4100 sequences are also produced in advance if necessary. */
4101
6e85d753
KH		 4102#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
4103 do { \
df7492f9 4104 int id = CHARSET_ID (charset); \
bf16eb23
KH		 4105 \
4106 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
4107 && id == charset_ascii) \
4108 { \
4109 id = charset_jisx0201_roman; \
4110 charset = CHARSET_FROM_ID (id); \
4111 } \
4112 \
df7492f9 4113 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 4114 { \
df7492f9
KH		 4115 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
4116 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753 4117 else \
df7492f9
KH		 4118 EMIT_ONE_BYTE (c1 | 0x80); \
4119 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 4120 break; \
4121 } \
df7492f9 4122 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 4123 { \
df7492f9 4124 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753
KH		 4125 break; \
4126 } \
df7492f9 4127 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 4128 { \
df7492f9 4129 EMIT_ONE_BYTE (c1 | 0x80); \
6e85d753
KH		 4130 break; \
4131 } \
6e85d753
KH		 4132 else \
4133 /* Since CHARSET is not yet invoked to any graphic planes, we \
4134 must invoke it, or, at first, designate it to some graphic \
4135 register. Then repeat the loop to actually produce the \
4136 character. */ \
df7492f9
KH		 4137 dst = encode_invocation_designation (charset, coding, dst, \
4138 &produced_chars); \
4ed46869
KH		 4139 } while (1)
4140
df7492f9 4141
f4dee582
RS		 4142/* Produce codes for a DIMENSION2 character whose character set is
4143 CHARSET and whose position-codes are C1 and C2. Designation and
4ed46869
KH		 4144 invocation codes are also produced in advance if necessary. */
4145
6e85d753
KH		 4146#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
4147 do { \
df7492f9 4148 int id = CHARSET_ID (charset); \
bf16eb23
KH		 4149 \
4150 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
4151 && id == charset_jisx0208) \
4152 { \
4153 id = charset_jisx0208_1978; \
4154 charset = CHARSET_FROM_ID (id); \
4155 } \
4156 \
df7492f9 4157 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 4158 { \
df7492f9
KH		 4159 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
4160 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753 4161 else \
df7492f9
KH		 4162 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
4163 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 4164 break; \
4165 } \
df7492f9 4166 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 4167 { \
df7492f9 4168 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753
KH		 4169 break; \
4170 } \
df7492f9 4171 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 4172 { \
df7492f9 4173 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
6e85d753
KH		 4174 break; \
4175 } \
6e85d753
KH		 4176 else \
4177 /* Since CHARSET is not yet invoked to any graphic planes, we \
4178 must invoke it, or, at first, designate it to some graphic \
4179 register. Then repeat the loop to actually produce the \
4180 character. */ \
df7492f9
KH		 4181 dst = encode_invocation_designation (charset, coding, dst, \
4182 &produced_chars); \
4ed46869
KH		 4183 } while (1)
4184
05e6f5dc 4185
df7492f9
KH		 4186#define ENCODE_ISO_CHARACTER(charset, c) \
4187 do { \
47664caa 4188 unsigned code = ENCODE_CHAR ((charset), (c)); \
df7492f9
KH		 4189 \
4190 if (CHARSET_DIMENSION (charset) == 1) \
4191 ENCODE_ISO_CHARACTER_DIMENSION1 ((charset), code); \
4192 else \
4193 ENCODE_ISO_CHARACTER_DIMENSION2 ((charset), code >> 8, code & 0xFF); \
84fbb8a0 4194 } while (0)
bdd9fb48 4195
05e6f5dc 4196
4ed46869 4197/* Produce designation and invocation codes at a place pointed by DST
df7492f9 4198 to use CHARSET. The element `spec.iso_2022' of *CODING is updated.
4ed46869
KH		 4199 Return new DST. */
4200
e2f1bab9 4201static unsigned char *
cf84bb53
JB		 4202encode_invocation_designation (struct charset *charset,
4203 struct coding_system *coding,
d311d28c 4204 unsigned char *dst, ptrdiff_t *p_nchars)
4ed46869 4205{
df7492f9 4206 int multibytep = coding->dst_multibyte;
d311d28c 4207 ptrdiff_t produced_chars = *p_nchars;
4ed46869 4208 int reg; /* graphic register number */
df7492f9 4209 int id = CHARSET_ID (charset);
4ed46869
KH		 4210
4211 /* At first, check designations. */
4212 for (reg = 0; reg < 4; reg++)
df7492f9 4213 if (id == CODING_ISO_DESIGNATION (coding, reg))
4ed46869
KH		 4214 break;
4215
4216 if (reg >= 4)
4217 {
4218 /* CHARSET is not yet designated to any graphic registers. */
4219 /* At first check the requested designation. */
df7492f9
KH		 4220 reg = CODING_ISO_REQUEST (coding, id);
4221 if (reg < 0)
1ba9e4ab
KH		 4222 /* Since CHARSET requests no special designation, designate it
4223 to graphic register 0. */
4ed46869
KH		 4224 reg = 0;
4225
4226 ENCODE_DESIGNATION (charset, reg, coding);
4227 }
4228
df7492f9
KH		 4229 if (CODING_ISO_INVOCATION (coding, 0) != reg
4230 && CODING_ISO_INVOCATION (coding, 1) != reg)
4ed46869
KH		 4231 {
4232 /* Since the graphic register REG is not invoked to any graphic
4233 planes, invoke it to graphic plane 0. */
4234 switch (reg)
4235 {
4236 case 0: /* graphic register 0 */
4237 ENCODE_SHIFT_IN;
4238 break;
4239
4240 case 1: /* graphic register 1 */
4241 ENCODE_SHIFT_OUT;
4242 break;
4243
4244 case 2: /* graphic register 2 */
df7492f9 4245 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 4246 ENCODE_SINGLE_SHIFT_2;
4247 else
4248 ENCODE_LOCKING_SHIFT_2;
4249 break;
4250
4251 case 3: /* graphic register 3 */
df7492f9 4252 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 4253 ENCODE_SINGLE_SHIFT_3;
4254 else
4255 ENCODE_LOCKING_SHIFT_3;
4256 break;
4257 }
4258 }
b73bfc1c 4259
df7492f9 4260 *p_nchars = produced_chars;
4ed46869
KH		 4261 return dst;
4262}
4263
4ed46869
KH		 4264
4265/* Produce codes for designation and invocation to reset the graphic
4266 planes and registers to initial state. */
df7492f9
KH		 4267#define ENCODE_RESET_PLANE_AND_REGISTER() \
4268 do { \
4269 int reg; \
4270 struct charset *charset; \
4271 \
4272 if (CODING_ISO_INVOCATION (coding, 0) != 0) \
4273 ENCODE_SHIFT_IN; \
4274 for (reg = 0; reg < 4; reg++) \
4275 if (CODING_ISO_INITIAL (coding, reg) >= 0 \
4276 && (CODING_ISO_DESIGNATION (coding, reg) \
4277 != CODING_ISO_INITIAL (coding, reg))) \
4278 { \
4279 charset = CHARSET_FROM_ID (CODING_ISO_INITIAL (coding, reg)); \
4280 ENCODE_DESIGNATION (charset, reg, coding); \
4281 } \
4ed46869
KH		 4282 } while (0)
4283
df7492f9 4284
bdd9fb48 4285/* Produce designation sequences of charsets in the line started from
b73bfc1c 4286 SRC to a place pointed by DST, and return updated DST.
bdd9fb48
KH		 4287
4288 If the current block ends before any end-of-line, we may fail to
d46c5b12
KH		 4289 find all the necessary designations. */
4290
b73bfc1c 4291static unsigned char *
cf84bb53 4292encode_designation_at_bol (struct coding_system *coding, int *charbuf,
461c2ab9 4293 unsigned char *dst)
e0e989f6 4294{
df7492f9 4295 struct charset *charset;
bdd9fb48
KH		 4296 /* Table of charsets to be designated to each graphic register. */
4297 int r[4];
df7492f9 4298 int c, found = 0, reg;
d311d28c 4299 ptrdiff_t produced_chars = 0;
df7492f9
KH		 4300 int multibytep = coding->dst_multibyte;
4301 Lisp_Object attrs;
4302 Lisp_Object charset_list;
4303
4304 attrs = CODING_ID_ATTRS (coding->id);
4305 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
4306 if (EQ (charset_list, Qiso_2022))
4307 charset_list = Viso_2022_charset_list;
bdd9fb48
KH		 4308
4309 for (reg = 0; reg < 4; reg++)
4310 r[reg] = -1;
4311
b73bfc1c 4312 while (found < 4)
e0e989f6 4313 {
df7492f9
KH		 4314 int id;
4315
4316 c = *charbuf++;
b73bfc1c
KH		 4317 if (c == '\n')
4318 break;
df7492f9
KH		 4319 charset = char_charset (c, charset_list, NULL);
4320 id = CHARSET_ID (charset);
4321 reg = CODING_ISO_REQUEST (coding, id);
4322 if (reg >= 0 && r[reg] < 0)
bdd9fb48
KH		 4323 {
4324 found++;
df7492f9 4325 r[reg] = id;
bdd9fb48 4326 }
bdd9fb48
KH		 4327 }
4328
4329 if (found)
4330 {
4331 for (reg = 0; reg < 4; reg++)
4332 if (r[reg] >= 0
df7492f9
KH		 4333 && CODING_ISO_DESIGNATION (coding, reg) != r[reg])
4334 ENCODE_DESIGNATION (CHARSET_FROM_ID (r[reg]), reg, coding);
e0e989f6 4335 }
b73bfc1c
KH		 4336
4337 return dst;
e0e989f6
KH		 4338}
4339
4ed46869
KH		 4340/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
4341
df7492f9 4342static int
971de7fb 4343encode_coding_iso_2022 (struct coding_system *coding)
4ed46869 4344{
df7492f9
KH		 4345 int multibytep = coding->dst_multibyte;
4346 int *charbuf = coding->charbuf;
4347 int *charbuf_end = charbuf + coding->charbuf_used;
4348 unsigned char *dst = coding->destination + coding->produced;
4349 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4350 int safe_room = 16;
4351 int bol_designation
4352 = (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL
4353 && CODING_ISO_BOL (coding));
d311d28c 4354 ptrdiff_t produced_chars = 0;
df7492f9
KH		 4355 Lisp_Object attrs, eol_type, charset_list;
4356 int ascii_compatible;
b73bfc1c 4357 int c;
ff0dacd7 4358 int preferred_charset_id = -1;
05e6f5dc 4359
24a73b0a 4360 CODING_GET_INFO (coding, attrs, charset_list);
0a9564cb 4361 eol_type = inhibit_eol_conversion ? Qunix : CODING_ID_EOL_TYPE (coding->id);
24a73b0a
KH		 4362 if (VECTORP (eol_type))
4363 eol_type = Qunix;
4364
004068e4 4365 setup_iso_safe_charsets (attrs);
ff0dacd7 4366 /* Charset list may have been changed. */
287c57d7 4367 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
1b3b981b 4368 coding->safe_charsets = SDATA (CODING_ATTR_SAFE_CHARSETS (attrs));
0eecad43 4369
a552b35a
KH		 4370 ascii_compatible
4371 = (! NILP (CODING_ATTR_ASCII_COMPAT (attrs))
4372 && ! (CODING_ISO_FLAGS (coding) & (CODING_ISO_FLAG_DESIGNATION
4373 | CODING_ISO_FLAG_LOCKING_SHIFT)));
bdd9fb48 4374
df7492f9 4375 while (charbuf < charbuf_end)
4ed46869 4376 {
df7492f9 4377 ASSURE_DESTINATION (safe_room);
b73bfc1c 4378
df7492f9 4379 if (bol_designation)
b73bfc1c 4380 {
df7492f9 4381 unsigned char *dst_prev = dst;
4ed46869 4382
bdd9fb48 4383 /* We have to produce designation sequences if any now. */
461c2ab9 4384 dst = encode_designation_at_bol (coding, charbuf, dst);
df7492f9
KH		 4385 bol_designation = 0;
4386 /* We are sure that designation sequences are all ASCII bytes. */
4387 produced_chars += dst - dst_prev;
e0e989f6
KH		 4388 }
4389
df7492f9 4390 c = *charbuf++;
ec6d2bb8 4391
ff0dacd7
KH		 4392 if (c < 0)
4393 {
4394 /* Handle an annotation. */
4395 switch (*charbuf)
ec6d2bb8 4396 {
ff0dacd7
KH		 4397 case CODING_ANNOTATE_COMPOSITION_MASK:
4398 /* Not yet implemented. */
4399 break;
4400 case CODING_ANNOTATE_CHARSET_MASK:
cf7dfdf5 4401 preferred_charset_id = charbuf[2];
ff0dacd7
KH		 4402 if (preferred_charset_id >= 0
4403 && NILP (Fmemq (make_number (preferred_charset_id),
4404 charset_list)))
4405 preferred_charset_id = -1;
4406 break;
4407 default:
4408 abort ();
4ed46869 4409 }
ff0dacd7
KH		 4410 charbuf += -c - 1;
4411 continue;
4ed46869 4412 }
ec6d2bb8 4413
b73bfc1c
KH		 4414 /* Now encode the character C. */
4415 if (c < 0x20 || c == 0x7F)
4416 {
df7492f9
KH		 4417 if (c == '\n'
4418 || (c == '\r' && EQ (eol_type, Qmac)))
19a8d9e0 4419 {
df7492f9
KH		 4420 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
4421 ENCODE_RESET_PLANE_AND_REGISTER ();
4422 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_INIT_AT_BOL)
b73bfc1c 4423 {
df7492f9
KH		 4424 int i;
4425
4426 for (i = 0; i < 4; i++)
4427 CODING_ISO_DESIGNATION (coding, i)
4428 = CODING_ISO_INITIAL (coding, i);
b73bfc1c 4429 }
df7492f9
KH		 4430 bol_designation
4431 = CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL;
19a8d9e0 4432 }
df7492f9
KH		 4433 else if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_CNTL)
4434 ENCODE_RESET_PLANE_AND_REGISTER ();
4435 EMIT_ONE_ASCII_BYTE (c);
4ed46869 4436 }
df7492f9 4437 else if (ASCII_CHAR_P (c))
88993dfd 4438 {
df7492f9
KH		 4439 if (ascii_compatible)
4440 EMIT_ONE_ASCII_BYTE (c);
93dec019 4441 else
19a8d9e0 4442 {
bf16eb23
KH		 4443 struct charset *charset = CHARSET_FROM_ID (charset_ascii);
4444 ENCODE_ISO_CHARACTER (charset, c);
19a8d9e0 4445 }
4ed46869 4446 }
16eafb5d 4447 else if (CHAR_BYTE8_P (c))
88993dfd 4448 {
16eafb5d
KH		 4449 c = CHAR_TO_BYTE8 (c);
4450 EMIT_ONE_BYTE (c);
88993dfd 4451 }
b73bfc1c 4452 else
df7492f9 4453 {
ff0dacd7 4454 struct charset *charset;
b73bfc1c 4455
ff0dacd7
KH		 4456 if (preferred_charset_id >= 0)
4457 {
4458 charset = CHARSET_FROM_ID (preferred_charset_id);
4459 if (! CHAR_CHARSET_P (c, charset))
4460 charset = char_charset (c, charset_list, NULL);
4461 }
4462 else
4463 charset = char_charset (c, charset_list, NULL);
df7492f9
KH		 4464 if (!charset)
4465 {
41cbe562
KH		 4466 if (coding->mode & CODING_MODE_SAFE_ENCODING)
4467 {
4468 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4469 charset = CHARSET_FROM_ID (charset_ascii);
4470 }
4471 else
4472 {
4473 c = coding->default_char;
4474 charset = char_charset (c, charset_list, NULL);
4475 }
df7492f9
KH		 4476 }
4477 ENCODE_ISO_CHARACTER (charset, c);
4478 }
84fbb8a0 4479 }
b73bfc1c 4480
df7492f9
KH		 4481 if (coding->mode & CODING_MODE_LAST_BLOCK
4482 && CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
4483 {
4484 ASSURE_DESTINATION (safe_room);
4485 ENCODE_RESET_PLANE_AND_REGISTER ();
4486 }
065e3595 4487 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4488 CODING_ISO_BOL (coding) = bol_designation;
4489 coding->produced_char += produced_chars;
4490 coding->produced = dst - coding->destination;
4491 return 0;
4ed46869
KH		 4492}
4493
4494\f
df7492f9 4495/*** 8,9. SJIS and BIG5 handlers ***/
4ed46869 4496
df7492f9 4497/* Although SJIS and BIG5 are not ISO's coding system, they are used
4ed46869
KH		 4498 quite widely. So, for the moment, Emacs supports them in the bare
4499 C code. But, in the future, they may be supported only by CCL. */
4500
4501/* SJIS is a coding system encoding three character sets: ASCII, right
4502 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
4503 as is. A character of charset katakana-jisx0201 is encoded by
4504 "position-code + 0x80". A character of charset japanese-jisx0208
4505 is encoded in 2-byte but two position-codes are divided and shifted
df7492f9 4506 so that it fit in the range below.
4ed46869
KH		 4507
4508 --- CODE RANGE of SJIS ---
4509 (character set) (range)
4510 ASCII 0x00 .. 0x7F
df7492f9 4511 KATAKANA-JISX0201 0xA0 .. 0xDF
c28a9453 4512 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
d14d03ac 4513 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
4ed46869
KH		 4514 -------------------------------
4515
4516*/
4517
4518/* BIG5 is a coding system encoding two character sets: ASCII and
4519 Big5. An ASCII character is encoded as is. Big5 is a two-byte
df7492f9 4520 character set and is encoded in two-byte.
4ed46869
KH		 4521
4522 --- CODE RANGE of BIG5 ---
4523 (character set) (range)
4524 ASCII 0x00 .. 0x7F
4525 Big5 (1st byte) 0xA1 .. 0xFE
4526 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
4527 --------------------------
4528
df7492f9 4529 */
4ed46869
KH		 4530
4531/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4532 Check if a text is encoded in SJIS. If it is, return
df7492f9 4533 CATEGORY_MASK_SJIS, else return 0. */
4ed46869 4534
0a28aafb 4535static int
cf84bb53
JB		 4536detect_coding_sjis (struct coding_system *coding,
4537 struct coding_detection_info *detect_info)
4ed46869 4538{
065e3595 4539 const unsigned char *src = coding->source, *src_base;
8f924df7 4540 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 4541 int multibytep = coding->src_multibyte;
d311d28c 4542 ptrdiff_t consumed_chars = 0;
df7492f9 4543 int found = 0;
b73bfc1c 4544 int c;
f07190ca
KH		 4545 Lisp_Object attrs, charset_list;
4546 int max_first_byte_of_2_byte_code;
4547
4548 CODING_GET_INFO (coding, attrs, charset_list);
4549 max_first_byte_of_2_byte_code
4550 = (XINT (Flength (charset_list)) > 3 ? 0xFC : 0xEF);
df7492f9 4551
ff0dacd7 4552 detect_info->checked |= CATEGORY_MASK_SJIS;
df7492f9
KH		 4553 /* A coding system of this category is always ASCII compatible. */
4554 src += coding->head_ascii;
4ed46869 4555
b73bfc1c 4556 while (1)
4ed46869 4557 {
065e3595 4558 src_base = src;
df7492f9 4559 ONE_MORE_BYTE (c);
682169fe
KH		 4560 if (c < 0x80)
4561 continue;
f07190ca
KH		 4562 if ((c >= 0x81 && c <= 0x9F)
4563 || (c >= 0xE0 && c <= max_first_byte_of_2_byte_code))
4ed46869 4564 {
df7492f9 4565 ONE_MORE_BYTE (c);
682169fe 4566 if (c < 0x40 || c == 0x7F || c > 0xFC)
df7492f9 4567 break;
ff0dacd7 4568 found = CATEGORY_MASK_SJIS;
4ed46869 4569 }
df7492f9 4570 else if (c >= 0xA0 && c < 0xE0)
ff0dacd7 4571 found = CATEGORY_MASK_SJIS;
df7492f9
KH		 4572 else
4573 break;
4ed46869 4574 }
ff0dacd7 4575 detect_info->rejected |= CATEGORY_MASK_SJIS;
df7492f9
KH		 4576 return 0;
4577
4578 no_more_source:
065e3595 4579 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 4580 {
ff0dacd7 4581 detect_info->rejected |= CATEGORY_MASK_SJIS;
89528eb3 4582 return 0;
4ed46869 4583 }
ff0dacd7
KH		 4584 detect_info->found |= found;
4585 return 1;
4ed46869
KH		 4586}
4587
4588/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4589 Check if a text is encoded in BIG5. If it is, return
df7492f9 4590 CATEGORY_MASK_BIG5, else return 0. */
4ed46869 4591
0a28aafb 4592static int
cf84bb53
JB		 4593detect_coding_big5 (struct coding_system *coding,
4594 struct coding_detection_info *detect_info)
4ed46869 4595{
065e3595 4596 const unsigned char *src = coding->source, *src_base;
8f924df7 4597 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 4598 int multibytep = coding->src_multibyte;
d311d28c 4599 ptrdiff_t consumed_chars = 0;
df7492f9 4600 int found = 0;
b73bfc1c 4601 int c;
fa42c37f 4602
ff0dacd7 4603 detect_info->checked |= CATEGORY_MASK_BIG5;
df7492f9
KH		 4604 /* A coding system of this category is always ASCII compatible. */
4605 src += coding->head_ascii;
fa42c37f 4606
b73bfc1c 4607 while (1)
fa42c37f 4608 {
065e3595 4609 src_base = src;
df7492f9
KH		 4610 ONE_MORE_BYTE (c);
4611 if (c < 0x80)
fa42c37f 4612 continue;
df7492f9 4613 if (c >= 0xA1)
fa42c37f 4614 {
df7492f9
KH		 4615 ONE_MORE_BYTE (c);
4616 if (c < 0x40 || (c >= 0x7F && c <= 0xA0))
fa42c37f 4617 return 0;
ff0dacd7 4618 found = CATEGORY_MASK_BIG5;
fa42c37f 4619 }
df7492f9
KH		 4620 else
4621 break;
fa42c37f 4622 }
ff0dacd7 4623 detect_info->rejected |= CATEGORY_MASK_BIG5;
fa42c37f 4624 return 0;
fa42c37f 4625
df7492f9 4626 no_more_source:
065e3595 4627 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 4628 {
ff0dacd7 4629 detect_info->rejected |= CATEGORY_MASK_BIG5;
89528eb3
KH		 4630 return 0;
4631 }
ff0dacd7
KH		 4632 detect_info->found |= found;
4633 return 1;
fa42c37f
KH		 4634}
4635
4ed46869
KH		 4636/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
4637 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
fa42c37f 4638
b73bfc1c 4639static void
971de7fb 4640decode_coding_sjis (struct coding_system *coding)
4ed46869 4641{
8f924df7
KH		 4642 const unsigned char *src = coding->source + coding->consumed;
4643 const unsigned char *src_end = coding->source + coding->src_bytes;
4644 const unsigned char *src_base;
69a80ea3 4645 int *charbuf = coding->charbuf + coding->charbuf_used;
ad1746f5 4646 /* We may produce one charset annotation in one loop and one more at
df80c7f0 4647 the end. */
69a80ea3 4648 int *charbuf_end
df80c7f0 4649 = coding->charbuf + coding->charbuf_size - (MAX_ANNOTATION_LENGTH * 2);
d311d28c 4650 ptrdiff_t consumed_chars = 0, consumed_chars_base;
df7492f9
KH		 4651 int multibytep = coding->src_multibyte;
4652 struct charset *charset_roman, *charset_kanji, *charset_kana;
57a47f8a 4653 struct charset *charset_kanji2;
24a73b0a 4654 Lisp_Object attrs, charset_list, val;
d311d28c
PE		 4655 ptrdiff_t char_offset = coding->produced_char;
4656 ptrdiff_t last_offset = char_offset;
ff0dacd7 4657 int last_id = charset_ascii;
2735d060 4658 int eol_dos =
0a9564cb 4659 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 4660 int byte_after_cr = -1;
a5d301df 4661
24a73b0a 4662 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4663
4664 val = charset_list;
4665 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
89528eb3 4666 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4667 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4668 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 4669
b73bfc1c 4670 while (1)
4ed46869 4671 {
df7492f9 4672 int c, c1;
24a73b0a 4673 struct charset *charset;
fa42c37f 4674
b73bfc1c 4675 src_base = src;
df7492f9 4676 consumed_chars_base = consumed_chars;
fa42c37f 4677
df7492f9 4678 if (charbuf >= charbuf_end)
b71f6f73
KH		 4679 {
4680 if (byte_after_cr >= 0)
4681 src_base--;
4682 break;
4683 }
df7492f9 4684
119852e7
KH		 4685 if (byte_after_cr >= 0)
4686 c = byte_after_cr, byte_after_cr = -1;
4687 else
4688 ONE_MORE_BYTE (c);
065e3595
KH		 4689 if (c < 0)
4690 goto invalid_code;
24a73b0a 4691 if (c < 0x80)
119852e7 4692 {
2735d060 4693 if (eol_dos && c == '\r')
119852e7
KH		 4694 ONE_MORE_BYTE (byte_after_cr);
4695 charset = charset_roman;
4696 }
57a47f8a 4697 else if (c == 0x80 || c == 0xA0)
8e921c4b 4698 goto invalid_code;
57a47f8a
KH		 4699 else if (c >= 0xA1 && c <= 0xDF)
4700 {
4701 /* SJIS -> JISX0201-Kana */
4702 c &= 0x7F;
4703 charset = charset_kana;
4704 }
4705 else if (c <= 0xEF)
df7492f9 4706 {
57a47f8a
KH		 4707 /* SJIS -> JISX0208 */
4708 ONE_MORE_BYTE (c1);
4709 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4710 goto invalid_code;
57a47f8a
KH		 4711 c = (c << 8) | c1;
4712 SJIS_TO_JIS (c);
4713 charset = charset_kanji;
4714 }
4715 else if (c <= 0xFC && charset_kanji2)
4716 {
c6876370 4717 /* SJIS -> JISX0213-2 */
57a47f8a
KH		 4718 ONE_MORE_BYTE (c1);
4719 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4720 goto invalid_code;
57a47f8a
KH		 4721 c = (c << 8) | c1;
4722 SJIS_TO_JIS2 (c);
4723 charset = charset_kanji2;
df7492f9 4724 }
57a47f8a
KH		 4725 else
4726 goto invalid_code;
24a73b0a
KH		 4727 if (charset->id != charset_ascii
4728 && last_id != charset->id)
4729 {
4730 if (last_id != charset_ascii)
69a80ea3 4731 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4732 last_id = charset->id;
4733 last_offset = char_offset;
4734 }
4735 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4736 *charbuf++ = c;
ff0dacd7 4737 char_offset++;
df7492f9 4738 continue;
b73bfc1c 4739
df7492f9
KH		 4740 invalid_code:
4741 src = src_base;
4742 consumed_chars = consumed_chars_base;
4743 ONE_MORE_BYTE (c);
065e3595 4744 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4745 char_offset++;
df7492f9
KH		 4746 coding->errors++;
4747 }
fa42c37f 4748
df7492f9 4749 no_more_source:
ff0dacd7 4750 if (last_id != charset_ascii)
69a80ea3 4751 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4752 coding->consumed_char += consumed_chars_base;
4753 coding->consumed = src_base - coding->source;
4754 coding->charbuf_used = charbuf - coding->charbuf;
fa42c37f
KH		 4755}
4756
b73bfc1c 4757static void
971de7fb 4758decode_coding_big5 (struct coding_system *coding)
4ed46869 4759{
8f924df7
KH		 4760 const unsigned char *src = coding->source + coding->consumed;
4761 const unsigned char *src_end = coding->source + coding->src_bytes;
4762 const unsigned char *src_base;
69a80ea3 4763 int *charbuf = coding->charbuf + coding->charbuf_used;
ad1746f5 4764 /* We may produce one charset annotation in one loop and one more at
df80c7f0 4765 the end. */
69a80ea3 4766 int *charbuf_end
df80c7f0 4767 = coding->charbuf + coding->charbuf_size - (MAX_ANNOTATION_LENGTH * 2);
d311d28c 4768 ptrdiff_t consumed_chars = 0, consumed_chars_base;
df7492f9
KH		 4769 int multibytep = coding->src_multibyte;
4770 struct charset *charset_roman, *charset_big5;
24a73b0a 4771 Lisp_Object attrs, charset_list, val;
d311d28c
PE		 4772 ptrdiff_t char_offset = coding->produced_char;
4773 ptrdiff_t last_offset = char_offset;
ff0dacd7 4774 int last_id = charset_ascii;
2735d060 4775 int eol_dos =
0a9564cb 4776 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 4777 int byte_after_cr = -1;
df7492f9 4778
24a73b0a 4779 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4780 val = charset_list;
4781 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4782 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4783
b73bfc1c 4784 while (1)
4ed46869 4785 {
df7492f9 4786 int c, c1;
24a73b0a 4787 struct charset *charset;
b73bfc1c
KH		 4788
4789 src_base = src;
df7492f9
KH		 4790 consumed_chars_base = consumed_chars;
4791
4792 if (charbuf >= charbuf_end)
b71f6f73
KH		 4793 {
4794 if (byte_after_cr >= 0)
4795 src_base--;
4796 break;
4797 }
df7492f9 4798
119852e7 4799 if (byte_after_cr >= 0)
14daee73 4800 c = byte_after_cr, byte_after_cr = -1;
119852e7
KH		 4801 else
4802 ONE_MORE_BYTE (c);
b73bfc1c 4803
065e3595
KH		 4804 if (c < 0)
4805 goto invalid_code;
24a73b0a 4806 if (c < 0x80)
119852e7 4807 {
2735d060 4808 if (eol_dos && c == '\r')
119852e7
KH		 4809 ONE_MORE_BYTE (byte_after_cr);
4810 charset = charset_roman;
4811 }
24a73b0a 4812 else
4ed46869 4813 {
24a73b0a
KH		 4814 /* BIG5 -> Big5 */
4815 if (c < 0xA1 || c > 0xFE)
4816 goto invalid_code;
4817 ONE_MORE_BYTE (c1);
4818 if (c1 < 0x40 || (c1 > 0x7E && c1 < 0xA1) || c1 > 0xFE)
4819 goto invalid_code;
4820 c = c << 8 | c1;
4821 charset = charset_big5;
4ed46869 4822 }
24a73b0a
KH		 4823 if (charset->id != charset_ascii
4824 && last_id != charset->id)
df7492f9 4825 {
24a73b0a 4826 if (last_id != charset_ascii)
69a80ea3 4827 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4828 last_id = charset->id;
4829 last_offset = char_offset;
4ed46869 4830 }
24a73b0a 4831 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4832 *charbuf++ = c;
ff0dacd7 4833 char_offset++;
fb88bf2d
KH		 4834 continue;
4835
df7492f9 4836 invalid_code:
4ed46869 4837 src = src_base;
df7492f9
KH		 4838 consumed_chars = consumed_chars_base;
4839 ONE_MORE_BYTE (c);
065e3595 4840 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4841 char_offset++;
df7492f9 4842 coding->errors++;
fb88bf2d 4843 }
d46c5b12 4844
df7492f9 4845 no_more_source:
ff0dacd7 4846 if (last_id != charset_ascii)
69a80ea3 4847 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4848 coding->consumed_char += consumed_chars_base;
4849 coding->consumed = src_base - coding->source;
4850 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 4851}
4852
4853/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
b73bfc1c
KH		 4854 This function can encode charsets `ascii', `katakana-jisx0201',
4855 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
4856 are sure that all these charsets are registered as official charset
4ed46869
KH		 4857 (i.e. do not have extended leading-codes). Characters of other
4858 charsets are produced without any encoding. If SJIS_P is 1, encode
4859 SJIS text, else encode BIG5 text. */
4860
df7492f9 4861static int
971de7fb 4862encode_coding_sjis (struct coding_system *coding)
4ed46869 4863{
df7492f9
KH		 4864 int multibytep = coding->dst_multibyte;
4865 int *charbuf = coding->charbuf;
4866 int *charbuf_end = charbuf + coding->charbuf_used;
4867 unsigned char *dst = coding->destination + coding->produced;
4868 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4869 int safe_room = 4;
d311d28c 4870 ptrdiff_t produced_chars = 0;
24a73b0a 4871 Lisp_Object attrs, charset_list, val;
df7492f9 4872 int ascii_compatible;
66ebf983 4873 struct charset *charset_kanji, *charset_kana;
57a47f8a 4874 struct charset *charset_kanji2;
df7492f9 4875 int c;
a5d301df 4876
24a73b0a 4877 CODING_GET_INFO (coding, attrs, charset_list);
66ebf983 4878 val = XCDR (charset_list);
df7492f9 4879 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4880 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4881 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4882
df7492f9 4883 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
93dec019 4884
df7492f9
KH		 4885 while (charbuf < charbuf_end)
4886 {
4887 ASSURE_DESTINATION (safe_room);
4888 c = *charbuf++;
b73bfc1c 4889 /* Now encode the character C. */
df7492f9
KH		 4890 if (ASCII_CHAR_P (c) && ascii_compatible)
4891 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4892 else if (CHAR_BYTE8_P (c))
4893 {
4894 c = CHAR_TO_BYTE8 (c);
4895 EMIT_ONE_BYTE (c);
4896 }
df7492f9 4897 else
b73bfc1c 4898 {
df7492f9
KH		 4899 unsigned code;
4900 struct charset *charset = char_charset (c, charset_list, &code);
4901
4902 if (!charset)
4ed46869 4903 {
41cbe562 4904 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4905 {
41cbe562
KH		 4906 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4907 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4908 }
41cbe562 4909 else
b73bfc1c 4910 {
41cbe562
KH		 4911 c = coding->default_char;
4912 charset = char_charset (c, charset_list, &code);
b73bfc1c 4913 }
b73bfc1c 4914 }
df7492f9
KH		 4915 if (code == CHARSET_INVALID_CODE (charset))
4916 abort ();
4917 if (charset == charset_kanji)
4918 {
4919 int c1, c2;
4920 JIS_TO_SJIS (code);
4921 c1 = code >> 8, c2 = code & 0xFF;
4922 EMIT_TWO_BYTES (c1, c2);
4923 }
4924 else if (charset == charset_kana)
4925 EMIT_ONE_BYTE (code | 0x80);
57a47f8a
KH		 4926 else if (charset_kanji2 && charset == charset_kanji2)
4927 {
4928 int c1, c2;
4929
4930 c1 = code >> 8;
f07190ca
KH		 4931 if (c1 == 0x21 || (c1 >= 0x23 && c1 <= 0x25)
4932 || c1 == 0x28
57a47f8a
KH		 4933 || (c1 >= 0x2C && c1 <= 0x2F) || c1 >= 0x6E)
4934 {
4935 JIS_TO_SJIS2 (code);
4936 c1 = code >> 8, c2 = code & 0xFF;
4937 EMIT_TWO_BYTES (c1, c2);
4938 }
4939 else
4940 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4941 }
df7492f9
KH		 4942 else
4943 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4944 }
4945 }
065e3595 4946 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4947 coding->produced_char += produced_chars;
4948 coding->produced = dst - coding->destination;
4949 return 0;
4950}
4951
4952static int
971de7fb 4953encode_coding_big5 (struct coding_system *coding)
df7492f9
KH		 4954{
4955 int multibytep = coding->dst_multibyte;
4956 int *charbuf = coding->charbuf;
4957 int *charbuf_end = charbuf + coding->charbuf_used;
4958 unsigned char *dst = coding->destination + coding->produced;
4959 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4960 int safe_room = 4;
d311d28c 4961 ptrdiff_t produced_chars = 0;
24a73b0a 4962 Lisp_Object attrs, charset_list, val;
df7492f9 4963 int ascii_compatible;
66ebf983 4964 struct charset *charset_big5;
df7492f9
KH		 4965 int c;
4966
24a73b0a 4967 CODING_GET_INFO (coding, attrs, charset_list);
66ebf983 4968 val = XCDR (charset_list);
df7492f9
KH		 4969 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4970 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4971
4972 while (charbuf < charbuf_end)
4973 {
4974 ASSURE_DESTINATION (safe_room);
4975 c = *charbuf++;
4976 /* Now encode the character C. */
4977 if (ASCII_CHAR_P (c) && ascii_compatible)
4978 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4979 else if (CHAR_BYTE8_P (c))
4980 {
4981 c = CHAR_TO_BYTE8 (c);
4982 EMIT_ONE_BYTE (c);
b73bfc1c
KH		 4983 }
4984 else
4985 {
df7492f9
KH		 4986 unsigned code;
4987 struct charset *charset = char_charset (c, charset_list, &code);
4988
4989 if (! charset)
b73bfc1c 4990 {
41cbe562 4991 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4992 {
41cbe562
KH		 4993 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4994 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4995 }
41cbe562 4996 else
0eecad43 4997 {
41cbe562
KH		 4998 c = coding->default_char;
4999 charset = char_charset (c, charset_list, &code);
0eecad43 5000 }
4ed46869 5001 }
df7492f9
KH		 5002 if (code == CHARSET_INVALID_CODE (charset))
5003 abort ();
5004 if (charset == charset_big5)
b73bfc1c 5005 {
df7492f9
KH		 5006 int c1, c2;
5007
5008 c1 = code >> 8, c2 = code & 0xFF;
5009 EMIT_TWO_BYTES (c1, c2);
b73bfc1c 5010 }
df7492f9
KH		 5011 else
5012 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4ed46869 5013 }
4ed46869 5014 }
065e3595 5015 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 5016 coding->produced_char += produced_chars;
5017 coding->produced = dst - coding->destination;
5018 return 0;
4ed46869
KH		 5019}
5020
5021\f
df7492f9 5022/*** 10. CCL handlers ***/
1397dc18
KH		 5023
5024/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
5025 Check if a text is encoded in a coding system of which
5026 encoder/decoder are written in CCL program. If it is, return
df7492f9 5027 CATEGORY_MASK_CCL, else return 0. */
1397dc18 5028
0a28aafb 5029static int
cf84bb53
JB		 5030detect_coding_ccl (struct coding_system *coding,
5031 struct coding_detection_info *detect_info)
1397dc18 5032{
065e3595 5033 const unsigned char *src = coding->source, *src_base;
8f924df7 5034 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 5035 int multibytep = coding->src_multibyte;
d311d28c 5036 ptrdiff_t consumed_chars = 0;
df7492f9 5037 int found = 0;
0e219d54 5038 unsigned char *valids;
d311d28c 5039 ptrdiff_t head_ascii = coding->head_ascii;
df7492f9
KH		 5040 Lisp_Object attrs;
5041
ff0dacd7
KH		 5042 detect_info->checked |= CATEGORY_MASK_CCL;
5043
df7492f9 5044 coding = &coding_categories[coding_category_ccl];
0e219d54 5045 valids = CODING_CCL_VALIDS (coding);
df7492f9
KH		 5046 attrs = CODING_ID_ATTRS (coding->id);
5047 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
5048 src += head_ascii;
1397dc18 5049
b73bfc1c 5050 while (1)
1397dc18 5051 {
df7492f9 5052 int c;
065e3595
KH		 5053
5054 src_base = src;
df7492f9 5055 ONE_MORE_BYTE (c);
065e3595 5056 if (c < 0 || ! valids[c])
df7492f9 5057 break;
ff0dacd7
KH		 5058 if ((valids[c] > 1))
5059 found = CATEGORY_MASK_CCL;
df7492f9 5060 }
ff0dacd7 5061 detect_info->rejected |= CATEGORY_MASK_CCL;
df7492f9
KH		 5062 return 0;
5063
5064 no_more_source:
ff0dacd7
KH		 5065 detect_info->found |= found;
5066 return 1;
df7492f9
KH		 5067}
5068
5069static void
971de7fb 5070decode_coding_ccl (struct coding_system *coding)
df7492f9 5071{
7c78e542 5072 const unsigned char *src = coding->source + coding->consumed;
8f924df7 5073 const unsigned char *src_end = coding->source + coding->src_bytes;
69a80ea3
KH		 5074 int *charbuf = coding->charbuf + coding->charbuf_used;
5075 int *charbuf_end = coding->charbuf + coding->charbuf_size;
d311d28c 5076 ptrdiff_t consumed_chars = 0;
df7492f9 5077 int multibytep = coding->src_multibyte;
d0396581 5078 struct ccl_program *ccl = &coding->spec.ccl->ccl;
df7492f9 5079 int source_charbuf[1024];
fbdc1721 5080 int source_byteidx[1025];
24a73b0a 5081 Lisp_Object attrs, charset_list;
df7492f9 5082
24a73b0a 5083 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 5084
d0396581 5085 while (1)
df7492f9 5086 {
7c78e542 5087 const unsigned char *p = src;
df7492f9
KH		 5088 int i = 0;
5089
5090 if (multibytep)
fbdc1721
KH		 5091 {
5092 while (i < 1024 && p < src_end)
5093 {
5094 source_byteidx[i] = p - src;
5095 source_charbuf[i++] = STRING_CHAR_ADVANCE (p);
5096 }
5097 source_byteidx[i] = p - src;
5098 }
df7492f9
KH		 5099 else
5100 while (i < 1024 && p < src_end)
5101 source_charbuf[i++] = *p++;
8f924df7 5102
df7492f9 5103 if (p == src_end && coding->mode & CODING_MODE_LAST_BLOCK)
d0396581
KH		 5104 ccl->last_block = 1;
5105 ccl_driver (ccl, source_charbuf, charbuf, i, charbuf_end - charbuf,
5106 charset_list);
5107 charbuf += ccl->produced;
fbdc1721 5108 if (multibytep)
d0396581 5109 src += source_byteidx[ccl->consumed];
df7492f9 5110 else
d0396581
KH		 5111 src += ccl->consumed;
5112 consumed_chars += ccl->consumed;
5113 if (p == src_end || ccl->status != CCL_STAT_SUSPEND_BY_SRC)
df7492f9
KH		 5114 break;
5115 }
5116
d0396581 5117 switch (ccl->status)
df7492f9
KH		 5118 {
5119 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 5120 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 5121 break;
5122 case CCL_STAT_SUSPEND_BY_DST:
d0396581 5123 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 5124 break;
5125 case CCL_STAT_QUIT:
5126 case CCL_STAT_INVALID_CMD:
065e3595 5127 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 5128 break;
5129 default:
065e3595 5130 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 5131 break;
5132 }
5133 coding->consumed_char += consumed_chars;
5134 coding->consumed = src - coding->source;
5135 coding->charbuf_used = charbuf - coding->charbuf;
5136}
5137
5138static int
971de7fb 5139encode_coding_ccl (struct coding_system *coding)
df7492f9 5140{
fb608df3 5141 struct ccl_program *ccl = &coding->spec.ccl->ccl;
df7492f9
KH		 5142 int multibytep = coding->dst_multibyte;
5143 int *charbuf = coding->charbuf;
5144 int *charbuf_end = charbuf + coding->charbuf_used;
5145 unsigned char *dst = coding->destination + coding->produced;
5146 unsigned char *dst_end = coding->destination + coding->dst_bytes;
df7492f9 5147 int destination_charbuf[1024];
d311d28c 5148 ptrdiff_t produced_chars = 0;
a53e2e89 5149 int i;
24a73b0a 5150 Lisp_Object attrs, charset_list;
df7492f9 5151
24a73b0a 5152 CODING_GET_INFO (coding, attrs, charset_list);
fb608df3
KH		 5153 if (coding->consumed_char == coding->src_chars
5154 && coding->mode & CODING_MODE_LAST_BLOCK)
5155 ccl->last_block = 1;
df7492f9 5156
8cffd3e7 5157 while (charbuf < charbuf_end)
df7492f9 5158 {
fb608df3 5159 ccl_driver (ccl, charbuf, destination_charbuf,
8cffd3e7 5160 charbuf_end - charbuf, 1024, charset_list);
df7492f9 5161 if (multibytep)
8cffd3e7 5162 {
fb608df3
KH		 5163 ASSURE_DESTINATION (ccl->produced * 2);
5164 for (i = 0; i < ccl->produced; i++)
8cffd3e7
KH		 5165 EMIT_ONE_BYTE (destination_charbuf[i] & 0xFF);
5166 }
df7492f9
KH		 5167 else
5168 {
fb608df3
KH		 5169 ASSURE_DESTINATION (ccl->produced);
5170 for (i = 0; i < ccl->produced; i++)
df7492f9 5171 *dst++ = destination_charbuf[i] & 0xFF;
fb608df3 5172 produced_chars += ccl->produced;
df7492f9 5173 }
fb608df3
KH		 5174 charbuf += ccl->consumed;
5175 if (ccl->status == CCL_STAT_QUIT
5176 || ccl->status == CCL_STAT_INVALID_CMD)
8cffd3e7 5177 break;
df7492f9
KH		 5178 }
5179
fb608df3 5180 switch (ccl->status)
df7492f9
KH		 5181 {
5182 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 5183 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 5184 break;
5185 case CCL_STAT_SUSPEND_BY_DST:
065e3595 5186 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 5187 break;
5188 case CCL_STAT_QUIT:
5189 case CCL_STAT_INVALID_CMD:
065e3595 5190 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 5191 break;
5192 default:
065e3595 5193 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 5194 break;
1397dc18 5195 }
df7492f9
KH		 5196
5197 coding->produced_char += produced_chars;
5198 coding->produced = dst - coding->destination;
5199 return 0;
1397dc18
KH		 5200}
5201
df7492f9 5202
1397dc18 5203\f
df7492f9 5204/*** 10, 11. no-conversion handlers ***/
4ed46869 5205
b73bfc1c 5206/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
4ed46869 5207
b73bfc1c 5208static void
971de7fb 5209decode_coding_raw_text (struct coding_system *coding)
4ed46869 5210{
2735d060 5211 int eol_dos =
0a9564cb 5212 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 5213
df7492f9 5214 coding->chars_at_source = 1;
119852e7
KH		 5215 coding->consumed_char = coding->src_chars;
5216 coding->consumed = coding->src_bytes;
2735d060 5217 if (eol_dos && coding->source[coding->src_bytes - 1] == '\r')
119852e7
KH		 5218 {
5219 coding->consumed_char--;
5220 coding->consumed--;
5221 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
5222 }
5223 else
5224 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 5225}
4ed46869 5226
df7492f9 5227static int
971de7fb 5228encode_coding_raw_text (struct coding_system *coding)
df7492f9
KH		 5229{
5230 int multibytep = coding->dst_multibyte;
5231 int *charbuf = coding->charbuf;
5232 int *charbuf_end = coding->charbuf + coding->charbuf_used;
5233 unsigned char *dst = coding->destination + coding->produced;
5234 unsigned char *dst_end = coding->destination + coding->dst_bytes;
d311d28c 5235 ptrdiff_t produced_chars = 0;
b73bfc1c
KH		 5236 int c;
5237
df7492f9 5238 if (multibytep)
b73bfc1c 5239 {
df7492f9 5240 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
4ed46869 5241
df7492f9
KH		 5242 if (coding->src_multibyte)
5243 while (charbuf < charbuf_end)
5244 {
5245 ASSURE_DESTINATION (safe_room);
5246 c = *charbuf++;
5247 if (ASCII_CHAR_P (c))
5248 EMIT_ONE_ASCII_BYTE (c);
5249 else if (CHAR_BYTE8_P (c))
5250 {
5251 c = CHAR_TO_BYTE8 (c);
5252 EMIT_ONE_BYTE (c);
5253 }
5254 else
5255 {
5256 unsigned char str[MAX_MULTIBYTE_LENGTH], *p0 = str, *p1 = str;
93dec019 5257
df7492f9 5258 CHAR_STRING_ADVANCE (c, p1);
8abc3f12 5259 do
9d123124
KH		 5260 {
5261 EMIT_ONE_BYTE (*p0);
5262 p0++;
5263 }
8abc3f12 5264 while (p0 < p1);
df7492f9
KH		 5265 }
5266 }
b73bfc1c 5267 else
df7492f9
KH		 5268 while (charbuf < charbuf_end)
5269 {
5270 ASSURE_DESTINATION (safe_room);
5271 c = *charbuf++;
5272 EMIT_ONE_BYTE (c);
5273 }
5274 }
5275 else
4ed46869 5276 {
df7492f9 5277 if (coding->src_multibyte)
d46c5b12 5278 {
df7492f9
KH		 5279 int safe_room = MAX_MULTIBYTE_LENGTH;
5280
5281 while (charbuf < charbuf_end)
d46c5b12 5282 {
df7492f9
KH		 5283 ASSURE_DESTINATION (safe_room);
5284 c = *charbuf++;
5285 if (ASCII_CHAR_P (c))
5286 *dst++ = c;
5287 else if (CHAR_BYTE8_P (c))
5288 *dst++ = CHAR_TO_BYTE8 (c);
b73bfc1c 5289 else
df7492f9 5290 CHAR_STRING_ADVANCE (c, dst);
d46c5b12
KH		 5291 }
5292 }
df7492f9
KH		 5293 else
5294 {
5295 ASSURE_DESTINATION (charbuf_end - charbuf);
5296 while (charbuf < charbuf_end && dst < dst_end)
5297 *dst++ = *charbuf++;
8f924df7 5298 }
319a3947 5299 produced_chars = dst - (coding->destination + coding->produced);
4ed46869 5300 }
065e3595 5301 record_conversion_result (coding, CODING_RESULT_SUCCESS);
a0ed9b27 5302 coding->produced_char += produced_chars;
df7492f9
KH		 5303 coding->produced = dst - coding->destination;
5304 return 0;
4ed46869
KH		 5305}
5306
ff0dacd7
KH		 5307/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
5308 Check if a text is encoded in a charset-based coding system. If it
5309 is, return 1, else return 0. */
5310
0a28aafb 5311static int
cf84bb53
JB		 5312detect_coding_charset (struct coding_system *coding,
5313 struct coding_detection_info *detect_info)
1397dc18 5314{
065e3595 5315 const unsigned char *src = coding->source, *src_base;
8f924df7 5316 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 5317 int multibytep = coding->src_multibyte;
d311d28c 5318 ptrdiff_t consumed_chars = 0;
07295713 5319 Lisp_Object attrs, valids, name;
584948ac 5320 int found = 0;
d311d28c 5321 ptrdiff_t head_ascii = coding->head_ascii;
07295713 5322 int check_latin_extra = 0;
1397dc18 5323
ff0dacd7
KH		 5324 detect_info->checked |= CATEGORY_MASK_CHARSET;
5325
df7492f9
KH		 5326 coding = &coding_categories[coding_category_charset];
5327 attrs = CODING_ID_ATTRS (coding->id);
5328 valids = AREF (attrs, coding_attr_charset_valids);
07295713 5329 name = CODING_ID_NAME (coding->id);
51b59d79 5330 if (strncmp (SSDATA (SYMBOL_NAME (name)),
237aabf4 5331 "iso-8859-", sizeof ("iso-8859-") - 1) == 0
51b59d79 5332 || strncmp (SSDATA (SYMBOL_NAME (name)),
237aabf4 5333 "iso-latin-", sizeof ("iso-latin-") - 1) == 0)
07295713 5334 check_latin_extra = 1;
237aabf4 5335
df7492f9 5336 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
716b3fa0 5337 src += head_ascii;
1397dc18 5338
b73bfc1c 5339 while (1)
1397dc18 5340 {
df7492f9 5341 int c;
716b3fa0
KH		 5342 Lisp_Object val;
5343 struct charset *charset;
5344 int dim, idx;
1397dc18 5345
065e3595 5346 src_base = src;
df7492f9 5347 ONE_MORE_BYTE (c);
065e3595
KH		 5348 if (c < 0)
5349 continue;
716b3fa0
KH		 5350 val = AREF (valids, c);
5351 if (NILP (val))
df7492f9 5352 break;
584948ac 5353 if (c >= 0x80)
07295713
KH		 5354 {
5355 if (c < 0xA0
237aabf4
JR		 5356 && check_latin_extra
5357 && (!VECTORP (Vlatin_extra_code_table)
9f0526cb 5358 || NILP (XVECTOR (Vlatin_extra_code_table)->contents[c])))
07295713
KH		 5359 break;
5360 found = CATEGORY_MASK_CHARSET;
5361 }
716b3fa0
KH		 5362 if (INTEGERP (val))
5363 {
5364 charset = CHARSET_FROM_ID (XFASTINT (val));
5365 dim = CHARSET_DIMENSION (charset);
5366 for (idx = 1; idx < dim; idx++)
5367 {
5368 if (src == src_end)
5369 goto too_short;
5370 ONE_MORE_BYTE (c);
2f9442b8
PE		 5371 if (c < charset->code_space[(dim - 1 - idx) * 4]
5372 || c > charset->code_space[(dim - 1 - idx) * 4 + 1])
716b3fa0
KH		 5373 break;
5374 }
5375 if (idx < dim)
5376 break;
5377 }
5378 else
5379 {
5380 idx = 1;
5381 for (; CONSP (val); val = XCDR (val))
5382 {
5383 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
5384 dim = CHARSET_DIMENSION (charset);
5385 while (idx < dim)
5386 {
5387 if (src == src_end)
5388 goto too_short;
5389 ONE_MORE_BYTE (c);
5390 if (c < charset->code_space[(dim - 1 - idx) * 4]
5391 || c > charset->code_space[(dim - 1 - idx) * 4 + 1])
5392 break;
5393 idx++;
5394 }
5395 if (idx == dim)
5396 {
5397 val = Qnil;
5398 break;
5399 }
5400 }
5401 if (CONSP (val))
5402 break;
5403 }
df7492f9 5404 }
716b3fa0 5405 too_short:
ff0dacd7 5406 detect_info->rejected |= CATEGORY_MASK_CHARSET;
df7492f9 5407 return 0;
4ed46869 5408
df7492f9 5409 no_more_source:
ff0dacd7
KH		 5410 detect_info->found |= found;
5411 return 1;
df7492f9 5412}
b73bfc1c 5413
b73bfc1c 5414static void
971de7fb 5415decode_coding_charset (struct coding_system *coding)
4ed46869 5416{
8f924df7
KH		 5417 const unsigned char *src = coding->source + coding->consumed;
5418 const unsigned char *src_end = coding->source + coding->src_bytes;
5419 const unsigned char *src_base;
69a80ea3 5420 int *charbuf = coding->charbuf + coding->charbuf_used;
ad1746f5 5421 /* We may produce one charset annotation in one loop and one more at
df80c7f0 5422 the end. */
69a80ea3 5423 int *charbuf_end
df80c7f0 5424 = coding->charbuf + coding->charbuf_size - (MAX_ANNOTATION_LENGTH * 2);
d311d28c 5425 ptrdiff_t consumed_chars = 0, consumed_chars_base;
df7492f9 5426 int multibytep = coding->src_multibyte;
66ebf983
PE		 5427 Lisp_Object attrs = CODING_ID_ATTRS (coding->id);
5428 Lisp_Object valids;
d311d28c
PE		 5429 ptrdiff_t char_offset = coding->produced_char;
5430 ptrdiff_t last_offset = char_offset;
ff0dacd7 5431 int last_id = charset_ascii;
2735d060 5432 int eol_dos =
0a9564cb 5433 !inhibit_eol_conversion && EQ (CODING_ID_EOL_TYPE (coding->id), Qdos);
119852e7 5434 int byte_after_cr = -1;
df7492f9 5435
4eb6d3f1 5436 valids = AREF (attrs, coding_attr_charset_valids);
b73bfc1c 5437
df7492f9 5438 while (1)
4ed46869 5439 {
4eb6d3f1 5440 int c;
24a73b0a
KH		 5441 Lisp_Object val;
5442 struct charset *charset;
5443 int dim;
5444 int len = 1;
5445 unsigned code;
df7492f9
KH		 5446
5447 src_base = src;
5448 consumed_chars_base = consumed_chars;
b73bfc1c 5449
df7492f9 5450 if (charbuf >= charbuf_end)
b71f6f73
KH		 5451 {
5452 if (byte_after_cr >= 0)
5453 src_base--;
5454 break;
5455 }
df7492f9 5456
119852e7
KH		 5457 if (byte_after_cr >= 0)
5458 {
5459 c = byte_after_cr;
5460 byte_after_cr = -1;
5461 }
5462 else
5463 {
5464 ONE_MORE_BYTE (c);
2735d060 5465 if (eol_dos && c == '\r')
119852e7
KH		 5466 ONE_MORE_BYTE (byte_after_cr);
5467 }
065e3595
KH		 5468 if (c < 0)
5469 goto invalid_code;
24a73b0a
KH		 5470 code = c;
5471
5472 val = AREF (valids, c);
1b17adfd 5473 if (! INTEGERP (val) && ! CONSP (val))
24a73b0a
KH		 5474 goto invalid_code;
5475 if (INTEGERP (val))
d46c5b12 5476 {
24a73b0a
KH		 5477 charset = CHARSET_FROM_ID (XFASTINT (val));
5478 dim = CHARSET_DIMENSION (charset);
5479 while (len < dim)
b73bfc1c 5480 {
24a73b0a
KH		 5481 ONE_MORE_BYTE (c);
5482 code = (code << 8) | c;
5483 len++;
b73bfc1c 5484 }
24a73b0a
KH		 5485 CODING_DECODE_CHAR (coding, src, src_base, src_end,
5486 charset, code, c);
d46c5b12 5487 }
df7492f9 5488 else
d46c5b12 5489 {
24a73b0a
KH		 5490 /* VAL is a list of charset IDs. It is assured that the
5491 list is sorted by charset dimensions (smaller one
5492 comes first). */
5493 while (CONSP (val))
4eb6d3f1 5494 {
24a73b0a 5495 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
c7c66a95 5496 dim = CHARSET_DIMENSION (charset);
f9d71dcd 5497 while (len < dim)
4eb6d3f1 5498 {
acb2a965
KH		 5499 ONE_MORE_BYTE (c);
5500 code = (code << 8) | c;
f9d71dcd 5501 len++;
4eb6d3f1 5502 }
24a73b0a
KH		 5503 CODING_DECODE_CHAR (coding, src, src_base,
5504 src_end, charset, code, c);
5505 if (c >= 0)
5506 break;
5507 val = XCDR (val);
ff0dacd7 5508 }
d46c5b12 5509 }
24a73b0a
KH		 5510 if (c < 0)
5511 goto invalid_code;
5512 if (charset->id != charset_ascii
5513 && last_id != charset->id)
5514 {
5515 if (last_id != charset_ascii)
69a80ea3 5516 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 5517 last_id = charset->id;
5518 last_offset = char_offset;
5519 }
5520
df7492f9 5521 *charbuf++ = c;
ff0dacd7 5522 char_offset++;
df7492f9
KH		 5523 continue;
5524
5525 invalid_code:
5526 src = src_base;
5527 consumed_chars = consumed_chars_base;
5528 ONE_MORE_BYTE (c);
065e3595 5529 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 5530 char_offset++;
df7492f9 5531 coding->errors++;
4ed46869
KH		 5532 }
5533
df7492f9 5534 no_more_source:
ff0dacd7 5535 if (last_id != charset_ascii)
69a80ea3 5536 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 5537 coding->consumed_char += consumed_chars_base;
5538 coding->consumed = src_base - coding->source;
5539 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 5540}
5541
df7492f9 5542static int
971de7fb 5543encode_coding_charset (struct coding_system *coding)
4ed46869 5544{
df7492f9
KH		 5545 int multibytep = coding->dst_multibyte;
5546 int *charbuf = coding->charbuf;
5547 int *charbuf_end = charbuf + coding->charbuf_used;
5548 unsigned char *dst = coding->destination + coding->produced;
5549 unsigned char *dst_end = coding->destination + coding->dst_bytes;
5550 int safe_room = MAX_MULTIBYTE_LENGTH;
d311d28c 5551 ptrdiff_t produced_chars = 0;
24a73b0a 5552 Lisp_Object attrs, charset_list;
df7492f9 5553 int ascii_compatible;
b73bfc1c 5554 int c;
b73bfc1c 5555
24a73b0a 5556 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 5557 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
fb88bf2d 5558
df7492f9 5559 while (charbuf < charbuf_end)
4ed46869 5560 {
4eb6d3f1 5561 struct charset *charset;
df7492f9 5562 unsigned code;
8f924df7 5563
df7492f9
KH		 5564 ASSURE_DESTINATION (safe_room);
5565 c = *charbuf++;
5566 if (ascii_compatible && ASCII_CHAR_P (c))
5567 EMIT_ONE_ASCII_BYTE (c);
16eafb5d 5568 else if (CHAR_BYTE8_P (c))
4ed46869 5569 {
16eafb5d
KH		 5570 c = CHAR_TO_BYTE8 (c);
5571 EMIT_ONE_BYTE (c);
d46c5b12 5572 }
d46c5b12 5573 else
b73bfc1c 5574 {
4eb6d3f1
KH		 5575 charset = char_charset (c, charset_list, &code);
5576 if (charset)
5577 {
5578 if (CHARSET_DIMENSION (charset) == 1)
5579 EMIT_ONE_BYTE (code);
5580 else if (CHARSET_DIMENSION (charset) == 2)
5581 EMIT_TWO_BYTES (code >> 8, code & 0xFF);
5582 else if (CHARSET_DIMENSION (charset) == 3)
5583 EMIT_THREE_BYTES (code >> 16, (code >> 8) & 0xFF, code & 0xFF);
5584 else
5585 EMIT_FOUR_BYTES (code >> 24, (code >> 16) & 0xFF,
5586 (code >> 8) & 0xFF, code & 0xFF);
5587 }
5588 else
41cbe562
KH		 5589 {
5590 if (coding->mode & CODING_MODE_SAFE_ENCODING)
5591 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
5592 else
5593 c = coding->default_char;
5594 EMIT_ONE_BYTE (c);
5595 }
4ed46869 5596 }
4ed46869
KH		 5597 }
5598
065e3595 5599 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 5600 coding->produced_char += produced_chars;
5601 coding->produced = dst - coding->destination;
5602 return 0;
4ed46869
KH		 5603}
5604
5605\f
1397dc18 5606/*** 7. C library functions ***/
4ed46869 5607
df7492f9
KH		 5608/* Setup coding context CODING from information about CODING_SYSTEM.
5609 If CODING_SYSTEM is nil, `no-conversion' is assumed. If
5610 CODING_SYSTEM is invalid, signal an error. */
4ed46869 5611
ec6d2bb8 5612void
971de7fb 5613setup_coding_system (Lisp_Object coding_system, struct coding_system *coding)
4ed46869 5614{
df7492f9
KH		 5615 Lisp_Object attrs;
5616 Lisp_Object eol_type;
5617 Lisp_Object coding_type;
4608c386 5618 Lisp_Object val;
4ed46869 5619
df7492f9 5620 if (NILP (coding_system))
ae6f73fa 5621 coding_system = Qundecided;
c07c8e12 5622
df7492f9 5623 CHECK_CODING_SYSTEM_GET_ID (coding_system, coding->id);
1f5dbf34 5624
df7492f9 5625 attrs = CODING_ID_ATTRS (coding->id);
0a9564cb 5626 eol_type = inhibit_eol_conversion ? Qunix : CODING_ID_EOL_TYPE (coding->id);
1f5dbf34 5627
df7492f9
KH		 5628 coding->mode = 0;
5629 coding->head_ascii = -1;
4a015c45
KH		 5630 if (VECTORP (eol_type))
5631 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5632 | CODING_REQUIRE_DETECTION_MASK);
5633 else if (! EQ (eol_type, Qunix))
5634 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5635 | CODING_REQUIRE_ENCODING_MASK);
5636 else
5637 coding->common_flags = 0;
5e5c78be
KH		 5638 if (! NILP (CODING_ATTR_POST_READ (attrs)))
5639 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5640 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
5641 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
8f924df7
KH		 5642 if (! NILP (CODING_ATTR_FOR_UNIBYTE (attrs)))
5643 coding->common_flags |= CODING_FOR_UNIBYTE_MASK;
1f5dbf34 5644
df7492f9 5645 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7 5646 coding->max_charset_id = SCHARS (val) - 1;
1b3b981b 5647 coding->safe_charsets = SDATA (val);
df7492f9 5648 coding->default_char = XINT (CODING_ATTR_DEFAULT_CHAR (attrs));
624bda09 5649 coding->carryover_bytes = 0;
4608c386 5650
df7492f9
KH		 5651 coding_type = CODING_ATTR_TYPE (attrs);
5652 if (EQ (coding_type, Qundecided))
d46c5b12 5653 {
df7492f9
KH		 5654 coding->detector = NULL;
5655 coding->decoder = decode_coding_raw_text;
5656 coding->encoder = encode_coding_raw_text;
5657 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 5658 }
df7492f9 5659 else if (EQ (coding_type, Qiso_2022))
d46c5b12 5660 {
df7492f9
KH		 5661 int i;
5662 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5663
5664 /* Invoke graphic register 0 to plane 0. */
5665 CODING_ISO_INVOCATION (coding, 0) = 0;
5666 /* Invoke graphic register 1 to plane 1 if we can use 8-bit. */
5667 CODING_ISO_INVOCATION (coding, 1)
5668 = (flags & CODING_ISO_FLAG_SEVEN_BITS ? -1 : 1);
5669 /* Setup the initial status of designation. */
5670 for (i = 0; i < 4; i++)
5671 CODING_ISO_DESIGNATION (coding, i) = CODING_ISO_INITIAL (coding, i);
5672 /* Not single shifting initially. */
5673 CODING_ISO_SINGLE_SHIFTING (coding) = 0;
5674 /* Beginning of buffer should also be regarded as bol. */
5675 CODING_ISO_BOL (coding) = 1;
5676 coding->detector = detect_coding_iso_2022;
5677 coding->decoder = decode_coding_iso_2022;
5678 coding->encoder = encode_coding_iso_2022;
5679 if (flags & CODING_ISO_FLAG_SAFE)
5680 coding->mode |= CODING_MODE_SAFE_ENCODING;
d46c5b12 5681 coding->common_flags
df7492f9
KH		 5682 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5683 | CODING_REQUIRE_FLUSHING_MASK);
5684 if (flags & CODING_ISO_FLAG_COMPOSITION)
5685 coding->common_flags |= CODING_ANNOTATE_COMPOSITION_MASK;
ff0dacd7
KH		 5686 if (flags & CODING_ISO_FLAG_DESIGNATION)
5687 coding->common_flags |= CODING_ANNOTATE_CHARSET_MASK;
df7492f9
KH		 5688 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5689 {
5690 setup_iso_safe_charsets (attrs);
5691 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7 5692 coding->max_charset_id = SCHARS (val) - 1;
1b3b981b 5693 coding->safe_charsets = SDATA (val);
df7492f9
KH		 5694 }
5695 CODING_ISO_FLAGS (coding) = flags;
e951386e
KH		 5696 CODING_ISO_CMP_STATUS (coding)->state = COMPOSING_NO;
5697 CODING_ISO_CMP_STATUS (coding)->method = COMPOSITION_NO;
5698 CODING_ISO_EXTSEGMENT_LEN (coding) = 0;
5699 CODING_ISO_EMBEDDED_UTF_8 (coding) = 0;
d46c5b12 5700 }
df7492f9 5701 else if (EQ (coding_type, Qcharset))
d46c5b12 5702 {
df7492f9
KH		 5703 coding->detector = detect_coding_charset;
5704 coding->decoder = decode_coding_charset;
5705 coding->encoder = encode_coding_charset;
d46c5b12 5706 coding->common_flags
df7492f9 5707 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
d46c5b12 5708 }
df7492f9 5709 else if (EQ (coding_type, Qutf_8))
d46c5b12 5710 {
a470d443
KH		 5711 val = AREF (attrs, coding_attr_utf_bom);
5712 CODING_UTF_8_BOM (coding) = (CONSP (val) ? utf_detect_bom
5713 : EQ (val, Qt) ? utf_with_bom
5714 : utf_without_bom);
df7492f9
KH		 5715 coding->detector = detect_coding_utf_8;
5716 coding->decoder = decode_coding_utf_8;
5717 coding->encoder = encode_coding_utf_8;
5718 coding->common_flags
5719 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
a470d443
KH		 5720 if (CODING_UTF_8_BOM (coding) == utf_detect_bom)
5721 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
df7492f9
KH		 5722 }
5723 else if (EQ (coding_type, Qutf_16))
5724 {
a470d443
KH		 5725 val = AREF (attrs, coding_attr_utf_bom);
5726 CODING_UTF_16_BOM (coding) = (CONSP (val) ? utf_detect_bom
5727 : EQ (val, Qt) ? utf_with_bom
5728 : utf_without_bom);
df7492f9 5729 val = AREF (attrs, coding_attr_utf_16_endian);
b49a1807 5730 CODING_UTF_16_ENDIAN (coding) = (EQ (val, Qbig) ? utf_16_big_endian
df7492f9 5731 : utf_16_little_endian);
e19c3639 5732 CODING_UTF_16_SURROGATE (coding) = 0;
df7492f9
KH		 5733 coding->detector = detect_coding_utf_16;
5734 coding->decoder = decode_coding_utf_16;
5735 coding->encoder = encode_coding_utf_16;
5736 coding->common_flags
5737 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
a470d443 5738 if (CODING_UTF_16_BOM (coding) == utf_detect_bom)
b49a1807 5739 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 5740 }
df7492f9 5741 else if (EQ (coding_type, Qccl))
4ed46869 5742 {
df7492f9
KH		 5743 coding->detector = detect_coding_ccl;
5744 coding->decoder = decode_coding_ccl;
5745 coding->encoder = encode_coding_ccl;
c952af22 5746 coding->common_flags
df7492f9
KH		 5747 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5748 | CODING_REQUIRE_FLUSHING_MASK);
5749 }
5750 else if (EQ (coding_type, Qemacs_mule))
5751 {
5752 coding->detector = detect_coding_emacs_mule;
5753 coding->decoder = decode_coding_emacs_mule;
5754 coding->encoder = encode_coding_emacs_mule;
c952af22 5755 coding->common_flags
df7492f9 5756 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
e951386e 5757 coding->spec.emacs_mule.full_support = 1;
df7492f9
KH		 5758 if (! NILP (AREF (attrs, coding_attr_emacs_mule_full))
5759 && ! EQ (CODING_ATTR_CHARSET_LIST (attrs), Vemacs_mule_charset_list))
5760 {
5761 Lisp_Object tail, safe_charsets;
5762 int max_charset_id = 0;
5763
5764 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5765 tail = XCDR (tail))
5766 if (max_charset_id < XFASTINT (XCAR (tail)))
5767 max_charset_id = XFASTINT (XCAR (tail));
1b3b981b
AS		 5768 safe_charsets = make_uninit_string (max_charset_id + 1);
5769 memset (SDATA (safe_charsets), 255, max_charset_id + 1);
df7492f9
KH		 5770 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5771 tail = XCDR (tail))
8f924df7 5772 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 5773 coding->max_charset_id = max_charset_id;
1b3b981b 5774 coding->safe_charsets = SDATA (safe_charsets);
e951386e 5775 coding->spec.emacs_mule.full_support = 1;
df7492f9 5776 }
e951386e
KH		 5777 coding->spec.emacs_mule.cmp_status.state = COMPOSING_NO;
5778 coding->spec.emacs_mule.cmp_status.method = COMPOSITION_NO;
df7492f9
KH		 5779 }
5780 else if (EQ (coding_type, Qshift_jis))
5781 {
5782 coding->detector = detect_coding_sjis;
5783 coding->decoder = decode_coding_sjis;
5784 coding->encoder = encode_coding_sjis;
c952af22 5785 coding->common_flags
df7492f9
KH		 5786 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5787 }
5788 else if (EQ (coding_type, Qbig5))
5789 {
5790 coding->detector = detect_coding_big5;
5791 coding->decoder = decode_coding_big5;
5792 coding->encoder = encode_coding_big5;
c952af22 5793 coding->common_flags
df7492f9
KH		 5794 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5795 }
5796 else /* EQ (coding_type, Qraw_text) */
ec6d2bb8 5797 {
df7492f9
KH		 5798 coding->detector = NULL;
5799 coding->decoder = decode_coding_raw_text;
5800 coding->encoder = encode_coding_raw_text;
ea29edf2
KH		 5801 if (! EQ (eol_type, Qunix))
5802 {
5803 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5804 if (! VECTORP (eol_type))
5805 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
5806 }
5807
4ed46869 5808 }
4ed46869 5809
df7492f9 5810 return;
4ed46869
KH		 5811}
5812
0ff61e78
KH		 5813/* Return a list of charsets supported by CODING. */
5814
5815Lisp_Object
971de7fb 5816coding_charset_list (struct coding_system *coding)
0ff61e78 5817{
35befdaa 5818 Lisp_Object attrs, charset_list;
0ff61e78
KH		 5819
5820 CODING_GET_INFO (coding, attrs, charset_list);
5821 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5822 {
5823 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5824
5825 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5826 charset_list = Viso_2022_charset_list;
5827 }
5828 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5829 {
5830 charset_list = Vemacs_mule_charset_list;
5831 }
5832 return charset_list;
5833}
5834
5835
e9f91ece
KH		 5836/* Return a list of charsets supported by CODING-SYSTEM. */
5837
5838Lisp_Object
971de7fb 5839coding_system_charset_list (Lisp_Object coding_system)
e9f91ece 5840{
d3411f89 5841 ptrdiff_t id;
e9f91ece
KH		 5842 Lisp_Object attrs, charset_list;
5843
5844 CHECK_CODING_SYSTEM_GET_ID (coding_system, id);
5845 attrs = CODING_ID_ATTRS (id);
5846
5847 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5848 {
5849 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5850
5851 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5852 charset_list = Viso_2022_charset_list;
5853 else
5854 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
5855 }
5856 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5857 {
5858 charset_list = Vemacs_mule_charset_list;
5859 }
5860 else
5861 {
5862 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
5863 }
5864 return charset_list;
5865}
5866
5867
df7492f9
KH		 5868/* Return raw-text or one of its subsidiaries that has the same
5869 eol_type as CODING-SYSTEM. */
ec6d2bb8 5870
df7492f9 5871Lisp_Object
971de7fb 5872raw_text_coding_system (Lisp_Object coding_system)
ec6d2bb8 5873{
0be8721c 5874 Lisp_Object spec, attrs;
df7492f9 5875 Lisp_Object eol_type, raw_text_eol_type;
ec6d2bb8 5876
d3e4cb56
KH		 5877 if (NILP (coding_system))
5878 return Qraw_text;
df7492f9
KH		 5879 spec = CODING_SYSTEM_SPEC (coding_system);
5880 attrs = AREF (spec, 0);
ec6d2bb8 5881
df7492f9
KH		 5882 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
5883 return coding_system;
ec6d2bb8 5884
df7492f9
KH		 5885 eol_type = AREF (spec, 2);
5886 if (VECTORP (eol_type))
5887 return Qraw_text;
5888 spec = CODING_SYSTEM_SPEC (Qraw_text);
5889 raw_text_eol_type = AREF (spec, 2);
5890 return (EQ (eol_type, Qunix) ? AREF (raw_text_eol_type, 0)
5891 : EQ (eol_type, Qdos) ? AREF (raw_text_eol_type, 1)
5892 : AREF (raw_text_eol_type, 2));
ec6d2bb8
KH		 5893}
5894
54f78171 5895
1911a33b
KH		 5896/* If CODING_SYSTEM doesn't specify end-of-line format, return one of
5897 the subsidiary that has the same eol-spec as PARENT (if it is not
5898 nil and specifies end-of-line format) or the system's setting
fcbcfb64 5899 (system_eol_type). */
df7492f9
KH		 5900
5901Lisp_Object
971de7fb 5902coding_inherit_eol_type (Lisp_Object coding_system, Lisp_Object parent)
54f78171 5903{
3e139625 5904 Lisp_Object spec, eol_type;
54f78171 5905
d3e4cb56
KH		 5906 if (NILP (coding_system))
5907 coding_system = Qraw_text;
df7492f9 5908 spec = CODING_SYSTEM_SPEC (coding_system);
df7492f9 5909 eol_type = AREF (spec, 2);
fcbcfb64 5910 if (VECTORP (eol_type))
df7492f9 5911 {
df7492f9
KH		 5912 Lisp_Object parent_eol_type;
5913
fcbcfb64
KH		 5914 if (! NILP (parent))
5915 {
5916 Lisp_Object parent_spec;
5917
4a015c45 5918 parent_spec = CODING_SYSTEM_SPEC (parent);
fcbcfb64 5919 parent_eol_type = AREF (parent_spec, 2);
1911a33b 5920 if (VECTORP (parent_eol_type))
4628bef1 5921 parent_eol_type = system_eol_type;
fcbcfb64
KH		 5922 }
5923 else
5924 parent_eol_type = system_eol_type;
df7492f9
KH		 5925 if (EQ (parent_eol_type, Qunix))
5926 coding_system = AREF (eol_type, 0);
5927 else if (EQ (parent_eol_type, Qdos))
5928 coding_system = AREF (eol_type, 1);
5929 else if (EQ (parent_eol_type, Qmac))
5930 coding_system = AREF (eol_type, 2);
54f78171 5931 }
df7492f9 5932 return coding_system;
54f78171
KH		 5933}
5934
fcaf8878
KH		 5935
5936/* Check if text-conversion and eol-conversion of CODING_SYSTEM are
5937 decided for writing to a process. If not, complement them, and
5938 return a new coding system. */
5939
5940Lisp_Object
4628bef1 5941complement_process_encoding_system (Lisp_Object coding_system)
fcaf8878 5942{
5886ec9c
KH		 5943 Lisp_Object coding_base = Qnil, eol_base = Qnil;
5944 Lisp_Object spec, attrs;
93d50df8 5945 int i;
fcaf8878 5946
93d50df8 5947 for (i = 0; i < 3; i++)
fcaf8878 5948 {
93d50df8
KH		 5949 if (i == 1)
5950 coding_system = CDR_SAFE (Vdefault_process_coding_system);
5951 else if (i == 2)
5952 coding_system = preferred_coding_system ();
5953 spec = CODING_SYSTEM_SPEC (coding_system);
5954 if (NILP (spec))
5955 continue;
5956 attrs = AREF (spec, 0);
5957 if (NILP (coding_base) && ! EQ (CODING_ATTR_TYPE (attrs), Qundecided))
5958 coding_base = CODING_ATTR_BASE_NAME (attrs);
5959 if (NILP (eol_base) && ! VECTORP (AREF (spec, 2)))
5960 eol_base = coding_system;
5961 if (! NILP (coding_base) && ! NILP (eol_base))
5962 break;
fcaf8878 5963 }
fcaf8878 5964
93d50df8
KH		 5965 if (i > 0)
5966 /* The original CODING_SYSTEM didn't specify text-conversion or
5967 eol-conversion. Be sure that we return a fully complemented
5968 coding system. */
5969 coding_system = coding_inherit_eol_type (coding_base, eol_base);
5970 return coding_system;
fcaf8878
KH		 5971}
5972
5973
4ed46869
KH		 5974/* Emacs has a mechanism to automatically detect a coding system if it
5975 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
5976 it's impossible to distinguish some coding systems accurately
5977 because they use the same range of codes. So, at first, coding
5978 systems are categorized into 7, those are:
5979
0ef69138 5980 o coding-category-emacs-mule
4ed46869
KH		 5981
5982 The category for a coding system which has the same code range
5983 as Emacs' internal format. Assigned the coding-system (Lisp
0ef69138 5984 symbol) `emacs-mule' by default.
4ed46869
KH		 5985
5986 o coding-category-sjis
5987
5988 The category for a coding system which has the same code range
5989 as SJIS. Assigned the coding-system (Lisp
7717c392 5990 symbol) `japanese-shift-jis' by default.
4ed46869
KH		 5991
5992 o coding-category-iso-7
5993
5994 The category for a coding system which has the same code range
7717c392 5995 as ISO2022 of 7-bit environment. This doesn't use any locking
d46c5b12
KH		 5996 shift and single shift functions. This can encode/decode all
5997 charsets. Assigned the coding-system (Lisp symbol)
5998 `iso-2022-7bit' by default.
5999
6000 o coding-category-iso-7-tight
6001
6002 Same as coding-category-iso-7 except that this can
6003 encode/decode only the specified charsets.
4ed46869
KH		 6004
6005 o coding-category-iso-8-1
6006
6007 The category for a coding system which has the same code range
6008 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 6009 for DIMENSION1 charset. This doesn't use any locking shift
6010 and single shift functions. Assigned the coding-system (Lisp
6011 symbol) `iso-latin-1' by default.
4ed46869
KH		 6012
6013 o coding-category-iso-8-2
6014
6015 The category for a coding system which has the same code range
6016 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 6017 for DIMENSION2 charset. This doesn't use any locking shift
6018 and single shift functions. Assigned the coding-system (Lisp
6019 symbol) `japanese-iso-8bit' by default.
4ed46869 6020
7717c392 6021 o coding-category-iso-7-else
4ed46869
KH		 6022
6023 The category for a coding system which has the same code range
ad1746f5 6024 as ISO2022 of 7-bit environment but uses locking shift or
7717c392
KH		 6025 single shift functions. Assigned the coding-system (Lisp
6026 symbol) `iso-2022-7bit-lock' by default.
6027
6028 o coding-category-iso-8-else
6029
6030 The category for a coding system which has the same code range
ad1746f5 6031 as ISO2022 of 8-bit environment but uses locking shift or
7717c392
KH		 6032 single shift functions. Assigned the coding-system (Lisp
6033 symbol) `iso-2022-8bit-ss2' by default.
4ed46869
KH		 6034
6035 o coding-category-big5
6036
6037 The category for a coding system which has the same code range
6038 as BIG5. Assigned the coding-system (Lisp symbol)
e0e989f6 6039 `cn-big5' by default.
4ed46869 6040
fa42c37f
KH		 6041 o coding-category-utf-8
6042
6043 The category for a coding system which has the same code range
6e76ae91 6044 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
fa42c37f
KH		 6045 symbol) `utf-8' by default.
6046
6047 o coding-category-utf-16-be
6048
6049 The category for a coding system in which a text has an
6050 Unicode signature (cf. Unicode Standard) in the order of BIG
6051 endian at the head. Assigned the coding-system (Lisp symbol)
6052 `utf-16-be' by default.
6053
6054 o coding-category-utf-16-le
6055
6056 The category for a coding system in which a text has an
6057 Unicode signature (cf. Unicode Standard) in the order of
6058 LITTLE endian at the head. Assigned the coding-system (Lisp
6059 symbol) `utf-16-le' by default.
6060
1397dc18
KH		 6061 o coding-category-ccl
6062
6063 The category for a coding system of which encoder/decoder is
6064 written in CCL programs. The default value is nil, i.e., no
6065 coding system is assigned.
6066
4ed46869
KH		 6067 o coding-category-binary
6068
6069 The category for a coding system not categorized in any of the
6070 above. Assigned the coding-system (Lisp symbol)
e0e989f6 6071 `no-conversion' by default.
4ed46869
KH		 6072
6073 Each of them is a Lisp symbol and the value is an actual
df7492f9 6074 `coding-system's (this is also a Lisp symbol) assigned by a user.
4ed46869
KH		 6075 What Emacs does actually is to detect a category of coding system.
6076 Then, it uses a `coding-system' assigned to it. If Emacs can't
df7492f9 6077 decide only one possible category, it selects a category of the
4ed46869
KH		 6078 highest priority. Priorities of categories are also specified by a
6079 user in a Lisp variable `coding-category-list'.
6080
6081*/
6082
df7492f9
KH		 6083#define EOL_SEEN_NONE 0
6084#define EOL_SEEN_LF 1
6085#define EOL_SEEN_CR 2
6086#define EOL_SEEN_CRLF 4
66cfb530 6087
ff0dacd7
KH		 6088/* Detect how end-of-line of a text of length SRC_BYTES pointed by
6089 SOURCE is encoded. If CATEGORY is one of
6090 coding_category_utf_16_XXXX, assume that CR and LF are encoded by
6091 two-byte, else they are encoded by one-byte.
6092
6093 Return one of EOL_SEEN_XXX. */
4ed46869 6094
bc4bc72a 6095#define MAX_EOL_CHECK_COUNT 3
d46c5b12
KH		 6096
6097static int
d311d28c 6098detect_eol (const unsigned char *source, ptrdiff_t src_bytes,
cf84bb53 6099 enum coding_category category)
4ed46869 6100{
f6cbaf43 6101 const unsigned char *src = source, *src_end = src + src_bytes;
4ed46869 6102 unsigned char c;
df7492f9
KH		 6103 int total = 0;
6104 int eol_seen = EOL_SEEN_NONE;
4ed46869 6105
89528eb3 6106 if ((1 << category) & CATEGORY_MASK_UTF_16)
4ed46869 6107 {
df7492f9 6108 int msb, lsb;
fa42c37f 6109
89528eb3
KH		 6110 msb = category == (coding_category_utf_16_le
6111 | coding_category_utf_16_le_nosig);
df7492f9 6112 lsb = 1 - msb;
fa42c37f 6113
df7492f9 6114 while (src + 1 < src_end)
fa42c37f 6115 {
df7492f9
KH		 6116 c = src[lsb];
6117 if (src[msb] == 0 && (c == '\n' || c == '\r'))
fa42c37f 6118 {
df7492f9
KH		 6119 int this_eol;
6120
6121 if (c == '\n')
6122 this_eol = EOL_SEEN_LF;
6123 else if (src + 3 >= src_end
6124 || src[msb + 2] != 0
6125 || src[lsb + 2] != '\n')
6126 this_eol = EOL_SEEN_CR;
fa42c37f 6127 else
75f4f1ac
EZ		 6128 {
6129 this_eol = EOL_SEEN_CRLF;
6130 src += 2;
6131 }
df7492f9
KH		 6132
6133 if (eol_seen == EOL_SEEN_NONE)
6134 /* This is the first end-of-line. */
6135 eol_seen = this_eol;
6136 else if (eol_seen != this_eol)
fa42c37f 6137 {
75f4f1ac
EZ		 6138 /* The found type is different from what found before.
6139 Allow for stray ^M characters in DOS EOL files. */
ef1b0ba7
SM		 6140 if ((eol_seen == EOL_SEEN_CR && this_eol == EOL_SEEN_CRLF)
6141 || (eol_seen == EOL_SEEN_CRLF
6142 && this_eol == EOL_SEEN_CR))
75f4f1ac
EZ		 6143 eol_seen = EOL_SEEN_CRLF;
6144 else
6145 {
6146 eol_seen = EOL_SEEN_LF;
6147 break;
6148 }
fa42c37f 6149 }
df7492f9
KH		 6150 if (++total == MAX_EOL_CHECK_COUNT)
6151 break;
fa42c37f 6152 }
df7492f9 6153 src += 2;
fa42c37f 6154 }
bcf26d6a 6155 }
d46c5b12 6156 else
ef1b0ba7
SM		 6157 while (src < src_end)
6158 {
6159 c = *src++;
6160 if (c == '\n' || c == '\r')
6161 {
6162 int this_eol;
d46c5b12 6163
ef1b0ba7
SM		 6164 if (c == '\n')
6165 this_eol = EOL_SEEN_LF;
6166 else if (src >= src_end || *src != '\n')
6167 this_eol = EOL_SEEN_CR;
6168 else
6169 this_eol = EOL_SEEN_CRLF, src++;
d46c5b12 6170
ef1b0ba7
SM		 6171 if (eol_seen == EOL_SEEN_NONE)
6172 /* This is the first end-of-line. */
6173 eol_seen = this_eol;
6174 else if (eol_seen != this_eol)
6175 {
6176 /* The found type is different from what found before.
6177 Allow for stray ^M characters in DOS EOL files. */
6178 if ((eol_seen == EOL_SEEN_CR && this_eol == EOL_SEEN_CRLF)
6179 || (eol_seen == EOL_SEEN_CRLF && this_eol == EOL_SEEN_CR))
6180 eol_seen = EOL_SEEN_CRLF;
6181 else
6182 {
6183 eol_seen = EOL_SEEN_LF;
6184 break;
6185 }
6186 }
6187 if (++total == MAX_EOL_CHECK_COUNT)
6188 break;
6189 }
6190 }
df7492f9 6191 return eol_seen;
73be902c
KH		 6192}
6193
df7492f9 6194
24a73b0a 6195static Lisp_Object
971de7fb 6196adjust_coding_eol_type (struct coding_system *coding, int eol_seen)
73be902c 6197{
0be8721c 6198 Lisp_Object eol_type;
8f924df7 6199
df7492f9
KH		 6200 eol_type = CODING_ID_EOL_TYPE (coding->id);
6201 if (eol_seen & EOL_SEEN_LF)
24a73b0a
KH		 6202 {
6203 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 0));
6204 eol_type = Qunix;
6205 }
6f197c07 6206 else if (eol_seen & EOL_SEEN_CRLF)
24a73b0a
KH		 6207 {
6208 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 1));
6209 eol_type = Qdos;
6210 }
6f197c07 6211 else if (eol_seen & EOL_SEEN_CR)
24a73b0a
KH		 6212 {
6213 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 2));
6214 eol_type = Qmac;
6215 }
6216 return eol_type;
d46c5b12 6217}
4ed46869 6218
df7492f9
KH		 6219/* Detect how a text specified in CODING is encoded. If a coding
6220 system is detected, update fields of CODING by the detected coding
6221 system. */
0a28aafb 6222
74ab6df5 6223static void
971de7fb 6224detect_coding (struct coding_system *coding)
d46c5b12 6225{
8f924df7 6226 const unsigned char *src, *src_end;
73cce38d 6227 int saved_mode = coding->mode;
d46c5b12 6228
df7492f9
KH		 6229 coding->consumed = coding->consumed_char = 0;
6230 coding->produced = coding->produced_char = 0;
6231 coding_set_source (coding);
1c3478b0 6232
df7492f9 6233 src_end = coding->source + coding->src_bytes;
c0e16b14 6234 coding->head_ascii = 0;
1c3478b0 6235
df7492f9
KH		 6236 /* If we have not yet decided the text encoding type, detect it
6237 now. */
6238 if (EQ (CODING_ATTR_TYPE (CODING_ID_ATTRS (coding->id)), Qundecided))
b73bfc1c 6239 {
df7492f9 6240 int c, i;
6cb21a4f 6241 struct coding_detection_info detect_info;
2f3cbb32 6242 int null_byte_found = 0, eight_bit_found = 0;
df7492f9 6243
6cb21a4f 6244 detect_info.checked = detect_info.found = detect_info.rejected = 0;
2f3cbb32 6245 for (src = coding->source; src < src_end; src++)
d46c5b12 6246 {
df7492f9 6247 c = *src;
6cb21a4f 6248 if (c & 0x80)
6cb21a4f 6249 {
2f3cbb32 6250 eight_bit_found = 1;
2f3cbb32
KH		 6251 if (null_byte_found)
6252 break;
6253 }
6254 else if (c < 0x20)
6255 {
6256 if ((c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
6257 && ! inhibit_iso_escape_detection
6258 && ! detect_info.checked)
6cb21a4f 6259 {
2f3cbb32
KH		 6260 if (detect_coding_iso_2022 (coding, &detect_info))
6261 {
6262 /* We have scanned the whole data. */
6263 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
c0e16b14
KH		 6264 {
6265 /* We didn't find an 8-bit code. We may
6266 have found a null-byte, but it's very
ce5b453a 6267 rare that a binary file conforms to
c0e16b14
KH		 6268 ISO-2022. */
6269 src = src_end;
6270 coding->head_ascii = src - coding->source;
6271 }
6272 detect_info.rejected |= ~CATEGORY_MASK_ISO_ESCAPE;
2f3cbb32
KH		 6273 break;
6274 }
6275 }
97b1b294 6276 else if (! c && !inhibit_null_byte_detection)
2f3cbb32
KH		 6277 {
6278 null_byte_found = 1;
6279 if (eight_bit_found)
6280 break;
6cb21a4f 6281 }
c006c0c8
KH		 6282 if (! eight_bit_found)
6283 coding->head_ascii++;
6cb21a4f 6284 }
c006c0c8 6285 else if (! eight_bit_found)
c0e16b14 6286 coding->head_ascii++;
d46c5b12 6287 }
df7492f9 6288
2f3cbb32
KH		 6289 if (null_byte_found || eight_bit_found
6290 || coding->head_ascii < coding->src_bytes
6cb21a4f 6291 || detect_info.found)
d46c5b12 6292 {
ff0dacd7
KH		 6293 enum coding_category category;
6294 struct coding_system *this;
df7492f9 6295
6cb21a4f
KH		 6296 if (coding->head_ascii == coding->src_bytes)
6297 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
6298 for (i = 0; i < coding_category_raw_text; i++)
6299 {
6300 category = coding_priorities[i];
6301 this = coding_categories + category;
6302 if (detect_info.found & (1 << category))
24a73b0a 6303 break;
6cb21a4f
KH		 6304 }
6305 else
2f3cbb32
KH		 6306 {
6307 if (null_byte_found)
ff0dacd7 6308 {
2f3cbb32
KH		 6309 detect_info.checked |= ~CATEGORY_MASK_UTF_16;
6310 detect_info.rejected |= ~CATEGORY_MASK_UTF_16;
ff0dacd7 6311 }
2f3cbb32
KH		 6312 for (i = 0; i < coding_category_raw_text; i++)
6313 {
6314 category = coding_priorities[i];
6315 this = coding_categories + category;
6316 if (this->id < 0)
6317 {
6318 /* No coding system of this category is defined. */
6319 detect_info.rejected |= (1 << category);
6320 }
6321 else if (category >= coding_category_raw_text)
6322 continue;
6323 else if (detect_info.checked & (1 << category))
6324 {
6325 if (detect_info.found & (1 << category))
6326 break;
6327 }
6328 else if ((*(this->detector)) (coding, &detect_info)
6329 && detect_info.found & (1 << category))
6330 {
6331 if (category == coding_category_utf_16_auto)
6332 {
6333 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
6334 category = coding_category_utf_16_le;
6335 else
6336 category = coding_category_utf_16_be;
6337 }
6338 break;
6339 }
6340 }
2f3cbb32 6341 }
c0e16b14
KH		 6342
6343 if (i < coding_category_raw_text)
6344 setup_coding_system (CODING_ID_NAME (this->id), coding);
6345 else if (null_byte_found)
6346 setup_coding_system (Qno_conversion, coding);
6347 else if ((detect_info.rejected & CATEGORY_MASK_ANY)
6348 == CATEGORY_MASK_ANY)
6349 setup_coding_system (Qraw_text, coding);
6350 else if (detect_info.rejected)
6351 for (i = 0; i < coding_category_raw_text; i++)
6352 if (! (detect_info.rejected & (1 << coding_priorities[i])))
6353 {
6354 this = coding_categories + coding_priorities[i];
6355 setup_coding_system (CODING_ID_NAME (this->id), coding);
6356 break;
6357 }
d46c5b12 6358 }
b73bfc1c 6359 }
a470d443
KH		 6360 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
6361 == coding_category_utf_8_auto)
6362 {
6363 Lisp_Object coding_systems;
6364 struct coding_detection_info detect_info;
6365
6366 coding_systems
6367 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_bom);
6368 detect_info.found = detect_info.rejected = 0;
6369 coding->head_ascii = 0;
6370 if (CONSP (coding_systems)
6371 && detect_coding_utf_8 (coding, &detect_info))
6372 {
6373 if (detect_info.found & CATEGORY_MASK_UTF_8_SIG)
6374 setup_coding_system (XCAR (coding_systems), coding);
6375 else
6376 setup_coding_system (XCDR (coding_systems), coding);
6377 }
6378 }
24a73b0a
KH		 6379 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
6380 == coding_category_utf_16_auto)
b49a1807
KH		 6381 {
6382 Lisp_Object coding_systems;
6383 struct coding_detection_info detect_info;
6384
6385 coding_systems
a470d443 6386 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_bom);
b49a1807 6387 detect_info.found = detect_info.rejected = 0;
a470d443 6388 coding->head_ascii = 0;
b49a1807 6389 if (CONSP (coding_systems)
24a73b0a 6390 && detect_coding_utf_16 (coding, &detect_info))
b49a1807
KH		 6391 {
6392 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
6393 setup_coding_system (XCAR (coding_systems), coding);
24a73b0a 6394 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
b49a1807
KH		 6395 setup_coding_system (XCDR (coding_systems), coding);
6396 }
6397 }
73cce38d 6398 coding->mode = saved_mode;
4ed46869 6399}
4ed46869 6400
d46c5b12 6401
aaaf0b1e 6402static void
971de7fb 6403decode_eol (struct coding_system *coding)
aaaf0b1e 6404{
24a73b0a
KH		 6405 Lisp_Object eol_type;
6406 unsigned char *p, *pbeg, *pend;
3ed051d4 6407
24a73b0a 6408 eol_type = CODING_ID_EOL_TYPE (coding->id);
0a9564cb 6409 if (EQ (eol_type, Qunix) || inhibit_eol_conversion)
24a73b0a
KH		 6410 return;
6411
6412 if (NILP (coding->dst_object))
6413 pbeg = coding->destination;
6414 else
6415 pbeg = BYTE_POS_ADDR (coding->dst_pos_byte);
6416 pend = pbeg + coding->produced;
6417
6418 if (VECTORP (eol_type))
aaaf0b1e 6419 {
df7492f9 6420 int eol_seen = EOL_SEEN_NONE;
4ed46869 6421
24a73b0a 6422 for (p = pbeg; p < pend; p++)
aaaf0b1e 6423 {
df7492f9
KH		 6424 if (*p == '\n')
6425 eol_seen |= EOL_SEEN_LF;
6426 else if (*p == '\r')
aaaf0b1e 6427 {
df7492f9 6428 if (p + 1 < pend && *(p + 1) == '\n')
aaaf0b1e 6429 {
df7492f9
KH		 6430 eol_seen |= EOL_SEEN_CRLF;
6431 p++;
aaaf0b1e 6432 }
aaaf0b1e 6433 else
df7492f9 6434 eol_seen |= EOL_SEEN_CR;
aaaf0b1e 6435 }
aaaf0b1e 6436 }
75f4f1ac
EZ		 6437 /* Handle DOS-style EOLs in a file with stray ^M characters. */
6438 if ((eol_seen & EOL_SEEN_CRLF) != 0
6439 && (eol_seen & EOL_SEEN_CR) != 0
6440 && (eol_seen & EOL_SEEN_LF) == 0)
6441 eol_seen = EOL_SEEN_CRLF;
6442 else if (eol_seen != EOL_SEEN_NONE
24a73b0a
KH		 6443 && eol_seen != EOL_SEEN_LF
6444 && eol_seen != EOL_SEEN_CRLF
6445 && eol_seen != EOL_SEEN_CR)
6446 eol_seen = EOL_SEEN_LF;
df7492f9 6447 if (eol_seen != EOL_SEEN_NONE)
24a73b0a 6448 eol_type = adjust_coding_eol_type (coding, eol_seen);
aaaf0b1e 6449 }
d46c5b12 6450
24a73b0a 6451 if (EQ (eol_type, Qmac))
27901516 6452 {
24a73b0a 6453 for (p = pbeg; p < pend; p++)
df7492f9
KH		 6454 if (*p == '\r')
6455 *p = '\n';
4ed46869 6456 }
24a73b0a 6457 else if (EQ (eol_type, Qdos))
df7492f9 6458 {
d311d28c 6459 ptrdiff_t n = 0;
b73bfc1c 6460
24a73b0a
KH		 6461 if (NILP (coding->dst_object))
6462 {
4347441b
KH		 6463 /* Start deleting '\r' from the tail to minimize the memory
6464 movement. */
24a73b0a
KH		 6465 for (p = pend - 2; p >= pbeg; p--)
6466 if (*p == '\r')
6467 {
72af86bd 6468 memmove (p, p + 1, pend-- - p - 1);
24a73b0a
KH		 6469 n++;
6470 }
6471 }
6472 else
6473 {
d311d28c
PE		 6474 ptrdiff_t pos_byte = coding->dst_pos_byte;
6475 ptrdiff_t pos = coding->dst_pos;
6476 ptrdiff_t pos_end = pos + coding->produced_char - 1;
4347441b
KH		 6477
6478 while (pos < pos_end)
6479 {
6480 p = BYTE_POS_ADDR (pos_byte);
6481 if (*p == '\r' && p[1] == '\n')
6482 {
6483 del_range_2 (pos, pos_byte, pos + 1, pos_byte + 1, 0);
6484 n++;
6485 pos_end--;
6486 }
6487 pos++;
69b8522d
KH		 6488 if (coding->dst_multibyte)
6489 pos_byte += BYTES_BY_CHAR_HEAD (*p);
6490 else
6491 pos_byte++;
4347441b 6492 }
24a73b0a
KH		 6493 }
6494 coding->produced -= n;
6495 coding->produced_char -= n;
aaaf0b1e 6496 }
4ed46869
KH		 6497}
6498
7d64c6ad 6499
a6f87d34
KH		 6500/* Return a translation table (or list of them) from coding system
6501 attribute vector ATTRS for encoding (ENCODEP is nonzero) or
6502 decoding (ENCODEP is zero). */
7d64c6ad 6503
e6a54062 6504static Lisp_Object
971de7fb 6505get_translation_table (Lisp_Object attrs, int encodep, int *max_lookup)
7d64c6ad
KH		 6506{
6507 Lisp_Object standard, translation_table;
09ee6fdd 6508 Lisp_Object val;
7d64c6ad 6509
4bed5909
CY		 6510 if (NILP (Venable_character_translation))
6511 {
6512 if (max_lookup)
6513 *max_lookup = 0;
6514 return Qnil;
6515 }
7d64c6ad
KH		 6516 if (encodep)
6517 translation_table = CODING_ATTR_ENCODE_TBL (attrs),
6518 standard = Vstandard_translation_table_for_encode;
6519 else
6520 translation_table = CODING_ATTR_DECODE_TBL (attrs),
6521 standard = Vstandard_translation_table_for_decode;
7d64c6ad 6522 if (NILP (translation_table))
09ee6fdd
KH		 6523 translation_table = standard;
6524 else
a6f87d34 6525 {
09ee6fdd
KH		 6526 if (SYMBOLP (translation_table))
6527 translation_table = Fget (translation_table, Qtranslation_table);
6528 else if (CONSP (translation_table))
6529 {
6530 translation_table = Fcopy_sequence (translation_table);
6531 for (val = translation_table; CONSP (val); val = XCDR (val))
6532 if (SYMBOLP (XCAR (val)))
6533 XSETCAR (val, Fget (XCAR (val), Qtranslation_table));
6534 }
6535 if (CHAR_TABLE_P (standard))
6536 {
6537 if (CONSP (translation_table))
6538 translation_table = nconc2 (translation_table,
6539 Fcons (standard, Qnil));
6540 else
6541 translation_table = Fcons (translation_table,
6542 Fcons (standard, Qnil));
6543 }
a6f87d34 6544 }
2170c8f0
KH		 6545
6546 if (max_lookup)
09ee6fdd 6547 {
2170c8f0
KH		 6548 *max_lookup = 1;
6549 if (CHAR_TABLE_P (translation_table)
6550 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (translation_table)) > 1)
6551 {
6552 val = XCHAR_TABLE (translation_table)->extras[1];
6553 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
6554 *max_lookup = XFASTINT (val);
6555 }
6556 else if (CONSP (translation_table))
6557 {
2735d060 6558 Lisp_Object tail;
09ee6fdd 6559
2170c8f0
KH		 6560 for (tail = translation_table; CONSP (tail); tail = XCDR (tail))
6561 if (CHAR_TABLE_P (XCAR (tail))
6562 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (XCAR (tail))) > 1)
6563 {
2735d060
PE		 6564 Lisp_Object tailval = XCHAR_TABLE (XCAR (tail))->extras[1];
6565 if (NATNUMP (tailval) && *max_lookup < XFASTINT (tailval))
6566 *max_lookup = XFASTINT (tailval);
2170c8f0
KH		 6567 }
6568 }
a6f87d34 6569 }
7d64c6ad
KH		 6570 return translation_table;
6571}
6572
09ee6fdd
KH		 6573#define LOOKUP_TRANSLATION_TABLE(table, c, trans) \
6574 do { \
6575 trans = Qnil; \
6576 if (CHAR_TABLE_P (table)) \
6577 { \
6578 trans = CHAR_TABLE_REF (table, c); \
6579 if (CHARACTERP (trans)) \
6580 c = XFASTINT (trans), trans = Qnil; \
6581 } \
6582 else if (CONSP (table)) \
6583 { \
6584 Lisp_Object tail; \
6585 \
6586 for (tail = table; CONSP (tail); tail = XCDR (tail)) \
6587 if (CHAR_TABLE_P (XCAR (tail))) \
6588 { \
6589 trans = CHAR_TABLE_REF (XCAR (tail), c); \
6590 if (CHARACTERP (trans)) \
6591 c = XFASTINT (trans), trans = Qnil; \
6592 else if (! NILP (trans)) \
6593 break; \
6594 } \
6595 } \
e6a54062
KH		 6596 } while (0)
6597
7d64c6ad 6598
e951386e
KH		 6599/* Return a translation of character(s) at BUF according to TRANS.
6600 TRANS is TO-CHAR or ((FROM . TO) ...) where
6601 FROM = [FROM-CHAR ...], TO is TO-CHAR or [TO-CHAR ...].
6602 The return value is TO-CHAR or ([FROM-CHAR ...] . TO) if a
6603 translation is found, and Qnil if not found..
6604 If BUF is too short to lookup characters in FROM, return Qt. */
6605
69a80ea3 6606static Lisp_Object
971de7fb 6607get_translation (Lisp_Object trans, int *buf, int *buf_end)
69a80ea3 6608{
e951386e
KH		 6609
6610 if (INTEGERP (trans))
6611 return trans;
6612 for (; CONSP (trans); trans = XCDR (trans))
69a80ea3 6613 {
e951386e
KH		 6614 Lisp_Object val = XCAR (trans);
6615 Lisp_Object from = XCAR (val);
2c6a9faa
PE		 6616 ptrdiff_t len = ASIZE (from);
6617 ptrdiff_t i;
69a80ea3 6618
e951386e 6619 for (i = 0; i < len; i++)
69a80ea3 6620 {
e951386e
KH		 6621 if (buf + i == buf_end)
6622 return Qt;
6623 if (XINT (AREF (from, i)) != buf[i])
6624 break;
69a80ea3 6625 }
e951386e
KH		 6626 if (i == len)
6627 return val;
69a80ea3 6628 }
e951386e 6629 return Qnil;
69a80ea3
KH		 6630}
6631
6632
d46c5b12 6633static int
cf84bb53
JB		 6634produce_chars (struct coding_system *coding, Lisp_Object translation_table,
6635 int last_block)
4ed46869 6636{
df7492f9
KH		 6637 unsigned char *dst = coding->destination + coding->produced;
6638 unsigned char *dst_end = coding->destination + coding->dst_bytes;
d311d28c
PE		 6639 ptrdiff_t produced;
6640 ptrdiff_t produced_chars = 0;
69a80ea3 6641 int carryover = 0;
4ed46869 6642
df7492f9 6643 if (! coding->chars_at_source)
4ed46869 6644 {
119852e7 6645 /* Source characters are in coding->charbuf. */
fba4576f
AS		 6646 int *buf = coding->charbuf;
6647 int *buf_end = buf + coding->charbuf_used;
4ed46869 6648
db274c7a
KH		 6649 if (EQ (coding->src_object, coding->dst_object))
6650 {
6651 coding_set_source (coding);
6652 dst_end = ((unsigned char *) coding->source) + coding->consumed;
6653 }
4ed46869 6654
df7492f9 6655 while (buf < buf_end)
4ed46869 6656 {
69a80ea3 6657 int c = *buf, i;
bc4bc72a 6658
df7492f9
KH		 6659 if (c >= 0)
6660 {
d311d28c 6661 ptrdiff_t from_nchars = 1, to_nchars = 1;
69a80ea3
KH		 6662 Lisp_Object trans = Qnil;
6663
09ee6fdd 6664 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 6665 if (! NILP (trans))
69a80ea3 6666 {
e951386e
KH		 6667 trans = get_translation (trans, buf, buf_end);
6668 if (INTEGERP (trans))
6669 c = XINT (trans);
6670 else if (CONSP (trans))
6671 {
6672 from_nchars = ASIZE (XCAR (trans));
6673 trans = XCDR (trans);
6674 if (INTEGERP (trans))
6675 c = XINT (trans);
6676 else
6677 {
6678 to_nchars = ASIZE (trans);
6679 c = XINT (AREF (trans, 0));
6680 }
6681 }
6682 else if (EQ (trans, Qt) && ! last_block)
69a80ea3 6683 break;
69a80ea3
KH		 6684 }
6685
5d009b3a 6686 if ((dst_end - dst) / MAX_MULTIBYTE_LENGTH < to_nchars)
69a80ea3 6687 {
5d009b3a
PE		 6688 if (((min (PTRDIFF_MAX, SIZE_MAX) - (buf_end - buf))
6689 / MAX_MULTIBYTE_LENGTH)
6690 < to_nchars)
6691 memory_full (SIZE_MAX);
69a80ea3
KH		 6692 dst = alloc_destination (coding,
6693 buf_end - buf
6694 + MAX_MULTIBYTE_LENGTH * to_nchars,
6695 dst);
db274c7a
KH		 6696 if (EQ (coding->src_object, coding->dst_object))
6697 {
6698 coding_set_source (coding);
e951386e
KH		 6699 dst_end = (((unsigned char *) coding->source)
6700 + coding->consumed);
db274c7a
KH		 6701 }
6702 else
6703 dst_end = coding->destination + coding->dst_bytes;
69a80ea3
KH		 6704 }
6705
433f7f87 6706 for (i = 0; i < to_nchars; i++)
69a80ea3 6707 {
433f7f87
KH		 6708 if (i > 0)
6709 c = XINT (AREF (trans, i));
69a80ea3
KH		 6710 if (coding->dst_multibyte
6711 || ! CHAR_BYTE8_P (c))
db274c7a 6712 CHAR_STRING_ADVANCE_NO_UNIFY (c, dst);
69a80ea3
KH		 6713 else
6714 *dst++ = CHAR_TO_BYTE8 (c);
6715 }
6716 produced_chars += to_nchars;
e951386e 6717 buf += from_nchars;
d46c5b12 6718 }
df7492f9 6719 else
69a80ea3
KH		 6720 /* This is an annotation datum. (-C) is the length. */
6721 buf += -c;
4ed46869 6722 }
69a80ea3 6723 carryover = buf_end - buf;
4ed46869 6724 }
fa42c37f 6725 else
fa42c37f 6726 {
119852e7 6727 /* Source characters are at coding->source. */
8f924df7 6728 const unsigned char *src = coding->source;
119852e7 6729 const unsigned char *src_end = src + coding->consumed;
4ed46869 6730
db274c7a
KH		 6731 if (EQ (coding->dst_object, coding->src_object))
6732 dst_end = (unsigned char *) src;
df7492f9 6733 if (coding->src_multibyte != coding->dst_multibyte)
fa42c37f 6734 {
df7492f9 6735 if (coding->src_multibyte)
fa42c37f 6736 {
71c81426 6737 int multibytep = 1;
d311d28c 6738 ptrdiff_t consumed_chars = 0;
d46c5b12 6739
df7492f9
KH		 6740 while (1)
6741 {
8f924df7 6742 const unsigned char *src_base = src;
df7492f9 6743 int c;
b73bfc1c 6744
df7492f9 6745 ONE_MORE_BYTE (c);
119852e7 6746 if (dst == dst_end)
df7492f9 6747 {
119852e7
KH		 6748 if (EQ (coding->src_object, coding->dst_object))
6749 dst_end = (unsigned char *) src;
6750 if (dst == dst_end)
df7492f9 6751 {
d311d28c 6752 ptrdiff_t offset = src - coding->source;
119852e7
KH		 6753
6754 dst = alloc_destination (coding, src_end - src + 1,
6755 dst);
6756 dst_end = coding->destination + coding->dst_bytes;
6757 coding_set_source (coding);
6758 src = coding->source + offset;
6759 src_end = coding->source + coding->src_bytes;
db274c7a
KH		 6760 if (EQ (coding->src_object, coding->dst_object))
6761 dst_end = (unsigned char *) src;
df7492f9 6762 }
df7492f9
KH		 6763 }
6764 *dst++ = c;
6765 produced_chars++;
6766 }
6767 no_more_source:
6768 ;
fa42c37f
KH		 6769 }
6770 else
df7492f9
KH		 6771 while (src < src_end)
6772 {
71c81426 6773 int multibytep = 1;
df7492f9 6774 int c = *src++;
b73bfc1c 6775
df7492f9
KH		 6776 if (dst >= dst_end - 1)
6777 {
2c78b7e1 6778 if (EQ (coding->src_object, coding->dst_object))
8f924df7 6779 dst_end = (unsigned char *) src;
2c78b7e1
KH		 6780 if (dst >= dst_end - 1)
6781 {
d311d28c
PE		 6782 ptrdiff_t offset = src - coding->source;
6783 ptrdiff_t more_bytes;
119852e7 6784
db274c7a
KH		 6785 if (EQ (coding->src_object, coding->dst_object))
6786 more_bytes = ((src_end - src) / 2) + 2;
6787 else
6788 more_bytes = src_end - src + 2;
6789 dst = alloc_destination (coding, more_bytes, dst);
2c78b7e1
KH		 6790 dst_end = coding->destination + coding->dst_bytes;
6791 coding_set_source (coding);
119852e7 6792 src = coding->source + offset;
2c78b7e1 6793 src_end = coding->source + coding->src_bytes;
db274c7a
KH		 6794 if (EQ (coding->src_object, coding->dst_object))
6795 dst_end = (unsigned char *) src;
2c78b7e1 6796 }
df7492f9
KH		 6797 }
6798 EMIT_ONE_BYTE (c);
6799 }
d46c5b12 6800 }
df7492f9
KH		 6801 else
6802 {
6803 if (!EQ (coding->src_object, coding->dst_object))
fa42c37f 6804 {
d311d28c 6805 ptrdiff_t require = coding->src_bytes - coding->dst_bytes;
4ed46869 6806
df7492f9 6807 if (require > 0)
fa42c37f 6808 {
d311d28c 6809 ptrdiff_t offset = src - coding->source;
df7492f9
KH		 6810
6811 dst = alloc_destination (coding, require, dst);
6812 coding_set_source (coding);
6813 src = coding->source + offset;
6814 src_end = coding->source + coding->src_bytes;
fa42c37f
KH		 6815 }
6816 }
119852e7 6817 produced_chars = coding->consumed_char;
df7492f9 6818 while (src < src_end)
14daee73 6819 *dst++ = *src++;
fa42c37f
KH		 6820 }
6821 }
6822
df7492f9 6823 produced = dst - (coding->destination + coding->produced);
284201e4 6824 if (BUFFERP (coding->dst_object) && produced_chars > 0)
df7492f9
KH		 6825 insert_from_gap (produced_chars, produced);
6826 coding->produced += produced;
6827 coding->produced_char += produced_chars;
69a80ea3 6828 return carryover;
fa42c37f
KH		 6829}
6830
ff0dacd7
KH		 6831/* Compose text in CODING->object according to the annotation data at
6832 CHARBUF. CHARBUF is an array:
e951386e 6833 [ -LENGTH ANNOTATION_MASK NCHARS NBYTES METHOD [ COMPONENTS... ] ]
df7492f9 6834 */
4ed46869 6835
55d4c1b2 6836static inline void
d311d28c 6837produce_composition (struct coding_system *coding, int *charbuf, ptrdiff_t pos)
4ed46869 6838{
df7492f9 6839 int len;
d311d28c 6840 ptrdiff_t to;
df7492f9 6841 enum composition_method method;
df7492f9 6842 Lisp_Object components;
fa42c37f 6843
e951386e 6844 len = -charbuf[0] - MAX_ANNOTATION_LENGTH;
69a80ea3 6845 to = pos + charbuf[2];
e951386e 6846 method = (enum composition_method) (charbuf[4]);
d46c5b12 6847
df7492f9
KH		 6848 if (method == COMPOSITION_RELATIVE)
6849 components = Qnil;
e951386e 6850 else
d46c5b12 6851 {
df7492f9 6852 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
e951386e 6853 int i, j;
b73bfc1c 6854
e951386e
KH		 6855 if (method == COMPOSITION_WITH_RULE)
6856 len = charbuf[2] * 3 - 2;
6857 charbuf += MAX_ANNOTATION_LENGTH;
6858 /* charbuf = [ CHRA ... CHAR] or [ CHAR -2 RULE ... CHAR ] */
6859 for (i = j = 0; i < len && charbuf[i] != -1; i++, j++)
9ffd559c 6860 {
e951386e
KH		 6861 if (charbuf[i] >= 0)
6862 args[j] = make_number (charbuf[i]);
6863 else
6864 {
6865 i++;
6866 args[j] = make_number (charbuf[i] % 0x100);
6867 }
9ffd559c 6868 }
e951386e 6869 components = (i == j ? Fstring (j, args) : Fvector (j, args));
d46c5b12 6870 }
69a80ea3 6871 compose_text (pos, to, components, Qnil, coding->dst_object);
d46c5b12
KH		 6872}
6873
d46c5b12 6874
ff0dacd7
KH		 6875/* Put `charset' property on text in CODING->object according to
6876 the annotation data at CHARBUF. CHARBUF is an array:
69a80ea3 6877 [ -LENGTH ANNOTATION_MASK NCHARS CHARSET-ID ]
ff0dacd7 6878 */
d46c5b12 6879
55d4c1b2 6880static inline void
d311d28c 6881produce_charset (struct coding_system *coding, int *charbuf, ptrdiff_t pos)
d46c5b12 6882{
d311d28c 6883 ptrdiff_t from = pos - charbuf[2];
69a80ea3 6884 struct charset *charset = CHARSET_FROM_ID (charbuf[3]);
b73bfc1c 6885
69a80ea3 6886 Fput_text_property (make_number (from), make_number (pos),
ff0dacd7
KH		 6887 Qcharset, CHARSET_NAME (charset),
6888 coding->dst_object);
d46c5b12
KH		 6889}
6890
d46c5b12 6891
df7492f9
KH		 6892#define CHARBUF_SIZE 0x4000
6893
6894#define ALLOC_CONVERSION_WORK_AREA(coding) \
6895 do { \
8510724d 6896 int size = CHARBUF_SIZE; \
df7492f9
KH		 6897 \
6898 coding->charbuf = NULL; \
6899 while (size > 1024) \
6900 { \
6901 coding->charbuf = (int *) alloca (sizeof (int) * size); \
6902 if (coding->charbuf) \
6903 break; \
6904 size >>= 1; \
6905 } \
6906 if (! coding->charbuf) \
6907 { \
065e3595 6908 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_MEM); \
df7492f9
KH		 6909 return coding->result; \
6910 } \
6911 coding->charbuf_size = size; \
6912 } while (0)
4ed46869 6913
d46c5b12
KH		 6914
6915static void
d311d28c 6916produce_annotation (struct coding_system *coding, ptrdiff_t pos)
d46c5b12 6917{
df7492f9
KH		 6918 int *charbuf = coding->charbuf;
6919 int *charbuf_end = charbuf + coding->charbuf_used;
d46c5b12 6920
ff0dacd7
KH		 6921 if (NILP (coding->dst_object))
6922 return;
d46c5b12 6923
df7492f9 6924 while (charbuf < charbuf_end)
a84f1519 6925 {
df7492f9 6926 if (*charbuf >= 0)
e951386e 6927 pos++, charbuf++;
d46c5b12 6928 else
d46c5b12 6929 {
df7492f9 6930 int len = -*charbuf;
e951386e
KH		 6931
6932 if (len > 2)
6933 switch (charbuf[1])
6934 {
6935 case CODING_ANNOTATE_COMPOSITION_MASK:
6936 produce_composition (coding, charbuf, pos);
6937 break;
6938 case CODING_ANNOTATE_CHARSET_MASK:
6939 produce_charset (coding, charbuf, pos);
6940 break;
6941 }
df7492f9 6942 charbuf += len;
d46c5b12 6943 }
a84f1519 6944 }
d46c5b12
KH		 6945}
6946
df7492f9
KH		 6947/* Decode the data at CODING->src_object into CODING->dst_object.
6948 CODING->src_object is a buffer, a string, or nil.
6949 CODING->dst_object is a buffer.
d46c5b12 6950
df7492f9
KH		 6951 If CODING->src_object is a buffer, it must be the current buffer.
6952 In this case, if CODING->src_pos is positive, it is a position of
6953 the source text in the buffer, otherwise, the source text is in the
6954 gap area of the buffer, and CODING->src_pos specifies the offset of
6955 the text from GPT (which must be the same as PT). If this is the
6956 same buffer as CODING->dst_object, CODING->src_pos must be
6957 negative.
d46c5b12 6958
b6828792 6959 If CODING->src_object is a string, CODING->src_pos is an index to
df7492f9 6960 that string.
d46c5b12 6961
df7492f9
KH		 6962 If CODING->src_object is nil, CODING->source must already point to
6963 the non-relocatable memory area. In this case, CODING->src_pos is
6964 an offset from CODING->source.
73be902c 6965
df7492f9
KH		 6966 The decoded data is inserted at the current point of the buffer
6967 CODING->dst_object.
6968*/
d46c5b12 6969
df7492f9 6970static int
971de7fb 6971decode_coding (struct coding_system *coding)
d46c5b12 6972{
df7492f9 6973 Lisp_Object attrs;
24a73b0a 6974 Lisp_Object undo_list;
7d64c6ad 6975 Lisp_Object translation_table;
d0396581 6976 struct ccl_spec cclspec;
69a80ea3
KH		 6977 int carryover;
6978 int i;
d46c5b12 6979
df7492f9
KH		 6980 if (BUFFERP (coding->src_object)
6981 && coding->src_pos > 0
6982 && coding->src_pos < GPT
6983 && coding->src_pos + coding->src_chars > GPT)
6984 move_gap_both (coding->src_pos, coding->src_pos_byte);
d46c5b12 6985
24a73b0a 6986 undo_list = Qt;
df7492f9 6987 if (BUFFERP (coding->dst_object))
1c3478b0 6988 {
df7492f9
KH		 6989 if (current_buffer != XBUFFER (coding->dst_object))
6990 set_buffer_internal (XBUFFER (coding->dst_object));
6991 if (GPT != PT)
6992 move_gap_both (PT, PT_BYTE);
4b4deea2
TT		 6993 undo_list = BVAR (current_buffer, undo_list);
6994 BVAR (current_buffer, undo_list) = Qt;
1c3478b0
KH		 6995 }
6996
df7492f9
KH		 6997 coding->consumed = coding->consumed_char = 0;
6998 coding->produced = coding->produced_char = 0;
6999 coding->chars_at_source = 0;
065e3595 7000 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 7001 coding->errors = 0;
1c3478b0 7002
df7492f9
KH		 7003 ALLOC_CONVERSION_WORK_AREA (coding);
7004
7005 attrs = CODING_ID_ATTRS (coding->id);
2170c8f0 7006 translation_table = get_translation_table (attrs, 0, NULL);
df7492f9 7007
69a80ea3 7008 carryover = 0;
d0396581
KH		 7009 if (coding->decoder == decode_coding_ccl)
7010 {
7011 coding->spec.ccl = &cclspec;
7012 setup_ccl_program (&cclspec.ccl, CODING_CCL_DECODER (coding));
7013 }
df7492f9 7014 do
b73bfc1c 7015 {
d311d28c 7016 ptrdiff_t pos = coding->dst_pos + coding->produced_char;
69a80ea3 7017
df7492f9
KH		 7018 coding_set_source (coding);
7019 coding->annotated = 0;
69a80ea3 7020 coding->charbuf_used = carryover;
df7492f9 7021 (*(coding->decoder)) (coding);
df7492f9 7022 coding_set_destination (coding);
69a80ea3 7023 carryover = produce_chars (coding, translation_table, 0);
df7492f9 7024 if (coding->annotated)
69a80ea3
KH		 7025 produce_annotation (coding, pos);
7026 for (i = 0; i < carryover; i++)
7027 coding->charbuf[i]
7028 = coding->charbuf[coding->charbuf_used - carryover + i];
d46c5b12 7029 }
d0396581
KH		 7030 while (coding->result == CODING_RESULT_INSUFFICIENT_DST
7031 || (coding->consumed < coding->src_bytes
7032 && (coding->result == CODING_RESULT_SUCCESS
7033 || coding->result == CODING_RESULT_INVALID_SRC)));
d46c5b12 7034
69a80ea3
KH		 7035 if (carryover > 0)
7036 {
7037 coding_set_destination (coding);
7038 coding->charbuf_used = carryover;
7039 produce_chars (coding, translation_table, 1);
7040 }
7041
df7492f9
KH		 7042 coding->carryover_bytes = 0;
7043 if (coding->consumed < coding->src_bytes)
d46c5b12 7044 {
df7492f9 7045 int nbytes = coding->src_bytes - coding->consumed;
8f924df7 7046 const unsigned char *src;
df7492f9
KH		 7047
7048 coding_set_source (coding);
7049 coding_set_destination (coding);
7050 src = coding->source + coding->consumed;
7051
7052 if (coding->mode & CODING_MODE_LAST_BLOCK)
1c3478b0 7053 {
df7492f9
KH		 7054 /* Flush out unprocessed data as binary chars. We are sure
7055 that the number of data is less than the size of
7056 coding->charbuf. */
065e3595 7057 coding->charbuf_used = 0;
b2dab6c8
JR		 7058 coding->chars_at_source = 0;
7059
df7492f9 7060 while (nbytes-- > 0)
1c3478b0 7061 {
df7492f9 7062 int c = *src++;
98725083 7063
1c91457d
KH		 7064 if (c & 0x80)
7065 c = BYTE8_TO_CHAR (c);
7066 coding->charbuf[coding->charbuf_used++] = c;
1c3478b0 7067 }
f6cbaf43 7068 produce_chars (coding, Qnil, 1);
d46c5b12 7069 }
d46c5b12 7070 else
df7492f9
KH		 7071 {
7072 /* Record unprocessed bytes in coding->carryover. We are
7073 sure that the number of data is less than the size of
7074 coding->carryover. */
7075 unsigned char *p = coding->carryover;
7076
f289d375
KH		 7077 if (nbytes > sizeof coding->carryover)
7078 nbytes = sizeof coding->carryover;
df7492f9
KH		 7079 coding->carryover_bytes = nbytes;
7080 while (nbytes-- > 0)
7081 *p++ = *src++;
1c3478b0 7082 }
df7492f9 7083 coding->consumed = coding->src_bytes;
b73bfc1c 7084 }
69f76525 7085
0a9564cb
EZ		 7086 if (! EQ (CODING_ID_EOL_TYPE (coding->id), Qunix)
7087 && !inhibit_eol_conversion)
4347441b 7088 decode_eol (coding);
24a73b0a
KH		 7089 if (BUFFERP (coding->dst_object))
7090 {
4b4deea2 7091 BVAR (current_buffer, undo_list) = undo_list;
24a73b0a
KH		 7092 record_insert (coding->dst_pos, coding->produced_char);
7093 }
73be902c 7094 return coding->result;
4ed46869
KH		 7095}
7096
aaaf0b1e 7097
e1c23804 7098/* Extract an annotation datum from a composition starting at POS and
ff0dacd7
KH		 7099 ending before LIMIT of CODING->src_object (buffer or string), store
7100 the data in BUF, set *STOP to a starting position of the next
7101 composition (if any) or to LIMIT, and return the address of the
7102 next element of BUF.
7103
7104 If such an annotation is not found, set *STOP to a starting
7105 position of a composition after POS (if any) or to LIMIT, and
7106 return BUF. */
7107
55d4c1b2 7108static inline int *
d311d28c 7109handle_composition_annotation (ptrdiff_t pos, ptrdiff_t limit,
cf84bb53 7110 struct coding_system *coding, int *buf,
d311d28c 7111 ptrdiff_t *stop)
aaaf0b1e 7112{
d311d28c 7113 ptrdiff_t start, end;
ff0dacd7 7114 Lisp_Object prop;
aaaf0b1e 7115
ff0dacd7
KH		 7116 if (! find_composition (pos, limit, &start, &end, &prop, coding->src_object)
7117 || end > limit)
7118 *stop = limit;
7119 else if (start > pos)
7120 *stop = start;
7121 else
aaaf0b1e 7122 {
ff0dacd7 7123 if (start == pos)
aaaf0b1e 7124 {
ff0dacd7
KH		 7125 /* We found a composition. Store the corresponding
7126 annotation data in BUF. */
7127 int *head = buf;
7128 enum composition_method method = COMPOSITION_METHOD (prop);
7129 int nchars = COMPOSITION_LENGTH (prop);
7130
e951386e 7131 ADD_COMPOSITION_DATA (buf, nchars, 0, method);
ff0dacd7 7132 if (method != COMPOSITION_RELATIVE)
aaaf0b1e 7133 {
ff0dacd7 7134 Lisp_Object components;
2c6a9faa 7135 ptrdiff_t i, len, i_byte;
ff0dacd7
KH		 7136
7137 components = COMPOSITION_COMPONENTS (prop);
7138 if (VECTORP (components))
aaaf0b1e 7139 {
77b37c05 7140 len = ASIZE (components);
ff0dacd7
KH		 7141 for (i = 0; i < len; i++)
7142 *buf++ = XINT (AREF (components, i));
aaaf0b1e 7143 }
ff0dacd7 7144 else if (STRINGP (components))
aaaf0b1e 7145 {
8f924df7 7146 len = SCHARS (components);
ff0dacd7
KH		 7147 i = i_byte = 0;
7148 while (i < len)
7149 {
7150 FETCH_STRING_CHAR_ADVANCE (*buf, components, i, i_byte);
7151 buf++;
7152 }
7153 }
7154 else if (INTEGERP (components))
7155 {
7156 len = 1;
7157 *buf++ = XINT (components);
7158 }
7159 else if (CONSP (components))
7160 {
7161 for (len = 0; CONSP (components);
7162 len++, components = XCDR (components))
7163 *buf++ = XINT (XCAR (components));
aaaf0b1e 7164 }
aaaf0b1e 7165 else
ff0dacd7
KH		 7166 abort ();
7167 *head -= len;
aaaf0b1e 7168 }
aaaf0b1e 7169 }
ff0dacd7
KH		 7170
7171 if (find_composition (end, limit, &start, &end, &prop,
7172 coding->src_object)
7173 && end <= limit)
7174 *stop = start;
7175 else
7176 *stop = limit;
aaaf0b1e 7177 }
ff0dacd7
KH		 7178 return buf;
7179}
7180
7181
e1c23804 7182/* Extract an annotation datum from a text property `charset' at POS of
ff0dacd7
KH		 7183 CODING->src_object (buffer of string), store the data in BUF, set
7184 *STOP to the position where the value of `charset' property changes
7185 (limiting by LIMIT), and return the address of the next element of
7186 BUF.
7187
7188 If the property value is nil, set *STOP to the position where the
7189 property value is non-nil (limiting by LIMIT), and return BUF. */
7190
55d4c1b2 7191static inline int *
d311d28c 7192handle_charset_annotation (ptrdiff_t pos, ptrdiff_t limit,
cf84bb53 7193 struct coding_system *coding, int *buf,
d311d28c 7194 ptrdiff_t *stop)
ff0dacd7
KH		 7195{
7196 Lisp_Object val, next;
7197 int id;
7198
7199 val = Fget_text_property (make_number (pos), Qcharset, coding->src_object);
7200 if (! NILP (val) && CHARSETP (val))
7201 id = XINT (CHARSET_SYMBOL_ID (val));
7202 else
7203 id = -1;
69a80ea3 7204 ADD_CHARSET_DATA (buf, 0, id);
ff0dacd7
KH		 7205 next = Fnext_single_property_change (make_number (pos), Qcharset,
7206 coding->src_object,
7207 make_number (limit));
7208 *stop = XINT (next);
7209 return buf;
7210}
7211
7212
df7492f9 7213static void
cf84bb53
JB		 7214consume_chars (struct coding_system *coding, Lisp_Object translation_table,
7215 int max_lookup)
df7492f9
KH		 7216{
7217 int *buf = coding->charbuf;
ff0dacd7 7218 int *buf_end = coding->charbuf + coding->charbuf_size;
7c78e542 7219 const unsigned char *src = coding->source + coding->consumed;
4776e638 7220 const unsigned char *src_end = coding->source + coding->src_bytes;
d311d28c
PE		 7221 ptrdiff_t pos = coding->src_pos + coding->consumed_char;
7222 ptrdiff_t end_pos = coding->src_pos + coding->src_chars;
df7492f9
KH		 7223 int multibytep = coding->src_multibyte;
7224 Lisp_Object eol_type;
7225 int c;
d311d28c 7226 ptrdiff_t stop, stop_composition, stop_charset;
09ee6fdd 7227 int *lookup_buf = NULL;
433f7f87
KH		 7228
7229 if (! NILP (translation_table))
09ee6fdd 7230 lookup_buf = alloca (sizeof (int) * max_lookup);
88993dfd 7231
0a9564cb 7232 eol_type = inhibit_eol_conversion ? Qunix : CODING_ID_EOL_TYPE (coding->id);
df7492f9
KH		 7233 if (VECTORP (eol_type))
7234 eol_type = Qunix;
88993dfd 7235
df7492f9
KH		 7236 /* Note: composition handling is not yet implemented. */
7237 coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
ec6d2bb8 7238
0b5670c9
KH		 7239 if (NILP (coding->src_object))
7240 stop = stop_composition = stop_charset = end_pos;
ff0dacd7 7241 else
0b5670c9
KH		 7242 {
7243 if (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK)
7244 stop = stop_composition = pos;
7245 else
7246 stop = stop_composition = end_pos;
7247 if (coding->common_flags & CODING_ANNOTATE_CHARSET_MASK)
7248 stop = stop_charset = pos;
7249 else
7250 stop_charset = end_pos;
7251 }
ec6d2bb8 7252
24a73b0a 7253 /* Compensate for CRLF and conversion. */
ff0dacd7 7254 buf_end -= 1 + MAX_ANNOTATION_LENGTH;
df7492f9 7255 while (buf < buf_end)
aaaf0b1e 7256 {
433f7f87
KH		 7257 Lisp_Object trans;
7258
df7492f9 7259 if (pos == stop)
ec6d2bb8 7260 {
df7492f9
KH		 7261 if (pos == end_pos)
7262 break;
ff0dacd7
KH		 7263 if (pos == stop_composition)
7264 buf = handle_composition_annotation (pos, end_pos, coding,
7265 buf, &stop_composition);
7266 if (pos == stop_charset)
7267 buf = handle_charset_annotation (pos, end_pos, coding,
7268 buf, &stop_charset);
7269 stop = (stop_composition < stop_charset
7270 ? stop_composition : stop_charset);
df7492f9
KH		 7271 }
7272
7273 if (! multibytep)
4776e638 7274 {
d311d28c 7275 int bytes;
aaaf0b1e 7276
4d1e6632
KH		 7277 if (coding->encoder == encode_coding_raw_text
7278 || coding->encoder == encode_coding_ccl)
ea29edf2
KH		 7279 c = *src++, pos++;
7280 else if ((bytes = MULTIBYTE_LENGTH (src, src_end)) > 0)
db274c7a 7281 c = STRING_CHAR_ADVANCE_NO_UNIFY (src), pos += bytes;
4776e638 7282 else
f03caae0 7283 c = BYTE8_TO_CHAR (*src), src++, pos++;
4776e638 7284 }
df7492f9 7285 else
db274c7a 7286 c = STRING_CHAR_ADVANCE_NO_UNIFY (src), pos++;
df7492f9
KH		 7287 if ((c == '\r') && (coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
7288 c = '\n';
7289 if (! EQ (eol_type, Qunix))
aaaf0b1e 7290 {
df7492f9 7291 if (c == '\n')
aaaf0b1e 7292 {
df7492f9
KH		 7293 if (EQ (eol_type, Qdos))
7294 *buf++ = '\r';
7295 else
7296 c = '\r';
aaaf0b1e
KH		 7297 }
7298 }
433f7f87 7299
e6a54062 7300 trans = Qnil;
09ee6fdd 7301 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 7302 if (NILP (trans))
433f7f87
KH		 7303 *buf++ = c;
7304 else
7305 {
2c6a9faa 7306 ptrdiff_t from_nchars = 1, to_nchars = 1;
433f7f87
KH		 7307 int *lookup_buf_end;
7308 const unsigned char *p = src;
7309 int i;
7310
7311 lookup_buf[0] = c;
7312 for (i = 1; i < max_lookup && p < src_end; i++)
7313 lookup_buf[i] = STRING_CHAR_ADVANCE (p);
7314 lookup_buf_end = lookup_buf + i;
e951386e
KH		 7315 trans = get_translation (trans, lookup_buf, lookup_buf_end);
7316 if (INTEGERP (trans))
7317 c = XINT (trans);
7318 else if (CONSP (trans))
7319 {
7320 from_nchars = ASIZE (XCAR (trans));
7321 trans = XCDR (trans);
7322 if (INTEGERP (trans))
7323 c = XINT (trans);
7324 else
7325 {
7326 to_nchars = ASIZE (trans);
2c6a9faa 7327 if (buf_end - buf < to_nchars)
e951386e
KH		 7328 break;
7329 c = XINT (AREF (trans, 0));
7330 }
7331 }
7332 else
433f7f87 7333 break;
e951386e 7334 *buf++ = c;
433f7f87
KH		 7335 for (i = 1; i < to_nchars; i++)
7336 *buf++ = XINT (AREF (trans, i));
7337 for (i = 1; i < from_nchars; i++, pos++)
7338 src += MULTIBYTE_LENGTH_NO_CHECK (src);
7339 }
aaaf0b1e 7340 }
ec6d2bb8 7341
df7492f9
KH		 7342 coding->consumed = src - coding->source;
7343 coding->consumed_char = pos - coding->src_pos;
7344 coding->charbuf_used = buf - coding->charbuf;
7345 coding->chars_at_source = 0;
aaaf0b1e
KH		 7346}
7347
4ed46869 7348
df7492f9
KH		 7349/* Encode the text at CODING->src_object into CODING->dst_object.
7350 CODING->src_object is a buffer or a string.
7351 CODING->dst_object is a buffer or nil.
7352
7353 If CODING->src_object is a buffer, it must be the current buffer.
7354 In this case, if CODING->src_pos is positive, it is a position of
7355 the source text in the buffer, otherwise. the source text is in the
7356 gap area of the buffer, and coding->src_pos specifies the offset of
7357 the text from GPT (which must be the same as PT). If this is the
7358 same buffer as CODING->dst_object, CODING->src_pos must be
7359 negative and CODING should not have `pre-write-conversion'.
7360
7361 If CODING->src_object is a string, CODING should not have
7362 `pre-write-conversion'.
7363
7364 If CODING->dst_object is a buffer, the encoded data is inserted at
7365 the current point of that buffer.
7366
7367 If CODING->dst_object is nil, the encoded data is placed at the
7368 memory area specified by CODING->destination. */
7369
7370static int
971de7fb 7371encode_coding (struct coding_system *coding)
4ed46869 7372{
df7492f9 7373 Lisp_Object attrs;
7d64c6ad 7374 Lisp_Object translation_table;
09ee6fdd 7375 int max_lookup;
fb608df3 7376 struct ccl_spec cclspec;
9861e777 7377
df7492f9 7378 attrs = CODING_ID_ATTRS (coding->id);
ea29edf2
KH		 7379 if (coding->encoder == encode_coding_raw_text)
7380 translation_table = Qnil, max_lookup = 0;
7381 else
7382 translation_table = get_translation_table (attrs, 1, &max_lookup);
4ed46869 7383
df7492f9 7384 if (BUFFERP (coding->dst_object))
8844fa83 7385 {
df7492f9
KH		 7386 set_buffer_internal (XBUFFER (coding->dst_object));
7387 coding->dst_multibyte
4b4deea2 7388 = ! NILP (BVAR (current_buffer, enable_multibyte_characters));
8844fa83 7389 }
4ed46869 7390
b73bfc1c 7391 coding->consumed = coding->consumed_char = 0;
df7492f9 7392 coding->produced = coding->produced_char = 0;
065e3595 7393 record_conversion_result (coding, CODING_RESULT_SUCCESS);
b73bfc1c 7394 coding->errors = 0;
b73bfc1c 7395
df7492f9 7396 ALLOC_CONVERSION_WORK_AREA (coding);
4ed46869 7397
fb608df3
KH		 7398 if (coding->encoder == encode_coding_ccl)
7399 {
7400 coding->spec.ccl = &cclspec;
7401 setup_ccl_program (&cclspec.ccl, CODING_CCL_ENCODER (coding));
7402 }
df7492f9
KH		 7403 do {
7404 coding_set_source (coding);
09ee6fdd 7405 consume_chars (coding, translation_table, max_lookup);
df7492f9
KH		 7406 coding_set_destination (coding);
7407 (*(coding->encoder)) (coding);
7408 } while (coding->consumed_char < coding->src_chars);
7409
284201e4 7410 if (BUFFERP (coding->dst_object) && coding->produced_char > 0)
df7492f9
KH		 7411 insert_from_gap (coding->produced_char, coding->produced);
7412
7413 return (coding->result);
ec6d2bb8
KH		 7414}
7415
fb88bf2d 7416
24a73b0a
KH		 7417/* Name (or base name) of work buffer for code conversion. */
7418static Lisp_Object Vcode_conversion_workbuf_name;
d46c5b12 7419
24a73b0a
KH		 7420/* A working buffer used by the top level conversion. Once it is
7421 created, it is never destroyed. It has the name
7422 Vcode_conversion_workbuf_name. The other working buffers are
7423 destroyed after the use is finished, and their names are modified
7424 versions of Vcode_conversion_workbuf_name. */
7425static Lisp_Object Vcode_conversion_reused_workbuf;
b73bfc1c 7426
24a73b0a
KH		 7427/* 1 iff Vcode_conversion_reused_workbuf is already in use. */
7428static int reused_workbuf_in_use;
4ed46869 7429
24a73b0a 7430
ad1746f5 7431/* Return a working buffer of code conversion. MULTIBYTE specifies the
24a73b0a 7432 multibyteness of returning buffer. */
b73bfc1c 7433
f6cbaf43 7434static Lisp_Object
971de7fb 7435make_conversion_work_buffer (int multibyte)
df7492f9 7436{
24a73b0a
KH		 7437 Lisp_Object name, workbuf;
7438 struct buffer *current;
4ed46869 7439
24a73b0a 7440 if (reused_workbuf_in_use++)
065e3595
KH		 7441 {
7442 name = Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name, Qnil);
7443 workbuf = Fget_buffer_create (name);
7444 }
df7492f9 7445 else
065e3595 7446 {
159bd5a2 7447 if (NILP (Fbuffer_live_p (Vcode_conversion_reused_workbuf)))
a993c7a1
KH		 7448 Vcode_conversion_reused_workbuf
7449 = Fget_buffer_create (Vcode_conversion_workbuf_name);
7450 workbuf = Vcode_conversion_reused_workbuf;
065e3595 7451 }
24a73b0a
KH		 7452 current = current_buffer;
7453 set_buffer_internal (XBUFFER (workbuf));
df36ff1f
CY		 7454 /* We can't allow modification hooks to run in the work buffer. For
7455 instance, directory_files_internal assumes that file decoding
7456 doesn't compile new regexps. */
7457 Fset (Fmake_local_variable (Qinhibit_modification_hooks), Qt);
3ed051d4 7458 Ferase_buffer ();
4b4deea2
TT		 7459 BVAR (current_buffer, undo_list) = Qt;
7460 BVAR (current_buffer, enable_multibyte_characters) = multibyte ? Qt : Qnil;
df7492f9 7461 set_buffer_internal (current);
24a73b0a 7462 return workbuf;
df7492f9 7463}
d46c5b12 7464
24a73b0a 7465
4776e638 7466static Lisp_Object
971de7fb 7467code_conversion_restore (Lisp_Object arg)
4776e638 7468{
24a73b0a 7469 Lisp_Object current, workbuf;
948bdcf3 7470 struct gcpro gcpro1;
24a73b0a 7471
948bdcf3 7472 GCPRO1 (arg);
24a73b0a
KH		 7473 current = XCAR (arg);
7474 workbuf = XCDR (arg);
7475 if (! NILP (workbuf))
7476 {
7477 if (EQ (workbuf, Vcode_conversion_reused_workbuf))
7478 reused_workbuf_in_use = 0;
7479 else if (! NILP (Fbuffer_live_p (workbuf)))
7480 Fkill_buffer (workbuf);
7481 }
7482 set_buffer_internal (XBUFFER (current));
948bdcf3 7483 UNGCPRO;
4776e638
KH		 7484 return Qnil;
7485}
b73bfc1c 7486
24a73b0a 7487Lisp_Object
971de7fb 7488code_conversion_save (int with_work_buf, int multibyte)
df7492f9 7489{
24a73b0a 7490 Lisp_Object workbuf = Qnil;
b73bfc1c 7491
4776e638 7492 if (with_work_buf)
24a73b0a
KH		 7493 workbuf = make_conversion_work_buffer (multibyte);
7494 record_unwind_protect (code_conversion_restore,
7495 Fcons (Fcurrent_buffer (), workbuf));
4776e638 7496 return workbuf;
df7492f9 7497}
d46c5b12 7498
df7492f9 7499int
cf84bb53 7500decode_coding_gap (struct coding_system *coding,
d311d28c 7501 ptrdiff_t chars, ptrdiff_t bytes)
df7492f9 7502{
d311d28c 7503 ptrdiff_t count = SPECPDL_INDEX ();
5e5c78be 7504 Lisp_Object attrs;
fb88bf2d 7505
24a73b0a 7506 code_conversion_save (0, 0);
ec6d2bb8 7507
24a73b0a 7508 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7509 coding->src_chars = chars;
7510 coding->src_bytes = bytes;
7511 coding->src_pos = -chars;
7512 coding->src_pos_byte = -bytes;
7513 coding->src_multibyte = chars < bytes;
24a73b0a 7514 coding->dst_object = coding->src_object;
df7492f9
KH		 7515 coding->dst_pos = PT;
7516 coding->dst_pos_byte = PT_BYTE;
4b4deea2 7517 coding->dst_multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters));
4ed46869 7518
df7492f9
KH		 7519 if (CODING_REQUIRE_DETECTION (coding))
7520 detect_coding (coding);
8f924df7 7521
9286b333 7522 coding->mode |= CODING_MODE_LAST_BLOCK;
287c57d7 7523 current_buffer->text->inhibit_shrinking = 1;
df7492f9 7524 decode_coding (coding);
287c57d7 7525 current_buffer->text->inhibit_shrinking = 0;
d46c5b12 7526
5e5c78be
KH		 7527 attrs = CODING_ID_ATTRS (coding->id);
7528 if (! NILP (CODING_ATTR_POST_READ (attrs)))
b73bfc1c 7529 {
d311d28c 7530 ptrdiff_t prev_Z = Z, prev_Z_BYTE = Z_BYTE;
5e5c78be
KH		 7531 Lisp_Object val;
7532
7533 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
5e5c78be
KH		 7534 val = call1 (CODING_ATTR_POST_READ (attrs),
7535 make_number (coding->produced_char));
5e5c78be
KH		 7536 CHECK_NATNUM (val);
7537 coding->produced_char += Z - prev_Z;
7538 coding->produced += Z_BYTE - prev_Z_BYTE;
b73bfc1c 7539 }
4ed46869 7540
df7492f9 7541 unbind_to (count, Qnil);
b73bfc1c
KH		 7542 return coding->result;
7543}
52d41803 7544
d46c5b12 7545
df7492f9
KH		 7546/* Decode the text in the range FROM/FROM_BYTE and TO/TO_BYTE in
7547 SRC_OBJECT into DST_OBJECT by coding context CODING.
b73bfc1c 7548
df7492f9 7549 SRC_OBJECT is a buffer, a string, or Qnil.
b73bfc1c 7550
df7492f9
KH		 7551 If it is a buffer, the text is at point of the buffer. FROM and TO
7552 are positions in the buffer.
b73bfc1c 7553
df7492f9
KH		 7554 If it is a string, the text is at the beginning of the string.
7555 FROM and TO are indices to the string.
4ed46869 7556
df7492f9
KH		 7557 If it is nil, the text is at coding->source. FROM and TO are
7558 indices to coding->source.
bb10be8b 7559
df7492f9 7560 DST_OBJECT is a buffer, Qt, or Qnil.
4ed46869 7561
df7492f9
KH		 7562 If it is a buffer, the decoded text is inserted at point of the
7563 buffer. If the buffer is the same as SRC_OBJECT, the source text
7564 is deleted.
4ed46869 7565
df7492f9
KH		 7566 If it is Qt, a string is made from the decoded text, and
7567 set in CODING->dst_object.
d46c5b12 7568
df7492f9 7569 If it is Qnil, the decoded text is stored at CODING->destination.
2cb26057 7570 The caller must allocate CODING->dst_bytes bytes at
df7492f9
KH		 7571 CODING->destination by xmalloc. If the decoded text is longer than
7572 CODING->dst_bytes, CODING->destination is relocated by xrealloc.
7573 */
d46c5b12 7574
df7492f9 7575void
cf84bb53
JB		 7576decode_coding_object (struct coding_system *coding,
7577 Lisp_Object src_object,
d311d28c
PE		 7578 ptrdiff_t from, ptrdiff_t from_byte,
7579 ptrdiff_t to, ptrdiff_t to_byte,
cf84bb53 7580 Lisp_Object dst_object)
d46c5b12 7581{
d311d28c 7582 ptrdiff_t count = SPECPDL_INDEX ();
c4a63b12 7583 unsigned char *destination IF_LINT (= NULL);
d311d28c
PE		 7584 ptrdiff_t dst_bytes IF_LINT (= 0);
7585 ptrdiff_t chars = to - from;
7586 ptrdiff_t bytes = to_byte - from_byte;
df7492f9 7587 Lisp_Object attrs;
c4a63b12 7588 int saved_pt = -1, saved_pt_byte IF_LINT (= 0);
64cedb0c 7589 int need_marker_adjustment = 0;
b3bfad50 7590 Lisp_Object old_deactivate_mark;
d46c5b12 7591
b3bfad50 7592 old_deactivate_mark = Vdeactivate_mark;
93dec019 7593
df7492f9 7594 if (NILP (dst_object))
d46c5b12 7595 {
df7492f9
KH		 7596 destination = coding->destination;
7597 dst_bytes = coding->dst_bytes;
d46c5b12 7598 }
93dec019 7599
df7492f9
KH		 7600 coding->src_object = src_object;
7601 coding->src_chars = chars;
7602 coding->src_bytes = bytes;
7603 coding->src_multibyte = chars < bytes;
70ad9fc4 7604
df7492f9 7605 if (STRINGP (src_object))
d46c5b12 7606 {
df7492f9
KH		 7607 coding->src_pos = from;
7608 coding->src_pos_byte = from_byte;
d46c5b12 7609 }
df7492f9 7610 else if (BUFFERP (src_object))
88993dfd 7611 {
df7492f9
KH		 7612 set_buffer_internal (XBUFFER (src_object));
7613 if (from != GPT)
7614 move_gap_both (from, from_byte);
7615 if (EQ (src_object, dst_object))
fb88bf2d 7616 {
64cedb0c
KH		 7617 struct Lisp_Marker *tail;
7618
7619 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7620 {
7621 tail->need_adjustment
7622 = tail->charpos == (tail->insertion_type ? from : to);
7623 need_marker_adjustment |= tail->need_adjustment;
7624 }
4776e638 7625 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9 7626 TEMP_SET_PT_BOTH (from, from_byte);
f4a3cc44 7627 current_buffer->text->inhibit_shrinking = 1;
df7492f9
KH		 7628 del_range_both (from, from_byte, to, to_byte, 1);
7629 coding->src_pos = -chars;
7630 coding->src_pos_byte = -bytes;
fb88bf2d 7631 }
df7492f9 7632 else
fb88bf2d 7633 {
df7492f9
KH		 7634 coding->src_pos = from;
7635 coding->src_pos_byte = from_byte;
fb88bf2d 7636 }
88993dfd
KH		 7637 }
7638
df7492f9
KH		 7639 if (CODING_REQUIRE_DETECTION (coding))
7640 detect_coding (coding);
7641 attrs = CODING_ID_ATTRS (coding->id);
d46c5b12 7642
2cb26057
KH		 7643 if (EQ (dst_object, Qt)
7644 || (! NILP (CODING_ATTR_POST_READ (attrs))
7645 && NILP (dst_object)))
b73bfc1c 7646 {
a1567c45
SM		 7647 coding->dst_multibyte = !CODING_FOR_UNIBYTE (coding);
7648 coding->dst_object = code_conversion_save (1, coding->dst_multibyte);
df7492f9
KH		 7649 coding->dst_pos = BEG;
7650 coding->dst_pos_byte = BEG_BYTE;
b73bfc1c 7651 }
df7492f9 7652 else if (BUFFERP (dst_object))
d46c5b12 7653 {
24a73b0a 7654 code_conversion_save (0, 0);
df7492f9
KH		 7655 coding->dst_object = dst_object;
7656 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
7657 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
7658 coding->dst_multibyte
4b4deea2 7659 = ! NILP (BVAR (XBUFFER (dst_object), enable_multibyte_characters));
d46c5b12
KH		 7660 }
7661 else
7662 {
24a73b0a 7663 code_conversion_save (0, 0);
df7492f9 7664 coding->dst_object = Qnil;
0154725e
SM		 7665 /* Most callers presume this will return a multibyte result, and they
7666 won't use `binary' or `raw-text' anyway, so let's not worry about
7667 CODING_FOR_UNIBYTE. */
bb555731 7668 coding->dst_multibyte = 1;
d46c5b12
KH		 7669 }
7670
df7492f9 7671 decode_coding (coding);
fa46990e 7672
df7492f9
KH		 7673 if (BUFFERP (coding->dst_object))
7674 set_buffer_internal (XBUFFER (coding->dst_object));
d46c5b12 7675
df7492f9 7676 if (! NILP (CODING_ATTR_POST_READ (attrs)))
d46c5b12 7677 {
b3bfad50 7678 struct gcpro gcpro1, gcpro2, gcpro3, gcpro4, gcpro5;
d311d28c 7679 ptrdiff_t prev_Z = Z, prev_Z_BYTE = Z_BYTE;
2b4f9037 7680 Lisp_Object val;
d46c5b12 7681
c0cc7f7f 7682 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
b3bfad50
KH		 7683 GCPRO5 (coding->src_object, coding->dst_object, src_object, dst_object,
7684 old_deactivate_mark);
d4850d67
KH		 7685 val = safe_call1 (CODING_ATTR_POST_READ (attrs),
7686 make_number (coding->produced_char));
df7492f9
KH		 7687 UNGCPRO;
7688 CHECK_NATNUM (val);
7689 coding->produced_char += Z - prev_Z;
7690 coding->produced += Z_BYTE - prev_Z_BYTE;
d46c5b12 7691 }
de79a6a5 7692
df7492f9 7693 if (EQ (dst_object, Qt))
ec6d2bb8 7694 {
df7492f9
KH		 7695 coding->dst_object = Fbuffer_string ();
7696 }
7697 else if (NILP (dst_object) && BUFFERP (coding->dst_object))
7698 {
7699 set_buffer_internal (XBUFFER (coding->dst_object));
7700 if (dst_bytes < coding->produced)
7701 {
b3bfad50 7702 destination = xrealloc (destination, coding->produced);
df7492f9
KH		 7703 if (! destination)
7704 {
065e3595 7705 record_conversion_result (coding,
ebaf11b6 7706 CODING_RESULT_INSUFFICIENT_MEM);
df7492f9
KH		 7707 unbind_to (count, Qnil);
7708 return;
7709 }
7710 if (BEGV < GPT && GPT < BEGV + coding->produced_char)
7711 move_gap_both (BEGV, BEGV_BYTE);
72af86bd 7712 memcpy (destination, BEGV_ADDR, coding->produced);
df7492f9 7713 coding->destination = destination;
d46c5b12 7714 }
ec6d2bb8 7715 }
b73bfc1c 7716
4776e638
KH		 7717 if (saved_pt >= 0)
7718 {
7719 /* This is the case of:
7720 (BUFFERP (src_object) && EQ (src_object, dst_object))
7721 As we have moved PT while replacing the original buffer
7722 contents, we must recover it now. */
7723 set_buffer_internal (XBUFFER (src_object));
f4a3cc44 7724 current_buffer->text->inhibit_shrinking = 0;
4776e638
KH		 7725 if (saved_pt < from)
7726 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7727 else if (saved_pt < from + chars)
7728 TEMP_SET_PT_BOTH (from, from_byte);
4b4deea2 7729 else if (! NILP (BVAR (current_buffer, enable_multibyte_characters)))
4776e638
KH		 7730 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7731 saved_pt_byte + (coding->produced - bytes));
7732 else
7733 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7734 saved_pt_byte + (coding->produced - bytes));
64cedb0c
KH		 7735
7736 if (need_marker_adjustment)
7737 {
7738 struct Lisp_Marker *tail;
7739
7740 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7741 if (tail->need_adjustment)
7742 {
7743 tail->need_adjustment = 0;
7744 if (tail->insertion_type)
7745 {
7746 tail->bytepos = from_byte;
7747 tail->charpos = from;
7748 }
7749 else
7750 {
7751 tail->bytepos = from_byte + coding->produced;
7752 tail->charpos
4b4deea2 7753 = (NILP (BVAR (current_buffer, enable_multibyte_characters))
64cedb0c
KH		 7754 ? tail->bytepos : from + coding->produced_char);
7755 }
7756 }
7757 }
d46c5b12 7758 }
4776e638 7759
b3bfad50 7760 Vdeactivate_mark = old_deactivate_mark;
065e3595 7761 unbind_to (count, coding->dst_object);
d46c5b12
KH		 7762}
7763
d46c5b12 7764
df7492f9 7765void
cf84bb53
JB		 7766encode_coding_object (struct coding_system *coding,
7767 Lisp_Object src_object,
d311d28c
PE		 7768 ptrdiff_t from, ptrdiff_t from_byte,
7769 ptrdiff_t to, ptrdiff_t to_byte,
cf84bb53 7770 Lisp_Object dst_object)
d46c5b12 7771{
d311d28c
PE		 7772 ptrdiff_t count = SPECPDL_INDEX ();
7773 ptrdiff_t chars = to - from;
7774 ptrdiff_t bytes = to_byte - from_byte;
df7492f9 7775 Lisp_Object attrs;
c4a63b12 7776 int saved_pt = -1, saved_pt_byte IF_LINT (= 0);
64cedb0c 7777 int need_marker_adjustment = 0;
c02d943b 7778 int kill_src_buffer = 0;
b3bfad50 7779 Lisp_Object old_deactivate_mark;
df7492f9 7780
b3bfad50 7781 old_deactivate_mark = Vdeactivate_mark;
df7492f9
KH		 7782
7783 coding->src_object = src_object;
7784 coding->src_chars = chars;
7785 coding->src_bytes = bytes;
7786 coding->src_multibyte = chars < bytes;
7787
7788 attrs = CODING_ID_ATTRS (coding->id);
7789
64cedb0c
KH		 7790 if (EQ (src_object, dst_object))
7791 {
7792 struct Lisp_Marker *tail;
7793
7794 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7795 {
7796 tail->need_adjustment
7797 = tail->charpos == (tail->insertion_type ? from : to);
7798 need_marker_adjustment |= tail->need_adjustment;
7799 }
7800 }
7801
df7492f9 7802 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
6bac5b12 7803 {
24a73b0a 7804 coding->src_object = code_conversion_save (1, coding->src_multibyte);
df7492f9
KH		 7805 set_buffer_internal (XBUFFER (coding->src_object));
7806 if (STRINGP (src_object))
7807 insert_from_string (src_object, from, from_byte, chars, bytes, 0);
7808 else if (BUFFERP (src_object))
7809 insert_from_buffer (XBUFFER (src_object), from, chars, 0);
7810 else
b68864e5 7811 insert_1_both ((char *) coding->source + from, chars, bytes, 0, 0, 0);
d46c5b12 7812
df7492f9
KH		 7813 if (EQ (src_object, dst_object))
7814 {
7815 set_buffer_internal (XBUFFER (src_object));
4776e638 7816 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9
KH		 7817 del_range_both (from, from_byte, to, to_byte, 1);
7818 set_buffer_internal (XBUFFER (coding->src_object));
7819 }
7820
d4850d67
KH		 7821 {
7822 Lisp_Object args[3];
b3bfad50 7823 struct gcpro gcpro1, gcpro2, gcpro3, gcpro4, gcpro5;
d4850d67 7824
b3bfad50
KH		 7825 GCPRO5 (coding->src_object, coding->dst_object, src_object, dst_object,
7826 old_deactivate_mark);
d4850d67
KH		 7827 args[0] = CODING_ATTR_PRE_WRITE (attrs);
7828 args[1] = make_number (BEG);
7829 args[2] = make_number (Z);
7830 safe_call (3, args);
b3bfad50 7831 UNGCPRO;
d4850d67 7832 }
c02d943b
KH		 7833 if (XBUFFER (coding->src_object) != current_buffer)
7834 kill_src_buffer = 1;
ac87bbef 7835 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 7836 if (BEG != GPT)
7837 move_gap_both (BEG, BEG_BYTE);
7838 coding->src_chars = Z - BEG;
7839 coding->src_bytes = Z_BYTE - BEG_BYTE;
7840 coding->src_pos = BEG;
7841 coding->src_pos_byte = BEG_BYTE;
7842 coding->src_multibyte = Z < Z_BYTE;
7843 }
7844 else if (STRINGP (src_object))
d46c5b12 7845 {
24a73b0a 7846 code_conversion_save (0, 0);
df7492f9
KH		 7847 coding->src_pos = from;
7848 coding->src_pos_byte = from_byte;
b73bfc1c 7849 }
df7492f9 7850 else if (BUFFERP (src_object))
b73bfc1c 7851 {
24a73b0a 7852 code_conversion_save (0, 0);
df7492f9 7853 set_buffer_internal (XBUFFER (src_object));
df7492f9 7854 if (EQ (src_object, dst_object))
d46c5b12 7855 {
4776e638 7856 saved_pt = PT, saved_pt_byte = PT_BYTE;
ff0dacd7
KH		 7857 coding->src_object = del_range_1 (from, to, 1, 1);
7858 coding->src_pos = 0;
7859 coding->src_pos_byte = 0;
d46c5b12 7860 }
df7492f9 7861 else
d46c5b12 7862 {
ff0dacd7
KH		 7863 if (from < GPT && to >= GPT)
7864 move_gap_both (from, from_byte);
df7492f9
KH		 7865 coding->src_pos = from;
7866 coding->src_pos_byte = from_byte;
d46c5b12 7867 }
d46c5b12 7868 }
4776e638 7869 else
24a73b0a 7870 code_conversion_save (0, 0);
d46c5b12 7871
df7492f9 7872 if (BUFFERP (dst_object))
88993dfd 7873 {
df7492f9 7874 coding->dst_object = dst_object;
28f67a95
KH		 7875 if (EQ (src_object, dst_object))
7876 {
7877 coding->dst_pos = from;
7878 coding->dst_pos_byte = from_byte;
7879 }
7880 else
7881 {
319a3947
KH		 7882 struct buffer *current = current_buffer;
7883
7884 set_buffer_temp (XBUFFER (dst_object));
7885 coding->dst_pos = PT;
7886 coding->dst_pos_byte = PT_BYTE;
7887 move_gap_both (coding->dst_pos, coding->dst_pos_byte);
7888 set_buffer_temp (current);
28f67a95 7889 }
df7492f9 7890 coding->dst_multibyte
4b4deea2 7891 = ! NILP (BVAR (XBUFFER (dst_object), enable_multibyte_characters));
88993dfd 7892 }
df7492f9 7893 else if (EQ (dst_object, Qt))
d46c5b12 7894 {
5d009b3a 7895 ptrdiff_t dst_bytes = max (1, coding->src_chars);
df7492f9 7896 coding->dst_object = Qnil;
5d009b3a
PE		 7897 coding->destination = (unsigned char *) xmalloc (dst_bytes);
7898 coding->dst_bytes = dst_bytes;
df7492f9 7899 coding->dst_multibyte = 0;
d46c5b12
KH		 7900 }
7901 else
7902 {
df7492f9
KH		 7903 coding->dst_object = Qnil;
7904 coding->dst_multibyte = 0;
d46c5b12
KH		 7905 }
7906
df7492f9 7907 encode_coding (coding);
d46c5b12 7908
df7492f9 7909 if (EQ (dst_object, Qt))
d46c5b12 7910 {
df7492f9
KH		 7911 if (BUFFERP (coding->dst_object))
7912 coding->dst_object = Fbuffer_string ();
7913 else
d46c5b12 7914 {
df7492f9
KH		 7915 coding->dst_object
7916 = make_unibyte_string ((char *) coding->destination,
7917 coding->produced);
7918 xfree (coding->destination);
d46c5b12 7919 }
4ed46869 7920 }
d46c5b12 7921
4776e638
KH		 7922 if (saved_pt >= 0)
7923 {
7924 /* This is the case of:
7925 (BUFFERP (src_object) && EQ (src_object, dst_object))
7926 As we have moved PT while replacing the original buffer
7927 contents, we must recover it now. */
7928 set_buffer_internal (XBUFFER (src_object));
7929 if (saved_pt < from)
7930 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7931 else if (saved_pt < from + chars)
7932 TEMP_SET_PT_BOTH (from, from_byte);
4b4deea2 7933 else if (! NILP (BVAR (current_buffer, enable_multibyte_characters)))
4776e638
KH		 7934 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7935 saved_pt_byte + (coding->produced - bytes));
d46c5b12 7936 else
4776e638
KH		 7937 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7938 saved_pt_byte + (coding->produced - bytes));
64cedb0c
KH		 7939
7940 if (need_marker_adjustment)
7941 {
7942 struct Lisp_Marker *tail;
7943
7944 for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next)
7945 if (tail->need_adjustment)
7946 {
7947 tail->need_adjustment = 0;
7948 if (tail->insertion_type)
7949 {
7950 tail->bytepos = from_byte;
7951 tail->charpos = from;
7952 }
7953 else
7954 {
7955 tail->bytepos = from_byte + coding->produced;
7956 tail->charpos
4b4deea2 7957 = (NILP (BVAR (current_buffer, enable_multibyte_characters))
64cedb0c
KH		 7958 ? tail->bytepos : from + coding->produced_char);
7959 }
7960 }
7961 }
4776e638
KH		 7962 }
7963
c02d943b
KH		 7964 if (kill_src_buffer)
7965 Fkill_buffer (coding->src_object);
b3bfad50
KH		 7966
7967 Vdeactivate_mark = old_deactivate_mark;
df7492f9 7968 unbind_to (count, Qnil);
b73bfc1c
KH		 7969}
7970
df7492f9 7971
b73bfc1c 7972Lisp_Object
971de7fb 7973preferred_coding_system (void)
b73bfc1c 7974{
df7492f9 7975 int id = coding_categories[coding_priorities[0]].id;
2391eaa4 7976
df7492f9 7977 return CODING_ID_NAME (id);
4ed46869
KH		 7978}
7979
7980\f
7981#ifdef emacs
1397dc18 7982/*** 8. Emacs Lisp library functions ***/
4ed46869 7983
a7ca3326 7984DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0,
48b0f3ae 7985 doc: /* Return t if OBJECT is nil or a coding-system.
df7492f9 7986See the documentation of `define-coding-system' for information
48b0f3ae 7987about coding-system objects. */)
5842a27b 7988 (Lisp_Object object)
4ed46869 7989{
d4a1d553
JB		 7990 if (NILP (object)
7991 || CODING_SYSTEM_ID (object) >= 0)
44e8490d 7992 return Qt;
d4a1d553
JB		 7993 if (! SYMBOLP (object)
7994 || NILP (Fget (object, Qcoding_system_define_form)))
44e8490d
KH		 7995 return Qnil;
7996 return Qt;
4ed46869
KH		 7997}
7998
a7ca3326 7999DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system,
9d991de8 8000 Sread_non_nil_coding_system, 1, 1, 0,
48b0f3ae 8001 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
5842a27b 8002 (Lisp_Object prompt)
4ed46869 8003{
e0e989f6 8004 Lisp_Object val;
9d991de8
RS		 8005 do
8006 {
4608c386
KH		 8007 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
8008 Qt, Qnil, Qcoding_system_history, Qnil, Qnil);
9d991de8 8009 }
8f924df7 8010 while (SCHARS (val) == 0);
e0e989f6 8011 return (Fintern (val, Qnil));
4ed46869
KH		 8012}
8013
a7ca3326 8014DEFUN ("read-coding-system", Fread_coding_system, Sread_coding_system, 1, 2, 0,
48b0f3ae 8015 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT.
c7183fb8
GM		 8016If the user enters null input, return second argument DEFAULT-CODING-SYSTEM.
8017Ignores case when completing coding systems (all Emacs coding systems
8018are lower-case). */)
5842a27b 8019 (Lisp_Object prompt, Lisp_Object default_coding_system)
4ed46869 8020{
f44d27ce 8021 Lisp_Object val;
d311d28c 8022 ptrdiff_t count = SPECPDL_INDEX ();
c7183fb8 8023
9b787f3e 8024 if (SYMBOLP (default_coding_system))
57d25e6f 8025 default_coding_system = SYMBOL_NAME (default_coding_system);
c7183fb8 8026 specbind (Qcompletion_ignore_case, Qt);
4608c386 8027 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
9b787f3e
RS		 8028 Qt, Qnil, Qcoding_system_history,
8029 default_coding_system, Qnil);
c7183fb8 8030 unbind_to (count, Qnil);
8f924df7 8031 return (SCHARS (val) == 0 ? Qnil : Fintern (val, Qnil));
4ed46869
KH		 8032}
8033
a7ca3326 8034DEFUN ("check-coding-system", Fcheck_coding_system, Scheck_coding_system,
4ed46869 8035 1, 1, 0,
48b0f3ae 8036 doc: /* Check validity of CODING-SYSTEM.
9ffd559c
KH		 8037If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
8038It is valid if it is nil or a symbol defined as a coding system by the
8039function `define-coding-system'. */)
5842a27b 8040 (Lisp_Object coding_system)
4ed46869 8041{
44e8490d
KH		 8042 Lisp_Object define_form;
8043
8044 define_form = Fget (coding_system, Qcoding_system_define_form);
8045 if (! NILP (define_form))
8046 {
8047 Fput (coding_system, Qcoding_system_define_form, Qnil);
8048 safe_eval (define_form);
8049 }
4ed46869
KH		 8050 if (!NILP (Fcoding_system_p (coding_system)))
8051 return coding_system;
fcad4ec4 8052 xsignal1 (Qcoding_system_error, coding_system);
4ed46869 8053}
df7492f9 8054
3a73fa5d 8055\f
89528eb3
KH		 8056/* Detect how the bytes at SRC of length SRC_BYTES are encoded. If
8057 HIGHEST is nonzero, return the coding system of the highest
ad1746f5 8058 priority among the detected coding systems. Otherwise return a
89528eb3
KH		 8059 list of detected coding systems sorted by their priorities. If
8060 MULTIBYTEP is nonzero, it is assumed that the bytes are in correct
8061 multibyte form but contains only ASCII and eight-bit chars.
8062 Otherwise, the bytes are raw bytes.
8063
8064 CODING-SYSTEM controls the detection as below:
8065
8066 If it is nil, detect both text-format and eol-format. If the
8067 text-format part of CODING-SYSTEM is already specified
8068 (e.g. `iso-latin-1'), detect only eol-format. If the eol-format
8069 part of CODING-SYSTEM is already specified (e.g. `undecided-unix'),
8070 detect only text-format. */
8071
d46c5b12 8072Lisp_Object
cf84bb53 8073detect_coding_system (const unsigned char *src,
d311d28c 8074 ptrdiff_t src_chars, ptrdiff_t src_bytes,
cf84bb53
JB		 8075 int highest, int multibytep,
8076 Lisp_Object coding_system)
4ed46869 8077{
8f924df7 8078 const unsigned char *src_end = src + src_bytes;
df7492f9 8079 Lisp_Object attrs, eol_type;
4533845d 8080 Lisp_Object val = Qnil;
df7492f9 8081 struct coding_system coding;
d3411f89 8082 ptrdiff_t id;
ff0dacd7 8083 struct coding_detection_info detect_info;
24a73b0a 8084 enum coding_category base_category;
2f3cbb32 8085 int null_byte_found = 0, eight_bit_found = 0;
b73bfc1c 8086
df7492f9
KH		 8087 if (NILP (coding_system))
8088 coding_system = Qundecided;
8089 setup_coding_system (coding_system, &coding);
8090 attrs = CODING_ID_ATTRS (coding.id);
8091 eol_type = CODING_ID_EOL_TYPE (coding.id);
89528eb3 8092 coding_system = CODING_ATTR_BASE_NAME (attrs);
4ed46869 8093
df7492f9 8094 coding.source = src;
24a73b0a 8095 coding.src_chars = src_chars;
df7492f9
KH		 8096 coding.src_bytes = src_bytes;
8097 coding.src_multibyte = multibytep;
8098 coding.consumed = 0;
89528eb3 8099 coding.mode |= CODING_MODE_LAST_BLOCK;
c0e16b14 8100 coding.head_ascii = 0;
d46c5b12 8101
ff0dacd7 8102 detect_info.checked = detect_info.found = detect_info.rejected = 0;
d46c5b12 8103
89528eb3 8104 /* At first, detect text-format if necessary. */
24a73b0a
KH		 8105 base_category = XINT (CODING_ATTR_CATEGORY (attrs));
8106 if (base_category == coding_category_undecided)
4ed46869 8107 {
c4a63b12
PE		 8108 enum coding_category category IF_LINT (= 0);
8109 struct coding_system *this IF_LINT (= NULL);
ff0dacd7 8110 int c, i;
88993dfd 8111
24a73b0a 8112 /* Skip all ASCII bytes except for a few ISO2022 controls. */
2f3cbb32 8113 for (; src < src_end; src++)
4ed46869 8114 {
df7492f9 8115 c = *src;
6cb21a4f 8116 if (c & 0x80)
6cb21a4f 8117 {
2f3cbb32 8118 eight_bit_found = 1;
2f3cbb32
KH		 8119 if (null_byte_found)
8120 break;
8121 }
c0e16b14 8122 else if (c < 0x20)
2f3cbb32
KH		 8123 {
8124 if ((c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
8125 && ! inhibit_iso_escape_detection
8126 && ! detect_info.checked)
6cb21a4f 8127 {
2f3cbb32
KH		 8128 if (detect_coding_iso_2022 (&coding, &detect_info))
8129 {
8130 /* We have scanned the whole data. */
8131 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
c0e16b14
KH		 8132 {
8133 /* We didn't find an 8-bit code. We may
8134 have found a null-byte, but it's very
8135 rare that a binary file confirm to
8136 ISO-2022. */
8137 src = src_end;
8138 coding.head_ascii = src - coding.source;
8139 }
8140 detect_info.rejected |= ~CATEGORY_MASK_ISO_ESCAPE;
2f3cbb32
KH		 8141 break;
8142 }
8143 }
97b1b294 8144 else if (! c && !inhibit_null_byte_detection)
2f3cbb32
KH		 8145 {
8146 null_byte_found = 1;
8147 if (eight_bit_found)
8148 break;
6cb21a4f 8149 }
c006c0c8
KH		 8150 if (! eight_bit_found)
8151 coding.head_ascii++;
6cb21a4f 8152 }
c006c0c8 8153 else if (! eight_bit_found)
c0e16b14 8154 coding.head_ascii++;
4ed46869 8155 }
88993dfd 8156
2f3cbb32
KH		 8157 if (null_byte_found || eight_bit_found
8158 || coding.head_ascii < coding.src_bytes
6cb21a4f
KH		 8159 || detect_info.found)
8160 {
2f3cbb32 8161 if (coding.head_ascii == coding.src_bytes)
6cb21a4f
KH		 8162 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
8163 for (i = 0; i < coding_category_raw_text; i++)
ff0dacd7 8164 {
6cb21a4f 8165 category = coding_priorities[i];
c7266f4a 8166 this = coding_categories + category;
6cb21a4f 8167 if (detect_info.found & (1 << category))
ff0dacd7
KH		 8168 break;
8169 }
6cb21a4f 8170 else
2f3cbb32
KH		 8171 {
8172 if (null_byte_found)
8173 {
8174 detect_info.checked |= ~CATEGORY_MASK_UTF_16;
8175 detect_info.rejected |= ~CATEGORY_MASK_UTF_16;
8176 }
8177 for (i = 0; i < coding_category_raw_text; i++)
8178 {
8179 category = coding_priorities[i];
8180 this = coding_categories + category;
6cb21a4f 8181
2f3cbb32
KH		 8182 if (this->id < 0)
8183 {
8184 /* No coding system of this category is defined. */
8185 detect_info.rejected |= (1 << category);
8186 }
8187 else if (category >= coding_category_raw_text)
8188 continue;
8189 else if (detect_info.checked & (1 << category))
8190 {
8191 if (highest
8192 && (detect_info.found & (1 << category)))
6cb21a4f 8193 break;
2f3cbb32
KH		 8194 }
8195 else if ((*(this->detector)) (&coding, &detect_info)
8196 && highest
8197 && (detect_info.found & (1 << category)))
8198 {
8199 if (category == coding_category_utf_16_auto)
8200 {
8201 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
8202 category = coding_category_utf_16_le;
8203 else
8204 category = coding_category_utf_16_be;
8205 }
8206 break;
8207 }
8208 }
8209 }
6cb21a4f 8210 }
ec6d2bb8 8211
4cddb209
KH		 8212 if ((detect_info.rejected & CATEGORY_MASK_ANY) == CATEGORY_MASK_ANY
8213 || null_byte_found)
ec6d2bb8 8214 {
ff0dacd7 8215 detect_info.found = CATEGORY_MASK_RAW_TEXT;
4cddb209 8216 id = CODING_SYSTEM_ID (Qno_conversion);
89528eb3
KH		 8217 val = Fcons (make_number (id), Qnil);
8218 }
ff0dacd7 8219 else if (! detect_info.rejected && ! detect_info.found)
89528eb3 8220 {
ff0dacd7 8221 detect_info.found = CATEGORY_MASK_ANY;
89528eb3
KH		 8222 id = coding_categories[coding_category_undecided].id;
8223 val = Fcons (make_number (id), Qnil);
8224 }
8225 else if (highest)
8226 {
ff0dacd7 8227 if (detect_info.found)
ec6d2bb8 8228 {
ff0dacd7
KH		 8229 detect_info.found = 1 << category;
8230 val = Fcons (make_number (this->id), Qnil);
8231 }
8232 else
8233 for (i = 0; i < coding_category_raw_text; i++)
8234 if (! (detect_info.rejected & (1 << coding_priorities[i])))
8235 {
8236 detect_info.found = 1 << coding_priorities[i];
8237 id = coding_categories[coding_priorities[i]].id;
8238 val = Fcons (make_number (id), Qnil);
8239 break;
8240 }
8241 }
89528eb3
KH		 8242 else
8243 {
ff0dacd7
KH		 8244 int mask = detect_info.rejected | detect_info.found;
8245 int found = 0;
ec6d2bb8 8246
89528eb3 8247 for (i = coding_category_raw_text - 1; i >= 0; i--)
ff0dacd7
KH		 8248 {
8249 category = coding_priorities[i];
8250 if (! (mask & (1 << category)))
ec6d2bb8 8251 {
ff0dacd7
KH		 8252 found |= 1 << category;
8253 id = coding_categories[category].id;
c7266f4a
KH		 8254 if (id >= 0)
8255 val = Fcons (make_number (id), val);
ff0dacd7
KH		 8256 }
8257 }
8258 for (i = coding_category_raw_text - 1; i >= 0; i--)
8259 {
8260 category = coding_priorities[i];
8261 if (detect_info.found & (1 << category))
8262 {
8263 id = coding_categories[category].id;
8264 val = Fcons (make_number (id), val);
ec6d2bb8 8265 }
ec6d2bb8 8266 }
ff0dacd7 8267 detect_info.found |= found;
ec6d2bb8 8268 }
ec6d2bb8 8269 }
a470d443
KH		 8270 else if (base_category == coding_category_utf_8_auto)
8271 {
8272 if (detect_coding_utf_8 (&coding, &detect_info))
8273 {
8274 struct coding_system *this;
8275
8276 if (detect_info.found & CATEGORY_MASK_UTF_8_SIG)
8277 this = coding_categories + coding_category_utf_8_sig;
8278 else
8279 this = coding_categories + coding_category_utf_8_nosig;
8280 val = Fcons (make_number (this->id), Qnil);
8281 }
8282 }
24a73b0a
KH		 8283 else if (base_category == coding_category_utf_16_auto)
8284 {
8285 if (detect_coding_utf_16 (&coding, &detect_info))
8286 {
24a73b0a
KH		 8287 struct coding_system *this;
8288
8289 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
8290 this = coding_categories + coding_category_utf_16_le;
8291 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
8292 this = coding_categories + coding_category_utf_16_be;
8293 else if (detect_info.rejected & CATEGORY_MASK_UTF_16_LE_NOSIG)
8294 this = coding_categories + coding_category_utf_16_be_nosig;
8295 else
8296 this = coding_categories + coding_category_utf_16_le_nosig;
8297 val = Fcons (make_number (this->id), Qnil);
8298 }
8299 }
df7492f9
KH		 8300 else
8301 {
ff0dacd7 8302 detect_info.found = 1 << XINT (CODING_ATTR_CATEGORY (attrs));
89528eb3 8303 val = Fcons (make_number (coding.id), Qnil);
4ed46869 8304 }
df7492f9 8305
89528eb3 8306 /* Then, detect eol-format if necessary. */
df7492f9 8307 {
4533845d 8308 int normal_eol = -1, utf_16_be_eol = -1, utf_16_le_eol = -1;
df7492f9
KH		 8309 Lisp_Object tail;
8310
89528eb3
KH		 8311 if (VECTORP (eol_type))
8312 {
ff0dacd7 8313 if (detect_info.found & ~CATEGORY_MASK_UTF_16)
2f3cbb32
KH		 8314 {
8315 if (null_byte_found)
8316 normal_eol = EOL_SEEN_LF;
8317 else
8318 normal_eol = detect_eol (coding.source, src_bytes,
8319 coding_category_raw_text);
8320 }
ff0dacd7
KH		 8321 if (detect_info.found & (CATEGORY_MASK_UTF_16_BE
8322 | CATEGORY_MASK_UTF_16_BE_NOSIG))
89528eb3
KH		 8323 utf_16_be_eol = detect_eol (coding.source, src_bytes,
8324 coding_category_utf_16_be);
ff0dacd7
KH		 8325 if (detect_info.found & (CATEGORY_MASK_UTF_16_LE
8326 | CATEGORY_MASK_UTF_16_LE_NOSIG))
89528eb3
KH		 8327 utf_16_le_eol = detect_eol (coding.source, src_bytes,
8328 coding_category_utf_16_le);
8329 }
8330 else
8331 {
8332 if (EQ (eol_type, Qunix))
8333 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_LF;
8334 else if (EQ (eol_type, Qdos))
8335 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CRLF;
8336 else
8337 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CR;
8338 }
8339
df7492f9
KH		 8340 for (tail = val; CONSP (tail); tail = XCDR (tail))
8341 {
89528eb3 8342 enum coding_category category;
df7492f9 8343 int this_eol;
89528eb3
KH		 8344
8345 id = XINT (XCAR (tail));
8346 attrs = CODING_ID_ATTRS (id);
8347 category = XINT (CODING_ATTR_CATEGORY (attrs));
8348 eol_type = CODING_ID_EOL_TYPE (id);
df7492f9
KH		 8349 if (VECTORP (eol_type))
8350 {
89528eb3
KH		 8351 if (category == coding_category_utf_16_be
8352 || category == coding_category_utf_16_be_nosig)
8353 this_eol = utf_16_be_eol;
8354 else if (category == coding_category_utf_16_le
8355 || category == coding_category_utf_16_le_nosig)
8356 this_eol = utf_16_le_eol;
df7492f9 8357 else
89528eb3
KH		 8358 this_eol = normal_eol;
8359
df7492f9
KH		 8360 if (this_eol == EOL_SEEN_LF)
8361 XSETCAR (tail, AREF (eol_type, 0));
8362 else if (this_eol == EOL_SEEN_CRLF)
8363 XSETCAR (tail, AREF (eol_type, 1));
8364 else if (this_eol == EOL_SEEN_CR)
8365 XSETCAR (tail, AREF (eol_type, 2));
89528eb3
KH		 8366 else
8367 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9 8368 }
89528eb3
KH		 8369 else
8370 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9
KH		 8371 }
8372 }
ec6d2bb8 8373
4533845d 8374 return (highest ? (CONSP (val) ? XCAR (val) : Qnil) : val);
ec6d2bb8
KH		 8375}
8376
ec6d2bb8 8377
d46c5b12
KH		 8378DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region,
8379 2, 3, 0,
48b0f3ae
PJ		 8380 doc: /* Detect coding system of the text in the region between START and END.
8381Return a list of possible coding systems ordered by priority.
b811c52b
KH		 8382The coding systems to try and their priorities follows what
8383the function `coding-system-priority-list' (which see) returns.
ec6d2bb8 8384
12e0131a 8385If only ASCII characters are found (except for such ISO-2022 control
d4a1d553
JB		 8386characters as ESC), it returns a list of single element `undecided'
8387or its subsidiary coding system according to a detected end-of-line
8388format.
ec6d2bb8 8389
48b0f3ae
PJ		 8390If optional argument HIGHEST is non-nil, return the coding system of
8391highest priority. */)
5842a27b 8392 (Lisp_Object start, Lisp_Object end, Lisp_Object highest)
d46c5b12 8393{
d311d28c
PE		 8394 ptrdiff_t from, to;
8395 ptrdiff_t from_byte, to_byte;
ec6d2bb8 8396
b7826503
PJ		 8397 CHECK_NUMBER_COERCE_MARKER (start);
8398 CHECK_NUMBER_COERCE_MARKER (end);
ec6d2bb8 8399
d46c5b12
KH		 8400 validate_region (&start, &end);
8401 from = XINT (start), to = XINT (end);
8402 from_byte = CHAR_TO_BYTE (from);
8403 to_byte = CHAR_TO_BYTE (to);
ec6d2bb8 8404
d46c5b12
KH		 8405 if (from < GPT && to >= GPT)
8406 move_gap_both (to, to_byte);
c210f766 8407
d46c5b12 8408 return detect_coding_system (BYTE_POS_ADDR (from_byte),
24a73b0a 8409 to - from, to_byte - from_byte,
0a28aafb 8410 !NILP (highest),
4b4deea2 8411 !NILP (BVAR (current_buffer
5d8ea120 8412 , enable_multibyte_characters)),
df7492f9 8413 Qnil);
ec6d2bb8
KH		 8414}
8415
d46c5b12
KH		 8416DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string,
8417 1, 2, 0,
48b0f3ae
PJ		 8418 doc: /* Detect coding system of the text in STRING.
8419Return a list of possible coding systems ordered by priority.
67ceab9d
KH		 8420The coding systems to try and their priorities follows what
8421the function `coding-system-priority-list' (which see) returns.
fb88bf2d 8422
12e0131a 8423If only ASCII characters are found (except for such ISO-2022 control
d4a1d553
JB		 8424characters as ESC), it returns a list of single element `undecided'
8425or its subsidiary coding system according to a detected end-of-line
8426format.
d46c5b12 8427
48b0f3ae
PJ		 8428If optional argument HIGHEST is non-nil, return the coding system of
8429highest priority. */)
5842a27b 8430 (Lisp_Object string, Lisp_Object highest)
d46c5b12 8431{
b7826503 8432 CHECK_STRING (string);
b73bfc1c 8433
24a73b0a
KH		 8434 return detect_coding_system (SDATA (string),
8435 SCHARS (string), SBYTES (string),
8f924df7 8436 !NILP (highest), STRING_MULTIBYTE (string),
df7492f9 8437 Qnil);
4ed46869 8438}
4ed46869 8439
b73bfc1c 8440
55d4c1b2 8441static inline int
971de7fb 8442char_encodable_p (int c, Lisp_Object attrs)
05e6f5dc 8443{
df7492f9 8444 Lisp_Object tail;
df7492f9 8445 struct charset *charset;
7d64c6ad 8446 Lisp_Object translation_table;
d46c5b12 8447
7d64c6ad 8448 translation_table = CODING_ATTR_TRANS_TBL (attrs);
a6f87d34 8449 if (! NILP (translation_table))
7d64c6ad 8450 c = translate_char (translation_table, c);
df7492f9
KH		 8451 for (tail = CODING_ATTR_CHARSET_LIST (attrs);
8452 CONSP (tail); tail = XCDR (tail))
e133c8fa 8453 {
df7492f9
KH		 8454 charset = CHARSET_FROM_ID (XINT (XCAR (tail)));
8455 if (CHAR_CHARSET_P (c, charset))
8456 break;
e133c8fa 8457 }
df7492f9 8458 return (! NILP (tail));
05e6f5dc 8459}
83fa074f 8460
fb88bf2d 8461
df7492f9
KH		 8462/* Return a list of coding systems that safely encode the text between
8463 START and END. If EXCLUDE is non-nil, it is a list of coding
8464 systems not to check. The returned list doesn't contain any such
48468dac 8465 coding systems. In any case, if the text contains only ASCII or is
df7492f9 8466 unibyte, return t. */
e077cc80 8467
df7492f9
KH		 8468DEFUN ("find-coding-systems-region-internal",
8469 Ffind_coding_systems_region_internal,
8470 Sfind_coding_systems_region_internal, 2, 3, 0,
8471 doc: /* Internal use only. */)
5842a27b 8472 (Lisp_Object start, Lisp_Object end, Lisp_Object exclude)
df7492f9
KH		 8473{
8474 Lisp_Object coding_attrs_list, safe_codings;
d311d28c 8475 ptrdiff_t start_byte, end_byte;
7c78e542 8476 const unsigned char *p, *pbeg, *pend;
df7492f9 8477 int c;
0e727afa 8478 Lisp_Object tail, elt, work_table;
d46c5b12 8479
df7492f9
KH		 8480 if (STRINGP (start))
8481 {
8482 if (!STRING_MULTIBYTE (start)
8f924df7 8483 || SCHARS (start) == SBYTES (start))
df7492f9
KH		 8484 return Qt;
8485 start_byte = 0;
8f924df7 8486 end_byte = SBYTES (start);
df7492f9
KH		 8487 }
8488 else
d46c5b12 8489 {
df7492f9
KH		 8490 CHECK_NUMBER_COERCE_MARKER (start);
8491 CHECK_NUMBER_COERCE_MARKER (end);
8492 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
8493 args_out_of_range (start, end);
4b4deea2 8494 if (NILP (BVAR (current_buffer, enable_multibyte_characters)))
df7492f9
KH		 8495 return Qt;
8496 start_byte = CHAR_TO_BYTE (XINT (start));
8497 end_byte = CHAR_TO_BYTE (XINT (end));
8498 if (XINT (end) - XINT (start) == end_byte - start_byte)
8499 return Qt;
d46c5b12 8500
e1c23804 8501 if (XINT (start) < GPT && XINT (end) > GPT)
d46c5b12 8502 {
e1c23804
DL		 8503 if ((GPT - XINT (start)) < (XINT (end) - GPT))
8504 move_gap_both (XINT (start), start_byte);
df7492f9 8505 else
e1c23804 8506 move_gap_both (XINT (end), end_byte);
d46c5b12
KH		 8507 }
8508 }
8509
df7492f9
KH		 8510 coding_attrs_list = Qnil;
8511 for (tail = Vcoding_system_list; CONSP (tail); tail = XCDR (tail))
8512 if (NILP (exclude)
8513 || NILP (Fmemq (XCAR (tail), exclude)))
8514 {
8515 Lisp_Object attrs;
d46c5b12 8516
df7492f9
KH		 8517 attrs = AREF (CODING_SYSTEM_SPEC (XCAR (tail)), 0);
8518 if (EQ (XCAR (tail), CODING_ATTR_BASE_NAME (attrs))
8519 && ! EQ (CODING_ATTR_TYPE (attrs), Qundecided))
7d64c6ad
KH		 8520 {
8521 ASET (attrs, coding_attr_trans_tbl,
2170c8f0 8522 get_translation_table (attrs, 1, NULL));
7d64c6ad
KH		 8523 coding_attrs_list = Fcons (attrs, coding_attrs_list);
8524 }
df7492f9 8525 }
d46c5b12 8526
df7492f9 8527 if (STRINGP (start))
8f924df7 8528 p = pbeg = SDATA (start);
df7492f9
KH		 8529 else
8530 p = pbeg = BYTE_POS_ADDR (start_byte);
8531 pend = p + (end_byte - start_byte);
b843d1ae 8532
df7492f9
KH		 8533 while (p < pend && ASCII_BYTE_P (*p)) p++;
8534 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
d46c5b12 8535
0e727afa 8536 work_table = Fmake_char_table (Qnil, Qnil);
05e6f5dc 8537 while (p < pend)
72d1a715 8538 {
df7492f9
KH		 8539 if (ASCII_BYTE_P (*p))
8540 p++;
72d1a715
RS		 8541 else
8542 {
df7492f9 8543 c = STRING_CHAR_ADVANCE (p);
0e727afa
YM		 8544 if (!NILP (char_table_ref (work_table, c)))
8545 /* This character was already checked. Ignore it. */
8546 continue;
12410ef1 8547
df7492f9
KH		 8548 charset_map_loaded = 0;
8549 for (tail = coding_attrs_list; CONSP (tail);)
8550 {
8551 elt = XCAR (tail);
8552 if (NILP (elt))
8553 tail = XCDR (tail);
8554 else if (char_encodable_p (c, elt))
8555 tail = XCDR (tail);
8556 else if (CONSP (XCDR (tail)))
8557 {
8558 XSETCAR (tail, XCAR (XCDR (tail)));
8559 XSETCDR (tail, XCDR (XCDR (tail)));
8560 }
8561 else
8562 {
8563 XSETCAR (tail, Qnil);
8564 tail = XCDR (tail);
8565 }
8566 }
8567 if (charset_map_loaded)
8568 {
d311d28c 8569 ptrdiff_t p_offset = p - pbeg, pend_offset = pend - pbeg;
05e6f5dc 8570
df7492f9 8571 if (STRINGP (start))
8f924df7 8572 pbeg = SDATA (start);
df7492f9
KH		 8573 else
8574 pbeg = BYTE_POS_ADDR (start_byte);
8575 p = pbeg + p_offset;
8576 pend = pbeg + pend_offset;
8577 }
0e727afa 8578 char_table_set (work_table, c, Qt);
df7492f9 8579 }
ec6d2bb8 8580 }
fb88bf2d 8581
988b3759 8582 safe_codings = list2 (Qraw_text, Qno_conversion);
df7492f9
KH		 8583 for (tail = coding_attrs_list; CONSP (tail); tail = XCDR (tail))
8584 if (! NILP (XCAR (tail)))
8585 safe_codings = Fcons (CODING_ATTR_BASE_NAME (XCAR (tail)), safe_codings);
ec6d2bb8 8586
05e6f5dc
KH		 8587 return safe_codings;
8588}
4956c225 8589
d46c5b12 8590
8f924df7
KH		 8591DEFUN ("unencodable-char-position", Funencodable_char_position,
8592 Sunencodable_char_position, 3, 5, 0,
8593 doc: /*
8594Return position of first un-encodable character in a region.
d4a1d553 8595START and END specify the region and CODING-SYSTEM specifies the
8f924df7 8596encoding to check. Return nil if CODING-SYSTEM does encode the region.
d46c5b12 8597
8f924df7
KH		 8598If optional 4th argument COUNT is non-nil, it specifies at most how
8599many un-encodable characters to search. In this case, the value is a
8600list of positions.
d46c5b12 8601
8f924df7
KH		 8602If optional 5th argument STRING is non-nil, it is a string to search
8603for un-encodable characters. In that case, START and END are indexes
8604to the string. */)
5842a27b 8605 (Lisp_Object start, Lisp_Object end, Lisp_Object coding_system, Lisp_Object count, Lisp_Object string)
8f924df7 8606{
d311d28c 8607 EMACS_INT n;
8f924df7 8608 struct coding_system coding;
7d64c6ad 8609 Lisp_Object attrs, charset_list, translation_table;
8f924df7 8610 Lisp_Object positions;
d311d28c 8611 ptrdiff_t from, to;
8f924df7
KH		 8612 const unsigned char *p, *stop, *pend;
8613 int ascii_compatible;
fb88bf2d 8614
8f924df7
KH		 8615 setup_coding_system (Fcheck_coding_system (coding_system), &coding);
8616 attrs = CODING_ID_ATTRS (coding.id);
8617 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
8618 return Qnil;
8619 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
8620 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2170c8f0 8621 translation_table = get_translation_table (attrs, 1, NULL);
fb88bf2d 8622
8f924df7
KH		 8623 if (NILP (string))
8624 {
8625 validate_region (&start, &end);
8626 from = XINT (start);
8627 to = XINT (end);
4b4deea2 8628 if (NILP (BVAR (current_buffer, enable_multibyte_characters))
8f924df7
KH		 8629 || (ascii_compatible
8630 && (to - from) == (CHAR_TO_BYTE (to) - (CHAR_TO_BYTE (from)))))
8631 return Qnil;
8632 p = CHAR_POS_ADDR (from);
8633 pend = CHAR_POS_ADDR (to);
8634 if (from < GPT && to >= GPT)
8635 stop = GPT_ADDR;
8636 else
8637 stop = pend;
8638 }
8639 else
8640 {
8641 CHECK_STRING (string);
8642 CHECK_NATNUM (start);
8643 CHECK_NATNUM (end);
d311d28c
PE		 8644 if (! (XINT (start) <= XINT (end) && XINT (end) <= SCHARS (string)))
8645 args_out_of_range_3 (string, start, end);
8f924df7
KH		 8646 from = XINT (start);
8647 to = XINT (end);
8f924df7
KH		 8648 if (! STRING_MULTIBYTE (string))
8649 return Qnil;
8650 p = SDATA (string) + string_char_to_byte (string, from);
8651 stop = pend = SDATA (string) + string_char_to_byte (string, to);
8652 if (ascii_compatible && (to - from) == (pend - p))
8653 return Qnil;
8654 }
f2558efd 8655
8f924df7
KH		 8656 if (NILP (count))
8657 n = 1;
8658 else
b73bfc1c 8659 {
8f924df7
KH		 8660 CHECK_NATNUM (count);
8661 n = XINT (count);
b73bfc1c
KH		 8662 }
8663
8f924df7
KH		 8664 positions = Qnil;
8665 while (1)
d46c5b12 8666 {
8f924df7 8667 int c;
ec6d2bb8 8668
8f924df7
KH		 8669 if (ascii_compatible)
8670 while (p < stop && ASCII_BYTE_P (*p))
8671 p++, from++;
8672 if (p >= stop)
0e79d667 8673 {
8f924df7
KH		 8674 if (p >= pend)
8675 break;
8676 stop = pend;
8677 p = GAP_END_ADDR;
0e79d667 8678 }
ec6d2bb8 8679
8f924df7
KH		 8680 c = STRING_CHAR_ADVANCE (p);
8681 if (! (ASCII_CHAR_P (c) && ascii_compatible)
7d64c6ad
KH		 8682 && ! char_charset (translate_char (translation_table, c),
8683 charset_list, NULL))
ec6d2bb8 8684 {
8f924df7
KH		 8685 positions = Fcons (make_number (from), positions);
8686 n--;
8687 if (n == 0)
8688 break;
ec6d2bb8
KH		 8689 }
8690
8f924df7
KH		 8691 from++;
8692 }
d46c5b12 8693
8f924df7
KH		 8694 return (NILP (count) ? Fcar (positions) : Fnreverse (positions));
8695}
d46c5b12 8696
d46c5b12 8697
df7492f9
KH		 8698DEFUN ("check-coding-systems-region", Fcheck_coding_systems_region,
8699 Scheck_coding_systems_region, 3, 3, 0,
8700 doc: /* Check if the region is encodable by coding systems.
d46c5b12 8701
df7492f9
KH		 8702START and END are buffer positions specifying the region.
8703CODING-SYSTEM-LIST is a list of coding systems to check.
d46c5b12 8704
df7492f9 8705The value is an alist ((CODING-SYSTEM POS0 POS1 ...) ...), where
d4a1d553 8706CODING-SYSTEM is a member of CODING-SYSTEM-LIST and can't encode the
df7492f9
KH		 8707whole region, POS0, POS1, ... are buffer positions where non-encodable
8708characters are found.
93dec019 8709
df7492f9
KH		 8710If all coding systems in CODING-SYSTEM-LIST can encode the region, the
8711value is nil.
93dec019 8712
df7492f9
KH		 8713START may be a string. In that case, check if the string is
8714encodable, and the value contains indices to the string instead of
5704f39a
KH		 8715buffer positions. END is ignored.
8716
4c1958f4 8717If the current buffer (or START if it is a string) is unibyte, the value
5704f39a 8718is nil. */)
5842a27b 8719 (Lisp_Object start, Lisp_Object end, Lisp_Object coding_system_list)
05e6f5dc 8720{
df7492f9 8721 Lisp_Object list;
d311d28c
PE		 8722 ptrdiff_t start_byte, end_byte;
8723 ptrdiff_t pos;
7c78e542 8724 const unsigned char *p, *pbeg, *pend;
df7492f9 8725 int c;
7d64c6ad 8726 Lisp_Object tail, elt, attrs;
70ad9fc4 8727
05e6f5dc
KH		 8728 if (STRINGP (start))
8729 {
df7492f9 8730 if (!STRING_MULTIBYTE (start)
4c1958f4 8731 || SCHARS (start) == SBYTES (start))
df7492f9
KH		 8732 return Qnil;
8733 start_byte = 0;
8f924df7 8734 end_byte = SBYTES (start);
df7492f9 8735 pos = 0;
d46c5b12 8736 }
05e6f5dc 8737 else
b73bfc1c 8738 {
b7826503
PJ		 8739 CHECK_NUMBER_COERCE_MARKER (start);
8740 CHECK_NUMBER_COERCE_MARKER (end);
05e6f5dc
KH		 8741 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
8742 args_out_of_range (start, end);
4b4deea2 8743 if (NILP (BVAR (current_buffer, enable_multibyte_characters)))
df7492f9
KH		 8744 return Qnil;
8745 start_byte = CHAR_TO_BYTE (XINT (start));
8746 end_byte = CHAR_TO_BYTE (XINT (end));
8747 if (XINT (end) - XINT (start) == end_byte - start_byte)
5704f39a 8748 return Qnil;
df7492f9 8749
e1c23804 8750 if (XINT (start) < GPT && XINT (end) > GPT)
b73bfc1c 8751 {
e1c23804
DL		 8752 if ((GPT - XINT (start)) < (XINT (end) - GPT))
8753 move_gap_both (XINT (start), start_byte);
df7492f9 8754 else
e1c23804 8755 move_gap_both (XINT (end), end_byte);
b73bfc1c 8756 }
e1c23804 8757 pos = XINT (start);
b73bfc1c 8758 }
7553d0e1 8759
df7492f9
KH		 8760 list = Qnil;
8761 for (tail = coding_system_list; CONSP (tail); tail = XCDR (tail))
12410ef1 8762 {
df7492f9 8763 elt = XCAR (tail);
7d64c6ad 8764 attrs = AREF (CODING_SYSTEM_SPEC (elt), 0);
2170c8f0
KH		 8765 ASET (attrs, coding_attr_trans_tbl,
8766 get_translation_table (attrs, 1, NULL));
7d64c6ad 8767 list = Fcons (Fcons (elt, Fcons (attrs, Qnil)), list);
12410ef1
KH		 8768 }
8769
df7492f9 8770 if (STRINGP (start))
8f924df7 8771 p = pbeg = SDATA (start);
72d1a715 8772 else
df7492f9
KH		 8773 p = pbeg = BYTE_POS_ADDR (start_byte);
8774 pend = p + (end_byte - start_byte);
4ed46869 8775
df7492f9
KH		 8776 while (p < pend && ASCII_BYTE_P (*p)) p++, pos++;
8777 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
ec6d2bb8 8778
df7492f9 8779 while (p < pend)
d46c5b12 8780 {
df7492f9
KH		 8781 if (ASCII_BYTE_P (*p))
8782 p++;
e133c8fa 8783 else
05e6f5dc 8784 {
df7492f9
KH		 8785 c = STRING_CHAR_ADVANCE (p);
8786
8787 charset_map_loaded = 0;
8788 for (tail = list; CONSP (tail); tail = XCDR (tail))
8789 {
8790 elt = XCDR (XCAR (tail));
8791 if (! char_encodable_p (c, XCAR (elt)))
8792 XSETCDR (elt, Fcons (make_number (pos), XCDR (elt)));
8793 }
8794 if (charset_map_loaded)
8795 {
d311d28c 8796 ptrdiff_t p_offset = p - pbeg, pend_offset = pend - pbeg;
df7492f9
KH		 8797
8798 if (STRINGP (start))
8f924df7 8799 pbeg = SDATA (start);
df7492f9
KH		 8800 else
8801 pbeg = BYTE_POS_ADDR (start_byte);
8802 p = pbeg + p_offset;
8803 pend = pbeg + pend_offset;
8804 }
05e6f5dc 8805 }
df7492f9 8806 pos++;
d46c5b12 8807 }
4ed46869 8808
df7492f9
KH		 8809 tail = list;
8810 list = Qnil;
8811 for (; CONSP (tail); tail = XCDR (tail))
ec6d2bb8 8812 {
df7492f9
KH		 8813 elt = XCAR (tail);
8814 if (CONSP (XCDR (XCDR (elt))))
8815 list = Fcons (Fcons (XCAR (elt), Fnreverse (XCDR (XCDR (elt)))),
8816 list);
ec6d2bb8 8817 }
2b4f9037 8818
df7492f9 8819 return list;
d46c5b12
KH		 8820}
8821
3fd9494b 8822
74ab6df5 8823static Lisp_Object
cf84bb53
JB		 8824code_convert_region (Lisp_Object start, Lisp_Object end,
8825 Lisp_Object coding_system, Lisp_Object dst_object,
8826 int encodep, int norecord)
4ed46869 8827{
3a73fa5d 8828 struct coding_system coding;
d311d28c 8829 ptrdiff_t from, from_byte, to, to_byte;
df7492f9 8830 Lisp_Object src_object;
4ed46869 8831
b7826503
PJ		 8832 CHECK_NUMBER_COERCE_MARKER (start);
8833 CHECK_NUMBER_COERCE_MARKER (end);
df7492f9
KH		 8834 if (NILP (coding_system))
8835 coding_system = Qno_conversion;
8836 else
8837 CHECK_CODING_SYSTEM (coding_system);
8838 src_object = Fcurrent_buffer ();
8839 if (NILP (dst_object))
8840 dst_object = src_object;
8841 else if (! EQ (dst_object, Qt))
8842 CHECK_BUFFER (dst_object);
3a73fa5d 8843
d46c5b12
KH		 8844 validate_region (&start, &end);
8845 from = XFASTINT (start);
df7492f9 8846 from_byte = CHAR_TO_BYTE (from);
d46c5b12 8847 to = XFASTINT (end);
df7492f9 8848 to_byte = CHAR_TO_BYTE (to);
764ca8da 8849
df7492f9
KH		 8850 setup_coding_system (coding_system, &coding);
8851 coding.mode |= CODING_MODE_LAST_BLOCK;
ec6d2bb8 8852
df7492f9
KH		 8853 if (encodep)
8854 encode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
8855 dst_object);
8856 else
8857 decode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
8858 dst_object);
8859 if (! norecord)
8860 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
ec6d2bb8 8861
df7492f9
KH		 8862 return (BUFFERP (dst_object)
8863 ? make_number (coding.produced_char)
8864 : coding.dst_object);
4031e2bf 8865}
78108bcd 8866
4ed46869 8867
4031e2bf 8868DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region,
df7492f9 8869 3, 4, "r\nzCoding system: ",
48b0f3ae 8870 doc: /* Decode the current region from the specified coding system.
df7492f9
KH		 8871When called from a program, takes four arguments:
8872 START, END, CODING-SYSTEM, and DESTINATION.
8873START and END are buffer positions.
8844fa83 8874
df7492f9 8875Optional 4th arguments DESTINATION specifies where the decoded text goes.
2354b80b 8876If nil, the region between START and END is replaced by the decoded text.
1560f91a
EZ		 8877If buffer, the decoded text is inserted in that buffer after point (point
8878does not move).
446dcd75 8879In those cases, the length of the decoded text is returned.
319a3947 8880If DESTINATION is t, the decoded text is returned.
8844fa83 8881
48b0f3ae
PJ		 8882This function sets `last-coding-system-used' to the precise coding system
8883used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
319a3947 8884not fully specified.) */)
5842a27b 8885 (Lisp_Object start, Lisp_Object end, Lisp_Object coding_system, Lisp_Object destination)
4031e2bf 8886{
df7492f9 8887 return code_convert_region (start, end, coding_system, destination, 0, 0);
3a73fa5d 8888}
8844fa83 8889
3a73fa5d 8890DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region,
df7492f9
KH		 8891 3, 4, "r\nzCoding system: ",
8892 doc: /* Encode the current region by specified coding system.
d4a1d553
JB		 8893When called from a program, takes four arguments:
8894 START, END, CODING-SYSTEM and DESTINATION.
8895START and END are buffer positions.
d46c5b12 8896
df7492f9
KH		 8897Optional 4th arguments DESTINATION specifies where the encoded text goes.
8898If nil, the region between START and END is replace by the encoded text.
1560f91a
EZ		 8899If buffer, the encoded text is inserted in that buffer after point (point
8900does not move).
446dcd75 8901In those cases, the length of the encoded text is returned.
319a3947 8902If DESTINATION is t, the encoded text is returned.
2391eaa4 8903
48b0f3ae
PJ		 8904This function sets `last-coding-system-used' to the precise coding system
8905used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
319a3947 8906not fully specified.) */)
5842a27b 8907 (Lisp_Object start, Lisp_Object end, Lisp_Object coding_system, Lisp_Object destination)
3a73fa5d 8908{
df7492f9 8909 return code_convert_region (start, end, coding_system, destination, 1, 0);
b73bfc1c
KH		 8910}
8911
8912Lisp_Object
6f704c76
DN		 8913code_convert_string (Lisp_Object string, Lisp_Object coding_system,
8914 Lisp_Object dst_object, int encodep, int nocopy, int norecord)
b73bfc1c 8915{
4031e2bf 8916 struct coding_system coding;
d311d28c 8917 ptrdiff_t chars, bytes;
ec6d2bb8 8918
b7826503 8919 CHECK_STRING (string);
d46c5b12 8920 if (NILP (coding_system))
4956c225 8921 {
df7492f9
KH		 8922 if (! norecord)
8923 Vlast_coding_system_used = Qno_conversion;
8924 if (NILP (dst_object))
8925 return (nocopy ? Fcopy_sequence (string) : string);
4956c225 8926 }
b73bfc1c 8927
df7492f9
KH		 8928 if (NILP (coding_system))
8929 coding_system = Qno_conversion;
8930 else
8931 CHECK_CODING_SYSTEM (coding_system);
8932 if (NILP (dst_object))
8933 dst_object = Qt;
8934 else if (! EQ (dst_object, Qt))
8935 CHECK_BUFFER (dst_object);
73be902c 8936
df7492f9 8937 setup_coding_system (coding_system, &coding);
d46c5b12 8938 coding.mode |= CODING_MODE_LAST_BLOCK;
8f924df7
KH		 8939 chars = SCHARS (string);
8940 bytes = SBYTES (string);
df7492f9
KH		 8941 if (encodep)
8942 encode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8943 else
8944 decode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8945 if (! norecord)
8946 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
73be902c 8947
df7492f9
KH		 8948 return (BUFFERP (dst_object)
8949 ? make_number (coding.produced_char)
8950 : coding.dst_object);
4ed46869 8951}
73be902c 8952
b73bfc1c 8953
ecec61c1 8954/* Encode or decode STRING according to CODING_SYSTEM.
ec6d2bb8 8955 Do not set Vlast_coding_system_used.
4ed46869 8956
ec6d2bb8
KH		 8957 This function is called only from macros DECODE_FILE and
8958 ENCODE_FILE, thus we ignore character composition. */
4ed46869 8959
ecec61c1 8960Lisp_Object
cf84bb53
JB		 8961code_convert_string_norecord (Lisp_Object string, Lisp_Object coding_system,
8962 int encodep)
4ed46869 8963{
0be8721c 8964 return code_convert_string (string, coding_system, Qt, encodep, 0, 1);
4ed46869
KH		 8965}
8966
4ed46869 8967
a7ca3326 8968DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string,
df7492f9
KH		 8969 2, 4, 0,
8970 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
8971
8972Optional third arg NOCOPY non-nil means it is OK to return STRING itself
8973if the decoding operation is trivial.
ecec61c1 8974
d4a1d553 8975Optional fourth arg BUFFER non-nil means that the decoded text is
1560f91a
EZ		 8976inserted in that buffer after point (point does not move). In this
8977case, the return value is the length of the decoded text.
ecec61c1 8978
df7492f9
KH		 8979This function sets `last-coding-system-used' to the precise coding system
8980used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
d4a1d553 8981not fully specified.) */)
5842a27b 8982 (Lisp_Object string, Lisp_Object coding_system, Lisp_Object nocopy, Lisp_Object buffer)
4ed46869 8983{
df7492f9
KH		 8984 return code_convert_string (string, coding_system, buffer,
8985 0, ! NILP (nocopy), 0);
4ed46869
KH		 8986}
8987
df7492f9
KH		 8988DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
8989 2, 4, 0,
8990 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
8991
8992Optional third arg NOCOPY non-nil means it is OK to return STRING
8993itself if the encoding operation is trivial.
8994
d4a1d553 8995Optional fourth arg BUFFER non-nil means that the encoded text is
1560f91a
EZ		 8996inserted in that buffer after point (point does not move). In this
8997case, the return value is the length of the encoded text.
df7492f9
KH		 8998
8999This function sets `last-coding-system-used' to the precise coding system
9000used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
9001not fully specified.) */)
5842a27b 9002 (Lisp_Object string, Lisp_Object coding_system, Lisp_Object nocopy, Lisp_Object buffer)
4ed46869 9003{
df7492f9 9004 return code_convert_string (string, coding_system, buffer,
4550efdf 9005 1, ! NILP (nocopy), 0);
4ed46869 9006}
df7492f9 9007
3a73fa5d 9008\f
4ed46869 9009DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0,
48b0f3ae
PJ		 9010 doc: /* Decode a Japanese character which has CODE in shift_jis encoding.
9011Return the corresponding character. */)
5842a27b 9012 (Lisp_Object code)
4ed46869 9013{
df7492f9
KH		 9014 Lisp_Object spec, attrs, val;
9015 struct charset *charset_roman, *charset_kanji, *charset_kana, *charset;
5fdb398c
PE		 9016 EMACS_INT ch;
9017 int c;
4ed46869 9018
df7492f9 9019 CHECK_NATNUM (code);
5fdb398c 9020 ch = XFASTINT (code);
df7492f9
KH		 9021 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
9022 attrs = AREF (spec, 0);
4ed46869 9023
5fdb398c 9024 if (ASCII_BYTE_P (ch)
df7492f9
KH		 9025 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
9026 return code;
4ed46869 9027
df7492f9
KH		 9028 val = CODING_ATTR_CHARSET_LIST (attrs);
9029 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
004068e4
KH		 9030 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
9031 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 9032
5fdb398c
PE		 9033 if (ch <= 0x7F)
9034 {
9035 c = ch;
9036 charset = charset_roman;
9037 }
9038 else if (ch >= 0xA0 && ch < 0xDF)
55ab7be3 9039 {
5fdb398c 9040 c = ch - 0x80;
df7492f9 9041 charset = charset_kana;
4ed46869 9042 }
55ab7be3 9043 else
4ed46869 9044 {
5fdb398c
PE		 9045 EMACS_INT c1 = ch >> 8;
9046 int c2 = ch & 0xFF;
df7492f9 9047
2735d060
PE		 9048 if (c1 < 0x81 || (c1 > 0x9F && c1 < 0xE0) || c1 > 0xEF
9049 || c2 < 0x40 || c2 == 0x7F || c2 > 0xFC)
c2982e87 9050 error ("Invalid code: %"pI"d", ch);
5fdb398c 9051 c = ch;
df7492f9
KH		 9052 SJIS_TO_JIS (c);
9053 charset = charset_kanji;
4ed46869 9054 }
df7492f9
KH		 9055 c = DECODE_CHAR (charset, c);
9056 if (c < 0)
c2982e87 9057 error ("Invalid code: %"pI"d", ch);
df7492f9 9058 return make_number (c);
93dec019 9059}
4ed46869 9060
48b0f3ae 9061
4ed46869 9062DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0,
8acf0c0e 9063 doc: /* Encode a Japanese character CH to shift_jis encoding.
48b0f3ae 9064Return the corresponding code in SJIS. */)
5842a27b 9065 (Lisp_Object ch)
4ed46869 9066{
df7492f9
KH		 9067 Lisp_Object spec, attrs, charset_list;
9068 int c;
9069 struct charset *charset;
9070 unsigned code;
48b0f3ae 9071
df7492f9
KH		 9072 CHECK_CHARACTER (ch);
9073 c = XFASTINT (ch);
9074 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
9075 attrs = AREF (spec, 0);
9076
9077 if (ASCII_CHAR_P (c)
9078 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
9079 return ch;
9080
9081 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
9082 charset = char_charset (c, charset_list, &code);
9083 if (code == CHARSET_INVALID_CODE (charset))
e6c3da20 9084 error ("Can't encode by shift_jis encoding: %c", c);
df7492f9
KH		 9085 JIS_TO_SJIS (code);
9086
9087 return make_number (code);
4ed46869
KH		 9088}
9089
9090DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0,
48b0f3ae
PJ		 9091 doc: /* Decode a Big5 character which has CODE in BIG5 coding system.
9092Return the corresponding character. */)
5842a27b 9093 (Lisp_Object code)
d46c5b12 9094{
df7492f9
KH		 9095 Lisp_Object spec, attrs, val;
9096 struct charset *charset_roman, *charset_big5, *charset;
5fdb398c 9097 EMACS_INT ch;
df7492f9 9098 int c;
6289dd10 9099
df7492f9 9100 CHECK_NATNUM (code);
5fdb398c 9101 ch = XFASTINT (code);
df7492f9
KH		 9102 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
9103 attrs = AREF (spec, 0);
4ed46869 9104
5fdb398c 9105 if (ASCII_BYTE_P (ch)
df7492f9
KH		 9106 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
9107 return code;
6289dd10 9108
df7492f9
KH		 9109 val = CODING_ATTR_CHARSET_LIST (attrs);
9110 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
9111 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
c210f766 9112
5fdb398c
PE		 9113 if (ch <= 0x7F)
9114 {
9115 c = ch;
9116 charset = charset_roman;
9117 }
c28a9453
KH		 9118 else
9119 {
5fdb398c
PE		 9120 EMACS_INT b1 = ch >> 8;
9121 int b2 = ch & 0x7F;
df7492f9
KH		 9122 if (b1 < 0xA1 || b1 > 0xFE
9123 || b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE)
c2982e87 9124 error ("Invalid code: %"pI"d", ch);
5fdb398c 9125 c = ch;
df7492f9 9126 charset = charset_big5;
c28a9453 9127 }
5fdb398c 9128 c = DECODE_CHAR (charset, c);
df7492f9 9129 if (c < 0)
c2982e87 9130 error ("Invalid code: %"pI"d", ch);
df7492f9 9131 return make_number (c);
d46c5b12 9132}
6289dd10 9133
4ed46869 9134DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0,
8acf0c0e 9135 doc: /* Encode the Big5 character CH to BIG5 coding system.
48b0f3ae 9136Return the corresponding character code in Big5. */)
5842a27b 9137 (Lisp_Object ch)
4ed46869 9138{
df7492f9
KH		 9139 Lisp_Object spec, attrs, charset_list;
9140 struct charset *charset;
9141 int c;
9142 unsigned code;
9143
9144 CHECK_CHARACTER (ch);
9145 c = XFASTINT (ch);
9146 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
9147 attrs = AREF (spec, 0);
9148 if (ASCII_CHAR_P (c)
9149 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
9150 return ch;
9151
9152 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
9153 charset = char_charset (c, charset_list, &code);
9154 if (code == CHARSET_INVALID_CODE (charset))
e6c3da20 9155 error ("Can't encode by Big5 encoding: %c", c);
df7492f9
KH		 9156
9157 return make_number (code);
4ed46869 9158}
48b0f3ae 9159
3a73fa5d 9160\f
002fdb44 9161DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal,
68bba4e4 9162 Sset_terminal_coding_system_internal, 1, 2, 0,
48b0f3ae 9163 doc: /* Internal use only. */)
5842a27b 9164 (Lisp_Object coding_system, Lisp_Object terminal)
4ed46869 9165{
b18fad6d
KH		 9166 struct terminal *term = get_terminal (terminal, 1);
9167 struct coding_system *terminal_coding = TERMINAL_TERMINAL_CODING (term);
b7826503 9168 CHECK_SYMBOL (coding_system);
b8299c66 9169 setup_coding_system (Fcheck_coding_system (coding_system), terminal_coding);
70c22245 9170 /* We had better not send unsafe characters to terminal. */
c73bd236 9171 terminal_coding->mode |= CODING_MODE_SAFE_ENCODING;
ad1746f5 9172 /* Character composition should be disabled. */
c73bd236 9173 terminal_coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b8299c66
KL		 9174 terminal_coding->src_multibyte = 1;
9175 terminal_coding->dst_multibyte = 0;
b18fad6d
KH		 9176 if (terminal_coding->common_flags & CODING_REQUIRE_ENCODING_MASK)
9177 term->charset_list = coding_charset_list (terminal_coding);
9178 else
6b4bb703 9179 term->charset_list = Fcons (make_number (charset_ascii), Qnil);
4ed46869
KH		 9180 return Qnil;
9181}
9182
c4825358
KH		 9183DEFUN ("set-safe-terminal-coding-system-internal",
9184 Fset_safe_terminal_coding_system_internal,
48b0f3ae 9185 Sset_safe_terminal_coding_system_internal, 1, 1, 0,
ddb67bdc 9186 doc: /* Internal use only. */)
5842a27b 9187 (Lisp_Object coding_system)
d46c5b12 9188{
b7826503 9189 CHECK_SYMBOL (coding_system);
c4825358
KH		 9190 setup_coding_system (Fcheck_coding_system (coding_system),
9191 &safe_terminal_coding);
ad1746f5 9192 /* Character composition should be disabled. */
df7492f9 9193 safe_terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b73bfc1c
KH		 9194 safe_terminal_coding.src_multibyte = 1;
9195 safe_terminal_coding.dst_multibyte = 0;
c4825358
KH		 9196 return Qnil;
9197}
4ed46869 9198
002fdb44 9199DEFUN ("terminal-coding-system", Fterminal_coding_system,
68bba4e4 9200 Sterminal_coding_system, 0, 1, 0,
6ed8eeff 9201 doc: /* Return coding system specified for terminal output on the given terminal.
708e05dc 9202TERMINAL may be a terminal object, a frame, or nil for the selected
6ed8eeff 9203frame's terminal device. */)
5842a27b 9204 (Lisp_Object terminal)
4ed46869 9205{
985773c9
MB		 9206 struct coding_system *terminal_coding
9207 = TERMINAL_TERMINAL_CODING (get_terminal (terminal, 1));
9208 Lisp_Object coding_system = CODING_ID_NAME (terminal_coding->id);
ae6f73fa 9209
ae6f73fa 9210 /* For backward compatibility, return nil if it is `undecided'. */
f75c90a9 9211 return (! EQ (coding_system, Qundecided) ? coding_system : Qnil);
4ed46869
KH		 9212}
9213
002fdb44 9214DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal,
68bba4e4 9215 Sset_keyboard_coding_system_internal, 1, 2, 0,
48b0f3ae 9216 doc: /* Internal use only. */)
5842a27b 9217 (Lisp_Object coding_system, Lisp_Object terminal)
4ed46869 9218{
6ed8eeff 9219 struct terminal *t = get_terminal (terminal, 1);
b7826503 9220 CHECK_SYMBOL (coding_system);
624bda09
KH		 9221 if (NILP (coding_system))
9222 coding_system = Qno_conversion;
9223 else
9224 Fcheck_coding_system (coding_system);
9225 setup_coding_system (coding_system, TERMINAL_KEYBOARD_CODING (t));
ad1746f5 9226 /* Character composition should be disabled. */
c73bd236
MB		 9227 TERMINAL_KEYBOARD_CODING (t)->common_flags
9228 &= ~CODING_ANNOTATE_COMPOSITION_MASK;
4ed46869
KH		 9229 return Qnil;
9230}
9231
9232DEFUN ("keyboard-coding-system",
985773c9 9233 Fkeyboard_coding_system, Skeyboard_coding_system, 0, 1, 0,
48b0f3ae 9234 doc: /* Return coding system specified for decoding keyboard input. */)
5842a27b 9235 (Lisp_Object terminal)
4ed46869 9236{
985773c9
MB		 9237 return CODING_ID_NAME (TERMINAL_KEYBOARD_CODING
9238 (get_terminal (terminal, 1))->id);
4ed46869
KH		 9239}
9240
4ed46869 9241\f
a7ca3326 9242DEFUN ("find-operation-coding-system", Ffind_operation_coding_system,
a5d301df 9243 Sfind_operation_coding_system, 1, MANY, 0,
48b0f3ae
PJ		 9244 doc: /* Choose a coding system for an operation based on the target name.
9245The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
9246DECODING-SYSTEM is the coding system to use for decoding
9247\(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
9248for encoding (in case OPERATION does encoding).
05e6f5dc 9249
48b0f3ae
PJ		 9250The first argument OPERATION specifies an I/O primitive:
9251 For file I/O, `insert-file-contents' or `write-region'.
9252 For process I/O, `call-process', `call-process-region', or `start-process'.
9253 For network I/O, `open-network-stream'.
05e6f5dc 9254
48b0f3ae
PJ		 9255The remaining arguments should be the same arguments that were passed
9256to the primitive. Depending on which primitive, one of those arguments
9257is selected as the TARGET. For example, if OPERATION does file I/O,
9258whichever argument specifies the file name is TARGET.
05e6f5dc 9259
48b0f3ae 9260TARGET has a meaning which depends on OPERATION:
b883cdb2 9261 For file I/O, TARGET is a file name (except for the special case below).
48b0f3ae 9262 For process I/O, TARGET is a process name.
d4a1d553 9263 For network I/O, TARGET is a service name or a port number.
05e6f5dc 9264
d4a1d553 9265This function looks up what is specified for TARGET in
48b0f3ae
PJ		 9266`file-coding-system-alist', `process-coding-system-alist',
9267or `network-coding-system-alist' depending on OPERATION.
9268They may specify a coding system, a cons of coding systems,
9269or a function symbol to call.
9270In the last case, we call the function with one argument,
9271which is a list of all the arguments given to this function.
1011c487
MB		 9272If the function can't decide a coding system, it can return
9273`undecided' so that the normal code-detection is performed.
48b0f3ae 9274
b883cdb2
MB		 9275If OPERATION is `insert-file-contents', the argument corresponding to
9276TARGET may be a cons (FILENAME . BUFFER). In that case, FILENAME is a
9277file name to look up, and BUFFER is a buffer that contains the file's
9278contents (not yet decoded). If `file-coding-system-alist' specifies a
9279function to call for FILENAME, that function should examine the
9280contents of BUFFER instead of reading the file.
9281
d918f936 9282usage: (find-operation-coding-system OPERATION ARGUMENTS...) */)
f66c7cf8 9283 (ptrdiff_t nargs, Lisp_Object *args)
6b89e3aa 9284{
4ed46869
KH		 9285 Lisp_Object operation, target_idx, target, val;
9286 register Lisp_Object chain;
177c0ea7 9287
4ed46869
KH		 9288 if (nargs < 2)
9289 error ("Too few arguments");
9290 operation = args[0];
9291 if (!SYMBOLP (operation)
d311d28c 9292 || (target_idx = Fget (operation, Qtarget_idx), !NATNUMP (target_idx)))
3ed051d4 9293 error ("Invalid first argument");
c5101a77 9294 if (nargs < 1 + XFASTINT (target_idx))
94dcfacf 9295 error ("Too few arguments for operation `%s'",
8f924df7 9296 SDATA (SYMBOL_NAME (operation)));
c5101a77 9297 target = args[XFASTINT (target_idx) + 1];
4ed46869 9298 if (!(STRINGP (target)
091a0ff0
KH		 9299 || (EQ (operation, Qinsert_file_contents) && CONSP (target)
9300 && STRINGP (XCAR (target)) && BUFFERP (XCDR (target)))
4ed46869 9301 || (EQ (operation, Qopen_network_stream) && INTEGERP (target))))
94dcfacf
EZ		 9302 error ("Invalid argument %"pI"d of operation `%s'",
9303 XFASTINT (target_idx) + 1, SDATA (SYMBOL_NAME (operation)));
091a0ff0
KH		 9304 if (CONSP (target))
9305 target = XCAR (target);
4ed46869 9306
2e34157c
RS		 9307 chain = ((EQ (operation, Qinsert_file_contents)
9308 || EQ (operation, Qwrite_region))
02ba4723 9309 ? Vfile_coding_system_alist
2e34157c 9310 : (EQ (operation, Qopen_network_stream)
02ba4723
KH		 9311 ? Vnetwork_coding_system_alist
9312 : Vprocess_coding_system_alist));
4ed46869
KH		 9313 if (NILP (chain))
9314 return Qnil;
9315
03699b14 9316 for (; CONSP (chain); chain = XCDR (chain))
6b89e3aa 9317 {
f44d27ce 9318 Lisp_Object elt;
6b89e3aa 9319
df7492f9 9320 elt = XCAR (chain);
4ed46869
KH		 9321 if (CONSP (elt)
9322 && ((STRINGP (target)
03699b14
KR		 9323 && STRINGP (XCAR (elt))
9324 && fast_string_match (XCAR (elt), target) >= 0)
9325 || (INTEGERP (target) && EQ (target, XCAR (elt)))))
6b89e3aa 9326 {
03699b14 9327 val = XCDR (elt);
b19fd4c5
KH		 9328 /* Here, if VAL is both a valid coding system and a valid
9329 function symbol, we return VAL as a coding system. */
02ba4723
KH		 9330 if (CONSP (val))
9331 return val;
9332 if (! SYMBOLP (val))
9333 return Qnil;
9334 if (! NILP (Fcoding_system_p (val)))
9335 return Fcons (val, val);
b19fd4c5 9336 if (! NILP (Ffboundp (val)))
6b89e3aa 9337 {
e2b97060
MB		 9338 /* We use call1 rather than safe_call1
9339 so as to get bug reports about functions called here
9340 which don't handle the current interface. */
9341 val = call1 (val, Flist (nargs, args));
b19fd4c5
KH		 9342 if (CONSP (val))
9343 return val;
9344 if (SYMBOLP (val) && ! NILP (Fcoding_system_p (val)))
9345 return Fcons (val, val);
6b89e3aa 9346 }
02ba4723 9347 return Qnil;
6b89e3aa
KH		 9348 }
9349 }
4ed46869 9350 return Qnil;
6b89e3aa
KH		 9351}
9352
df7492f9 9353DEFUN ("set-coding-system-priority", Fset_coding_system_priority,
a3f6ee6d 9354 Sset_coding_system_priority, 0, MANY, 0,
da7db224 9355 doc: /* Assign higher priority to the coding systems given as arguments.
d4a1d553 9356If multiple coding systems belong to the same category,
a3181084
DL		 9357all but the first one are ignored.
9358
d4a1d553 9359usage: (set-coding-system-priority &rest coding-systems) */)
f66c7cf8 9360 (ptrdiff_t nargs, Lisp_Object *args)
df7492f9 9361{
f66c7cf8 9362 ptrdiff_t i, j;
df7492f9
KH		 9363 int changed[coding_category_max];
9364 enum coding_category priorities[coding_category_max];
9365
72af86bd 9366 memset (changed, 0, sizeof changed);
6b89e3aa 9367
df7492f9 9368 for (i = j = 0; i < nargs; i++)
6b89e3aa 9369 {
df7492f9
KH		 9370 enum coding_category category;
9371 Lisp_Object spec, attrs;
6b89e3aa 9372
df7492f9
KH		 9373 CHECK_CODING_SYSTEM_GET_SPEC (args[i], spec);
9374 attrs = AREF (spec, 0);
9375 category = XINT (CODING_ATTR_CATEGORY (attrs));
9376 if (changed[category])
9377 /* Ignore this coding system because a coding system of the
9378 same category already had a higher priority. */
9379 continue;
9380 changed[category] = 1;
9381 priorities[j++] = category;
9382 if (coding_categories[category].id >= 0
9383 && ! EQ (args[i], CODING_ID_NAME (coding_categories[category].id)))
9384 setup_coding_system (args[i], &coding_categories[category]);
ff563fce 9385 Fset (AREF (Vcoding_category_table, category), args[i]);
df7492f9 9386 }
6b89e3aa 9387
df7492f9
KH		 9388 /* Now we have decided top J priorities. Reflect the order of the
9389 original priorities to the remaining priorities. */
6b89e3aa 9390
df7492f9 9391 for (i = j, j = 0; i < coding_category_max; i++, j++)
6b89e3aa 9392 {
df7492f9
KH		 9393 while (j < coding_category_max
9394 && changed[coding_priorities[j]])
9395 j++;
9396 if (j == coding_category_max)
9397 abort ();
9398 priorities[i] = coding_priorities[j];
9399 }
6b89e3aa 9400
72af86bd 9401 memcpy (coding_priorities, priorities, sizeof priorities);
177c0ea7 9402
ff563fce
KH		 9403 /* Update `coding-category-list'. */
9404 Vcoding_category_list = Qnil;
c5101a77 9405 for (i = coding_category_max; i-- > 0; )
ff563fce
KH		 9406 Vcoding_category_list
9407 = Fcons (AREF (Vcoding_category_table, priorities[i]),
9408 Vcoding_category_list);
6b89e3aa 9409
df7492f9 9410 return Qnil;
6b89e3aa
KH		 9411}
9412
df7492f9
KH		 9413DEFUN ("coding-system-priority-list", Fcoding_system_priority_list,
9414 Scoding_system_priority_list, 0, 1, 0,
da7db224 9415 doc: /* Return a list of coding systems ordered by their priorities.
b811c52b
KH		 9416The list contains a subset of coding systems; i.e. coding systems
9417assigned to each coding category (see `coding-category-list').
9418
da7db224 9419HIGHESTP non-nil means just return the highest priority one. */)
5842a27b 9420 (Lisp_Object highestp)
d46c5b12
KH		 9421{
9422 int i;
df7492f9 9423 Lisp_Object val;
6b89e3aa 9424
df7492f9 9425 for (i = 0, val = Qnil; i < coding_category_max; i++)
d46c5b12 9426 {
df7492f9
KH		 9427 enum coding_category category = coding_priorities[i];
9428 int id = coding_categories[category].id;
9429 Lisp_Object attrs;
068a9dbd 9430
df7492f9
KH		 9431 if (id < 0)
9432 continue;
9433 attrs = CODING_ID_ATTRS (id);
9434 if (! NILP (highestp))
9435 return CODING_ATTR_BASE_NAME (attrs);
9436 val = Fcons (CODING_ATTR_BASE_NAME (attrs), val);
9437 }
9438 return Fnreverse (val);
9439}
068a9dbd 9440
91433552 9441static const char *const suffixes[] = { "-unix", "-dos", "-mac" };
068a9dbd
KH		 9442
9443static Lisp_Object
971de7fb 9444make_subsidiaries (Lisp_Object base)
068a9dbd 9445{
df7492f9 9446 Lisp_Object subsidiaries;
1bfdaf10 9447 ptrdiff_t base_name_len = SBYTES (SYMBOL_NAME (base));
df7492f9
KH		 9448 char *buf = (char *) alloca (base_name_len + 6);
9449 int i;
068a9dbd 9450
72af86bd 9451 memcpy (buf, SDATA (SYMBOL_NAME (base)), base_name_len);
df7492f9
KH		 9452 subsidiaries = Fmake_vector (make_number (3), Qnil);
9453 for (i = 0; i < 3; i++)
068a9dbd 9454 {
1bfdaf10 9455 strcpy (buf + base_name_len, suffixes[i]);
df7492f9 9456 ASET (subsidiaries, i, intern (buf));
068a9dbd 9457 }
df7492f9 9458 return subsidiaries;
068a9dbd
KH		 9459}
9460
9461
df7492f9
KH		 9462DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal,
9463 Sdefine_coding_system_internal, coding_arg_max, MANY, 0,
1fcd6c8b
DL		 9464 doc: /* For internal use only.
9465usage: (define-coding-system-internal ...) */)
f66c7cf8 9466 (ptrdiff_t nargs, Lisp_Object *args)
068a9dbd 9467{
df7492f9
KH		 9468 Lisp_Object name;
9469 Lisp_Object spec_vec; /* [ ATTRS ALIASE EOL_TYPE ] */
9470 Lisp_Object attrs; /* Vector of attributes. */
9471 Lisp_Object eol_type;
9472 Lisp_Object aliases;
9473 Lisp_Object coding_type, charset_list, safe_charsets;
9474 enum coding_category category;
9475 Lisp_Object tail, val;
9476 int max_charset_id = 0;
9477 int i;
068a9dbd 9478
df7492f9
KH		 9479 if (nargs < coding_arg_max)
9480 goto short_args;
068a9dbd 9481
df7492f9 9482 attrs = Fmake_vector (make_number (coding_attr_last_index), Qnil);
068a9dbd 9483
df7492f9
KH		 9484 name = args[coding_arg_name];
9485 CHECK_SYMBOL (name);
9486 CODING_ATTR_BASE_NAME (attrs) = name;
068a9dbd 9487
df7492f9
KH		 9488 val = args[coding_arg_mnemonic];
9489 if (! STRINGP (val))
9490 CHECK_CHARACTER (val);
9491 CODING_ATTR_MNEMONIC (attrs) = val;
068a9dbd 9492
df7492f9
KH		 9493 coding_type = args[coding_arg_coding_type];
9494 CHECK_SYMBOL (coding_type);
9495 CODING_ATTR_TYPE (attrs) = coding_type;
068a9dbd 9496
df7492f9
KH		 9497 charset_list = args[coding_arg_charset_list];
9498 if (SYMBOLP (charset_list))
9499 {
9500 if (EQ (charset_list, Qiso_2022))
9501 {
9502 if (! EQ (coding_type, Qiso_2022))
9503 error ("Invalid charset-list");
9504 charset_list = Viso_2022_charset_list;
9505 }
9506 else if (EQ (charset_list, Qemacs_mule))
9507 {
9508 if (! EQ (coding_type, Qemacs_mule))
9509 error ("Invalid charset-list");
9510 charset_list = Vemacs_mule_charset_list;
9511 }
9512 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
d311d28c
PE		 9513 {
9514 if (! RANGED_INTEGERP (0, XCAR (tail), INT_MAX - 1))
9515 error ("Invalid charset-list");
9516 if (max_charset_id < XFASTINT (XCAR (tail)))
9517 max_charset_id = XFASTINT (XCAR (tail));
9518 }
df7492f9 9519 }
068a9dbd
KH		 9520 else
9521 {
df7492f9 9522 charset_list = Fcopy_sequence (charset_list);
985773c9 9523 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
068a9dbd 9524 {
df7492f9
KH		 9525 struct charset *charset;
9526
985773c9 9527 val = XCAR (tail);
df7492f9
KH		 9528 CHECK_CHARSET_GET_CHARSET (val, charset);
9529 if (EQ (coding_type, Qiso_2022)
9530 ? CHARSET_ISO_FINAL (charset) < 0
9531 : EQ (coding_type, Qemacs_mule)
9532 ? CHARSET_EMACS_MULE_ID (charset) < 0
9533 : 0)
9534 error ("Can't handle charset `%s'",
8f924df7 9535 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9 9536
8f924df7 9537 XSETCAR (tail, make_number (charset->id));
df7492f9
KH		 9538 if (max_charset_id < charset->id)
9539 max_charset_id = charset->id;
068a9dbd
KH		 9540 }
9541 }
df7492f9 9542 CODING_ATTR_CHARSET_LIST (attrs) = charset_list;
068a9dbd 9543
1b3b981b
AS		 9544 safe_charsets = make_uninit_string (max_charset_id + 1);
9545 memset (SDATA (safe_charsets), 255, max_charset_id + 1);
df7492f9 9546 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8f924df7 9547 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 9548 CODING_ATTR_SAFE_CHARSETS (attrs) = safe_charsets;
068a9dbd 9549
584948ac 9550 CODING_ATTR_ASCII_COMPAT (attrs) = args[coding_arg_ascii_compatible_p];
3a73fa5d 9551
df7492f9 9552 val = args[coding_arg_decode_translation_table];
a6f87d34 9553 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 9554 CHECK_SYMBOL (val);
df7492f9 9555 CODING_ATTR_DECODE_TBL (attrs) = val;
3a73fa5d 9556
df7492f9 9557 val = args[coding_arg_encode_translation_table];
a6f87d34 9558 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 9559 CHECK_SYMBOL (val);
df7492f9 9560 CODING_ATTR_ENCODE_TBL (attrs) = val;
d46c5b12 9561
df7492f9
KH		 9562 val = args[coding_arg_post_read_conversion];
9563 CHECK_SYMBOL (val);
9564 CODING_ATTR_POST_READ (attrs) = val;
d46c5b12 9565
df7492f9
KH		 9566 val = args[coding_arg_pre_write_conversion];
9567 CHECK_SYMBOL (val);
9568 CODING_ATTR_PRE_WRITE (attrs) = val;
3a73fa5d 9569
df7492f9
KH		 9570 val = args[coding_arg_default_char];
9571 if (NILP (val))
9572 CODING_ATTR_DEFAULT_CHAR (attrs) = make_number (' ');
9573 else
9574 {
8f924df7 9575 CHECK_CHARACTER (val);
df7492f9
KH		 9576 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
9577 }
4031e2bf 9578
8f924df7
KH		 9579 val = args[coding_arg_for_unibyte];
9580 CODING_ATTR_FOR_UNIBYTE (attrs) = NILP (val) ? Qnil : Qt;
3a73fa5d 9581
df7492f9
KH		 9582 val = args[coding_arg_plist];
9583 CHECK_LIST (val);
9584 CODING_ATTR_PLIST (attrs) = val;
3a73fa5d 9585
df7492f9
KH		 9586 if (EQ (coding_type, Qcharset))
9587 {
c7c66a95
KH		 9588 /* Generate a lisp vector of 256 elements. Each element is nil,
9589 integer, or a list of charset IDs.
3a73fa5d 9590
c7c66a95
KH		 9591 If Nth element is nil, the byte code N is invalid in this
9592 coding system.
4ed46869 9593
c7c66a95
KH		 9594 If Nth element is a number NUM, N is the first byte of a
9595 charset whose ID is NUM.
4ed46869 9596
c7c66a95
KH		 9597 If Nth element is a list of charset IDs, N is the first byte
9598 of one of them. The list is sorted by dimensions of the
ad1746f5 9599 charsets. A charset of smaller dimension comes first. */
df7492f9 9600 val = Fmake_vector (make_number (256), Qnil);
4ed46869 9601
5c99c2e6 9602 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
df7492f9 9603 {
c7c66a95
KH		 9604 struct charset *charset = CHARSET_FROM_ID (XFASTINT (XCAR (tail)));
9605 int dim = CHARSET_DIMENSION (charset);
9606 int idx = (dim - 1) * 4;
4ed46869 9607
5c99c2e6 9608 if (CHARSET_ASCII_COMPATIBLE_P (charset))
584948ac 9609 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
4031e2bf 9610
15d143f7
KH		 9611 for (i = charset->code_space[idx];
9612 i <= charset->code_space[idx + 1]; i++)
9613 {
c7c66a95
KH		 9614 Lisp_Object tmp, tmp2;
9615 int dim2;
ec6d2bb8 9616
c7c66a95
KH		 9617 tmp = AREF (val, i);
9618 if (NILP (tmp))
9619 tmp = XCAR (tail);
9620 else if (NUMBERP (tmp))
9621 {
9622 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (tmp)));
9623 if (dim < dim2)
c7c66a95 9624 tmp = Fcons (XCAR (tail), Fcons (tmp, Qnil));
f9d71dcd
KH		 9625 else
9626 tmp = Fcons (tmp, Fcons (XCAR (tail), Qnil));
c7c66a95 9627 }
15d143f7 9628 else
c7c66a95
KH		 9629 {
9630 for (tmp2 = tmp; CONSP (tmp2); tmp2 = XCDR (tmp2))
9631 {
9632 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (XCAR (tmp2))));
9633 if (dim < dim2)
9634 break;
9635 }
9636 if (NILP (tmp2))
9637 tmp = nconc2 (tmp, Fcons (XCAR (tail), Qnil));
9638 else
9639 {
9640 XSETCDR (tmp2, Fcons (XCAR (tmp2), XCDR (tmp2)));
9641 XSETCAR (tmp2, XCAR (tail));
9642 }
9643 }
9644 ASET (val, i, tmp);
15d143f7 9645 }
df7492f9
KH		 9646 }
9647 ASET (attrs, coding_attr_charset_valids, val);
9648 category = coding_category_charset;
9649 }
9650 else if (EQ (coding_type, Qccl))
9651 {
9652 Lisp_Object valids;
ecec61c1 9653
df7492f9
KH		 9654 if (nargs < coding_arg_ccl_max)
9655 goto short_args;
ecec61c1 9656
df7492f9
KH		 9657 val = args[coding_arg_ccl_decoder];
9658 CHECK_CCL_PROGRAM (val);
9659 if (VECTORP (val))
9660 val = Fcopy_sequence (val);
9661 ASET (attrs, coding_attr_ccl_decoder, val);
ecec61c1 9662
df7492f9
KH		 9663 val = args[coding_arg_ccl_encoder];
9664 CHECK_CCL_PROGRAM (val);
9665 if (VECTORP (val))
9666 val = Fcopy_sequence (val);
9667 ASET (attrs, coding_attr_ccl_encoder, val);
ecec61c1 9668
df7492f9
KH		 9669 val = args[coding_arg_ccl_valids];
9670 valids = Fmake_string (make_number (256), make_number (0));
9671 for (tail = val; !NILP (tail); tail = Fcdr (tail))
9672 {
8dcbea82 9673 int from, to;
ecec61c1 9674
df7492f9
KH		 9675 val = Fcar (tail);
9676 if (INTEGERP (val))
8dcbea82 9677 {
d311d28c 9678 if (! (0 <= XINT (val) && XINT (val) <= 255))
8dcbea82 9679 args_out_of_range_3 (val, make_number (0), make_number (255));
d311d28c 9680 from = to = XINT (val);
8dcbea82 9681 }
df7492f9
KH		 9682 else
9683 {
df7492f9 9684 CHECK_CONS (val);
8f924df7 9685 CHECK_NATNUM_CAR (val);
d311d28c
PE		 9686 CHECK_NUMBER_CDR (val);
9687 if (XINT (XCAR (val)) > 255)
8dcbea82
KH		 9688 args_out_of_range_3 (XCAR (val),
9689 make_number (0), make_number (255));
d311d28c
PE		 9690 from = XINT (XCAR (val));
9691 if (! (from <= XINT (XCDR (val)) && XINT (XCDR (val)) <= 255))
8dcbea82
KH		 9692 args_out_of_range_3 (XCDR (val),
9693 XCAR (val), make_number (255));
d311d28c 9694 to = XINT (XCDR (val));
df7492f9 9695 }
8dcbea82 9696 for (i = from; i <= to; i++)
8f924df7 9697 SSET (valids, i, 1);
df7492f9
KH		 9698 }
9699 ASET (attrs, coding_attr_ccl_valids, valids);
4ed46869 9700
df7492f9 9701 category = coding_category_ccl;
55ab7be3 9702 }
df7492f9 9703 else if (EQ (coding_type, Qutf_16))
55ab7be3 9704 {
df7492f9 9705 Lisp_Object bom, endian;
4ed46869 9706
584948ac 9707 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
4ed46869 9708
df7492f9
KH		 9709 if (nargs < coding_arg_utf16_max)
9710 goto short_args;
4ed46869 9711
df7492f9
KH		 9712 bom = args[coding_arg_utf16_bom];
9713 if (! NILP (bom) && ! EQ (bom, Qt))
9714 {
9715 CHECK_CONS (bom);
8f924df7
KH		 9716 val = XCAR (bom);
9717 CHECK_CODING_SYSTEM (val);
9718 val = XCDR (bom);
9719 CHECK_CODING_SYSTEM (val);
df7492f9 9720 }
a470d443 9721 ASET (attrs, coding_attr_utf_bom, bom);
df7492f9
KH		 9722
9723 endian = args[coding_arg_utf16_endian];
b49a1807
KH		 9724 CHECK_SYMBOL (endian);
9725 if (NILP (endian))
9726 endian = Qbig;
9727 else if (! EQ (endian, Qbig) && ! EQ (endian, Qlittle))
8f924df7 9728 error ("Invalid endian: %s", SDATA (SYMBOL_NAME (endian)));
df7492f9
KH		 9729 ASET (attrs, coding_attr_utf_16_endian, endian);
9730
9731 category = (CONSP (bom)
9732 ? coding_category_utf_16_auto
9733 : NILP (bom)
b49a1807 9734 ? (EQ (endian, Qbig)
df7492f9
KH		 9735 ? coding_category_utf_16_be_nosig
9736 : coding_category_utf_16_le_nosig)
b49a1807 9737 : (EQ (endian, Qbig)
df7492f9
KH		 9738 ? coding_category_utf_16_be
9739 : coding_category_utf_16_le));
9740 }
9741 else if (EQ (coding_type, Qiso_2022))
9742 {
9743 Lisp_Object initial, reg_usage, request, flags;
1397dc18 9744
df7492f9
KH		 9745 if (nargs < coding_arg_iso2022_max)
9746 goto short_args;
9747
9748 initial = Fcopy_sequence (args[coding_arg_iso2022_initial]);
9749 CHECK_VECTOR (initial);
9750 for (i = 0; i < 4; i++)
9751 {
9752 val = Faref (initial, make_number (i));
9753 if (! NILP (val))
9754 {
584948ac
KH		 9755 struct charset *charset;
9756
9757 CHECK_CHARSET_GET_CHARSET (val, charset);
9758 ASET (initial, i, make_number (CHARSET_ID (charset)));
9759 if (i == 0 && CHARSET_ASCII_COMPATIBLE_P (charset))
9760 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9761 }
9762 else
9763 ASET (initial, i, make_number (-1));
9764 }
9765
9766 reg_usage = args[coding_arg_iso2022_reg_usage];
9767 CHECK_CONS (reg_usage);
8f924df7
KH		 9768 CHECK_NUMBER_CAR (reg_usage);
9769 CHECK_NUMBER_CDR (reg_usage);
df7492f9
KH		 9770
9771 request = Fcopy_sequence (args[coding_arg_iso2022_request]);
9772 for (tail = request; ! NILP (tail); tail = Fcdr (tail))
1397dc18 9773 {
df7492f9 9774 int id;
2735d060 9775 Lisp_Object tmp1;
df7492f9
KH		 9776
9777 val = Fcar (tail);
9778 CHECK_CONS (val);
2735d060
PE		 9779 tmp1 = XCAR (val);
9780 CHECK_CHARSET_GET_ID (tmp1, id);
8f924df7 9781 CHECK_NATNUM_CDR (val);
df7492f9 9782 if (XINT (XCDR (val)) >= 4)
c2982e87 9783 error ("Invalid graphic register number: %"pI"d", XINT (XCDR (val)));
8f924df7 9784 XSETCAR (val, make_number (id));
1397dc18 9785 }
4ed46869 9786
df7492f9
KH		 9787 flags = args[coding_arg_iso2022_flags];
9788 CHECK_NATNUM (flags);
d311d28c 9789 i = XINT (flags) & INT_MAX;
df7492f9 9790 if (EQ (args[coding_arg_charset_list], Qiso_2022))
d311d28c
PE		 9791 i |= CODING_ISO_FLAG_FULL_SUPPORT;
9792 flags = make_number (i);
df7492f9
KH		 9793
9794 ASET (attrs, coding_attr_iso_initial, initial);
9795 ASET (attrs, coding_attr_iso_usage, reg_usage);
9796 ASET (attrs, coding_attr_iso_request, request);
9797 ASET (attrs, coding_attr_iso_flags, flags);
9798 setup_iso_safe_charsets (attrs);
9799
9800 if (i & CODING_ISO_FLAG_SEVEN_BITS)
9801 category = ((i & (CODING_ISO_FLAG_LOCKING_SHIFT
9802 | CODING_ISO_FLAG_SINGLE_SHIFT))
9803 ? coding_category_iso_7_else
9804 : EQ (args[coding_arg_charset_list], Qiso_2022)
9805 ? coding_category_iso_7
9806 : coding_category_iso_7_tight);
9807 else
9808 {
9809 int id = XINT (AREF (initial, 1));
9810
c6fb6e98 9811 category = (((i & CODING_ISO_FLAG_LOCKING_SHIFT)
df7492f9
KH		 9812 || EQ (args[coding_arg_charset_list], Qiso_2022)
9813 || id < 0)
9814 ? coding_category_iso_8_else
9815 : (CHARSET_DIMENSION (CHARSET_FROM_ID (id)) == 1)
9816 ? coding_category_iso_8_1
9817 : coding_category_iso_8_2);
9818 }
0ce7886f
KH		 9819 if (category != coding_category_iso_8_1
9820 && category != coding_category_iso_8_2)
9821 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
df7492f9
KH		 9822 }
9823 else if (EQ (coding_type, Qemacs_mule))
c28a9453 9824 {
df7492f9
KH		 9825 if (EQ (args[coding_arg_charset_list], Qemacs_mule))
9826 ASET (attrs, coding_attr_emacs_mule_full, Qt);
584948ac 9827 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9 9828 category = coding_category_emacs_mule;
c28a9453 9829 }
df7492f9 9830 else if (EQ (coding_type, Qshift_jis))
c28a9453 9831 {
df7492f9
KH		 9832
9833 struct charset *charset;
9834
7d64c6ad 9835 if (XINT (Flength (charset_list)) != 3
6e07c25f 9836 && XINT (Flength (charset_list)) != 4)
7d64c6ad 9837 error ("There should be three or four charsets");
df7492f9
KH		 9838
9839 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9840 if (CHARSET_DIMENSION (charset) != 1)
9841 error ("Dimension of charset %s is not one",
8f924df7 9842 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 9843 if (CHARSET_ASCII_COMPATIBLE_P (charset))
9844 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9845
9846 charset_list = XCDR (charset_list);
9847 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9848 if (CHARSET_DIMENSION (charset) != 1)
9849 error ("Dimension of charset %s is not one",
8f924df7 9850 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9
KH		 9851
9852 charset_list = XCDR (charset_list);
9853 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9854 if (CHARSET_DIMENSION (charset) != 2)
7d64c6ad
KH		 9855 error ("Dimension of charset %s is not two",
9856 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9857
9858 charset_list = XCDR (charset_list);
2b917a06
KH		 9859 if (! NILP (charset_list))
9860 {
9861 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9862 if (CHARSET_DIMENSION (charset) != 2)
9863 error ("Dimension of charset %s is not two",
9864 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9865 }
df7492f9
KH		 9866
9867 category = coding_category_sjis;
9868 Vsjis_coding_system = name;
c28a9453 9869 }
df7492f9
KH		 9870 else if (EQ (coding_type, Qbig5))
9871 {
9872 struct charset *charset;
4ed46869 9873
df7492f9
KH		 9874 if (XINT (Flength (charset_list)) != 2)
9875 error ("There should be just two charsets");
9876
9877 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9878 if (CHARSET_DIMENSION (charset) != 1)
9879 error ("Dimension of charset %s is not one",
8f924df7 9880 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 9881 if (CHARSET_ASCII_COMPATIBLE_P (charset))
9882 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 9883
9884 charset_list = XCDR (charset_list);
9885 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9886 if (CHARSET_DIMENSION (charset) != 2)
9887 error ("Dimension of charset %s is not two",
8f924df7 9888 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
4ed46869 9889
df7492f9
KH		 9890 category = coding_category_big5;
9891 Vbig5_coding_system = name;
9892 }
9893 else if (EQ (coding_type, Qraw_text))
c28a9453 9894 {
584948ac
KH		 9895 category = coding_category_raw_text;
9896 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
c28a9453 9897 }
df7492f9 9898 else if (EQ (coding_type, Qutf_8))
4ed46869 9899 {
a470d443
KH		 9900 Lisp_Object bom;
9901
584948ac 9902 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
a470d443
KH		 9903
9904 if (nargs < coding_arg_utf8_max)
9905 goto short_args;
9906
9907 bom = args[coding_arg_utf8_bom];
9908 if (! NILP (bom) && ! EQ (bom, Qt))
9909 {
9910 CHECK_CONS (bom);
9911 val = XCAR (bom);
9912 CHECK_CODING_SYSTEM (val);
9913 val = XCDR (bom);
9914 CHECK_CODING_SYSTEM (val);
9915 }
9916 ASET (attrs, coding_attr_utf_bom, bom);
9917
9918 category = (CONSP (bom) ? coding_category_utf_8_auto
9919 : NILP (bom) ? coding_category_utf_8_nosig
9920 : coding_category_utf_8_sig);
4ed46869 9921 }
df7492f9
KH		 9922 else if (EQ (coding_type, Qundecided))
9923 category = coding_category_undecided;
4ed46869 9924 else
df7492f9 9925 error ("Invalid coding system type: %s",
8f924df7 9926 SDATA (SYMBOL_NAME (coding_type)));
4ed46869 9927
df7492f9 9928 CODING_ATTR_CATEGORY (attrs) = make_number (category);
01378f49
KH		 9929 CODING_ATTR_PLIST (attrs)
9930 = Fcons (QCcategory, Fcons (AREF (Vcoding_category_table, category),
9931 CODING_ATTR_PLIST (attrs)));
35befdaa 9932 CODING_ATTR_PLIST (attrs)
3ed051d4 9933 = Fcons (QCascii_compatible_p,
35befdaa
KH		 9934 Fcons (CODING_ATTR_ASCII_COMPAT (attrs),
9935 CODING_ATTR_PLIST (attrs)));
c4825358 9936
df7492f9
KH		 9937 eol_type = args[coding_arg_eol_type];
9938 if (! NILP (eol_type)
9939 && ! EQ (eol_type, Qunix)
9940 && ! EQ (eol_type, Qdos)
9941 && ! EQ (eol_type, Qmac))
9942 error ("Invalid eol-type");
4ed46869 9943
df7492f9 9944 aliases = Fcons (name, Qnil);
4ed46869 9945
df7492f9
KH		 9946 if (NILP (eol_type))
9947 {
9948 eol_type = make_subsidiaries (name);
9949 for (i = 0; i < 3; i++)
1397dc18 9950 {
df7492f9
KH		 9951 Lisp_Object this_spec, this_name, this_aliases, this_eol_type;
9952
9953 this_name = AREF (eol_type, i);
9954 this_aliases = Fcons (this_name, Qnil);
9955 this_eol_type = (i == 0 ? Qunix : i == 1 ? Qdos : Qmac);
9956 this_spec = Fmake_vector (make_number (3), attrs);
9957 ASET (this_spec, 1, this_aliases);
9958 ASET (this_spec, 2, this_eol_type);
9959 Fputhash (this_name, this_spec, Vcoding_system_hash_table);
9960 Vcoding_system_list = Fcons (this_name, Vcoding_system_list);
583f71ca
KH		 9961 val = Fassoc (Fsymbol_name (this_name), Vcoding_system_alist);
9962 if (NILP (val))
9963 Vcoding_system_alist
9964 = Fcons (Fcons (Fsymbol_name (this_name), Qnil),
9965 Vcoding_system_alist);
1397dc18 9966 }
d46c5b12 9967 }
4ed46869 9968
df7492f9
KH		 9969 spec_vec = Fmake_vector (make_number (3), attrs);
9970 ASET (spec_vec, 1, aliases);
9971 ASET (spec_vec, 2, eol_type);
48b0f3ae 9972
df7492f9
KH		 9973 Fputhash (name, spec_vec, Vcoding_system_hash_table);
9974 Vcoding_system_list = Fcons (name, Vcoding_system_list);
583f71ca
KH		 9975 val = Fassoc (Fsymbol_name (name), Vcoding_system_alist);
9976 if (NILP (val))
9977 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (name), Qnil),
9978 Vcoding_system_alist);
48b0f3ae 9979
df7492f9
KH		 9980 {
9981 int id = coding_categories[category].id;
48b0f3ae 9982
df7492f9
KH		 9983 if (id < 0 || EQ (name, CODING_ID_NAME (id)))
9984 setup_coding_system (name, &coding_categories[category]);
9985 }
48b0f3ae 9986
d46c5b12 9987 return Qnil;
48b0f3ae 9988
df7492f9
KH		 9989 short_args:
9990 return Fsignal (Qwrong_number_of_arguments,
9991 Fcons (intern ("define-coding-system-internal"),
9992 make_number (nargs)));
d46c5b12 9993}
4ed46869 9994
d6925f38 9995
a6f87d34
KH		 9996DEFUN ("coding-system-put", Fcoding_system_put, Scoding_system_put,
9997 3, 3, 0,
9998 doc: /* Change value in CODING-SYSTEM's property list PROP to VAL. */)
5842a27b 9999 (Lisp_Object coding_system, Lisp_Object prop, Lisp_Object val)
a6f87d34 10000{
3dbe7859 10001 Lisp_Object spec, attrs;
a6f87d34
KH		 10002
10003 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
10004 attrs = AREF (spec, 0);
10005 if (EQ (prop, QCmnemonic))
10006 {
10007 if (! STRINGP (val))
10008 CHECK_CHARACTER (val);
10009 CODING_ATTR_MNEMONIC (attrs) = val;
10010 }
2133e2d1 10011 else if (EQ (prop, QCdefault_char))
a6f87d34
KH		 10012 {
10013 if (NILP (val))
10014 val = make_number (' ');
10015 else
10016 CHECK_CHARACTER (val);
10017 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
10018 }
10019 else if (EQ (prop, QCdecode_translation_table))
10020 {
10021 if (! CHAR_TABLE_P (val) && ! CONSP (val))
10022 CHECK_SYMBOL (val);
10023 CODING_ATTR_DECODE_TBL (attrs) = val;
10024 }
10025 else if (EQ (prop, QCencode_translation_table))
10026 {
10027 if (! CHAR_TABLE_P (val) && ! CONSP (val))
10028 CHECK_SYMBOL (val);
10029 CODING_ATTR_ENCODE_TBL (attrs) = val;
10030 }
10031 else if (EQ (prop, QCpost_read_conversion))
10032 {
10033 CHECK_SYMBOL (val);
10034 CODING_ATTR_POST_READ (attrs) = val;
10035 }
10036 else if (EQ (prop, QCpre_write_conversion))
10037 {
10038 CHECK_SYMBOL (val);
10039 CODING_ATTR_PRE_WRITE (attrs) = val;
10040 }
35befdaa
KH		 10041 else if (EQ (prop, QCascii_compatible_p))
10042 {
10043 CODING_ATTR_ASCII_COMPAT (attrs) = val;
10044 }
a6f87d34
KH		 10045
10046 CODING_ATTR_PLIST (attrs)
10047 = Fplist_put (CODING_ATTR_PLIST (attrs), prop, val);
10048 return val;
10049}
10050
10051
df7492f9
KH		 10052DEFUN ("define-coding-system-alias", Fdefine_coding_system_alias,
10053 Sdefine_coding_system_alias, 2, 2, 0,
10054 doc: /* Define ALIAS as an alias for CODING-SYSTEM. */)
5842a27b 10055 (Lisp_Object alias, Lisp_Object coding_system)
66cfb530 10056{
583f71ca 10057 Lisp_Object spec, aliases, eol_type, val;
4ed46869 10058
df7492f9
KH		 10059 CHECK_SYMBOL (alias);
10060 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
10061 aliases = AREF (spec, 1);
d4a1d553 10062 /* ALIASES should be a list of length more than zero, and the first
d6925f38
KH		 10063 element is a base coding system. Append ALIAS at the tail of the
10064 list. */
df7492f9
KH		 10065 while (!NILP (XCDR (aliases)))
10066 aliases = XCDR (aliases);
8f924df7 10067 XSETCDR (aliases, Fcons (alias, Qnil));
4ed46869 10068
df7492f9
KH		 10069 eol_type = AREF (spec, 2);
10070 if (VECTORP (eol_type))
4ed46869 10071 {
df7492f9
KH		 10072 Lisp_Object subsidiaries;
10073 int i;
4ed46869 10074
df7492f9
KH		 10075 subsidiaries = make_subsidiaries (alias);
10076 for (i = 0; i < 3; i++)
10077 Fdefine_coding_system_alias (AREF (subsidiaries, i),
10078 AREF (eol_type, i));
4ed46869 10079 }
df7492f9
KH		 10080
10081 Fputhash (alias, spec, Vcoding_system_hash_table);
d6925f38 10082 Vcoding_system_list = Fcons (alias, Vcoding_system_list);
583f71ca
KH		 10083 val = Fassoc (Fsymbol_name (alias), Vcoding_system_alist);
10084 if (NILP (val))
10085 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (alias), Qnil),
10086 Vcoding_system_alist);
66cfb530 10087
4ed46869
KH		 10088 return Qnil;
10089}
10090
a7ca3326 10091DEFUN ("coding-system-base", Fcoding_system_base, Scoding_system_base,
df7492f9
KH		 10092 1, 1, 0,
10093 doc: /* Return the base of CODING-SYSTEM.
da7db224 10094Any alias or subsidiary coding system is not a base coding system. */)
5842a27b 10095 (Lisp_Object coding_system)
d46c5b12 10096{
df7492f9 10097 Lisp_Object spec, attrs;
d46c5b12 10098
df7492f9
KH		 10099 if (NILP (coding_system))
10100 return (Qno_conversion);
10101 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
10102 attrs = AREF (spec, 0);
10103 return CODING_ATTR_BASE_NAME (attrs);
10104}
1397dc18 10105
df7492f9
KH		 10106DEFUN ("coding-system-plist", Fcoding_system_plist, Scoding_system_plist,
10107 1, 1, 0,
10108 doc: "Return the property list of CODING-SYSTEM.")
5842a27b 10109 (Lisp_Object coding_system)
df7492f9
KH		 10110{
10111 Lisp_Object spec, attrs;
1397dc18 10112
df7492f9
KH		 10113 if (NILP (coding_system))
10114 coding_system = Qno_conversion;
10115 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
10116 attrs = AREF (spec, 0);
10117 return CODING_ATTR_PLIST (attrs);
d46c5b12
KH		 10118}
10119
df7492f9
KH		 10120
10121DEFUN ("coding-system-aliases", Fcoding_system_aliases, Scoding_system_aliases,
10122 1, 1, 0,
da7db224 10123 doc: /* Return the list of aliases of CODING-SYSTEM. */)
5842a27b 10124 (Lisp_Object coding_system)
66cfb530 10125{
df7492f9 10126 Lisp_Object spec;
84d60297 10127
df7492f9
KH		 10128 if (NILP (coding_system))
10129 coding_system = Qno_conversion;
10130 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
da7db224 10131 return AREF (spec, 1);
df7492f9 10132}
66cfb530 10133
a7ca3326 10134DEFUN ("coding-system-eol-type", Fcoding_system_eol_type,
df7492f9
KH		 10135 Scoding_system_eol_type, 1, 1, 0,
10136 doc: /* Return eol-type of CODING-SYSTEM.
d4a1d553 10137An eol-type is an integer 0, 1, 2, or a vector of coding systems.
66cfb530 10138
df7492f9
KH		 10139Integer values 0, 1, and 2 indicate a format of end-of-line; LF, CRLF,
10140and CR respectively.
66cfb530 10141
df7492f9
KH		 10142A vector value indicates that a format of end-of-line should be
10143detected automatically. Nth element of the vector is the subsidiary
10144coding system whose eol-type is N. */)
5842a27b 10145 (Lisp_Object coding_system)
6b89e3aa 10146{
df7492f9
KH		 10147 Lisp_Object spec, eol_type;
10148 int n;
6b89e3aa 10149
df7492f9
KH		 10150 if (NILP (coding_system))
10151 coding_system = Qno_conversion;
10152 if (! CODING_SYSTEM_P (coding_system))
10153 return Qnil;
10154 spec = CODING_SYSTEM_SPEC (coding_system);
10155 eol_type = AREF (spec, 2);
10156 if (VECTORP (eol_type))
10157 return Fcopy_sequence (eol_type);
10158 n = EQ (eol_type, Qunix) ? 0 : EQ (eol_type, Qdos) ? 1 : 2;
10159 return make_number (n);
6b89e3aa
KH		 10160}
10161
4ed46869
KH		 10162#endif /* emacs */
10163
10164\f
1397dc18 10165/*** 9. Post-amble ***/
4ed46869 10166
dfcf069d 10167void
971de7fb 10168init_coding_once (void)
4ed46869
KH		 10169{
10170 int i;
10171
df7492f9
KH		 10172 for (i = 0; i < coding_category_max; i++)
10173 {
10174 coding_categories[i].id = -1;
10175 coding_priorities[i] = i;
10176 }
4ed46869
KH		 10177
10178 /* ISO2022 specific initialize routine. */
10179 for (i = 0; i < 0x20; i++)
b73bfc1c 10180 iso_code_class[i] = ISO_control_0;
4ed46869
KH		 10181 for (i = 0x21; i < 0x7F; i++)
10182 iso_code_class[i] = ISO_graphic_plane_0;
10183 for (i = 0x80; i < 0xA0; i++)
b73bfc1c 10184 iso_code_class[i] = ISO_control_1;
4ed46869
KH		 10185 for (i = 0xA1; i < 0xFF; i++)
10186 iso_code_class[i] = ISO_graphic_plane_1;
10187 iso_code_class[0x20] = iso_code_class[0x7F] = ISO_0x20_or_0x7F;
10188 iso_code_class[0xA0] = iso_code_class[0xFF] = ISO_0xA0_or_0xFF;
4ed46869
KH		 10189 iso_code_class[ISO_CODE_SO] = ISO_shift_out;
10190 iso_code_class[ISO_CODE_SI] = ISO_shift_in;
10191 iso_code_class[ISO_CODE_SS2_7] = ISO_single_shift_2_7;
10192 iso_code_class[ISO_CODE_ESC] = ISO_escape;
10193 iso_code_class[ISO_CODE_SS2] = ISO_single_shift_2;
10194 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3;
10195 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer;
10196
df7492f9
KH		 10197 for (i = 0; i < 256; i++)
10198 {
10199 emacs_mule_bytes[i] = 1;
10200 }
7c78e542
KH		 10201 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_11] = 3;
10202 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_12] = 3;
10203 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_21] = 4;
10204 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_22] = 4;
e0e989f6
KH		 10205}
10206
10207#ifdef emacs
10208
dfcf069d 10209void
971de7fb 10210syms_of_coding (void)
e0e989f6 10211{
df7492f9 10212 staticpro (&Vcoding_system_hash_table);
8f924df7
KH		 10213 {
10214 Lisp_Object args[2];
10215 args[0] = QCtest;
10216 args[1] = Qeq;
10217 Vcoding_system_hash_table = Fmake_hash_table (2, args);
10218 }
df7492f9
KH		 10219
10220 staticpro (&Vsjis_coding_system);
10221 Vsjis_coding_system = Qnil;
e0e989f6 10222
df7492f9
KH		 10223 staticpro (&Vbig5_coding_system);
10224 Vbig5_coding_system = Qnil;
10225
24a73b0a
KH		 10226 staticpro (&Vcode_conversion_reused_workbuf);
10227 Vcode_conversion_reused_workbuf = Qnil;
10228
10229 staticpro (&Vcode_conversion_workbuf_name);
d67b4f80 10230 Vcode_conversion_workbuf_name = make_pure_c_string (" *code-conversion-work*");
e0e989f6 10231
24a73b0a 10232 reused_workbuf_in_use = 0;
df7492f9
KH		 10233
10234 DEFSYM (Qcharset, "charset");
10235 DEFSYM (Qtarget_idx, "target-idx");
10236 DEFSYM (Qcoding_system_history, "coding-system-history");
bb0115a2
RS		 10237 Fset (Qcoding_system_history, Qnil);
10238
9ce27fde 10239 /* Target FILENAME is the first argument. */
e0e989f6 10240 Fput (Qinsert_file_contents, Qtarget_idx, make_number (0));
9ce27fde 10241 /* Target FILENAME is the third argument. */
e0e989f6
KH		 10242 Fput (Qwrite_region, Qtarget_idx, make_number (2));
10243
df7492f9 10244 DEFSYM (Qcall_process, "call-process");
9ce27fde 10245 /* Target PROGRAM is the first argument. */
e0e989f6
KH		 10246 Fput (Qcall_process, Qtarget_idx, make_number (0));
10247
df7492f9 10248 DEFSYM (Qcall_process_region, "call-process-region");
9ce27fde 10249 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 10250 Fput (Qcall_process_region, Qtarget_idx, make_number (2));
10251
df7492f9 10252 DEFSYM (Qstart_process, "start-process");
9ce27fde 10253 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 10254 Fput (Qstart_process, Qtarget_idx, make_number (2));
10255
df7492f9 10256 DEFSYM (Qopen_network_stream, "open-network-stream");
9ce27fde 10257 /* Target SERVICE is the fourth argument. */
e0e989f6
KH		 10258 Fput (Qopen_network_stream, Qtarget_idx, make_number (3));
10259
df7492f9
KH		 10260 DEFSYM (Qcoding_system, "coding-system");
10261 DEFSYM (Qcoding_aliases, "coding-aliases");
4ed46869 10262
df7492f9
KH		 10263 DEFSYM (Qeol_type, "eol-type");
10264 DEFSYM (Qunix, "unix");
10265 DEFSYM (Qdos, "dos");
4ed46869 10266
df7492f9
KH		 10267 DEFSYM (Qbuffer_file_coding_system, "buffer-file-coding-system");
10268 DEFSYM (Qpost_read_conversion, "post-read-conversion");
10269 DEFSYM (Qpre_write_conversion, "pre-write-conversion");
10270 DEFSYM (Qdefault_char, "default-char");
10271 DEFSYM (Qundecided, "undecided");
10272 DEFSYM (Qno_conversion, "no-conversion");
10273 DEFSYM (Qraw_text, "raw-text");
4ed46869 10274
df7492f9 10275 DEFSYM (Qiso_2022, "iso-2022");
4ed46869 10276
df7492f9 10277 DEFSYM (Qutf_8, "utf-8");
8f924df7 10278 DEFSYM (Qutf_8_emacs, "utf-8-emacs");
27901516 10279
df7492f9 10280 DEFSYM (Qutf_16, "utf-16");
df7492f9
KH		 10281 DEFSYM (Qbig, "big");
10282 DEFSYM (Qlittle, "little");
27901516 10283
df7492f9
KH		 10284 DEFSYM (Qshift_jis, "shift-jis");
10285 DEFSYM (Qbig5, "big5");
4ed46869 10286
df7492f9 10287 DEFSYM (Qcoding_system_p, "coding-system-p");
4ed46869 10288
df7492f9 10289 DEFSYM (Qcoding_system_error, "coding-system-error");
4ed46869 10290 Fput (Qcoding_system_error, Qerror_conditions,
d67b4f80 10291 pure_cons (Qcoding_system_error, pure_cons (Qerror, Qnil)));
4ed46869 10292 Fput (Qcoding_system_error, Qerror_message,
d67b4f80 10293 make_pure_c_string ("Invalid coding system"));
4ed46869 10294
05e6f5dc
KH		 10295 /* Intern this now in case it isn't already done.
10296 Setting this variable twice is harmless.
10297 But don't staticpro it here--that is done in alloc.c. */
d67b4f80 10298 Qchar_table_extra_slots = intern_c_string ("char-table-extra-slots");
70c22245 10299
df7492f9 10300 DEFSYM (Qtranslation_table, "translation-table");
433f7f87 10301 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (2));
df7492f9
KH		 10302 DEFSYM (Qtranslation_table_id, "translation-table-id");
10303 DEFSYM (Qtranslation_table_for_decode, "translation-table-for-decode");
10304 DEFSYM (Qtranslation_table_for_encode, "translation-table-for-encode");
1397dc18 10305
df7492f9 10306 DEFSYM (Qvalid_codes, "valid-codes");
9ce27fde 10307
df7492f9 10308 DEFSYM (Qemacs_mule, "emacs-mule");
d46c5b12 10309
01378f49 10310 DEFSYM (QCcategory, ":category");
a6f87d34 10311 DEFSYM (QCmnemonic, ":mnemonic");
2133e2d1 10312 DEFSYM (QCdefault_char, ":default-char");
a6f87d34
KH		 10313 DEFSYM (QCdecode_translation_table, ":decode-translation-table");
10314 DEFSYM (QCencode_translation_table, ":encode-translation-table");
10315 DEFSYM (QCpost_read_conversion, ":post-read-conversion");
10316 DEFSYM (QCpre_write_conversion, ":pre-write-conversion");
35befdaa 10317 DEFSYM (QCascii_compatible_p, ":ascii-compatible-p");
01378f49 10318
df7492f9
KH		 10319 Vcoding_category_table
10320 = Fmake_vector (make_number (coding_category_max), Qnil);
10321 staticpro (&Vcoding_category_table);
10322 /* Followings are target of code detection. */
10323 ASET (Vcoding_category_table, coding_category_iso_7,
d67b4f80 10324 intern_c_string ("coding-category-iso-7"));
df7492f9 10325 ASET (Vcoding_category_table, coding_category_iso_7_tight,
d67b4f80 10326 intern_c_string ("coding-category-iso-7-tight"));
df7492f9 10327 ASET (Vcoding_category_table, coding_category_iso_8_1,
d67b4f80 10328 intern_c_string ("coding-category-iso-8-1"));
df7492f9 10329 ASET (Vcoding_category_table, coding_category_iso_8_2,
d67b4f80 10330 intern_c_string ("coding-category-iso-8-2"));
df7492f9 10331 ASET (Vcoding_category_table, coding_category_iso_7_else,
d67b4f80 10332 intern_c_string ("coding-category-iso-7-else"));
df7492f9 10333 ASET (Vcoding_category_table, coding_category_iso_8_else,
d67b4f80 10334 intern_c_string ("coding-category-iso-8-else"));
a470d443 10335 ASET (Vcoding_category_table, coding_category_utf_8_auto,
d67b4f80 10336 intern_c_string ("coding-category-utf-8-auto"));
a470d443 10337 ASET (Vcoding_category_table, coding_category_utf_8_nosig,
d67b4f80 10338 intern_c_string ("coding-category-utf-8"));
a470d443 10339 ASET (Vcoding_category_table, coding_category_utf_8_sig,
d67b4f80 10340 intern_c_string ("coding-category-utf-8-sig"));
df7492f9 10341 ASET (Vcoding_category_table, coding_category_utf_16_be,
d67b4f80 10342 intern_c_string ("coding-category-utf-16-be"));
ff563fce 10343 ASET (Vcoding_category_table, coding_category_utf_16_auto,
d67b4f80 10344 intern_c_string ("coding-category-utf-16-auto"));
df7492f9 10345 ASET (Vcoding_category_table, coding_category_utf_16_le,
d67b4f80 10346 intern_c_string ("coding-category-utf-16-le"));
df7492f9 10347 ASET (Vcoding_category_table, coding_category_utf_16_be_nosig,
d67b4f80 10348 intern_c_string ("coding-category-utf-16-be-nosig"));
df7492f9 10349 ASET (Vcoding_category_table, coding_category_utf_16_le_nosig,
d67b4f80 10350 intern_c_string ("coding-category-utf-16-le-nosig"));
df7492f9 10351 ASET (Vcoding_category_table, coding_category_charset,
d67b4f80 10352 intern_c_string ("coding-category-charset"));
df7492f9 10353 ASET (Vcoding_category_table, coding_category_sjis,
d67b4f80 10354 intern_c_string ("coding-category-sjis"));
df7492f9 10355 ASET (Vcoding_category_table, coding_category_big5,
d67b4f80 10356 intern_c_string ("coding-category-big5"));
df7492f9 10357 ASET (Vcoding_category_table, coding_category_ccl,
d67b4f80 10358 intern_c_string ("coding-category-ccl"));
df7492f9 10359 ASET (Vcoding_category_table, coding_category_emacs_mule,
d67b4f80 10360 intern_c_string ("coding-category-emacs-mule"));
df7492f9
KH		 10361 /* Followings are NOT target of code detection. */
10362 ASET (Vcoding_category_table, coding_category_raw_text,
d67b4f80 10363 intern_c_string ("coding-category-raw-text"));
df7492f9 10364 ASET (Vcoding_category_table, coding_category_undecided,
d67b4f80 10365 intern_c_string ("coding-category-undecided"));
ecf488bc 10366
065e3595
KH		 10367 DEFSYM (Qinsufficient_source, "insufficient-source");
10368 DEFSYM (Qinconsistent_eol, "inconsistent-eol");
10369 DEFSYM (Qinvalid_source, "invalid-source");
10370 DEFSYM (Qinterrupted, "interrupted");
10371 DEFSYM (Qinsufficient_memory, "insufficient-memory");
44e8490d 10372 DEFSYM (Qcoding_system_define_form, "coding-system-define-form");
065e3595 10373
4ed46869
KH		 10374 defsubr (&Scoding_system_p);
10375 defsubr (&Sread_coding_system);
10376 defsubr (&Sread_non_nil_coding_system);
10377 defsubr (&Scheck_coding_system);
10378 defsubr (&Sdetect_coding_region);
d46c5b12 10379 defsubr (&Sdetect_coding_string);
05e6f5dc 10380 defsubr (&Sfind_coding_systems_region_internal);
068a9dbd 10381 defsubr (&Sunencodable_char_position);
df7492f9 10382 defsubr (&Scheck_coding_systems_region);
4ed46869
KH		 10383 defsubr (&Sdecode_coding_region);
10384 defsubr (&Sencode_coding_region);
10385 defsubr (&Sdecode_coding_string);
10386 defsubr (&Sencode_coding_string);
10387 defsubr (&Sdecode_sjis_char);
10388 defsubr (&Sencode_sjis_char);
10389 defsubr (&Sdecode_big5_char);
10390 defsubr (&Sencode_big5_char);
1ba9e4ab 10391 defsubr (&Sset_terminal_coding_system_internal);
c4825358 10392 defsubr (&Sset_safe_terminal_coding_system_internal);
4ed46869 10393 defsubr (&Sterminal_coding_system);
1ba9e4ab 10394 defsubr (&Sset_keyboard_coding_system_internal);
4ed46869 10395 defsubr (&Skeyboard_coding_system);
a5d301df 10396 defsubr (&Sfind_operation_coding_system);
df7492f9 10397 defsubr (&Sset_coding_system_priority);
6b89e3aa 10398 defsubr (&Sdefine_coding_system_internal);
df7492f9 10399 defsubr (&Sdefine_coding_system_alias);
a6f87d34 10400 defsubr (&Scoding_system_put);
df7492f9
KH		 10401 defsubr (&Scoding_system_base);
10402 defsubr (&Scoding_system_plist);
10403 defsubr (&Scoding_system_aliases);
10404 defsubr (&Scoding_system_eol_type);
10405 defsubr (&Scoding_system_priority_list);
4ed46869 10406
29208e82 10407 DEFVAR_LISP ("coding-system-list", Vcoding_system_list,
48b0f3ae
PJ		 10408 doc: /* List of coding systems.
10409
10410Do not alter the value of this variable manually. This variable should be
df7492f9 10411updated by the functions `define-coding-system' and
48b0f3ae 10412`define-coding-system-alias'. */);
4608c386
KH		 10413 Vcoding_system_list = Qnil;
10414
29208e82 10415 DEFVAR_LISP ("coding-system-alist", Vcoding_system_alist,
48b0f3ae
PJ		 10416 doc: /* Alist of coding system names.
10417Each element is one element list of coding system name.
446dcd75 10418This variable is given to `completing-read' as COLLECTION argument.
48b0f3ae
PJ		 10419
10420Do not alter the value of this variable manually. This variable should be
10421updated by the functions `make-coding-system' and
10422`define-coding-system-alias'. */);
4608c386
KH		 10423 Vcoding_system_alist = Qnil;
10424
29208e82 10425 DEFVAR_LISP ("coding-category-list", Vcoding_category_list,
48b0f3ae
PJ		 10426 doc: /* List of coding-categories (symbols) ordered by priority.
10427
10428On detecting a coding system, Emacs tries code detection algorithms
10429associated with each coding-category one by one in this order. When
10430one algorithm agrees with a byte sequence of source text, the coding
0ec31faf
KH		 10431system bound to the corresponding coding-category is selected.
10432
448e17d6 10433Don't modify this variable directly, but use `set-coding-system-priority'. */);
4ed46869
KH		 10434 {
10435 int i;
10436
10437 Vcoding_category_list = Qnil;
df7492f9 10438 for (i = coding_category_max - 1; i >= 0; i--)
4ed46869 10439 Vcoding_category_list
d46c5b12
KH		 10440 = Fcons (XVECTOR (Vcoding_category_table)->contents[i],
10441 Vcoding_category_list);
4ed46869
KH		 10442 }
10443
29208e82 10444 DEFVAR_LISP ("coding-system-for-read", Vcoding_system_for_read,
48b0f3ae
PJ		 10445 doc: /* Specify the coding system for read operations.
10446It is useful to bind this variable with `let', but do not set it globally.
10447If the value is a coding system, it is used for decoding on read operation.
446dcd75
JB		 10448If not, an appropriate element is used from one of the coding system alists.
10449There are three such tables: `file-coding-system-alist',
48b0f3ae 10450`process-coding-system-alist', and `network-coding-system-alist'. */);
4ed46869
KH		 10451 Vcoding_system_for_read = Qnil;
10452
29208e82 10453 DEFVAR_LISP ("coding-system-for-write", Vcoding_system_for_write,
48b0f3ae
PJ		 10454 doc: /* Specify the coding system for write operations.
10455Programs bind this variable with `let', but you should not set it globally.
10456If the value is a coding system, it is used for encoding of output,
10457when writing it to a file and when sending it to a file or subprocess.
10458
10459If this does not specify a coding system, an appropriate element
446dcd75
JB		 10460is used from one of the coding system alists.
10461There are three such tables: `file-coding-system-alist',
48b0f3ae
PJ		 10462`process-coding-system-alist', and `network-coding-system-alist'.
10463For output to files, if the above procedure does not specify a coding system,
10464the value of `buffer-file-coding-system' is used. */);
4ed46869
KH		 10465 Vcoding_system_for_write = Qnil;
10466
29208e82 10467 DEFVAR_LISP ("last-coding-system-used", Vlast_coding_system_used,
df7492f9
KH		 10468 doc: /*
10469Coding system used in the latest file or process I/O. */);
4ed46869
KH		 10470 Vlast_coding_system_used = Qnil;
10471
29208e82 10472 DEFVAR_LISP ("last-code-conversion-error", Vlast_code_conversion_error,
065e3595
KH		 10473 doc: /*
10474Error status of the last code conversion.
10475
10476When an error was detected in the last code conversion, this variable
10477is set to one of the following symbols.
10478 `insufficient-source'
10479 `inconsistent-eol'
10480 `invalid-source'
10481 `interrupted'
10482 `insufficient-memory'
10483When no error was detected, the value doesn't change. So, to check
10484the error status of a code conversion by this variable, you must
10485explicitly set this variable to nil before performing code
10486conversion. */);
10487 Vlast_code_conversion_error = Qnil;
10488
29208e82 10489 DEFVAR_BOOL ("inhibit-eol-conversion", inhibit_eol_conversion,
df7492f9
KH		 10490 doc: /*
10491*Non-nil means always inhibit code conversion of end-of-line format.
48b0f3ae
PJ		 10492See info node `Coding Systems' and info node `Text and Binary' concerning
10493such conversion. */);
9ce27fde
KH		 10494 inhibit_eol_conversion = 0;
10495
29208e82 10496 DEFVAR_BOOL ("inherit-process-coding-system", inherit_process_coding_system,
df7492f9
KH		 10497 doc: /*
10498Non-nil means process buffer inherits coding system of process output.
48b0f3ae
PJ		 10499Bind it to t if the process output is to be treated as if it were a file
10500read from some filesystem. */);
ed29121d
EZ		 10501 inherit_process_coding_system = 0;
10502
29208e82 10503 DEFVAR_LISP ("file-coding-system-alist", Vfile_coding_system_alist,
df7492f9
KH		 10504 doc: /*
10505Alist to decide a coding system to use for a file I/O operation.
48b0f3ae
PJ		 10506The format is ((PATTERN . VAL) ...),
10507where PATTERN is a regular expression matching a file name,
10508VAL is a coding system, a cons of coding systems, or a function symbol.
10509If VAL is a coding system, it is used for both decoding and encoding
10510the file contents.
10511If VAL is a cons of coding systems, the car part is used for decoding,
10512and the cdr part is used for encoding.
10513If VAL is a function symbol, the function must return a coding system
2c53e699
KH		 10514or a cons of coding systems which are used as above. The function is
10515called with an argument that is a list of the arguments with which
5a0bbd9a
KH		 10516`find-operation-coding-system' was called. If the function can't decide
10517a coding system, it can return `undecided' so that the normal
10518code-detection is performed.
48b0f3ae
PJ		 10519
10520See also the function `find-operation-coding-system'
10521and the variable `auto-coding-alist'. */);
02ba4723
KH		 10522 Vfile_coding_system_alist = Qnil;
10523
29208e82 10524 DEFVAR_LISP ("process-coding-system-alist", Vprocess_coding_system_alist,
df7492f9
KH		 10525 doc: /*
10526Alist to decide a coding system to use for a process I/O operation.
48b0f3ae
PJ		 10527The format is ((PATTERN . VAL) ...),
10528where PATTERN is a regular expression matching a program name,
10529VAL is a coding system, a cons of coding systems, or a function symbol.
10530If VAL is a coding system, it is used for both decoding what received
10531from the program and encoding what sent to the program.
10532If VAL is a cons of coding systems, the car part is used for decoding,
10533and the cdr part is used for encoding.
10534If VAL is a function symbol, the function must return a coding system
10535or a cons of coding systems which are used as above.
10536
10537See also the function `find-operation-coding-system'. */);
02ba4723
KH		 10538 Vprocess_coding_system_alist = Qnil;
10539
29208e82 10540 DEFVAR_LISP ("network-coding-system-alist", Vnetwork_coding_system_alist,
df7492f9
KH		 10541 doc: /*
10542Alist to decide a coding system to use for a network I/O operation.
48b0f3ae
PJ		 10543The format is ((PATTERN . VAL) ...),
10544where PATTERN is a regular expression matching a network service name
10545or is a port number to connect to,
10546VAL is a coding system, a cons of coding systems, or a function symbol.
10547If VAL is a coding system, it is used for both decoding what received
10548from the network stream and encoding what sent to the network stream.
10549If VAL is a cons of coding systems, the car part is used for decoding,
10550and the cdr part is used for encoding.
10551If VAL is a function symbol, the function must return a coding system
10552or a cons of coding systems which are used as above.
10553
10554See also the function `find-operation-coding-system'. */);
02ba4723 10555 Vnetwork_coding_system_alist = Qnil;
4ed46869 10556
29208e82 10557 DEFVAR_LISP ("locale-coding-system", Vlocale_coding_system,
75205970
RS		 10558 doc: /* Coding system to use with system messages.
10559Also used for decoding keyboard input on X Window system. */);
68c45bf0
PE		 10560 Vlocale_coding_system = Qnil;
10561
005f0d35 10562 /* The eol mnemonics are reset in startup.el system-dependently. */
29208e82 10563 DEFVAR_LISP ("eol-mnemonic-unix", eol_mnemonic_unix,
df7492f9
KH		 10564 doc: /*
10565*String displayed in mode line for UNIX-like (LF) end-of-line format. */);
d67b4f80 10566 eol_mnemonic_unix = make_pure_c_string (":");
4ed46869 10567
29208e82 10568 DEFVAR_LISP ("eol-mnemonic-dos", eol_mnemonic_dos,
df7492f9
KH		 10569 doc: /*
10570*String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
d67b4f80 10571 eol_mnemonic_dos = make_pure_c_string ("\\");
4ed46869 10572
29208e82 10573 DEFVAR_LISP ("eol-mnemonic-mac", eol_mnemonic_mac,
df7492f9
KH		 10574 doc: /*
10575*String displayed in mode line for MAC-like (CR) end-of-line format. */);
d67b4f80 10576 eol_mnemonic_mac = make_pure_c_string ("/");
4ed46869 10577
29208e82 10578 DEFVAR_LISP ("eol-mnemonic-undecided", eol_mnemonic_undecided,
df7492f9
KH		 10579 doc: /*
10580*String displayed in mode line when end-of-line format is not yet determined. */);
d67b4f80 10581 eol_mnemonic_undecided = make_pure_c_string (":");
4ed46869 10582
29208e82 10583 DEFVAR_LISP ("enable-character-translation", Venable_character_translation,
df7492f9
KH		 10584 doc: /*
10585*Non-nil enables character translation while encoding and decoding. */);
84fbb8a0 10586 Venable_character_translation = Qt;
bdd9fb48 10587
f967223b 10588 DEFVAR_LISP ("standard-translation-table-for-decode",
29208e82 10589 Vstandard_translation_table_for_decode,
48b0f3ae 10590 doc: /* Table for translating characters while decoding. */);
f967223b 10591 Vstandard_translation_table_for_decode = Qnil;
bdd9fb48 10592
f967223b 10593 DEFVAR_LISP ("standard-translation-table-for-encode",
29208e82 10594 Vstandard_translation_table_for_encode,
48b0f3ae 10595 doc: /* Table for translating characters while encoding. */);
f967223b 10596 Vstandard_translation_table_for_encode = Qnil;
4ed46869 10597
29208e82 10598 DEFVAR_LISP ("charset-revision-table", Vcharset_revision_table,
48b0f3ae
PJ		 10599 doc: /* Alist of charsets vs revision numbers.
10600While encoding, if a charset (car part of an element) is found,
df7492f9
KH		 10601designate it with the escape sequence identifying revision (cdr part
10602of the element). */);
10603 Vcharset_revision_table = Qnil;
02ba4723
KH		 10604
10605 DEFVAR_LISP ("default-process-coding-system",
29208e82 10606 Vdefault_process_coding_system,
48b0f3ae
PJ		 10607 doc: /* Cons of coding systems used for process I/O by default.
10608The car part is used for decoding a process output,
10609the cdr part is used for encoding a text to be sent to a process. */);
02ba4723 10610 Vdefault_process_coding_system = Qnil;
c4825358 10611
29208e82 10612 DEFVAR_LISP ("latin-extra-code-table", Vlatin_extra_code_table,
df7492f9
KH		 10613 doc: /*
10614Table of extra Latin codes in the range 128..159 (inclusive).
48b0f3ae
PJ		 10615This is a vector of length 256.
10616If Nth element is non-nil, the existence of code N in a file
10617\(or output of subprocess) doesn't prevent it to be detected as
10618a coding system of ISO 2022 variant which has a flag
10619`accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
10620or reading output of a subprocess.
446dcd75 10621Only 128th through 159th elements have a meaning. */);
3f003981 10622 Vlatin_extra_code_table = Fmake_vector (make_number (256), Qnil);
d46c5b12
KH		 10623
10624 DEFVAR_LISP ("select-safe-coding-system-function",
29208e82 10625 Vselect_safe_coding_system_function,
df7492f9
KH		 10626 doc: /*
10627Function to call to select safe coding system for encoding a text.
48b0f3ae
PJ		 10628
10629If set, this function is called to force a user to select a proper
10630coding system which can encode the text in the case that a default
fdecf907
GM		 10631coding system used in each operation can't encode the text. The
10632function should take care that the buffer is not modified while
10633the coding system is being selected.
48b0f3ae
PJ		 10634
10635The default value is `select-safe-coding-system' (which see). */);
d46c5b12
KH		 10636 Vselect_safe_coding_system_function = Qnil;
10637
5d5bf4d8 10638 DEFVAR_BOOL ("coding-system-require-warning",
29208e82 10639 coding_system_require_warning,
5d5bf4d8 10640 doc: /* Internal use only.
6b89e3aa
KH		 10641If non-nil, on writing a file, `select-safe-coding-system-function' is
10642called even if `coding-system-for-write' is non-nil. The command
10643`universal-coding-system-argument' binds this variable to t temporarily. */);
5d5bf4d8
KH		 10644 coding_system_require_warning = 0;
10645
10646
22ab2303 10647 DEFVAR_BOOL ("inhibit-iso-escape-detection",
29208e82 10648 inhibit_iso_escape_detection,
df7492f9 10649 doc: /*
97b1b294 10650If non-nil, Emacs ignores ISO-2022 escape sequences during code detection.
48b0f3ae 10651
97b1b294
EZ		 10652When Emacs reads text, it tries to detect how the text is encoded.
10653This code detection is sensitive to escape sequences. If Emacs sees
10654a valid ISO-2022 escape sequence, it assumes the text is encoded in one
10655of the ISO2022 encodings, and decodes text by the corresponding coding
10656system (e.g. `iso-2022-7bit').
48b0f3ae
PJ		 10657
10658However, there may be a case that you want to read escape sequences in
10659a file as is. In such a case, you can set this variable to non-nil.
97b1b294
EZ		 10660Then the code detection will ignore any escape sequences, and no text is
10661detected as encoded in some ISO-2022 encoding. The result is that all
48b0f3ae
PJ		 10662escape sequences become visible in a buffer.
10663
10664The default value is nil, and it is strongly recommended not to change
10665it. That is because many Emacs Lisp source files that contain
10666non-ASCII characters are encoded by the coding system `iso-2022-7bit'
10667in Emacs's distribution, and they won't be decoded correctly on
10668reading if you suppress escape sequence detection.
10669
10670The other way to read escape sequences in a file without decoding is
97b1b294 10671to explicitly specify some coding system that doesn't use ISO-2022
48b0f3ae 10672escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
74383408 10673 inhibit_iso_escape_detection = 0;
002fdb44 10674
97b1b294 10675 DEFVAR_BOOL ("inhibit-null-byte-detection",
29208e82 10676 inhibit_null_byte_detection,
97b1b294
EZ		 10677 doc: /* If non-nil, Emacs ignores null bytes on code detection.
10678By default, Emacs treats it as binary data, and does not attempt to
10679decode it. The effect is as if you specified `no-conversion' for
10680reading that text.
10681
10682Set this to non-nil when a regular text happens to include null bytes.
10683Examples are Index nodes of Info files and null-byte delimited output
10684from GNU Find and GNU Grep. Emacs will then ignore the null bytes and
10685decode text as usual. */);
10686 inhibit_null_byte_detection = 0;
10687
29208e82 10688 DEFVAR_LISP ("translation-table-for-input", Vtranslation_table_for_input,
15c8f9d1 10689 doc: /* Char table for translating self-inserting characters.
446dcd75 10690This is applied to the result of input methods, not their input.
8434d0b8
EZ		 10691See also `keyboard-translate-table'.
10692
10693Use of this variable for character code unification was rendered
10694obsolete in Emacs 23.1 and later, since Unicode is now the basis of
10695internal character representation. */);
002fdb44 10696 Vtranslation_table_for_input = Qnil;
8f924df7 10697
2c78b7e1
KH		 10698 {
10699 Lisp_Object args[coding_arg_max];
8f924df7 10700 Lisp_Object plist[16];
2c78b7e1
KH		 10701 int i;
10702
10703 for (i = 0; i < coding_arg_max; i++)
10704 args[i] = Qnil;
10705
d67b4f80 10706 plist[0] = intern_c_string (":name");
2c78b7e1 10707 plist[1] = args[coding_arg_name] = Qno_conversion;
d67b4f80 10708 plist[2] = intern_c_string (":mnemonic");
2c78b7e1 10709 plist[3] = args[coding_arg_mnemonic] = make_number ('=');
d67b4f80 10710 plist[4] = intern_c_string (":coding-type");
2c78b7e1 10711 plist[5] = args[coding_arg_coding_type] = Qraw_text;
d67b4f80 10712 plist[6] = intern_c_string (":ascii-compatible-p");
2c78b7e1 10713 plist[7] = args[coding_arg_ascii_compatible_p] = Qt;
d67b4f80 10714 plist[8] = intern_c_string (":default-char");
2c78b7e1 10715 plist[9] = args[coding_arg_default_char] = make_number (0);
d67b4f80 10716 plist[10] = intern_c_string (":for-unibyte");
8f924df7 10717 plist[11] = args[coding_arg_for_unibyte] = Qt;
d67b4f80
DN		 10718 plist[12] = intern_c_string (":docstring");
10719 plist[13] = make_pure_c_string ("Do no conversion.\n\
2c78b7e1
KH		 10720\n\
10721When you visit a file with this coding, the file is read into a\n\
10722unibyte buffer as is, thus each byte of a file is treated as a\n\
10723character.");
d67b4f80 10724 plist[14] = intern_c_string (":eol-type");
8f924df7
KH		 10725 plist[15] = args[coding_arg_eol_type] = Qunix;
10726 args[coding_arg_plist] = Flist (16, plist);
2c78b7e1 10727 Fdefine_coding_system_internal (coding_arg_max, args);
ae6f73fa
KH		 10728
10729 plist[1] = args[coding_arg_name] = Qundecided;
10730 plist[3] = args[coding_arg_mnemonic] = make_number ('-');
10731 plist[5] = args[coding_arg_coding_type] = Qundecided;
10732 /* This is already set.
35befdaa 10733 plist[7] = args[coding_arg_ascii_compatible_p] = Qt; */
d67b4f80 10734 plist[8] = intern_c_string (":charset-list");
ae6f73fa
KH		 10735 plist[9] = args[coding_arg_charset_list] = Fcons (Qascii, Qnil);
10736 plist[11] = args[coding_arg_for_unibyte] = Qnil;
d67b4f80 10737 plist[13] = make_pure_c_string ("No conversion on encoding, automatic conversion on decoding.");
ae6f73fa
KH		 10738 plist[15] = args[coding_arg_eol_type] = Qnil;
10739 args[coding_arg_plist] = Flist (16, plist);
10740 Fdefine_coding_system_internal (coding_arg_max, args);
2c78b7e1
KH		 10741 }
10742
2c78b7e1 10743 setup_coding_system (Qno_conversion, &safe_terminal_coding);
ff563fce
KH		 10744
10745 {
10746 int i;
10747
10748 for (i = 0; i < coding_category_max; i++)
10749 Fset (AREF (Vcoding_category_table, i), Qno_conversion);
10750 }
1a4990fb 10751#if defined (DOS_NT)
fcbcfb64
KH		 10752 system_eol_type = Qdos;
10753#else
10754 system_eol_type = Qunix;
10755#endif
10756 staticpro (&system_eol_type);
4ed46869
KH		 10757}
10758
68c45bf0 10759char *
971de7fb 10760emacs_strerror (int error_number)
68c45bf0
PE		 10761{
10762 char *str;
10763
ca9c0567 10764 synchronize_system_messages_locale ();
68c45bf0
PE		 10765 str = strerror (error_number);
10766
10767 if (! NILP (Vlocale_coding_system))
10768 {
10769 Lisp_Object dec = code_convert_string_norecord (build_string (str),
10770 Vlocale_coding_system,
10771 0);
51b59d79 10772 str = SSDATA (dec);
68c45bf0
PE		 10773 }
10774
10775 return str;
10776}
10777
4ed46869 10778#endif /* emacs */
Unnamed repository; edit this file 'description' to name the repository.