HCoop / bpt / emacs.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
(decode_coding): Fix handling of invalid bytes.
[bpt/emacs.git] / src / coding.c
CommitLineData
9542cb1f 1/* Coding system handler (conversion, detection, etc).
4a2f9c6a 2 Copyright (C) 1995, 1997, 1998 Electrotechnical Laboratory, JAPAN.
8f924df7 3 Licensed to the Free Software Foundation.
0b5538bd 4 Copyright (C) 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc.
8f924df7 5 Copyright (C) 2003
df7492f9
KH		 6 National Institute of Advanced Industrial Science and Technology (AIST)
7 Registration Number H13PRO009
4ed46869 8
369314dc
KH		 9This file is part of GNU Emacs.
10
11GNU Emacs is free software; you can redistribute it and/or modify
12it under the terms of the GNU General Public License as published by
13the Free Software Foundation; either version 2, or (at your option)
14any later version.
4ed46869 15
369314dc
KH		 16GNU Emacs is distributed in the hope that it will be useful,
17but WITHOUT ANY WARRANTY; without even the implied warranty of
18MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19GNU General Public License for more details.
4ed46869 20
369314dc
KH		 21You should have received a copy of the GNU General Public License
22along with GNU Emacs; see the file COPYING. If not, write to
4fc5845f
LK		 23the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
24Boston, MA 02110-1301, USA. */
4ed46869
KH		 25
26/*** TABLE OF CONTENTS ***
27
b73bfc1c 28 0. General comments
4ed46869 29 1. Preamble
df7492f9
KH		 30 2. Emacs' internal format (emacs-utf-8) handlers
31 3. UTF-8 handlers
32 4. UTF-16 handlers
33 5. Charset-base coding systems handlers
34 6. emacs-mule (old Emacs' internal format) handlers
35 7. ISO2022 handlers
36 8. Shift-JIS and BIG5 handlers
37 9. CCL handlers
38 10. C library functions
39 11. Emacs Lisp library functions
40 12. Postamble
4ed46869
KH		 41
42*/
43
df7492f9 44/*** 0. General comments ***
b73bfc1c
KH		 45
46
df7492f9 47CODING SYSTEM
4ed46869 48
5bad0796
DL		 49 A coding system is an object for an encoding mechanism that contains
50 information about how to convert byte sequences to character
e19c3639
KH		 51 sequences and vice versa. When we say "decode", it means converting
52 a byte sequence of a specific coding system into a character
53 sequence that is represented by Emacs' internal coding system
54 `emacs-utf-8', and when we say "encode", it means converting a
55 character sequence of emacs-utf-8 to a byte sequence of a specific
0ef69138 56 coding system.
4ed46869 57
e19c3639
KH		 58 In Emacs Lisp, a coding system is represented by a Lisp symbol. In
59 C level, a coding system is represented by a vector of attributes
5bad0796 60 stored in the hash table Vcharset_hash_table. The conversion from
e19c3639
KH		 61 coding system symbol to attributes vector is done by looking up
62 Vcharset_hash_table by the symbol.
4ed46869 63
e19c3639 64 Coding systems are classified into the following types depending on
5bad0796 65 the encoding mechanism. Here's a brief description of the types.
4ed46869 66
df7492f9
KH		 67 o UTF-8
68
69 o UTF-16
70
71 o Charset-base coding system
72
73 A coding system defined by one or more (coded) character sets.
5bad0796 74 Decoding and encoding are done by a code converter defined for each
df7492f9
KH		 75 character set.
76
5bad0796 77 o Old Emacs internal format (emacs-mule)
df7492f9 78
5bad0796 79 The coding system adopted by old versions of Emacs (20 and 21).
4ed46869 80
df7492f9 81 o ISO2022-base coding system
4ed46869
KH		 82
83 The most famous coding system for multiple character sets. X's
df7492f9
KH		 84 Compound Text, various EUCs (Extended Unix Code), and coding systems
85 used in the Internet communication such as ISO-2022-JP are all
86 variants of ISO2022.
4ed46869 87
df7492f9 88 o SJIS (or Shift-JIS or MS-Kanji-Code)
93dec019 89
4ed46869
KH		 90 A coding system to encode character sets: ASCII, JISX0201, and
91 JISX0208. Widely used for PC's in Japan. Details are described in
df7492f9 92 section 8.
4ed46869 93
df7492f9 94 o BIG5
4ed46869 95
df7492f9 96 A coding system to encode character sets: ASCII and Big5. Widely
cfb43547 97 used for Chinese (mainly in Taiwan and Hong Kong). Details are
df7492f9
KH		 98 described in section 8. In this file, when we write "big5" (all
99 lowercase), we mean the coding system, and when we write "Big5"
100 (capitalized), we mean the character set.
4ed46869 101
df7492f9 102 o CCL
27901516 103
5bad0796 104 If a user wants to decode/encode text encoded in a coding system
df7492f9
KH		 105 not listed above, he can supply a decoder and an encoder for it in
106 CCL (Code Conversion Language) programs. Emacs executes the CCL
107 program while decoding/encoding.
27901516 108
df7492f9 109 o Raw-text
4ed46869 110
5a936b46 111 A coding system for text containing raw eight-bit data. Emacs
5bad0796 112 treats each byte of source text as a character (except for
df7492f9 113 end-of-line conversion).
4ed46869 114
df7492f9
KH		 115 o No-conversion
116
117 Like raw text, but don't do end-of-line conversion.
4ed46869 118
4ed46869 119
df7492f9 120END-OF-LINE FORMAT
4ed46869 121
5bad0796 122 How text end-of-line is encoded depends on operating system. For
df7492f9 123 instance, Unix's format is just one byte of LF (line-feed) code,
f4dee582 124 whereas DOS's format is two-byte sequence of `carriage-return' and
d46c5b12
KH		 125 `line-feed' codes. MacOS's format is usually one byte of
126 `carriage-return'.
4ed46869 127
cfb43547 128 Since text character encoding and end-of-line encoding are
df7492f9
KH		 129 independent, any coding system described above can take any format
130 of end-of-line (except for no-conversion).
4ed46869 131
e19c3639
KH		 132STRUCT CODING_SYSTEM
133
134 Before using a coding system for code conversion (i.e. decoding and
135 encoding), we setup a structure of type `struct coding_system'.
136 This structure keeps various information about a specific code
5bad0796 137 conversion (e.g. the location of source and destination data).
4ed46869
KH		 138
139*/
140
df7492f9
KH		 141/* COMMON MACROS */
142
143
4ed46869
KH		 144/*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
145
df7492f9 146 These functions check if a byte sequence specified as a source in
ff0dacd7
KH		 147 CODING conforms to the format of XXX, and update the members of
148 DETECT_INFO.
df7492f9 149
ff0dacd7 150 Return 1 if the byte sequence conforms to XXX, otherwise return 0.
df7492f9
KH		 151
152 Below is the template of these functions. */
153
4ed46869 154#if 0
df7492f9 155static int
ff0dacd7 156detect_coding_XXX (coding, detect_info)
df7492f9 157 struct coding_system *coding;
ff0dacd7 158 struct coding_detection_info *detect_info;
4ed46869 159{
f1d34bca
MB		 160 const unsigned char *src = coding->source;
161 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 162 int multibytep = coding->src_multibyte;
ff0dacd7 163 int consumed_chars = 0;
df7492f9
KH		 164 int found = 0;
165 ...;
166
167 while (1)
168 {
169 /* Get one byte from the source. If the souce is exausted, jump
170 to no_more_source:. */
171 ONE_MORE_BYTE (c);
ff0dacd7
KH		 172
173 if (! __C_conforms_to_XXX___ (c))
174 break;
175 if (! __C_strongly_suggests_XXX__ (c))
176 found = CATEGORY_MASK_XXX;
df7492f9 177 }
ff0dacd7
KH		 178 /* The byte sequence is invalid for XXX. */
179 detect_info->rejected |= CATEGORY_MASK_XXX;
df7492f9 180 return 0;
ff0dacd7 181
df7492f9 182 no_more_source:
ff0dacd7
KH		 183 /* The source exausted successfully. */
184 detect_info->found |= found;
df7492f9 185 return 1;
4ed46869
KH		 186}
187#endif
188
189/*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
190
df7492f9
KH		 191 These functions decode a byte sequence specified as a source by
192 CODING. The resulting multibyte text goes to a place pointed to by
193 CODING->charbuf, the length of which should not exceed
194 CODING->charbuf_size;
d46c5b12 195
df7492f9
KH		 196 These functions set the information of original and decoded texts in
197 CODING->consumed, CODING->consumed_char, and CODING->charbuf_used.
198 They also set CODING->result to one of CODING_RESULT_XXX indicating
199 how the decoding is finished.
d46c5b12 200
df7492f9 201 Below is the template of these functions. */
d46c5b12 202
4ed46869 203#if 0
b73bfc1c 204static void
df7492f9 205decode_coding_XXXX (coding)
4ed46869 206 struct coding_system *coding;
4ed46869 207{
f1d34bca
MB		 208 const unsigned char *src = coding->source + coding->consumed;
209 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 210 /* SRC_BASE remembers the start position in source in each loop.
211 The loop will be exited when there's not enough source code, or
212 when there's no room in CHARBUF for a decoded character. */
f1d34bca 213 const unsigned char *src_base;
df7492f9 214 /* A buffer to produce decoded characters. */
69a80ea3
KH		 215 int *charbuf = coding->charbuf + coding->charbuf_used;
216 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 217 int multibytep = coding->src_multibyte;
218
219 while (1)
220 {
221 src_base = src;
222 if (charbuf < charbuf_end)
223 /* No more room to produce a decoded character. */
224 break;
225 ONE_MORE_BYTE (c);
226 /* Decode it. */
227 }
228
229 no_more_source:
230 if (src_base < src_end
231 && coding->mode & CODING_MODE_LAST_BLOCK)
232 /* If the source ends by partial bytes to construct a character,
233 treat them as eight-bit raw data. */
234 while (src_base < src_end && charbuf < charbuf_end)
235 *charbuf++ = *src_base++;
236 /* Remember how many bytes and characters we consumed. If the
237 source is multibyte, the bytes and chars are not identical. */
238 coding->consumed = coding->consumed_char = src_base - coding->source;
239 /* Remember how many characters we produced. */
240 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 241}
242#endif
243
244/*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
245
df7492f9
KH		 246 These functions encode SRC_BYTES length text at SOURCE of Emacs'
247 internal multibyte format by CODING. The resulting byte sequence
b73bfc1c
KH		 248 goes to a place pointed to by DESTINATION, the length of which
249 should not exceed DST_BYTES.
d46c5b12 250
df7492f9
KH		 251 These functions set the information of original and encoded texts in
252 the members produced, produced_char, consumed, and consumed_char of
253 the structure *CODING. They also set the member result to one of
254 CODING_RESULT_XXX indicating how the encoding finished.
d46c5b12 255
df7492f9
KH		 256 DST_BYTES zero means that source area and destination area are
257 overlapped, which means that we can produce a encoded text until it
258 reaches at the head of not-yet-encoded source text.
d46c5b12 259
df7492f9 260 Below is a template of these functions. */
4ed46869 261#if 0
b73bfc1c 262static void
df7492f9 263encode_coding_XXX (coding)
4ed46869 264 struct coding_system *coding;
4ed46869 265{
df7492f9
KH		 266 int multibytep = coding->dst_multibyte;
267 int *charbuf = coding->charbuf;
268 int *charbuf_end = charbuf->charbuf + coding->charbuf_used;
269 unsigned char *dst = coding->destination + coding->produced;
270 unsigned char *dst_end = coding->destination + coding->dst_bytes;
271 unsigned char *adjusted_dst_end = dst_end - _MAX_BYTES_PRODUCED_IN_LOOP_;
272 int produced_chars = 0;
273
274 for (; charbuf < charbuf_end && dst < adjusted_dst_end; charbuf++)
275 {
276 int c = *charbuf;
277 /* Encode C into DST, and increment DST. */
278 }
279 label_no_more_destination:
280 /* How many chars and bytes we produced. */
281 coding->produced_char += produced_chars;
282 coding->produced = dst - coding->destination;
4ed46869
KH		 283}
284#endif
285
4ed46869
KH		 286\f
287/*** 1. Preamble ***/
288
68c45bf0 289#include <config.h>
4ed46869
KH		 290#include <stdio.h>
291
4ed46869
KH		 292#include "lisp.h"
293#include "buffer.h"
df7492f9 294#include "character.h"
4ed46869
KH		 295#include "charset.h"
296#include "ccl.h"
df7492f9 297#include "composite.h"
4ed46869
KH		 298#include "coding.h"
299#include "window.h"
4ed46869 300
df7492f9 301Lisp_Object Vcoding_system_hash_table;
4ed46869 302
df7492f9 303Lisp_Object Qcoding_system, Qcoding_aliases, Qeol_type;
1965cb73
DL		 304Lisp_Object Qunix, Qdos;
305extern Lisp_Object Qmac; /* frame.c */
4ed46869
KH		 306Lisp_Object Qbuffer_file_coding_system;
307Lisp_Object Qpost_read_conversion, Qpre_write_conversion;
df7492f9 308Lisp_Object Qdefault_char;
27901516 309Lisp_Object Qno_conversion, Qundecided;
df7492f9 310Lisp_Object Qcharset, Qiso_2022, Qutf_8, Qutf_16, Qshift_jis, Qbig5;
b49a1807 311Lisp_Object Qbig, Qlittle;
bb0115a2 312Lisp_Object Qcoding_system_history;
1397dc18 313Lisp_Object Qvalid_codes;
a6f87d34
KH		 314Lisp_Object QCcategory, QCmnemonic, QCdefalut_char;
315Lisp_Object QCdecode_translation_table, QCencode_translation_table;
316Lisp_Object QCpost_read_conversion, QCpre_write_conversion;
35befdaa 317Lisp_Object QCascii_compatible_p;
4ed46869
KH		 318
319extern Lisp_Object Qinsert_file_contents, Qwrite_region;
387f6ba5 320Lisp_Object Qcall_process, Qcall_process_region;
4ed46869
KH		 321Lisp_Object Qstart_process, Qopen_network_stream;
322Lisp_Object Qtarget_idx;
323
065e3595
KH		 324Lisp_Object Qinsufficient_source, Qinconsistent_eol, Qinvalid_source;
325Lisp_Object Qinterrupted, Qinsufficient_memory;
326
5d5bf4d8
KH		 327int coding_system_require_warning;
328
d46c5b12
KH		 329Lisp_Object Vselect_safe_coding_system_function;
330
7722baf9
EZ		 331/* Mnemonic string for each format of end-of-line. */
332Lisp_Object eol_mnemonic_unix, eol_mnemonic_dos, eol_mnemonic_mac;
333/* Mnemonic string to indicate format of end-of-line is not yet
4ed46869 334 decided. */
7722baf9 335Lisp_Object eol_mnemonic_undecided;
4ed46869
KH		 336
337#ifdef emacs
338
4608c386
KH		 339Lisp_Object Vcoding_system_list, Vcoding_system_alist;
340
341Lisp_Object Qcoding_system_p, Qcoding_system_error;
4ed46869 342
d46c5b12
KH		 343/* Coding system emacs-mule and raw-text are for converting only
344 end-of-line format. */
345Lisp_Object Qemacs_mule, Qraw_text;
8f924df7 346Lisp_Object Qutf_8_emacs;
ecf488bc 347
4ed46869
KH		 348/* Coding-systems are handed between Emacs Lisp programs and C internal
349 routines by the following three variables. */
350/* Coding-system for reading files and receiving data from process. */
351Lisp_Object Vcoding_system_for_read;
352/* Coding-system for writing files and sending data to process. */
353Lisp_Object Vcoding_system_for_write;
354/* Coding-system actually used in the latest I/O. */
355Lisp_Object Vlast_coding_system_used;
065e3595
KH		 356/* Set to non-nil when an error is detected while code conversion. */
357Lisp_Object Vlast_code_conversion_error;
c4825358 358/* A vector of length 256 which contains information about special
94487c4e 359 Latin codes (especially for dealing with Microsoft codes). */
3f003981 360Lisp_Object Vlatin_extra_code_table;
c4825358 361
9ce27fde
KH		 362/* Flag to inhibit code conversion of end-of-line format. */
363int inhibit_eol_conversion;
364
74383408
KH		 365/* Flag to inhibit ISO2022 escape sequence detection. */
366int inhibit_iso_escape_detection;
367
ed29121d
EZ		 368/* Flag to make buffer-file-coding-system inherit from process-coding. */
369int inherit_process_coding_system;
370
c4825358 371/* Coding system to be used to encode text for terminal display. */
4ed46869
KH		 372struct coding_system terminal_coding;
373
c4825358
KH		 374/* Coding system to be used to encode text for terminal display when
375 terminal coding system is nil. */
376struct coding_system safe_terminal_coding;
377
378/* Coding system of what is sent from terminal keyboard. */
4ed46869
KH		 379struct coding_system keyboard_coding;
380
02ba4723
KH		 381Lisp_Object Vfile_coding_system_alist;
382Lisp_Object Vprocess_coding_system_alist;
383Lisp_Object Vnetwork_coding_system_alist;
4ed46869 384
68c45bf0
PE		 385Lisp_Object Vlocale_coding_system;
386
4ed46869
KH		 387#endif /* emacs */
388
f967223b
KH		 389/* Flag to tell if we look up translation table on character code
390 conversion. */
84fbb8a0 391Lisp_Object Venable_character_translation;
f967223b
KH		 392/* Standard translation table to look up on decoding (reading). */
393Lisp_Object Vstandard_translation_table_for_decode;
394/* Standard translation table to look up on encoding (writing). */
395Lisp_Object Vstandard_translation_table_for_encode;
84fbb8a0 396
f967223b
KH		 397Lisp_Object Qtranslation_table;
398Lisp_Object Qtranslation_table_id;
399Lisp_Object Qtranslation_table_for_decode;
400Lisp_Object Qtranslation_table_for_encode;
4ed46869
KH		 401
402/* Alist of charsets vs revision number. */
df7492f9 403static Lisp_Object Vcharset_revision_table;
4ed46869 404
02ba4723
KH		 405/* Default coding systems used for process I/O. */
406Lisp_Object Vdefault_process_coding_system;
407
002fdb44
DL		 408/* Char table for translating Quail and self-inserting input. */
409Lisp_Object Vtranslation_table_for_input;
410
df7492f9
KH		 411/* Two special coding systems. */
412Lisp_Object Vsjis_coding_system;
413Lisp_Object Vbig5_coding_system;
414
df7492f9
KH		 415/* ISO2022 section */
416
417#define CODING_ISO_INITIAL(coding, reg) \
418 (XINT (AREF (AREF (CODING_ID_ATTRS ((coding)->id), \
419 coding_attr_iso_initial), \
420 reg)))
421
422
423#define CODING_ISO_REQUEST(coding, charset_id) \
424 ((charset_id <= (coding)->max_charset_id \
425 ? (coding)->safe_charsets[charset_id] \
426 : -1))
427
428
429#define CODING_ISO_FLAGS(coding) \
430 ((coding)->spec.iso_2022.flags)
431#define CODING_ISO_DESIGNATION(coding, reg) \
432 ((coding)->spec.iso_2022.current_designation[reg])
433#define CODING_ISO_INVOCATION(coding, plane) \
434 ((coding)->spec.iso_2022.current_invocation[plane])
435#define CODING_ISO_SINGLE_SHIFTING(coding) \
436 ((coding)->spec.iso_2022.single_shifting)
437#define CODING_ISO_BOL(coding) \
438 ((coding)->spec.iso_2022.bol)
439#define CODING_ISO_INVOKED_CHARSET(coding, plane) \
440 CODING_ISO_DESIGNATION ((coding), CODING_ISO_INVOCATION ((coding), (plane)))
441
442/* Control characters of ISO2022. */
443 /* code */ /* function */
444#define ISO_CODE_LF 0x0A /* line-feed */
445#define ISO_CODE_CR 0x0D /* carriage-return */
446#define ISO_CODE_SO 0x0E /* shift-out */
447#define ISO_CODE_SI 0x0F /* shift-in */
448#define ISO_CODE_SS2_7 0x19 /* single-shift-2 for 7-bit code */
449#define ISO_CODE_ESC 0x1B /* escape */
450#define ISO_CODE_SS2 0x8E /* single-shift-2 */
451#define ISO_CODE_SS3 0x8F /* single-shift-3 */
452#define ISO_CODE_CSI 0x9B /* control-sequence-introducer */
453
454/* All code (1-byte) of ISO2022 is classified into one of the
455 followings. */
456enum iso_code_class_type
457 {
458 ISO_control_0, /* Control codes in the range
459 0x00..0x1F and 0x7F, except for the
460 following 5 codes. */
df7492f9
KH		 461 ISO_shift_out, /* ISO_CODE_SO (0x0E) */
462 ISO_shift_in, /* ISO_CODE_SI (0x0F) */
463 ISO_single_shift_2_7, /* ISO_CODE_SS2_7 (0x19) */
464 ISO_escape, /* ISO_CODE_SO (0x1B) */
465 ISO_control_1, /* Control codes in the range
466 0x80..0x9F, except for the
467 following 3 codes. */
468 ISO_single_shift_2, /* ISO_CODE_SS2 (0x8E) */
469 ISO_single_shift_3, /* ISO_CODE_SS3 (0x8F) */
470 ISO_control_sequence_introducer, /* ISO_CODE_CSI (0x9B) */
471 ISO_0x20_or_0x7F, /* Codes of the values 0x20 or 0x7F. */
472 ISO_graphic_plane_0, /* Graphic codes in the range 0x21..0x7E. */
473 ISO_0xA0_or_0xFF, /* Codes of the values 0xA0 or 0xFF. */
474 ISO_graphic_plane_1 /* Graphic codes in the range 0xA1..0xFE. */
475 };
05e6f5dc 476
df7492f9
KH		 477/** The macros CODING_ISO_FLAG_XXX defines a flag bit of the
478 `iso-flags' attribute of an iso2022 coding system. */
05e6f5dc 479
df7492f9
KH		 480/* If set, produce long-form designation sequence (e.g. ESC $ ( A)
481 instead of the correct short-form sequence (e.g. ESC $ A). */
482#define CODING_ISO_FLAG_LONG_FORM 0x0001
93dec019 483
df7492f9
KH		 484/* If set, reset graphic planes and registers at end-of-line to the
485 initial state. */
486#define CODING_ISO_FLAG_RESET_AT_EOL 0x0002
05e6f5dc 487
df7492f9
KH		 488/* If set, reset graphic planes and registers before any control
489 characters to the initial state. */
490#define CODING_ISO_FLAG_RESET_AT_CNTL 0x0004
05e6f5dc 491
df7492f9
KH		 492/* If set, encode by 7-bit environment. */
493#define CODING_ISO_FLAG_SEVEN_BITS 0x0008
4ed46869 494
df7492f9
KH		 495/* If set, use locking-shift function. */
496#define CODING_ISO_FLAG_LOCKING_SHIFT 0x0010
b73bfc1c 497
df7492f9
KH		 498/* If set, use single-shift function. Overwrite
499 CODING_ISO_FLAG_LOCKING_SHIFT. */
500#define CODING_ISO_FLAG_SINGLE_SHIFT 0x0020
b73bfc1c 501
df7492f9
KH		 502/* If set, use designation escape sequence. */
503#define CODING_ISO_FLAG_DESIGNATION 0x0040
b73bfc1c 504
df7492f9
KH		 505/* If set, produce revision number sequence. */
506#define CODING_ISO_FLAG_REVISION 0x0080
b73bfc1c 507
df7492f9
KH		 508/* If set, produce ISO6429's direction specifying sequence. */
509#define CODING_ISO_FLAG_DIRECTION 0x0100
f4dee582 510
df7492f9
KH		 511/* If set, assume designation states are reset at beginning of line on
512 output. */
513#define CODING_ISO_FLAG_INIT_AT_BOL 0x0200
4ed46869 514
df7492f9
KH		 515/* If set, designation sequence should be placed at beginning of line
516 on output. */
517#define CODING_ISO_FLAG_DESIGNATE_AT_BOL 0x0400
aa72b389 518
df7492f9
KH		 519/* If set, do not encode unsafe charactes on output. */
520#define CODING_ISO_FLAG_SAFE 0x0800
aa72b389 521
df7492f9
KH		 522/* If set, extra latin codes (128..159) are accepted as a valid code
523 on input. */
524#define CODING_ISO_FLAG_LATIN_EXTRA 0x1000
aa72b389 525
df7492f9 526#define CODING_ISO_FLAG_COMPOSITION 0x2000
aa72b389 527
df7492f9 528#define CODING_ISO_FLAG_EUC_TW_SHIFT 0x4000
aa72b389 529
bf16eb23 530#define CODING_ISO_FLAG_USE_ROMAN 0x8000
aa72b389 531
bf16eb23 532#define CODING_ISO_FLAG_USE_OLDJIS 0x10000
aa72b389 533
bf16eb23 534#define CODING_ISO_FLAG_FULL_SUPPORT 0x100000
aa72b389 535
df7492f9
KH		 536/* A character to be produced on output if encoding of the original
537 character is prohibited by CODING_ISO_FLAG_SAFE. */
538#define CODING_INHIBIT_CHARACTER_SUBSTITUTION '?'
aa72b389 539
4ed46869 540
df7492f9
KH		 541/* UTF-16 section */
542#define CODING_UTF_16_BOM(coding) \
543 ((coding)->spec.utf_16.bom)
4ed46869 544
df7492f9
KH		 545#define CODING_UTF_16_ENDIAN(coding) \
546 ((coding)->spec.utf_16.endian)
4ed46869 547
df7492f9
KH		 548#define CODING_UTF_16_SURROGATE(coding) \
549 ((coding)->spec.utf_16.surrogate)
4ed46869 550
4ed46869 551
df7492f9
KH		 552/* CCL section */
553#define CODING_CCL_DECODER(coding) \
554 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_decoder)
555#define CODING_CCL_ENCODER(coding) \
556 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_encoder)
557#define CODING_CCL_VALIDS(coding) \
8f924df7 558 (SDATA (AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_valids)))
4ed46869 559
5a936b46 560/* Index for each coding category in `coding_categories' */
4ed46869 561
df7492f9
KH		 562enum coding_category
563 {
564 coding_category_iso_7,
565 coding_category_iso_7_tight,
566 coding_category_iso_8_1,
567 coding_category_iso_8_2,
568 coding_category_iso_7_else,
569 coding_category_iso_8_else,
570 coding_category_utf_8,
571 coding_category_utf_16_auto,
572 coding_category_utf_16_be,
573 coding_category_utf_16_le,
574 coding_category_utf_16_be_nosig,
575 coding_category_utf_16_le_nosig,
576 coding_category_charset,
577 coding_category_sjis,
578 coding_category_big5,
579 coding_category_ccl,
580 coding_category_emacs_mule,
581 /* All above are targets of code detection. */
582 coding_category_raw_text,
583 coding_category_undecided,
584 coding_category_max
585 };
586
587/* Definitions of flag bits used in detect_coding_XXXX. */
588#define CATEGORY_MASK_ISO_7 (1 << coding_category_iso_7)
589#define CATEGORY_MASK_ISO_7_TIGHT (1 << coding_category_iso_7_tight)
590#define CATEGORY_MASK_ISO_8_1 (1 << coding_category_iso_8_1)
591#define CATEGORY_MASK_ISO_8_2 (1 << coding_category_iso_8_2)
592#define CATEGORY_MASK_ISO_7_ELSE (1 << coding_category_iso_7_else)
593#define CATEGORY_MASK_ISO_8_ELSE (1 << coding_category_iso_8_else)
594#define CATEGORY_MASK_UTF_8 (1 << coding_category_utf_8)
b49a1807 595#define CATEGORY_MASK_UTF_16_AUTO (1 << coding_category_utf_16_auto)
df7492f9
KH		 596#define CATEGORY_MASK_UTF_16_BE (1 << coding_category_utf_16_be)
597#define CATEGORY_MASK_UTF_16_LE (1 << coding_category_utf_16_le)
598#define CATEGORY_MASK_UTF_16_BE_NOSIG (1 << coding_category_utf_16_be_nosig)
599#define CATEGORY_MASK_UTF_16_LE_NOSIG (1 << coding_category_utf_16_le_nosig)
600#define CATEGORY_MASK_CHARSET (1 << coding_category_charset)
601#define CATEGORY_MASK_SJIS (1 << coding_category_sjis)
602#define CATEGORY_MASK_BIG5 (1 << coding_category_big5)
603#define CATEGORY_MASK_CCL (1 << coding_category_ccl)
604#define CATEGORY_MASK_EMACS_MULE (1 << coding_category_emacs_mule)
ff0dacd7 605#define CATEGORY_MASK_RAW_TEXT (1 << coding_category_raw_text)
df7492f9
KH		 606
607/* This value is returned if detect_coding_mask () find nothing other
608 than ASCII characters. */
609#define CATEGORY_MASK_ANY \
610 (CATEGORY_MASK_ISO_7 \
611 | CATEGORY_MASK_ISO_7_TIGHT \
612 | CATEGORY_MASK_ISO_8_1 \
613 | CATEGORY_MASK_ISO_8_2 \
614 | CATEGORY_MASK_ISO_7_ELSE \
615 | CATEGORY_MASK_ISO_8_ELSE \
616 | CATEGORY_MASK_UTF_8 \
617 | CATEGORY_MASK_UTF_16_BE \
618 | CATEGORY_MASK_UTF_16_LE \
619 | CATEGORY_MASK_UTF_16_BE_NOSIG \
620 | CATEGORY_MASK_UTF_16_LE_NOSIG \
621 | CATEGORY_MASK_CHARSET \
622 | CATEGORY_MASK_SJIS \
623 | CATEGORY_MASK_BIG5 \
624 | CATEGORY_MASK_CCL \
625 | CATEGORY_MASK_EMACS_MULE)
626
627
628#define CATEGORY_MASK_ISO_7BIT \
629 (CATEGORY_MASK_ISO_7 | CATEGORY_MASK_ISO_7_TIGHT)
630
631#define CATEGORY_MASK_ISO_8BIT \
632 (CATEGORY_MASK_ISO_8_1 | CATEGORY_MASK_ISO_8_2)
633
634#define CATEGORY_MASK_ISO_ELSE \
635 (CATEGORY_MASK_ISO_7_ELSE | CATEGORY_MASK_ISO_8_ELSE)
636
637#define CATEGORY_MASK_ISO_ESCAPE \
638 (CATEGORY_MASK_ISO_7 \
639 | CATEGORY_MASK_ISO_7_TIGHT \
640 | CATEGORY_MASK_ISO_7_ELSE \
641 | CATEGORY_MASK_ISO_8_ELSE)
642
643#define CATEGORY_MASK_ISO \
644 ( CATEGORY_MASK_ISO_7BIT \
645 | CATEGORY_MASK_ISO_8BIT \
646 | CATEGORY_MASK_ISO_ELSE)
647
648#define CATEGORY_MASK_UTF_16 \
649 (CATEGORY_MASK_UTF_16_BE \
650 | CATEGORY_MASK_UTF_16_LE \
651 | CATEGORY_MASK_UTF_16_BE_NOSIG \
652 | CATEGORY_MASK_UTF_16_LE_NOSIG)
653
654
655/* List of symbols `coding-category-xxx' ordered by priority. This
656 variable is exposed to Emacs Lisp. */
657static Lisp_Object Vcoding_category_list;
658
659/* Table of coding categories (Lisp symbols). This variable is for
660 internal use oly. */
661static Lisp_Object Vcoding_category_table;
662
663/* Table of coding-categories ordered by priority. */
664static enum coding_category coding_priorities[coding_category_max];
665
666/* Nth element is a coding context for the coding system bound to the
667 Nth coding category. */
668static struct coding_system coding_categories[coding_category_max];
669
df7492f9
KH		 670/*** Commonly used macros and functions ***/
671
672#ifndef min
673#define min(a, b) ((a) < (b) ? (a) : (b))
674#endif
675#ifndef max
676#define max(a, b) ((a) > (b) ? (a) : (b))
677#endif
4ed46869 678
24a73b0a
KH		 679#define CODING_GET_INFO(coding, attrs, charset_list) \
680 do { \
681 (attrs) = CODING_ID_ATTRS ((coding)->id); \
682 (charset_list) = CODING_ATTR_CHARSET_LIST (attrs); \
df7492f9 683 } while (0)
4ed46869 684
4ed46869 685
df7492f9
KH		 686/* Safely get one byte from the source text pointed by SRC which ends
687 at SRC_END, and set C to that byte. If there are not enough bytes
065e3595
KH		 688 in the source, it jumps to `no_more_source'. If multibytep is
689 nonzero, and a multibyte character is found at SRC, set C to the
690 negative value of the character code. The caller should declare
691 and set these variables appropriately in advance:
692 src, src_end, multibytep */
aa72b389 693
065e3595
KH		 694#define ONE_MORE_BYTE(c) \
695 do { \
696 if (src == src_end) \
697 { \
698 if (src_base < src) \
699 record_conversion_result \
700 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
701 goto no_more_source; \
702 } \
703 c = *src++; \
704 if (multibytep && (c & 0x80)) \
705 { \
706 if ((c & 0xFE) == 0xC0) \
707 c = ((c & 1) << 6) | *src++; \
708 else \
709 { \
35befdaa
KH		 710 src--; \
711 c = - string_char (src, &src, NULL); \
065e3595
KH		 712 record_conversion_result \
713 (coding, CODING_RESULT_INVALID_SRC); \
714 } \
715 } \
716 consumed_chars++; \
aa72b389
KH		 717 } while (0)
718
aa72b389 719
065e3595
KH		 720#define ONE_MORE_BYTE_NO_CHECK(c) \
721 do { \
722 c = *src++; \
723 if (multibytep && (c & 0x80)) \
724 { \
725 if ((c & 0xFE) == 0xC0) \
726 c = ((c & 1) << 6) | *src++; \
727 else \
728 { \
35befdaa
KH		 729 src--; \
730 c = - string_char (src, &src, NULL); \
065e3595
KH		 731 record_conversion_result \
732 (coding, CODING_RESULT_INVALID_SRC); \
733 } \
734 } \
735 consumed_chars++; \
aa72b389
KH		 736 } while (0)
737
aa72b389 738
df7492f9
KH		 739/* Store a byte C in the place pointed by DST and increment DST to the
740 next free point, and increment PRODUCED_CHARS. The caller should
741 assure that C is 0..127, and declare and set the variable `dst'
742 appropriately in advance.
743*/
aa72b389
KH		 744
745
df7492f9
KH		 746#define EMIT_ONE_ASCII_BYTE(c) \
747 do { \
748 produced_chars++; \
749 *dst++ = (c); \
b6871cc7 750 } while (0)
aa72b389
KH		 751
752
df7492f9 753/* Like EMIT_ONE_ASCII_BYTE byt store two bytes; C1 and C2. */
aa72b389 754
df7492f9
KH		 755#define EMIT_TWO_ASCII_BYTES(c1, c2) \
756 do { \
757 produced_chars += 2; \
758 *dst++ = (c1), *dst++ = (c2); \
759 } while (0)
aa72b389
KH		 760
761
df7492f9
KH		 762/* Store a byte C in the place pointed by DST and increment DST to the
763 next free point, and increment PRODUCED_CHARS. If MULTIBYTEP is
764 nonzero, store in an appropriate multibyte from. The caller should
765 declare and set the variables `dst' and `multibytep' appropriately
766 in advance. */
767
768#define EMIT_ONE_BYTE(c) \
769 do { \
770 produced_chars++; \
771 if (multibytep) \
772 { \
773 int ch = (c); \
774 if (ch >= 0x80) \
775 ch = BYTE8_TO_CHAR (ch); \
776 CHAR_STRING_ADVANCE (ch, dst); \
777 } \
778 else \
779 *dst++ = (c); \
aa72b389 780 } while (0)
aa72b389 781
aa72b389 782
df7492f9 783/* Like EMIT_ONE_BYTE, but emit two bytes; C1 and C2. */
aa72b389 784
e19c3639
KH		 785#define EMIT_TWO_BYTES(c1, c2) \
786 do { \
787 produced_chars += 2; \
788 if (multibytep) \
789 { \
790 int ch; \
791 \
792 ch = (c1); \
793 if (ch >= 0x80) \
794 ch = BYTE8_TO_CHAR (ch); \
795 CHAR_STRING_ADVANCE (ch, dst); \
796 ch = (c2); \
797 if (ch >= 0x80) \
798 ch = BYTE8_TO_CHAR (ch); \
799 CHAR_STRING_ADVANCE (ch, dst); \
800 } \
801 else \
802 { \
803 *dst++ = (c1); \
804 *dst++ = (c2); \
805 } \
aa72b389
KH		 806 } while (0)
807
808
df7492f9
KH		 809#define EMIT_THREE_BYTES(c1, c2, c3) \
810 do { \
811 EMIT_ONE_BYTE (c1); \
812 EMIT_TWO_BYTES (c2, c3); \
813 } while (0)
aa72b389 814
aa72b389 815
df7492f9
KH		 816#define EMIT_FOUR_BYTES(c1, c2, c3, c4) \
817 do { \
818 EMIT_TWO_BYTES (c1, c2); \
819 EMIT_TWO_BYTES (c3, c4); \
820 } while (0)
aa72b389 821
aa72b389 822
f6cbaf43
KH		 823/* Prototypes for static functions. */
824static void record_conversion_result P_ ((struct coding_system *coding,
825 enum coding_result_code result));
826static int detect_coding_utf_8 P_ ((struct coding_system *,
827 struct coding_detection_info *info));
828static void decode_coding_utf_8 P_ ((struct coding_system *));
829static int encode_coding_utf_8 P_ ((struct coding_system *));
830
831static int detect_coding_utf_16 P_ ((struct coding_system *,
832 struct coding_detection_info *info));
833static void decode_coding_utf_16 P_ ((struct coding_system *));
834static int encode_coding_utf_16 P_ ((struct coding_system *));
835
836static int detect_coding_iso_2022 P_ ((struct coding_system *,
837 struct coding_detection_info *info));
838static void decode_coding_iso_2022 P_ ((struct coding_system *));
839static int encode_coding_iso_2022 P_ ((struct coding_system *));
840
841static int detect_coding_emacs_mule P_ ((struct coding_system *,
842 struct coding_detection_info *info));
843static void decode_coding_emacs_mule P_ ((struct coding_system *));
844static int encode_coding_emacs_mule P_ ((struct coding_system *));
845
846static int detect_coding_sjis P_ ((struct coding_system *,
847 struct coding_detection_info *info));
848static void decode_coding_sjis P_ ((struct coding_system *));
849static int encode_coding_sjis P_ ((struct coding_system *));
850
851static int detect_coding_big5 P_ ((struct coding_system *,
852 struct coding_detection_info *info));
853static void decode_coding_big5 P_ ((struct coding_system *));
854static int encode_coding_big5 P_ ((struct coding_system *));
855
856static int detect_coding_ccl P_ ((struct coding_system *,
857 struct coding_detection_info *info));
858static void decode_coding_ccl P_ ((struct coding_system *));
859static int encode_coding_ccl P_ ((struct coding_system *));
860
861static void decode_coding_raw_text P_ ((struct coding_system *));
862static int encode_coding_raw_text P_ ((struct coding_system *));
863
864static void coding_set_source P_ ((struct coding_system *));
865static void coding_set_destination P_ ((struct coding_system *));
866static void coding_alloc_by_realloc P_ ((struct coding_system *, EMACS_INT));
867static void coding_alloc_by_making_gap P_ ((struct coding_system *,
868 EMACS_INT));
869static unsigned char *alloc_destination P_ ((struct coding_system *,
870 EMACS_INT, unsigned char *));
871static void setup_iso_safe_charsets P_ ((Lisp_Object));
872static unsigned char *encode_designation_at_bol P_ ((struct coding_system *,
873 int *, int *,
874 unsigned char *));
875static int detect_eol P_ ((const unsigned char *,
876 EMACS_INT, enum coding_category));
877static Lisp_Object adjust_coding_eol_type P_ ((struct coding_system *, int));
878static void decode_eol P_ ((struct coding_system *));
879static Lisp_Object get_translation_table P_ ((Lisp_Object, int, int *));
880static Lisp_Object get_translation P_ ((Lisp_Object, int *, int *,
881 int, int *, int *));
882static int produce_chars P_ ((struct coding_system *, Lisp_Object, int));
883static INLINE void produce_composition P_ ((struct coding_system *, int *,
884 EMACS_INT));
885static INLINE void produce_charset P_ ((struct coding_system *, int *,
886 EMACS_INT));
887static void produce_annotation P_ ((struct coding_system *, EMACS_INT));
888static int decode_coding P_ ((struct coding_system *));
889static INLINE int *handle_composition_annotation P_ ((EMACS_INT, EMACS_INT,
890 struct coding_system *,
891 int *, EMACS_INT *));
892static INLINE int *handle_charset_annotation P_ ((EMACS_INT, EMACS_INT,
893 struct coding_system *,
894 int *, EMACS_INT *));
895static void consume_chars P_ ((struct coding_system *, Lisp_Object, int));
896static int encode_coding P_ ((struct coding_system *));
897static Lisp_Object make_conversion_work_buffer P_ ((int));
898static Lisp_Object code_conversion_restore P_ ((Lisp_Object));
899static INLINE int char_encodable_p P_ ((int, Lisp_Object));
900static Lisp_Object make_subsidiaries P_ ((Lisp_Object));
901
065e3595
KH		 902static void
903record_conversion_result (struct coding_system *coding,
904 enum coding_result_code result)
905{
906 coding->result = result;
907 switch (result)
908 {
909 case CODING_RESULT_INSUFFICIENT_SRC:
910 Vlast_code_conversion_error = Qinsufficient_source;
911 break;
912 case CODING_RESULT_INCONSISTENT_EOL:
913 Vlast_code_conversion_error = Qinconsistent_eol;
914 break;
915 case CODING_RESULT_INVALID_SRC:
916 Vlast_code_conversion_error = Qinvalid_source;
917 break;
918 case CODING_RESULT_INTERRUPT:
919 Vlast_code_conversion_error = Qinterrupted;
920 break;
921 case CODING_RESULT_INSUFFICIENT_MEM:
922 Vlast_code_conversion_error = Qinsufficient_memory;
923 break;
35befdaa
KH		 924 default:
925 Vlast_code_conversion_error = intern ("Unknown error");
065e3595
KH		 926 }
927}
928
df7492f9
KH		 929#define CODING_DECODE_CHAR(coding, src, src_base, src_end, charset, code, c) \
930 do { \
931 charset_map_loaded = 0; \
932 c = DECODE_CHAR (charset, code); \
933 if (charset_map_loaded) \
934 { \
8f924df7 935 const unsigned char *orig = coding->source; \
df7492f9
KH		 936 EMACS_INT offset; \
937 \
938 coding_set_source (coding); \
939 offset = coding->source - orig; \
940 src += offset; \
941 src_base += offset; \
942 src_end += offset; \
943 } \
aa72b389
KH		 944 } while (0)
945
946
df7492f9
KH		 947#define ASSURE_DESTINATION(bytes) \
948 do { \
949 if (dst + (bytes) >= dst_end) \
950 { \
951 int more_bytes = charbuf_end - charbuf + (bytes); \
952 \
953 dst = alloc_destination (coding, more_bytes, dst); \
954 dst_end = coding->destination + coding->dst_bytes; \
955 } \
956 } while (0)
aa72b389 957
aa72b389 958
aa72b389 959
df7492f9
KH		 960static void
961coding_set_source (coding)
aa72b389 962 struct coding_system *coding;
aa72b389 963{
df7492f9
KH		 964 if (BUFFERP (coding->src_object))
965 {
2cb26057 966 struct buffer *buf = XBUFFER (coding->src_object);
aa72b389 967
df7492f9 968 if (coding->src_pos < 0)
2cb26057 969 coding->source = BUF_GAP_END_ADDR (buf) + coding->src_pos_byte;
df7492f9 970 else
2cb26057 971 coding->source = BUF_BYTE_ADDRESS (buf, coding->src_pos_byte);
aa72b389 972 }
df7492f9 973 else if (STRINGP (coding->src_object))
aa72b389 974 {
8f924df7 975 coding->source = SDATA (coding->src_object) + coding->src_pos_byte;
aa72b389 976 }
df7492f9
KH		 977 else
978 /* Otherwise, the source is C string and is never relocated
979 automatically. Thus we don't have to update anything. */
980 ;
981}
aa72b389 982
df7492f9
KH		 983static void
984coding_set_destination (coding)
985 struct coding_system *coding;
986{
987 if (BUFFERP (coding->dst_object))
aa72b389 988 {
df7492f9 989 if (coding->src_pos < 0)
aa72b389 990 {
28f67a95
KH		 991 coding->destination = BEG_ADDR + coding->dst_pos_byte - 1;
992 coding->dst_bytes = (GAP_END_ADDR
993 - (coding->src_bytes - coding->consumed)
994 - coding->destination);
aa72b389 995 }
df7492f9 996 else
28f67a95
KH		 997 {
998 /* We are sure that coding->dst_pos_byte is before the gap
999 of the buffer. */
1000 coding->destination = (BUF_BEG_ADDR (XBUFFER (coding->dst_object))
1001 + coding->dst_pos_byte - 1);
1002 coding->dst_bytes = (BUF_GAP_END_ADDR (XBUFFER (coding->dst_object))
1003 - coding->destination);
1004 }
df7492f9
KH		 1005 }
1006 else
1007 /* Otherwise, the destination is C string and is never relocated
1008 automatically. Thus we don't have to update anything. */
1009 ;
1010}
1011
1012
1013static void
1014coding_alloc_by_realloc (coding, bytes)
1015 struct coding_system *coding;
1016 EMACS_INT bytes;
1017{
1018 coding->destination = (unsigned char *) xrealloc (coding->destination,
1019 coding->dst_bytes + bytes);
1020 coding->dst_bytes += bytes;
1021}
1022
1023static void
1024coding_alloc_by_making_gap (coding, bytes)
1025 struct coding_system *coding;
1026 EMACS_INT bytes;
1027{
2c78b7e1
KH		 1028 if (BUFFERP (coding->dst_object)
1029 && EQ (coding->src_object, coding->dst_object))
df7492f9
KH		 1030 {
1031 EMACS_INT add = coding->src_bytes - coding->consumed;
1032
1033 GAP_SIZE -= add; ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
1034 make_gap (bytes);
1035 GAP_SIZE += add; ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
1036 }
1037 else
1038 {
2c78b7e1
KH		 1039 Lisp_Object this_buffer;
1040
1041 this_buffer = Fcurrent_buffer ();
df7492f9
KH		 1042 set_buffer_internal (XBUFFER (coding->dst_object));
1043 make_gap (bytes);
1044 set_buffer_internal (XBUFFER (this_buffer));
aa72b389 1045 }
df7492f9 1046}
8f924df7 1047
df7492f9
KH		 1048
1049static unsigned char *
1050alloc_destination (coding, nbytes, dst)
1051 struct coding_system *coding;
3e139625 1052 EMACS_INT nbytes;
df7492f9
KH		 1053 unsigned char *dst;
1054{
1055 EMACS_INT offset = dst - coding->destination;
1056
1057 if (BUFFERP (coding->dst_object))
1058 coding_alloc_by_making_gap (coding, nbytes);
aa72b389 1059 else
df7492f9 1060 coding_alloc_by_realloc (coding, nbytes);
065e3595 1061 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1062 coding_set_destination (coding);
1063 dst = coding->destination + offset;
1064 return dst;
1065}
aa72b389 1066
ff0dacd7
KH		 1067/** Macros for annotations. */
1068
1069/* Maximum length of annotation data (sum of annotations for
1070 composition and charset). */
69a80ea3 1071#define MAX_ANNOTATION_LENGTH (4 + (MAX_COMPOSITION_COMPONENTS * 2) - 1 + 4)
ff0dacd7
KH		 1072
1073/* An annotation data is stored in the array coding->charbuf in this
1074 format:
69a80ea3 1075 [ -LENGTH ANNOTATION_MASK NCHARS ... ]
ff0dacd7
KH		 1076 LENGTH is the number of elements in the annotation.
1077 ANNOTATION_MASK is one of CODING_ANNOTATE_XXX_MASK.
69a80ea3 1078 NCHARS is the number of characters in the text annotated.
ff0dacd7
KH		 1079
1080 The format of the following elements depend on ANNOTATION_MASK.
1081
1082 In the case of CODING_ANNOTATE_COMPOSITION_MASK, these elements
1083 follows:
1084 ... METHOD [ COMPOSITION-COMPONENTS ... ]
1085 METHOD is one of enum composition_method.
1086 Optionnal COMPOSITION-COMPONENTS are characters and composition
1087 rules.
1088
1089 In the case of CODING_ANNOTATE_CHARSET_MASK, one element CHARSET-ID
1090 follows. */
1091
69a80ea3 1092#define ADD_ANNOTATION_DATA(buf, len, mask, nchars) \
ff0dacd7
KH		 1093 do { \
1094 *(buf)++ = -(len); \
1095 *(buf)++ = (mask); \
69a80ea3 1096 *(buf)++ = (nchars); \
ff0dacd7
KH		 1097 coding->annotated = 1; \
1098 } while (0);
1099
69a80ea3
KH		 1100#define ADD_COMPOSITION_DATA(buf, nchars, method) \
1101 do { \
1102 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_COMPOSITION_MASK, nchars); \
1103 *buf++ = method; \
ff0dacd7
KH		 1104 } while (0)
1105
1106
69a80ea3
KH		 1107#define ADD_CHARSET_DATA(buf, nchars, id) \
1108 do { \
1109 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_CHARSET_MASK, nchars); \
1110 *buf++ = id; \
ff0dacd7
KH		 1111 } while (0)
1112
df7492f9
KH		 1113\f
1114/*** 2. Emacs' internal format (emacs-utf-8) ***/
1115
1116
1117
1118\f
1119/*** 3. UTF-8 ***/
1120
1121/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1122 Check if a text is encoded in UTF-8. If it is, return 1, else
1123 return 0. */
df7492f9
KH		 1124
1125#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
1126#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
1127#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
1128#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
1129#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
1130#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
1131
1132static int
ff0dacd7 1133detect_coding_utf_8 (coding, detect_info)
df7492f9 1134 struct coding_system *coding;
ff0dacd7 1135 struct coding_detection_info *detect_info;
df7492f9 1136{
065e3595 1137 const unsigned char *src = coding->source, *src_base;
8f924df7 1138 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 1139 int multibytep = coding->src_multibyte;
1140 int consumed_chars = 0;
1141 int found = 0;
1142
ff0dacd7 1143 detect_info->checked |= CATEGORY_MASK_UTF_8;
df7492f9
KH		 1144 /* A coding system of this category is always ASCII compatible. */
1145 src += coding->head_ascii;
1146
1147 while (1)
aa72b389 1148 {
df7492f9 1149 int c, c1, c2, c3, c4;
aa72b389 1150
065e3595 1151 src_base = src;
df7492f9 1152 ONE_MORE_BYTE (c);
065e3595 1153 if (c < 0 || UTF_8_1_OCTET_P (c))
df7492f9
KH		 1154 continue;
1155 ONE_MORE_BYTE (c1);
065e3595 1156 if (c1 < 0 || ! UTF_8_EXTRA_OCTET_P (c1))
df7492f9
KH		 1157 break;
1158 if (UTF_8_2_OCTET_LEADING_P (c))
aa72b389 1159 {
ff0dacd7 1160 found = CATEGORY_MASK_UTF_8;
df7492f9 1161 continue;
aa72b389 1162 }
df7492f9 1163 ONE_MORE_BYTE (c2);
065e3595 1164 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1165 break;
1166 if (UTF_8_3_OCTET_LEADING_P (c))
aa72b389 1167 {
ff0dacd7 1168 found = CATEGORY_MASK_UTF_8;
df7492f9 1169 continue;
aa72b389 1170 }
df7492f9 1171 ONE_MORE_BYTE (c3);
065e3595 1172 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1173 break;
1174 if (UTF_8_4_OCTET_LEADING_P (c))
aa72b389 1175 {
ff0dacd7 1176 found = CATEGORY_MASK_UTF_8;
df7492f9
KH		 1177 continue;
1178 }
1179 ONE_MORE_BYTE (c4);
065e3595 1180 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1181 break;
1182 if (UTF_8_5_OCTET_LEADING_P (c))
1183 {
ff0dacd7 1184 found = CATEGORY_MASK_UTF_8;
df7492f9
KH		 1185 continue;
1186 }
1187 break;
aa72b389 1188 }
ff0dacd7 1189 detect_info->rejected |= CATEGORY_MASK_UTF_8;
df7492f9 1190 return 0;
aa72b389 1191
df7492f9 1192 no_more_source:
065e3595 1193 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
aa72b389 1194 {
ff0dacd7 1195 detect_info->rejected |= CATEGORY_MASK_UTF_8;
89528eb3 1196 return 0;
aa72b389 1197 }
ff0dacd7
KH		 1198 detect_info->found |= found;
1199 return 1;
aa72b389
KH		 1200}
1201
4ed46869 1202
b73bfc1c 1203static void
df7492f9 1204decode_coding_utf_8 (coding)
b73bfc1c 1205 struct coding_system *coding;
b73bfc1c 1206{
8f924df7
KH		 1207 const unsigned char *src = coding->source + coding->consumed;
1208 const unsigned char *src_end = coding->source + coding->src_bytes;
1209 const unsigned char *src_base;
69a80ea3
KH		 1210 int *charbuf = coding->charbuf + coding->charbuf_used;
1211 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 1212 int consumed_chars = 0, consumed_chars_base;
1213 int multibytep = coding->src_multibyte;
24a73b0a 1214 Lisp_Object attr, charset_list;
4ed46869 1215
24a73b0a 1216 CODING_GET_INFO (coding, attr, charset_list);
df7492f9
KH		 1217
1218 while (1)
b73bfc1c 1219 {
df7492f9 1220 int c, c1, c2, c3, c4, c5;
ec6d2bb8 1221
df7492f9
KH		 1222 src_base = src;
1223 consumed_chars_base = consumed_chars;
4af310db 1224
df7492f9
KH		 1225 if (charbuf >= charbuf_end)
1226 break;
1227
1228 ONE_MORE_BYTE (c1);
065e3595
KH		 1229 if (c1 < 0)
1230 {
1231 c = - c1;
1232 }
1233 else if (UTF_8_1_OCTET_P(c1))
df7492f9
KH		 1234 {
1235 c = c1;
4af310db 1236 }
df7492f9 1237 else
4af310db 1238 {
df7492f9 1239 ONE_MORE_BYTE (c2);
065e3595 1240 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
df7492f9
KH		 1241 goto invalid_code;
1242 if (UTF_8_2_OCTET_LEADING_P (c1))
4af310db 1243 {
b0edb2c5
DL		 1244 c = ((c1 & 0x1F) << 6) | (c2 & 0x3F);
1245 /* Reject overlong sequences here and below. Encoders
1246 producing them are incorrect, they can be misleading,
1247 and they mess up read/write invariance. */
1248 if (c < 128)
1249 goto invalid_code;
4af310db 1250 }
df7492f9 1251 else
aa72b389 1252 {
df7492f9 1253 ONE_MORE_BYTE (c3);
065e3595 1254 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
df7492f9
KH		 1255 goto invalid_code;
1256 if (UTF_8_3_OCTET_LEADING_P (c1))
b0edb2c5
DL		 1257 {
1258 c = (((c1 & 0xF) << 12)
1259 | ((c2 & 0x3F) << 6) | (c3 & 0x3F));
72fe1301
DL		 1260 if (c < 0x800
1261 || (c >= 0xd800 && c < 0xe000)) /* surrogates (invalid) */
b0edb2c5
DL		 1262 goto invalid_code;
1263 }
df7492f9
KH		 1264 else
1265 {
1266 ONE_MORE_BYTE (c4);
065e3595 1267 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
df7492f9
KH		 1268 goto invalid_code;
1269 if (UTF_8_4_OCTET_LEADING_P (c1))
b0edb2c5 1270 {
df7492f9
KH		 1271 c = (((c1 & 0x7) << 18) | ((c2 & 0x3F) << 12)
1272 | ((c3 & 0x3F) << 6) | (c4 & 0x3F));
b0edb2c5
DL		 1273 if (c < 0x10000)
1274 goto invalid_code;
1275 }
df7492f9
KH		 1276 else
1277 {
1278 ONE_MORE_BYTE (c5);
065e3595 1279 if (c5 < 0 || ! UTF_8_EXTRA_OCTET_P (c5))
df7492f9
KH		 1280 goto invalid_code;
1281 if (UTF_8_5_OCTET_LEADING_P (c1))
1282 {
1283 c = (((c1 & 0x3) << 24) | ((c2 & 0x3F) << 18)
1284 | ((c3 & 0x3F) << 12) | ((c4 & 0x3F) << 6)
1285 | (c5 & 0x3F));
b0edb2c5 1286 if ((c > MAX_CHAR) || (c < 0x200000))
df7492f9
KH		 1287 goto invalid_code;
1288 }
1289 else
1290 goto invalid_code;
1291 }
1292 }
aa72b389 1293 }
b73bfc1c 1294 }
df7492f9
KH		 1295
1296 *charbuf++ = c;
1297 continue;
1298
1299 invalid_code:
1300 src = src_base;
1301 consumed_chars = consumed_chars_base;
1302 ONE_MORE_BYTE (c);
1303 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
1304 coding->errors++;
aa72b389
KH		 1305 }
1306
df7492f9
KH		 1307 no_more_source:
1308 coding->consumed_char += consumed_chars_base;
1309 coding->consumed = src_base - coding->source;
1310 coding->charbuf_used = charbuf - coding->charbuf;
1311}
1312
1313
1314static int
1315encode_coding_utf_8 (coding)
1316 struct coding_system *coding;
1317{
1318 int multibytep = coding->dst_multibyte;
1319 int *charbuf = coding->charbuf;
1320 int *charbuf_end = charbuf + coding->charbuf_used;
1321 unsigned char *dst = coding->destination + coding->produced;
1322 unsigned char *dst_end = coding->destination + coding->dst_bytes;
e19c3639 1323 int produced_chars = 0;
df7492f9
KH		 1324 int c;
1325
1326 if (multibytep)
aa72b389 1327 {
df7492f9
KH		 1328 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
1329
1330 while (charbuf < charbuf_end)
b73bfc1c 1331 {
df7492f9 1332 unsigned char str[MAX_MULTIBYTE_LENGTH], *p, *pend = str;
8f924df7 1333
df7492f9
KH		 1334 ASSURE_DESTINATION (safe_room);
1335 c = *charbuf++;
28f67a95
KH		 1336 if (CHAR_BYTE8_P (c))
1337 {
1338 c = CHAR_TO_BYTE8 (c);
1339 EMIT_ONE_BYTE (c);
1340 }
1341 else
1342 {
1343 CHAR_STRING_ADVANCE (c, pend);
1344 for (p = str; p < pend; p++)
1345 EMIT_ONE_BYTE (*p);
1346 }
b73bfc1c 1347 }
aa72b389 1348 }
df7492f9
KH		 1349 else
1350 {
1351 int safe_room = MAX_MULTIBYTE_LENGTH;
1352
1353 while (charbuf < charbuf_end)
b73bfc1c 1354 {
df7492f9
KH		 1355 ASSURE_DESTINATION (safe_room);
1356 c = *charbuf++;
f03caae0
KH		 1357 if (CHAR_BYTE8_P (c))
1358 *dst++ = CHAR_TO_BYTE8 (c);
1359 else
1360 dst += CHAR_STRING (c, dst);
df7492f9 1361 produced_chars++;
4ed46869
KH		 1362 }
1363 }
065e3595 1364 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1365 coding->produced_char += produced_chars;
1366 coding->produced = dst - coding->destination;
1367 return 0;
4ed46869
KH		 1368}
1369
b73bfc1c 1370
df7492f9 1371/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1372 Check if a text is encoded in one of UTF-16 based coding systems.
1373 If it is, return 1, else return 0. */
aa72b389 1374
df7492f9
KH		 1375#define UTF_16_HIGH_SURROGATE_P(val) \
1376 (((val) & 0xFC00) == 0xD800)
1377
1378#define UTF_16_LOW_SURROGATE_P(val) \
1379 (((val) & 0xFC00) == 0xDC00)
93dec019 1380
df7492f9
KH		 1381#define UTF_16_INVALID_P(val) \
1382 (((val) == 0xFFFE) \
1383 || ((val) == 0xFFFF) \
1384 || UTF_16_LOW_SURROGATE_P (val))
aa72b389 1385
aa72b389 1386
df7492f9 1387static int
ff0dacd7 1388detect_coding_utf_16 (coding, detect_info)
aa72b389 1389 struct coding_system *coding;
ff0dacd7 1390 struct coding_detection_info *detect_info;
aa72b389 1391{
8f924df7
KH		 1392 const unsigned char *src = coding->source, *src_base = src;
1393 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 1394 int multibytep = coding->src_multibyte;
1395 int consumed_chars = 0;
1396 int c1, c2;
aa72b389 1397
ff0dacd7 1398 detect_info->checked |= CATEGORY_MASK_UTF_16;
ff0dacd7 1399 if (coding->mode & CODING_MODE_LAST_BLOCK
24a73b0a 1400 && (coding->src_chars & 1))
ff0dacd7
KH		 1401 {
1402 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1403 return 0;
1404 }
24a73b0a 1405
df7492f9
KH		 1406 ONE_MORE_BYTE (c1);
1407 ONE_MORE_BYTE (c2);
df7492f9 1408 if ((c1 == 0xFF) && (c2 == 0xFE))
aa72b389 1409 {
b49a1807
KH		 1410 detect_info->found |= (CATEGORY_MASK_UTF_16_LE
1411 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1412 detect_info->rejected |= (CATEGORY_MASK_UTF_16_BE
1413 | CATEGORY_MASK_UTF_16_BE_NOSIG
1414 | CATEGORY_MASK_UTF_16_LE_NOSIG);
aa72b389 1415 }
df7492f9 1416 else if ((c1 == 0xFE) && (c2 == 0xFF))
ff0dacd7 1417 {
b49a1807
KH		 1418 detect_info->found |= (CATEGORY_MASK_UTF_16_BE
1419 | CATEGORY_MASK_UTF_16_AUTO);
24a73b0a
KH		 1420 detect_info->rejected |= (CATEGORY_MASK_UTF_16_LE
1421 | CATEGORY_MASK_UTF_16_BE_NOSIG
1422 | CATEGORY_MASK_UTF_16_LE_NOSIG);
1423 }
065e3595 1424 else if (c1 >= 0 && c2 >= 0)
24a73b0a 1425 {
24a73b0a
KH		 1426 detect_info->rejected
1427 |= (CATEGORY_MASK_UTF_16_BE | CATEGORY_MASK_UTF_16_LE);
ff0dacd7 1428 }
df7492f9 1429 no_more_source:
ff0dacd7 1430 return 1;
df7492f9 1431}
aa72b389 1432
df7492f9
KH		 1433static void
1434decode_coding_utf_16 (coding)
1435 struct coding_system *coding;
1436{
8f924df7
KH		 1437 const unsigned char *src = coding->source + coding->consumed;
1438 const unsigned char *src_end = coding->source + coding->src_bytes;
1439 const unsigned char *src_base;
69a80ea3
KH		 1440 int *charbuf = coding->charbuf + coding->charbuf_used;
1441 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 1442 int consumed_chars = 0, consumed_chars_base;
1443 int multibytep = coding->src_multibyte;
1444 enum utf_16_bom_type bom = CODING_UTF_16_BOM (coding);
1445 enum utf_16_endian_type endian = CODING_UTF_16_ENDIAN (coding);
1446 int surrogate = CODING_UTF_16_SURROGATE (coding);
24a73b0a 1447 Lisp_Object attr, charset_list;
df7492f9 1448
24a73b0a 1449 CODING_GET_INFO (coding, attr, charset_list);
df7492f9 1450
b49a1807 1451 if (bom == utf_16_with_bom)
aa72b389 1452 {
df7492f9 1453 int c, c1, c2;
4af310db 1454
aa72b389 1455 src_base = src;
df7492f9
KH		 1456 ONE_MORE_BYTE (c1);
1457 ONE_MORE_BYTE (c2);
e19c3639 1458 c = (c1 << 8) | c2;
aa72b389 1459
b49a1807
KH		 1460 if (endian == utf_16_big_endian
1461 ? c != 0xFEFF : c != 0xFFFE)
aa72b389 1462 {
b49a1807
KH		 1463 /* The first two bytes are not BOM. Treat them as bytes
1464 for a normal character. */
1465 src = src_base;
1466 coding->errors++;
aa72b389 1467 }
b49a1807
KH		 1468 CODING_UTF_16_BOM (coding) = utf_16_without_bom;
1469 }
1470 else if (bom == utf_16_detect_bom)
1471 {
1472 /* We have already tried to detect BOM and failed in
1473 detect_coding. */
1474 CODING_UTF_16_BOM (coding) = utf_16_without_bom;
df7492f9 1475 }
aa72b389 1476
df7492f9
KH		 1477 while (1)
1478 {
1479 int c, c1, c2;
1480
1481 src_base = src;
1482 consumed_chars_base = consumed_chars;
1483
1484 if (charbuf + 2 >= charbuf_end)
1485 break;
1486
1487 ONE_MORE_BYTE (c1);
065e3595
KH		 1488 if (c1 < 0)
1489 {
1490 *charbuf++ = -c1;
1491 continue;
1492 }
df7492f9 1493 ONE_MORE_BYTE (c2);
065e3595
KH		 1494 if (c2 < 0)
1495 {
1496 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
1497 *charbuf++ = -c2;
1498 continue;
1499 }
df7492f9 1500 c = (endian == utf_16_big_endian
e19c3639 1501 ? ((c1 << 8) | c2) : ((c2 << 8) | c1));
df7492f9 1502 if (surrogate)
fd3ae0b9 1503 {
df7492f9 1504 if (! UTF_16_LOW_SURROGATE_P (c))
fd3ae0b9 1505 {
df7492f9
KH		 1506 if (endian == utf_16_big_endian)
1507 c1 = surrogate >> 8, c2 = surrogate & 0xFF;
1508 else
1509 c1 = surrogate & 0xFF, c2 = surrogate >> 8;
1510 *charbuf++ = c1;
1511 *charbuf++ = c2;
1512 coding->errors++;
1513 if (UTF_16_HIGH_SURROGATE_P (c))
1514 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
fd3ae0b9 1515 else
df7492f9 1516 *charbuf++ = c;
fd3ae0b9
KH		 1517 }
1518 else
df7492f9
KH		 1519 {
1520 c = ((surrogate - 0xD800) << 10) | (c - 0xDC00);
1521 CODING_UTF_16_SURROGATE (coding) = surrogate = 0;
29f7ffd0 1522 *charbuf++ = 0x10000 + c;
df7492f9 1523 }
fd3ae0b9 1524 }
aa72b389 1525 else
df7492f9
KH		 1526 {
1527 if (UTF_16_HIGH_SURROGATE_P (c))
1528 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1529 else
1530 *charbuf++ = c;
8f924df7 1531 }
aa72b389 1532 }
df7492f9
KH		 1533
1534 no_more_source:
1535 coding->consumed_char += consumed_chars_base;
1536 coding->consumed = src_base - coding->source;
1537 coding->charbuf_used = charbuf - coding->charbuf;
aa72b389 1538}
b73bfc1c 1539
df7492f9
KH		 1540static int
1541encode_coding_utf_16 (coding)
1542 struct coding_system *coding;
1543{
1544 int multibytep = coding->dst_multibyte;
1545 int *charbuf = coding->charbuf;
1546 int *charbuf_end = charbuf + coding->charbuf_used;
1547 unsigned char *dst = coding->destination + coding->produced;
1548 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1549 int safe_room = 8;
1550 enum utf_16_bom_type bom = CODING_UTF_16_BOM (coding);
1551 int big_endian = CODING_UTF_16_ENDIAN (coding) == utf_16_big_endian;
1552 int produced_chars = 0;
24a73b0a 1553 Lisp_Object attrs, charset_list;
df7492f9 1554 int c;
4ed46869 1555
24a73b0a 1556 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 1557
b49a1807 1558 if (bom != utf_16_without_bom)
df7492f9
KH		 1559 {
1560 ASSURE_DESTINATION (safe_room);
1561 if (big_endian)
df7492f9 1562 EMIT_TWO_BYTES (0xFE, 0xFF);
880cf180
KH		 1563 else
1564 EMIT_TWO_BYTES (0xFF, 0xFE);
df7492f9
KH		 1565 CODING_UTF_16_BOM (coding) = utf_16_without_bom;
1566 }
1567
1568 while (charbuf < charbuf_end)
1569 {
1570 ASSURE_DESTINATION (safe_room);
1571 c = *charbuf++;
e19c3639
KH		 1572 if (c >= MAX_UNICODE_CHAR)
1573 c = coding->default_char;
df7492f9
KH		 1574
1575 if (c < 0x10000)
1576 {
1577 if (big_endian)
1578 EMIT_TWO_BYTES (c >> 8, c & 0xFF);
1579 else
1580 EMIT_TWO_BYTES (c & 0xFF, c >> 8);
1581 }
1582 else
1583 {
1584 int c1, c2;
1585
1586 c -= 0x10000;
1587 c1 = (c >> 10) + 0xD800;
1588 c2 = (c & 0x3FF) + 0xDC00;
1589 if (big_endian)
1590 EMIT_FOUR_BYTES (c1 >> 8, c1 & 0xFF, c2 >> 8, c2 & 0xFF);
1591 else
1592 EMIT_FOUR_BYTES (c1 & 0xFF, c1 >> 8, c2 & 0xFF, c2 >> 8);
1593 }
1594 }
065e3595 1595 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 1596 coding->produced = dst - coding->destination;
1597 coding->produced_char += produced_chars;
1598 return 0;
1599}
1600
1601\f
1602/*** 6. Old Emacs' internal format (emacs-mule) ***/
1603
1604/* Emacs' internal format for representation of multiple character
1605 sets is a kind of multi-byte encoding, i.e. characters are
1606 represented by variable-length sequences of one-byte codes.
1607
1608 ASCII characters and control characters (e.g. `tab', `newline') are
1609 represented by one-byte sequences which are their ASCII codes, in
1610 the range 0x00 through 0x7F.
1611
1612 8-bit characters of the range 0x80..0x9F are represented by
1613 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
1614 code + 0x20).
1615
1616 8-bit characters of the range 0xA0..0xFF are represented by
1617 one-byte sequences which are their 8-bit code.
1618
1619 The other characters are represented by a sequence of `base
1620 leading-code', optional `extended leading-code', and one or two
1621 `position-code's. The length of the sequence is determined by the
1622 base leading-code. Leading-code takes the range 0x81 through 0x9D,
1623 whereas extended leading-code and position-code take the range 0xA0
1624 through 0xFF. See `charset.h' for more details about leading-code
1625 and position-code.
1626
1627 --- CODE RANGE of Emacs' internal format ---
1628 character set range
1629 ------------- -----
1630 ascii 0x00..0x7F
1631 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
1632 eight-bit-graphic 0xA0..0xBF
1633 ELSE 0x81..0x9D + [0xA0..0xFF]+
1634 ---------------------------------------------
1635
1636 As this is the internal character representation, the format is
1637 usually not used externally (i.e. in a file or in a data sent to a
1638 process). But, it is possible to have a text externally in this
1639 format (i.e. by encoding by the coding system `emacs-mule').
1640
1641 In that case, a sequence of one-byte codes has a slightly different
1642 form.
1643
1644 At first, all characters in eight-bit-control are represented by
1645 one-byte sequences which are their 8-bit code.
1646
1647 Next, character composition data are represented by the byte
1648 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
1649 where,
1650 METHOD is 0xF0 plus one of composition method (enum
1651 composition_method),
1652
1653 BYTES is 0xA0 plus a byte length of this composition data,
1654
1655 CHARS is 0x20 plus a number of characters composed by this
1656 data,
1657
1658 COMPONENTs are characters of multibye form or composition
1659 rules encoded by two-byte of ASCII codes.
1660
1661 In addition, for backward compatibility, the following formats are
1662 also recognized as composition data on decoding.
1663
1664 0x80 MSEQ ...
1665 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
1666
1667 Here,
1668 MSEQ is a multibyte form but in these special format:
1669 ASCII: 0xA0 ASCII_CODE+0x80,
1670 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
1671 RULE is a one byte code of the range 0xA0..0xF0 that
1672 represents a composition rule.
1673 */
1674
1675char emacs_mule_bytes[256];
1676
df7492f9 1677int
ff0dacd7 1678emacs_mule_char (coding, src, nbytes, nchars, id)
df7492f9 1679 struct coding_system *coding;
065e3595 1680 const unsigned char *src;
ff0dacd7 1681 int *nbytes, *nchars, *id;
df7492f9 1682{
8f924df7
KH		 1683 const unsigned char *src_end = coding->source + coding->src_bytes;
1684 const unsigned char *src_base = src;
df7492f9 1685 int multibytep = coding->src_multibyte;
df7492f9
KH		 1686 struct charset *charset;
1687 unsigned code;
1688 int c;
1689 int consumed_chars = 0;
1690
1691 ONE_MORE_BYTE (c);
065e3595 1692 if (c < 0)
df7492f9 1693 {
065e3595
KH		 1694 c = -c;
1695 charset = emacs_mule_charset[0];
1696 }
1697 else
1698 {
1699 switch (emacs_mule_bytes[c])
b73bfc1c 1700 {
065e3595 1701 case 2:
df7492f9
KH		 1702 if (! (charset = emacs_mule_charset[c]))
1703 goto invalid_code;
1704 ONE_MORE_BYTE (c);
9ffd559c 1705 if (c < 0xA0)
065e3595 1706 goto invalid_code;
df7492f9 1707 code = c & 0x7F;
065e3595
KH		 1708 break;
1709
1710 case 3:
1711 if (c == EMACS_MULE_LEADING_CODE_PRIVATE_11
1712 || c == EMACS_MULE_LEADING_CODE_PRIVATE_12)
1713 {
1714 ONE_MORE_BYTE (c);
9ffd559c 1715 if (c < 0xA0 || ! (charset = emacs_mule_charset[c]))
065e3595
KH		 1716 goto invalid_code;
1717 ONE_MORE_BYTE (c);
9ffd559c 1718 if (c < 0xA0)
065e3595
KH		 1719 goto invalid_code;
1720 code = c & 0x7F;
1721 }
1722 else
1723 {
1724 if (! (charset = emacs_mule_charset[c]))
1725 goto invalid_code;
1726 ONE_MORE_BYTE (c);
9ffd559c 1727 if (c < 0xA0)
065e3595
KH		 1728 goto invalid_code;
1729 code = (c & 0x7F) << 8;
1730 ONE_MORE_BYTE (c);
9ffd559c 1731 if (c < 0xA0)
065e3595
KH		 1732 goto invalid_code;
1733 code |= c & 0x7F;
1734 }
1735 break;
1736
1737 case 4:
1738 ONE_MORE_BYTE (c);
1739 if (c < 0 || ! (charset = emacs_mule_charset[c]))
df7492f9
KH		 1740 goto invalid_code;
1741 ONE_MORE_BYTE (c);
9ffd559c 1742 if (c < 0xA0)
065e3595 1743 goto invalid_code;
781d7a48 1744 code = (c & 0x7F) << 8;
df7492f9 1745 ONE_MORE_BYTE (c);
9ffd559c 1746 if (c < 0xA0)
065e3595 1747 goto invalid_code;
df7492f9 1748 code |= c & 0x7F;
065e3595 1749 break;
df7492f9 1750
065e3595
KH		 1751 case 1:
1752 code = c;
1753 charset = CHARSET_FROM_ID (ASCII_BYTE_P (code)
1754 ? charset_ascii : charset_eight_bit);
1755 break;
df7492f9 1756
065e3595
KH		 1757 default:
1758 abort ();
1759 }
1760 c = DECODE_CHAR (charset, code);
1761 if (c < 0)
1762 goto invalid_code;
df7492f9 1763 }
df7492f9
KH		 1764 *nbytes = src - src_base;
1765 *nchars = consumed_chars;
ff0dacd7
KH		 1766 if (id)
1767 *id = charset->id;
df7492f9
KH		 1768 return c;
1769
1770 no_more_source:
1771 return -2;
1772
1773 invalid_code:
1774 return -1;
1775}
1776
1777
1778/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 1779 Check if a text is encoded in `emacs-mule'. If it is, return 1,
1780 else return 0. */
df7492f9
KH		 1781
1782static int
ff0dacd7 1783detect_coding_emacs_mule (coding, detect_info)
df7492f9 1784 struct coding_system *coding;
ff0dacd7 1785 struct coding_detection_info *detect_info;
df7492f9 1786{
065e3595 1787 const unsigned char *src = coding->source, *src_base;
8f924df7 1788 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 1789 int multibytep = coding->src_multibyte;
1790 int consumed_chars = 0;
1791 int c;
1792 int found = 0;
1793
ff0dacd7 1794 detect_info->checked |= CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 1795 /* A coding system of this category is always ASCII compatible. */
1796 src += coding->head_ascii;
1797
1798 while (1)
1799 {
065e3595 1800 src_base = src;
df7492f9 1801 ONE_MORE_BYTE (c);
065e3595
KH		 1802 if (c < 0)
1803 continue;
df7492f9
KH		 1804 if (c == 0x80)
1805 {
1806 /* Perhaps the start of composite character. We simple skip
1807 it because analyzing it is too heavy for detecting. But,
1808 at least, we check that the composite character
1809 constitues of more than 4 bytes. */
8f924df7 1810 const unsigned char *src_base;
df7492f9
KH		 1811
1812 repeat:
1813 src_base = src;
1814 do
1815 {
1816 ONE_MORE_BYTE (c);
1817 }
1818 while (c >= 0xA0);
1819
1820 if (src - src_base <= 4)
1821 break;
ff0dacd7 1822 found = CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 1823 if (c == 0x80)
1824 goto repeat;
b73bfc1c 1825 }
df7492f9
KH		 1826
1827 if (c < 0x80)
b73bfc1c 1828 {
df7492f9
KH		 1829 if (c < 0x20
1830 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO))
1831 break;
1832 }
1833 else
1834 {
0e219d54 1835 int more_bytes = emacs_mule_bytes[*src_base] - 1;
df7492f9 1836
0e219d54 1837 while (more_bytes > 0)
df7492f9
KH		 1838 {
1839 ONE_MORE_BYTE (c);
0e219d54
KH		 1840 if (c < 0xA0)
1841 {
1842 src--; /* Unread the last byte. */
1843 break;
1844 }
1845 more_bytes--;
df7492f9 1846 }
0e219d54 1847 if (more_bytes != 0)
df7492f9 1848 break;
ff0dacd7 1849 found = CATEGORY_MASK_EMACS_MULE;
4ed46869
KH		 1850 }
1851 }
ff0dacd7 1852 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
df7492f9
KH		 1853 return 0;
1854
1855 no_more_source:
065e3595 1856 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 1857 {
ff0dacd7 1858 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
89528eb3
KH		 1859 return 0;
1860 }
ff0dacd7
KH		 1861 detect_info->found |= found;
1862 return 1;
4ed46869
KH		 1863}
1864
b73bfc1c 1865
df7492f9
KH		 1866/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1867
1868/* Decode a character represented as a component of composition
1869 sequence of Emacs 20/21 style at SRC. Set C to that character and
1870 update SRC to the head of next character (or an encoded composition
1871 rule). If SRC doesn't points a composition component, set C to -1.
1872 If SRC points an invalid byte sequence, global exit by a return
1873 value 0. */
1874
1875#define DECODE_EMACS_MULE_COMPOSITION_CHAR(buf) \
1876 if (1) \
1877 { \
1878 int c; \
1879 int nbytes, nchars; \
1880 \
1881 if (src == src_end) \
1882 break; \
ff0dacd7 1883 c = emacs_mule_char (coding, src, &nbytes, &nchars, NULL);\
df7492f9
KH		 1884 if (c < 0) \
1885 { \
1886 if (c == -2) \
1887 break; \
1888 goto invalid_code; \
1889 } \
1890 *buf++ = c; \
1891 src += nbytes; \
1892 consumed_chars += nchars; \
1893 } \
1894 else
1895
1896
1897/* Decode a composition rule represented as a component of composition
781d7a48
KH		 1898 sequence of Emacs 20 style at SRC. Store the decoded rule in *BUF,
1899 and increment BUF. If SRC points an invalid byte sequence, set C
1900 to -1. */
df7492f9 1901
781d7a48 1902#define DECODE_EMACS_MULE_COMPOSITION_RULE_20(buf) \
df7492f9
KH		 1903 do { \
1904 int c, gref, nref; \
1905 \
781d7a48 1906 if (src >= src_end) \
df7492f9
KH		 1907 goto invalid_code; \
1908 ONE_MORE_BYTE_NO_CHECK (c); \
781d7a48 1909 c -= 0x20; \
df7492f9
KH		 1910 if (c < 0 || c >= 81) \
1911 goto invalid_code; \
1912 \
1913 gref = c / 9, nref = c % 9; \
1914 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
1915 } while (0)
1916
1917
781d7a48
KH		 1918/* Decode a composition rule represented as a component of composition
1919 sequence of Emacs 21 style at SRC. Store the decoded rule in *BUF,
1920 and increment BUF. If SRC points an invalid byte sequence, set C
1921 to -1. */
1922
1923#define DECODE_EMACS_MULE_COMPOSITION_RULE_21(buf) \
1924 do { \
1925 int gref, nref; \
1926 \
1927 if (src + 1>= src_end) \
1928 goto invalid_code; \
1929 ONE_MORE_BYTE_NO_CHECK (gref); \
1930 gref -= 0x20; \
1931 ONE_MORE_BYTE_NO_CHECK (nref); \
1932 nref -= 0x20; \
1933 if (gref < 0 || gref >= 81 \
1934 || nref < 0 || nref >= 81) \
1935 goto invalid_code; \
1936 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
1937 } while (0)
1938
1939
df7492f9 1940#define DECODE_EMACS_MULE_21_COMPOSITION(c) \
aa72b389 1941 do { \
df7492f9 1942 /* Emacs 21 style format. The first three bytes at SRC are \
781d7a48 1943 (METHOD - 0xF2), (BYTES - 0xA0), (CHARS - 0xA0), where BYTES is \
df7492f9
KH		 1944 the byte length of this composition information, CHARS is the \
1945 number of characters composed by this composition. */ \
781d7a48
KH		 1946 enum composition_method method = c - 0xF2; \
1947 int *charbuf_base = charbuf; \
df7492f9
KH		 1948 int consumed_chars_limit; \
1949 int nbytes, nchars; \
1950 \
1951 ONE_MORE_BYTE (c); \
065e3595
KH		 1952 if (c < 0) \
1953 goto invalid_code; \
df7492f9
KH		 1954 nbytes = c - 0xA0; \
1955 if (nbytes < 3) \
1956 goto invalid_code; \
1957 ONE_MORE_BYTE (c); \
065e3595
KH		 1958 if (c < 0) \
1959 goto invalid_code; \
df7492f9 1960 nchars = c - 0xA0; \
69a80ea3 1961 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
df7492f9
KH		 1962 consumed_chars_limit = consumed_chars_base + nbytes; \
1963 if (method != COMPOSITION_RELATIVE) \
aa72b389 1964 { \
df7492f9
KH		 1965 int i = 0; \
1966 while (consumed_chars < consumed_chars_limit) \
aa72b389 1967 { \
df7492f9 1968 if (i % 2 && method != COMPOSITION_WITH_ALTCHARS) \
781d7a48 1969 DECODE_EMACS_MULE_COMPOSITION_RULE_21 (charbuf); \
df7492f9
KH		 1970 else \
1971 DECODE_EMACS_MULE_COMPOSITION_CHAR (charbuf); \
781d7a48 1972 i++; \
aa72b389 1973 } \
df7492f9
KH		 1974 if (consumed_chars < consumed_chars_limit) \
1975 goto invalid_code; \
781d7a48 1976 charbuf_base[0] -= i; \
aa72b389
KH		 1977 } \
1978 } while (0)
93dec019 1979
aa72b389 1980
df7492f9
KH		 1981#define DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION(c) \
1982 do { \
1983 /* Emacs 20 style format for relative composition. */ \
1984 /* Store multibyte form of characters to be composed. */ \
ff0dacd7 1985 enum composition_method method = COMPOSITION_RELATIVE; \
df7492f9
KH		 1986 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
1987 int *buf = components; \
1988 int i, j; \
1989 \
1990 src = src_base; \
1991 ONE_MORE_BYTE (c); /* skip 0x80 */ \
1992 for (i = 0; i < MAX_COMPOSITION_COMPONENTS; i++) \
1993 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
1994 if (i < 2) \
1995 goto invalid_code; \
69a80ea3 1996 ADD_COMPOSITION_DATA (charbuf, i, method); \
df7492f9
KH		 1997 for (j = 0; j < i; j++) \
1998 *charbuf++ = components[j]; \
1999 } while (0)
2000
2001
2002#define DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION(c) \
2003 do { \
2004 /* Emacs 20 style format for rule-base composition. */ \
2005 /* Store multibyte form of characters to be composed. */ \
ff0dacd7 2006 enum composition_method method = COMPOSITION_WITH_RULE; \
df7492f9
KH		 2007 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
2008 int *buf = components; \
2009 int i, j; \
2010 \
2011 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2012 for (i = 0; i < MAX_COMPOSITION_COMPONENTS; i++) \
2013 { \
781d7a48 2014 DECODE_EMACS_MULE_COMPOSITION_RULE_20 (buf); \
df7492f9
KH		 2015 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2016 } \
2017 if (i < 1 || (buf - components) % 2 == 0) \
2018 goto invalid_code; \
2019 if (charbuf + i + (i / 2) + 1 < charbuf_end) \
2020 goto no_more_source; \
69a80ea3 2021 ADD_COMPOSITION_DATA (buf, i, method); \
df7492f9
KH		 2022 for (j = 0; j < i; j++) \
2023 *charbuf++ = components[j]; \
2024 for (j = 0; j < i; j += 2) \
2025 *charbuf++ = components[j]; \
2026 } while (0)
2027
aa72b389
KH		 2028
2029static void
df7492f9 2030decode_coding_emacs_mule (coding)
aa72b389 2031 struct coding_system *coding;
aa72b389 2032{
8f924df7
KH		 2033 const unsigned char *src = coding->source + coding->consumed;
2034 const unsigned char *src_end = coding->source + coding->src_bytes;
2035 const unsigned char *src_base;
69a80ea3
KH		 2036 int *charbuf = coding->charbuf + coding->charbuf_used;
2037 int *charbuf_end
2038 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9 2039 int consumed_chars = 0, consumed_chars_base;
df7492f9 2040 int multibytep = coding->src_multibyte;
24a73b0a 2041 Lisp_Object attrs, charset_list;
ff0dacd7
KH		 2042 int char_offset = coding->produced_char;
2043 int last_offset = char_offset;
2044 int last_id = charset_ascii;
aa72b389 2045
24a73b0a 2046 CODING_GET_INFO (coding, attrs, charset_list);
aa72b389 2047
aa72b389
KH		 2048 while (1)
2049 {
df7492f9
KH		 2050 int c;
2051
aa72b389 2052 src_base = src;
df7492f9
KH		 2053 consumed_chars_base = consumed_chars;
2054
2055 if (charbuf >= charbuf_end)
2056 break;
aa72b389 2057
df7492f9 2058 ONE_MORE_BYTE (c);
065e3595
KH		 2059 if (c < 0)
2060 {
2061 *charbuf++ = -c;
2062 char_offset++;
2063 }
2064 else if (c < 0x80)
aa72b389 2065 {
df7492f9
KH		 2066 *charbuf++ = c;
2067 char_offset++;
aa72b389 2068 }
df7492f9
KH		 2069 else if (c == 0x80)
2070 {
df7492f9 2071 ONE_MORE_BYTE (c);
065e3595
KH		 2072 if (c < 0)
2073 goto invalid_code;
781d7a48
KH		 2074 if (c - 0xF2 >= COMPOSITION_RELATIVE
2075 && c - 0xF2 <= COMPOSITION_WITH_RULE_ALTCHARS)
df7492f9
KH		 2076 DECODE_EMACS_MULE_21_COMPOSITION (c);
2077 else if (c < 0xC0)
2078 DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION (c);
2079 else if (c == 0xFF)
2080 DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION (c);
2081 else
2082 goto invalid_code;
2083 }
2084 else if (c < 0xA0 && emacs_mule_bytes[c] > 1)
2085 {
2086 int nbytes, nchars;
ff0dacd7
KH		 2087 int id;
2088
781d7a48
KH		 2089 src = src_base;
2090 consumed_chars = consumed_chars_base;
ff0dacd7 2091 c = emacs_mule_char (coding, src, &nbytes, &nchars, &id);
df7492f9
KH		 2092 if (c < 0)
2093 {
2094 if (c == -2)
2095 break;
2096 goto invalid_code;
2097 }
ff0dacd7
KH		 2098 if (last_id != id)
2099 {
2100 if (last_id != charset_ascii)
69a80ea3 2101 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
ff0dacd7
KH		 2102 last_id = id;
2103 last_offset = char_offset;
2104 }
df7492f9 2105 *charbuf++ = c;
781d7a48
KH		 2106 src += nbytes;
2107 consumed_chars += nchars;
df7492f9
KH		 2108 char_offset++;
2109 }
2110 continue;
2111
2112 invalid_code:
2113 src = src_base;
2114 consumed_chars = consumed_chars_base;
2115 ONE_MORE_BYTE (c);
2116 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 2117 char_offset++;
df7492f9
KH		 2118 coding->errors++;
2119 }
2120
2121 no_more_source:
ff0dacd7 2122 if (last_id != charset_ascii)
69a80ea3 2123 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 2124 coding->consumed_char += consumed_chars_base;
2125 coding->consumed = src_base - coding->source;
2126 coding->charbuf_used = charbuf - coding->charbuf;
2127}
2128
2129
2130#define EMACS_MULE_LEADING_CODES(id, codes) \
2131 do { \
2132 if (id < 0xA0) \
2133 codes[0] = id, codes[1] = 0; \
2134 else if (id < 0xE0) \
2135 codes[0] = 0x9A, codes[1] = id; \
2136 else if (id < 0xF0) \
2137 codes[0] = 0x9B, codes[1] = id; \
2138 else if (id < 0xF5) \
2139 codes[0] = 0x9C, codes[1] = id; \
2140 else \
2141 codes[0] = 0x9D, codes[1] = id; \
2142 } while (0);
2143
aa72b389 2144
df7492f9
KH		 2145static int
2146encode_coding_emacs_mule (coding)
2147 struct coding_system *coding;
2148{
2149 int multibytep = coding->dst_multibyte;
2150 int *charbuf = coding->charbuf;
2151 int *charbuf_end = charbuf + coding->charbuf_used;
2152 unsigned char *dst = coding->destination + coding->produced;
2153 unsigned char *dst_end = coding->destination + coding->dst_bytes;
2154 int safe_room = 8;
df7492f9 2155 int produced_chars = 0;
24a73b0a 2156 Lisp_Object attrs, charset_list;
df7492f9 2157 int c;
ff0dacd7 2158 int preferred_charset_id = -1;
df7492f9 2159
24a73b0a 2160 CODING_GET_INFO (coding, attrs, charset_list);
eccb6815
KH		 2161 if (! EQ (charset_list, Vemacs_mule_charset_list))
2162 {
2163 CODING_ATTR_CHARSET_LIST (attrs)
2164 = charset_list = Vemacs_mule_charset_list;
2165 }
df7492f9
KH		 2166
2167 while (charbuf < charbuf_end)
2168 {
2169 ASSURE_DESTINATION (safe_room);
2170 c = *charbuf++;
ff0dacd7
KH		 2171
2172 if (c < 0)
2173 {
2174 /* Handle an annotation. */
2175 switch (*charbuf)
2176 {
2177 case CODING_ANNOTATE_COMPOSITION_MASK:
2178 /* Not yet implemented. */
2179 break;
2180 case CODING_ANNOTATE_CHARSET_MASK:
2181 preferred_charset_id = charbuf[3];
2182 if (preferred_charset_id >= 0
2183 && NILP (Fmemq (make_number (preferred_charset_id),
2184 charset_list)))
2185 preferred_charset_id = -1;
2186 break;
2187 default:
2188 abort ();
2189 }
2190 charbuf += -c - 1;
2191 continue;
2192 }
2193
df7492f9
KH		 2194 if (ASCII_CHAR_P (c))
2195 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 2196 else if (CHAR_BYTE8_P (c))
2197 {
2198 c = CHAR_TO_BYTE8 (c);
2199 EMIT_ONE_BYTE (c);
2200 }
df7492f9 2201 else
aa72b389 2202 {
df7492f9
KH		 2203 struct charset *charset;
2204 unsigned code;
2205 int dimension;
2206 int emacs_mule_id;
2207 unsigned char leading_codes[2];
2208
ff0dacd7
KH		 2209 if (preferred_charset_id >= 0)
2210 {
2211 charset = CHARSET_FROM_ID (preferred_charset_id);
2212 if (! CHAR_CHARSET_P (c, charset))
2213 charset = char_charset (c, charset_list, NULL);
2214 }
2215 else
2216 charset = char_charset (c, charset_list, &code);
df7492f9
KH		 2217 if (! charset)
2218 {
2219 c = coding->default_char;
2220 if (ASCII_CHAR_P (c))
2221 {
2222 EMIT_ONE_ASCII_BYTE (c);
2223 continue;
2224 }
2225 charset = char_charset (c, charset_list, &code);
2226 }
2227 dimension = CHARSET_DIMENSION (charset);
2228 emacs_mule_id = CHARSET_EMACS_MULE_ID (charset);
2229 EMACS_MULE_LEADING_CODES (emacs_mule_id, leading_codes);
2230 EMIT_ONE_BYTE (leading_codes[0]);
2231 if (leading_codes[1])
2232 EMIT_ONE_BYTE (leading_codes[1]);
2233 if (dimension == 1)
1fa663f9 2234 EMIT_ONE_BYTE (code | 0x80);
aa72b389 2235 else
df7492f9 2236 {
1fa663f9 2237 code |= 0x8080;
df7492f9
KH		 2238 EMIT_ONE_BYTE (code >> 8);
2239 EMIT_ONE_BYTE (code & 0xFF);
2240 }
aa72b389 2241 }
aa72b389 2242 }
065e3595 2243 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 2244 coding->produced_char += produced_chars;
2245 coding->produced = dst - coding->destination;
2246 return 0;
aa72b389 2247}
b73bfc1c 2248
4ed46869 2249\f
df7492f9 2250/*** 7. ISO2022 handlers ***/
4ed46869
KH		 2251
2252/* The following note describes the coding system ISO2022 briefly.
39787efd 2253 Since the intention of this note is to help understand the
5a936b46 2254 functions in this file, some parts are NOT ACCURATE or are OVERLY
39787efd 2255 SIMPLIFIED. For thorough understanding, please refer to the
5a936b46 2256 original document of ISO2022. This is equivalent to the standard
cfb43547 2257 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
4ed46869
KH		 2258
2259 ISO2022 provides many mechanisms to encode several character sets
cfb43547 2260 in 7-bit and 8-bit environments. For 7-bit environments, all text
39787efd
KH		 2261 is encoded using bytes less than 128. This may make the encoded
2262 text a little bit longer, but the text passes more easily through
cfb43547 2263 several types of gateway, some of which strip off the MSB (Most
8ca3766a 2264 Significant Bit).
b73bfc1c 2265
cfb43547
DL		 2266 There are two kinds of character sets: control character sets and
2267 graphic character sets. The former contain control characters such
4ed46869 2268 as `newline' and `escape' to provide control functions (control
39787efd 2269 functions are also provided by escape sequences). The latter
cfb43547 2270 contain graphic characters such as 'A' and '-'. Emacs recognizes
4ed46869
KH		 2271 two control character sets and many graphic character sets.
2272
2273 Graphic character sets are classified into one of the following
39787efd
KH		 2274 four classes, according to the number of bytes (DIMENSION) and
2275 number of characters in one dimension (CHARS) of the set:
2276 - DIMENSION1_CHARS94
2277 - DIMENSION1_CHARS96
2278 - DIMENSION2_CHARS94
2279 - DIMENSION2_CHARS96
2280
2281 In addition, each character set is assigned an identification tag,
cfb43547 2282 unique for each set, called the "final character" (denoted as <F>
39787efd
KH		 2283 hereafter). The <F> of each character set is decided by ECMA(*)
2284 when it is registered in ISO. The code range of <F> is 0x30..0x7F
2285 (0x30..0x3F are for private use only).
4ed46869
KH		 2286
2287 Note (*): ECMA = European Computer Manufacturers Association
2288
cfb43547 2289 Here are examples of graphic character sets [NAME(<F>)]:
4ed46869
KH		 2290 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
2291 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
2292 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
2293 o DIMENSION2_CHARS96 -- none for the moment
2294
39787efd 2295 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
4ed46869
KH		 2296 C0 [0x00..0x1F] -- control character plane 0
2297 GL [0x20..0x7F] -- graphic character plane 0
2298 C1 [0x80..0x9F] -- control character plane 1
2299 GR [0xA0..0xFF] -- graphic character plane 1
2300
2301 A control character set is directly designated and invoked to C0 or
39787efd
KH		 2302 C1 by an escape sequence. The most common case is that:
2303 - ISO646's control character set is designated/invoked to C0, and
2304 - ISO6429's control character set is designated/invoked to C1,
2305 and usually these designations/invocations are omitted in encoded
2306 text. In a 7-bit environment, only C0 can be used, and a control
2307 character for C1 is encoded by an appropriate escape sequence to
2308 fit into the environment. All control characters for C1 are
2309 defined to have corresponding escape sequences.
4ed46869
KH		 2310
2311 A graphic character set is at first designated to one of four
2312 graphic registers (G0 through G3), then these graphic registers are
2313 invoked to GL or GR. These designations and invocations can be
2314 done independently. The most common case is that G0 is invoked to
39787efd
KH		 2315 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
2316 these invocations and designations are omitted in encoded text.
2317 In a 7-bit environment, only GL can be used.
4ed46869 2318
39787efd
KH		 2319 When a graphic character set of CHARS94 is invoked to GL, codes
2320 0x20 and 0x7F of the GL area work as control characters SPACE and
2321 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
2322 be used.
4ed46869
KH		 2323
2324 There are two ways of invocation: locking-shift and single-shift.
2325 With locking-shift, the invocation lasts until the next different
39787efd
KH		 2326 invocation, whereas with single-shift, the invocation affects the
2327 following character only and doesn't affect the locking-shift
2328 state. Invocations are done by the following control characters or
2329 escape sequences:
4ed46869
KH		 2330
2331 ----------------------------------------------------------------------
39787efd 2332 abbrev function cntrl escape seq description
4ed46869 2333 ----------------------------------------------------------------------
39787efd
KH		 2334 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
2335 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
2336 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
2337 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
2338 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
2339 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
2340 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
2341 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
2342 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
4ed46869 2343 ----------------------------------------------------------------------
39787efd
KH		 2344 (*) These are not used by any known coding system.
2345
2346 Control characters for these functions are defined by macros
2347 ISO_CODE_XXX in `coding.h'.
4ed46869 2348
39787efd 2349 Designations are done by the following escape sequences:
4ed46869
KH		 2350 ----------------------------------------------------------------------
2351 escape sequence description
2352 ----------------------------------------------------------------------
2353 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
2354 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
2355 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
2356 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
2357 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
2358 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
2359 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
2360 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
2361 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
2362 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
2363 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
2364 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
2365 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
2366 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
2367 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
2368 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
2369 ----------------------------------------------------------------------
2370
2371 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
39787efd 2372 of dimension 1, chars 94, and final character <F>, etc...
4ed46869
KH		 2373
2374 Note (*): Although these designations are not allowed in ISO2022,
2375 Emacs accepts them on decoding, and produces them on encoding
39787efd 2376 CHARS96 character sets in a coding system which is characterized as
4ed46869
KH		 2377 7-bit environment, non-locking-shift, and non-single-shift.
2378
2379 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
df7492f9 2380 '(' must be omitted. We refer to this as "short-form" hereafter.
4ed46869 2381
cfb43547 2382 Now you may notice that there are a lot of ways of encoding the
39787efd
KH		 2383 same multilingual text in ISO2022. Actually, there exist many
2384 coding systems such as Compound Text (used in X11's inter client
8ca3766a
DL		 2385 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
2386 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
4ed46869
KH		 2387 localized platforms), and all of these are variants of ISO2022.
2388
2389 In addition to the above, Emacs handles two more kinds of escape
2390 sequences: ISO6429's direction specification and Emacs' private
2391 sequence for specifying character composition.
2392
39787efd 2393 ISO6429's direction specification takes the following form:
4ed46869
KH		 2394 o CSI ']' -- end of the current direction
2395 o CSI '0' ']' -- end of the current direction
2396 o CSI '1' ']' -- start of left-to-right text
2397 o CSI '2' ']' -- start of right-to-left text
2398 The control character CSI (0x9B: control sequence introducer) is
39787efd
KH		 2399 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
2400
2401 Character composition specification takes the following form:
ec6d2bb8
KH		 2402 o ESC '0' -- start relative composition
2403 o ESC '1' -- end composition
2404 o ESC '2' -- start rule-base composition (*)
2405 o ESC '3' -- start relative composition with alternate chars (**)
2406 o ESC '4' -- start rule-base composition with alternate chars (**)
b73bfc1c 2407 Since these are not standard escape sequences of any ISO standard,
cfb43547 2408 the use of them with these meanings is restricted to Emacs only.
ec6d2bb8 2409
5a936b46
DL		 2410 (*) This form is used only in Emacs 20.7 and older versions,
2411 but newer versions can safely decode it.
cfb43547 2412 (**) This form is used only in Emacs 21.1 and newer versions,
5a936b46 2413 and older versions can't decode it.
ec6d2bb8 2414
cfb43547 2415 Here's a list of example usages of these composition escape
b73bfc1c 2416 sequences (categorized by `enum composition_method').
ec6d2bb8 2417
b73bfc1c 2418 COMPOSITION_RELATIVE:
ec6d2bb8 2419 ESC 0 CHAR [ CHAR ] ESC 1
8ca3766a 2420 COMPOSITION_WITH_RULE:
ec6d2bb8 2421 ESC 2 CHAR [ RULE CHAR ] ESC 1
b73bfc1c 2422 COMPOSITION_WITH_ALTCHARS:
ec6d2bb8 2423 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
b73bfc1c 2424 COMPOSITION_WITH_RULE_ALTCHARS:
ec6d2bb8 2425 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
4ed46869
KH		 2426
2427enum iso_code_class_type iso_code_class[256];
2428
df7492f9
KH		 2429#define SAFE_CHARSET_P(coding, id) \
2430 ((id) <= (coding)->max_charset_id \
2431 && (coding)->safe_charsets[id] >= 0)
2432
2433
2434#define SHIFT_OUT_OK(category) \
2435 (CODING_ISO_INITIAL (&coding_categories[category], 1) >= 0)
2436
2437static void
f0064e1f
DL		 2438setup_iso_safe_charsets (attrs)
2439 Lisp_Object attrs;
df7492f9
KH		 2440{
2441 Lisp_Object charset_list, safe_charsets;
2442 Lisp_Object request;
2443 Lisp_Object reg_usage;
2444 Lisp_Object tail;
2445 int reg94, reg96;
2446 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
2447 int max_charset_id;
2448
2449 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2450 if ((flags & CODING_ISO_FLAG_FULL_SUPPORT)
2451 && ! EQ (charset_list, Viso_2022_charset_list))
2452 {
2453 CODING_ATTR_CHARSET_LIST (attrs)
2454 = charset_list = Viso_2022_charset_list;
2455 ASET (attrs, coding_attr_safe_charsets, Qnil);
2456 }
2457
2458 if (STRINGP (AREF (attrs, coding_attr_safe_charsets)))
2459 return;
2460
2461 max_charset_id = 0;
2462 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2463 {
2464 int id = XINT (XCAR (tail));
2465 if (max_charset_id < id)
2466 max_charset_id = id;
2467 }
d46c5b12 2468
df7492f9
KH		 2469 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
2470 make_number (255));
2471 request = AREF (attrs, coding_attr_iso_request);
2472 reg_usage = AREF (attrs, coding_attr_iso_usage);
2473 reg94 = XINT (XCAR (reg_usage));
2474 reg96 = XINT (XCDR (reg_usage));
2475
2476 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2477 {
2478 Lisp_Object id;
2479 Lisp_Object reg;
2480 struct charset *charset;
2481
2482 id = XCAR (tail);
2483 charset = CHARSET_FROM_ID (XINT (id));
bf16eb23 2484 reg = Fcdr (Fassq (id, request));
df7492f9 2485 if (! NILP (reg))
8f924df7 2486 SSET (safe_charsets, XINT (id), XINT (reg));
df7492f9
KH		 2487 else if (charset->iso_chars_96)
2488 {
2489 if (reg96 < 4)
8f924df7 2490 SSET (safe_charsets, XINT (id), reg96);
df7492f9
KH		 2491 }
2492 else
2493 {
2494 if (reg94 < 4)
8f924df7 2495 SSET (safe_charsets, XINT (id), reg94);
df7492f9
KH		 2496 }
2497 }
2498 ASET (attrs, coding_attr_safe_charsets, safe_charsets);
2499}
d46c5b12 2500
b6871cc7 2501
4ed46869 2502/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
ff0dacd7
KH		 2503 Check if a text is encoded in one of ISO-2022 based codig systems.
2504 If it is, return 1, else return 0. */
4ed46869 2505
0a28aafb 2506static int
ff0dacd7 2507detect_coding_iso_2022 (coding, detect_info)
df7492f9 2508 struct coding_system *coding;
ff0dacd7 2509 struct coding_detection_info *detect_info;
4ed46869 2510{
8f924df7
KH		 2511 const unsigned char *src = coding->source, *src_base = src;
2512 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9 2513 int multibytep = coding->src_multibyte;
ff0dacd7 2514 int single_shifting = 0;
df7492f9
KH		 2515 int id;
2516 int c, c1;
2517 int consumed_chars = 0;
2518 int i;
ff0dacd7
KH		 2519 int rejected = 0;
2520 int found = 0;
2521
2522 detect_info->checked |= CATEGORY_MASK_ISO;
df7492f9
KH		 2523
2524 for (i = coding_category_iso_7; i <= coding_category_iso_8_else; i++)
2525 {
2526 struct coding_system *this = &(coding_categories[i]);
2527 Lisp_Object attrs, val;
2528
2529 attrs = CODING_ID_ATTRS (this->id);
2530 if (CODING_ISO_FLAGS (this) & CODING_ISO_FLAG_FULL_SUPPORT
2531 && ! EQ (CODING_ATTR_SAFE_CHARSETS (attrs), Viso_2022_charset_list))
2532 setup_iso_safe_charsets (attrs);
2533 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 2534 this->max_charset_id = SCHARS (val) - 1;
2535 this->safe_charsets = (char *) SDATA (val);
df7492f9
KH		 2536 }
2537
2538 /* A coding system of this category is always ASCII compatible. */
2539 src += coding->head_ascii;
3f003981 2540
ff0dacd7 2541 while (rejected != CATEGORY_MASK_ISO)
4ed46869 2542 {
065e3595 2543 src_base = src;
df7492f9 2544 ONE_MORE_BYTE (c);
4ed46869
KH		 2545 switch (c)
2546 {
2547 case ISO_CODE_ESC:
74383408
KH		 2548 if (inhibit_iso_escape_detection)
2549 break;
f46869e4 2550 single_shifting = 0;
df7492f9 2551 ONE_MORE_BYTE (c);
d46c5b12 2552 if (c >= '(' && c <= '/')
4ed46869 2553 {
bf9cdd4e 2554 /* Designation sequence for a charset of dimension 1. */
df7492f9 2555 ONE_MORE_BYTE (c1);
d46c5b12 2556 if (c1 < ' ' || c1 >= 0x80
df7492f9 2557 || (id = iso_charset_table[0][c >= ','][c1]) < 0)
d46c5b12
KH		 2558 /* Invalid designation sequence. Just ignore. */
2559 break;
bf9cdd4e
KH		 2560 }
2561 else if (c == '$')
2562 {
2563 /* Designation sequence for a charset of dimension 2. */
df7492f9 2564 ONE_MORE_BYTE (c);
bf9cdd4e
KH		 2565 if (c >= '@' && c <= 'B')
2566 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
ff0dacd7 2567 id = iso_charset_table[1][0][c];
bf9cdd4e 2568 else if (c >= '(' && c <= '/')
bcf26d6a 2569 {
df7492f9 2570 ONE_MORE_BYTE (c1);
d46c5b12 2571 if (c1 < ' ' || c1 >= 0x80
df7492f9 2572 || (id = iso_charset_table[1][c >= ','][c1]) < 0)
d46c5b12
KH		 2573 /* Invalid designation sequence. Just ignore. */
2574 break;
bcf26d6a 2575 }
bf9cdd4e 2576 else
ff0dacd7 2577 /* Invalid designation sequence. Just ignore it. */
d46c5b12
KH		 2578 break;
2579 }
ae9ff118 2580 else if (c == 'N' || c == 'O')
d46c5b12 2581 {
ae9ff118 2582 /* ESC <Fe> for SS2 or SS3. */
ff0dacd7
KH		 2583 single_shifting = 1;
2584 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
d46c5b12 2585 break;
4ed46869 2586 }
ec6d2bb8
KH		 2587 else if (c >= '0' && c <= '4')
2588 {
2589 /* ESC <Fp> for start/end composition. */
ff0dacd7 2590 found |= CATEGORY_MASK_ISO;
ec6d2bb8
KH		 2591 break;
2592 }
bf9cdd4e 2593 else
df7492f9 2594 {
ff0dacd7 2595 /* Invalid escape sequence. Just ignore it. */
df7492f9
KH		 2596 break;
2597 }
d46c5b12
KH		 2598
2599 /* We found a valid designation sequence for CHARSET. */
ff0dacd7 2600 rejected |= CATEGORY_MASK_ISO_8BIT;
df7492f9
KH		 2601 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7],
2602 id))
ff0dacd7 2603 found |= CATEGORY_MASK_ISO_7;
d46c5b12 2604 else
ff0dacd7 2605 rejected |= CATEGORY_MASK_ISO_7;
df7492f9
KH		 2606 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_tight],
2607 id))
ff0dacd7 2608 found |= CATEGORY_MASK_ISO_7_TIGHT;
d46c5b12 2609 else
ff0dacd7 2610 rejected |= CATEGORY_MASK_ISO_7_TIGHT;
df7492f9
KH		 2611 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_else],
2612 id))
ff0dacd7 2613 found |= CATEGORY_MASK_ISO_7_ELSE;
ae9ff118 2614 else
ff0dacd7 2615 rejected |= CATEGORY_MASK_ISO_7_ELSE;
df7492f9
KH		 2616 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_8_else],
2617 id))
ff0dacd7 2618 found |= CATEGORY_MASK_ISO_8_ELSE;
ae9ff118 2619 else
ff0dacd7 2620 rejected |= CATEGORY_MASK_ISO_8_ELSE;
4ed46869
KH		 2621 break;
2622
4ed46869 2623 case ISO_CODE_SO:
d46c5b12 2624 case ISO_CODE_SI:
ff0dacd7 2625 /* Locking shift out/in. */
74383408
KH		 2626 if (inhibit_iso_escape_detection)
2627 break;
f46869e4 2628 single_shifting = 0;
ff0dacd7
KH		 2629 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
2630 found |= CATEGORY_MASK_ISO_ELSE;
d46c5b12
KH		 2631 break;
2632
4ed46869 2633 case ISO_CODE_CSI:
ff0dacd7 2634 /* Control sequence introducer. */
f46869e4 2635 single_shifting = 0;
ff0dacd7
KH		 2636 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2637 found |= CATEGORY_MASK_ISO_8_ELSE;
2638 goto check_extra_latin;
2639
4ed46869
KH		 2640 case ISO_CODE_SS2:
2641 case ISO_CODE_SS3:
ff0dacd7
KH		 2642 /* Single shift. */
2643 if (inhibit_iso_escape_detection)
2644 break;
75e2a253 2645 single_shifting = 0;
ff0dacd7
KH		 2646 rejected |= CATEGORY_MASK_ISO_7BIT;
2647 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
2648 & CODING_ISO_FLAG_SINGLE_SHIFT)
75e2a253 2649 found |= CATEGORY_MASK_ISO_8_1, single_shifting = 1;
ff0dacd7
KH		 2650 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
2651 & CODING_ISO_FLAG_SINGLE_SHIFT)
75e2a253
KH		 2652 found |= CATEGORY_MASK_ISO_8_2, single_shifting = 1;
2653 if (single_shifting)
2654 break;
ff0dacd7 2655 goto check_extra_latin;
4ed46869
KH		 2656
2657 default:
065e3595
KH		 2658 if (c < 0)
2659 continue;
4ed46869 2660 if (c < 0x80)
f46869e4
KH		 2661 {
2662 single_shifting = 0;
2663 break;
2664 }
ff0dacd7 2665 if (c >= 0xA0)
c4825358 2666 {
ff0dacd7
KH		 2667 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2668 found |= CATEGORY_MASK_ISO_8_1;
f46869e4 2669 /* Check the length of succeeding codes of the range
ff0dacd7
KH		 2670 0xA0..0FF. If the byte length is even, we include
2671 CATEGORY_MASK_ISO_8_2 in `found'. We can check this
2672 only when we are not single shifting. */
2673 if (! single_shifting
2674 && ! (rejected & CATEGORY_MASK_ISO_8_2))
f46869e4 2675 {
e17de821 2676 int i = 1;
b73bfc1c
KH		 2677 while (src < src_end)
2678 {
df7492f9 2679 ONE_MORE_BYTE (c);
b73bfc1c
KH		 2680 if (c < 0xA0)
2681 break;
2682 i++;
2683 }
2684
2685 if (i & 1 && src < src_end)
ff0dacd7 2686 rejected |= CATEGORY_MASK_ISO_8_2;
f46869e4 2687 else
ff0dacd7 2688 found |= CATEGORY_MASK_ISO_8_2;
f46869e4 2689 }
ff0dacd7 2690 break;
4ed46869 2691 }
ff0dacd7
KH		 2692 check_extra_latin:
2693 single_shifting = 0;
2694 if (! VECTORP (Vlatin_extra_code_table)
2695 || NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
2696 {
2697 rejected = CATEGORY_MASK_ISO;
2698 break;
2699 }
2700 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
2701 & CODING_ISO_FLAG_LATIN_EXTRA)
2702 found |= CATEGORY_MASK_ISO_8_1;
2703 else
2704 rejected |= CATEGORY_MASK_ISO_8_1;
75e2a253 2705 rejected |= CATEGORY_MASK_ISO_8_2;
4ed46869
KH		 2706 }
2707 }
ff0dacd7
KH		 2708 detect_info->rejected |= CATEGORY_MASK_ISO;
2709 return 0;
4ed46869 2710
df7492f9 2711 no_more_source:
ff0dacd7
KH		 2712 detect_info->rejected |= rejected;
2713 detect_info->found |= (found & ~rejected);
df7492f9 2714 return 1;
4ed46869 2715}
ec6d2bb8 2716
4ed46869 2717
134b9549
KH		 2718/* Set designation state into CODING. Set CHARS_96 to -1 if the
2719 escape sequence should be kept. */
df7492f9
KH		 2720#define DECODE_DESIGNATION(reg, dim, chars_96, final) \
2721 do { \
2722 int id, prev; \
2723 \
2724 if (final < '0' || final >= 128 \
2725 || ((id = ISO_CHARSET_TABLE (dim, chars_96, final)) < 0) \
2726 || !SAFE_CHARSET_P (coding, id)) \
2727 { \
2728 CODING_ISO_DESIGNATION (coding, reg) = -2; \
134b9549
KH		 2729 chars_96 = -1; \
2730 break; \
df7492f9
KH		 2731 } \
2732 prev = CODING_ISO_DESIGNATION (coding, reg); \
bf16eb23
KH		 2733 if (id == charset_jisx0201_roman) \
2734 { \
2735 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
2736 id = charset_ascii; \
2737 } \
2738 else if (id == charset_jisx0208_1978) \
2739 { \
2740 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
2741 id = charset_jisx0208; \
2742 } \
df7492f9
KH		 2743 CODING_ISO_DESIGNATION (coding, reg) = id; \
2744 /* If there was an invalid designation to REG previously, and this \
2745 designation is ASCII to REG, we should keep this designation \
2746 sequence. */ \
2747 if (prev == -2 && id == charset_ascii) \
134b9549 2748 chars_96 = -1; \
4ed46869
KH		 2749 } while (0)
2750
d46c5b12 2751
df7492f9
KH		 2752#define MAYBE_FINISH_COMPOSITION() \
2753 do { \
2754 int i; \
2755 if (composition_state == COMPOSING_NO) \
2756 break; \
2757 /* It is assured that we have enough room for producing \
2758 characters stored in the table `components'. */ \
2759 if (charbuf + component_idx > charbuf_end) \
2760 goto no_more_source; \
2761 composition_state = COMPOSING_NO; \
2762 if (method == COMPOSITION_RELATIVE \
2763 || method == COMPOSITION_WITH_ALTCHARS) \
2764 { \
2765 for (i = 0; i < component_idx; i++) \
2766 *charbuf++ = components[i]; \
2767 char_offset += component_idx; \
2768 } \
2769 else \
2770 { \
2771 for (i = 0; i < component_idx; i += 2) \
2772 *charbuf++ = components[i]; \
2773 char_offset += (component_idx / 2) + 1; \
2774 } \
2775 } while (0)
2776
d46c5b12 2777
aa72b389
KH		 2778/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
2779 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
2780 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
df7492f9
KH		 2781 ESC 3 : altchar composition : ESC 3 CHAR ... ESC 0 CHAR ... ESC 1
2782 ESC 4 : alt&rule composition : ESC 4 CHAR RULE ... CHAR ESC 0 CHAR ... ESC 1
aa72b389 2783 */
ec6d2bb8 2784
df7492f9
KH		 2785#define DECODE_COMPOSITION_START(c1) \
2786 do { \
2787 if (c1 == '0' \
781d7a48 2788 && composition_state == COMPOSING_COMPONENT_RULE) \
df7492f9
KH		 2789 { \
2790 component_len = component_idx; \
2791 composition_state = COMPOSING_CHAR; \
2792 } \
2793 else \
2794 { \
8f924df7 2795 const unsigned char *p; \
df7492f9
KH		 2796 \
2797 MAYBE_FINISH_COMPOSITION (); \
2798 if (charbuf + MAX_COMPOSITION_COMPONENTS > charbuf_end) \
2799 goto no_more_source; \
2800 for (p = src; p < src_end - 1; p++) \
2801 if (*p == ISO_CODE_ESC && p[1] == '1') \
2802 break; \
2803 if (p == src_end - 1) \
2804 { \
2805 if (coding->mode & CODING_MODE_LAST_BLOCK) \
2806 goto invalid_code; \
2807 goto no_more_source; \
2808 } \
2809 \
2810 /* This is surely the start of a composition. */ \
2811 method = (c1 == '0' ? COMPOSITION_RELATIVE \
2812 : c1 == '2' ? COMPOSITION_WITH_RULE \
2813 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
2814 : COMPOSITION_WITH_RULE_ALTCHARS); \
2815 composition_state = (c1 <= '2' ? COMPOSING_CHAR \
2816 : COMPOSING_COMPONENT_CHAR); \
2817 component_idx = component_len = 0; \
2818 } \
ec6d2bb8
KH		 2819 } while (0)
2820
ec6d2bb8 2821
df7492f9
KH		 2822/* Handle compositoin end sequence ESC 1. */
2823
2824#define DECODE_COMPOSITION_END() \
ec6d2bb8 2825 do { \
df7492f9
KH		 2826 int nchars = (component_len > 0 ? component_idx - component_len \
2827 : method == COMPOSITION_RELATIVE ? component_idx \
2828 : (component_idx + 1) / 2); \
2829 int i; \
2830 int *saved_charbuf = charbuf; \
2831 \
69a80ea3 2832 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
df7492f9 2833 if (method != COMPOSITION_RELATIVE) \
ec6d2bb8 2834 { \
df7492f9
KH		 2835 if (component_len == 0) \
2836 for (i = 0; i < component_idx; i++) \
2837 *charbuf++ = components[i]; \
2838 else \
2839 for (i = 0; i < component_len; i++) \
2840 *charbuf++ = components[i]; \
2841 *saved_charbuf = saved_charbuf - charbuf; \
ec6d2bb8 2842 } \
df7492f9
KH		 2843 if (method == COMPOSITION_WITH_RULE) \
2844 for (i = 0; i < component_idx; i += 2, char_offset++) \
2845 *charbuf++ = components[i]; \
ec6d2bb8 2846 else \
df7492f9
KH		 2847 for (i = component_len; i < component_idx; i++, char_offset++) \
2848 *charbuf++ = components[i]; \
2849 coding->annotated = 1; \
2850 composition_state = COMPOSING_NO; \
ec6d2bb8
KH		 2851 } while (0)
2852
df7492f9 2853
ec6d2bb8
KH		 2854/* Decode a composition rule from the byte C1 (and maybe one more byte
2855 from SRC) and store one encoded composition rule in
2856 coding->cmp_data. */
2857
2858#define DECODE_COMPOSITION_RULE(c1) \
2859 do { \
ec6d2bb8
KH		 2860 (c1) -= 32; \
2861 if (c1 < 81) /* old format (before ver.21) */ \
2862 { \
2863 int gref = (c1) / 9; \
2864 int nref = (c1) % 9; \
2865 if (gref == 4) gref = 10; \
2866 if (nref == 4) nref = 10; \
df7492f9 2867 c1 = COMPOSITION_ENCODE_RULE (gref, nref); \
ec6d2bb8 2868 } \
b73bfc1c 2869 else if (c1 < 93) /* new format (after ver.21) */ \
ec6d2bb8
KH		 2870 { \
2871 ONE_MORE_BYTE (c2); \
df7492f9 2872 c1 = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
ec6d2bb8 2873 } \
df7492f9
KH		 2874 else \
2875 c1 = 0; \
ec6d2bb8 2876 } while (0)
88993dfd 2877
d46c5b12 2878
4ed46869
KH		 2879/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
2880
b73bfc1c 2881static void
df7492f9 2882decode_coding_iso_2022 (coding)
4ed46869 2883 struct coding_system *coding;
4ed46869 2884{
8f924df7
KH		 2885 const unsigned char *src = coding->source + coding->consumed;
2886 const unsigned char *src_end = coding->source + coding->src_bytes;
2887 const unsigned char *src_base;
69a80ea3 2888 int *charbuf = coding->charbuf + coding->charbuf_used;
ff0dacd7 2889 int *charbuf_end
69a80ea3 2890 = coding->charbuf + coding->charbuf_size - 4 - MAX_ANNOTATION_LENGTH;
df7492f9 2891 int consumed_chars = 0, consumed_chars_base;
df7492f9 2892 int multibytep = coding->src_multibyte;
4ed46869 2893 /* Charsets invoked to graphic plane 0 and 1 respectively. */
df7492f9
KH		 2894 int charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
2895 int charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
134b9549 2896 int charset_id_2, charset_id_3;
df7492f9
KH		 2897 struct charset *charset;
2898 int c;
2899 /* For handling composition sequence. */
2900#define COMPOSING_NO 0
2901#define COMPOSING_CHAR 1
2902#define COMPOSING_RULE 2
2903#define COMPOSING_COMPONENT_CHAR 3
2904#define COMPOSING_COMPONENT_RULE 4
2905
2906 int composition_state = COMPOSING_NO;
2907 enum composition_method method;
2908 int components[MAX_COMPOSITION_COMPONENTS * 2 + 1];
2909 int component_idx;
2910 int component_len;
24a73b0a 2911 Lisp_Object attrs, charset_list;
ff0dacd7
KH		 2912 int char_offset = coding->produced_char;
2913 int last_offset = char_offset;
2914 int last_id = charset_ascii;
df7492f9 2915
24a73b0a 2916 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 2917 setup_iso_safe_charsets (attrs);
b73bfc1c
KH		 2918
2919 while (1)
4ed46869 2920 {
463f5630 2921 int c1, c2;
b73bfc1c
KH		 2922
2923 src_base = src;
df7492f9
KH		 2924 consumed_chars_base = consumed_chars;
2925
2926 if (charbuf >= charbuf_end)
2927 break;
2928
b73bfc1c 2929 ONE_MORE_BYTE (c1);
065e3595
KH		 2930 if (c1 < 0)
2931 goto invalid_code;
4ed46869 2932
98725083 2933 /* We produce at most one character. */
4ed46869
KH		 2934 switch (iso_code_class [c1])
2935 {
2936 case ISO_0x20_or_0x7F:
df7492f9 2937 if (composition_state != COMPOSING_NO)
ec6d2bb8 2938 {
df7492f9
KH		 2939 if (composition_state == COMPOSING_RULE
2940 || composition_state == COMPOSING_COMPONENT_RULE)
2941 {
2942 DECODE_COMPOSITION_RULE (c1);
2943 components[component_idx++] = c1;
2944 composition_state--;
2945 continue;
2946 }
4ed46869 2947 }
df7492f9
KH		 2948 if (charset_id_0 < 0
2949 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_0)))
781d7a48
KH		 2950 /* This is SPACE or DEL. */
2951 charset = CHARSET_FROM_ID (charset_ascii);
2952 else
2953 charset = CHARSET_FROM_ID (charset_id_0);
2954 break;
4ed46869
KH		 2955
2956 case ISO_graphic_plane_0:
781d7a48 2957 if (composition_state != COMPOSING_NO)
b73bfc1c 2958 {
781d7a48
KH		 2959 if (composition_state == COMPOSING_RULE
2960 || composition_state == COMPOSING_COMPONENT_RULE)
2961 {
2962 DECODE_COMPOSITION_RULE (c1);
2963 components[component_idx++] = c1;
2964 composition_state--;
2965 continue;
2966 }
b73bfc1c 2967 }
134b9549
KH		 2968 if (charset_id_0 < 0)
2969 charset = CHARSET_FROM_ID (charset_ascii);
2970 else
2971 charset = CHARSET_FROM_ID (charset_id_0);
4ed46869
KH		 2972 break;
2973
2974 case ISO_0xA0_or_0xFF:
df7492f9
KH		 2975 if (charset_id_1 < 0
2976 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_1))
2977 || CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS)
2978 goto invalid_code;
4ed46869
KH		 2979 /* This is a graphic character, we fall down ... */
2980
2981 case ISO_graphic_plane_1:
df7492f9
KH		 2982 if (charset_id_1 < 0)
2983 goto invalid_code;
2984 charset = CHARSET_FROM_ID (charset_id_1);
4ed46869
KH		 2985 break;
2986
df7492f9
KH		 2987 case ISO_control_0:
2988 MAYBE_FINISH_COMPOSITION ();
2989 charset = CHARSET_FROM_ID (charset_ascii);
4ed46869
KH		 2990 break;
2991
df7492f9
KH		 2992 case ISO_control_1:
2993 MAYBE_FINISH_COMPOSITION ();
2994 goto invalid_code;
2995
4ed46869 2996 case ISO_shift_out:
df7492f9
KH		 2997 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
2998 || CODING_ISO_DESIGNATION (coding, 1) < 0)
2999 goto invalid_code;
3000 CODING_ISO_INVOCATION (coding, 0) = 1;
3001 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3002 continue;
4ed46869
KH		 3003
3004 case ISO_shift_in:
df7492f9
KH		 3005 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT))
3006 goto invalid_code;
3007 CODING_ISO_INVOCATION (coding, 0) = 0;
3008 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3009 continue;
4ed46869
KH		 3010
3011 case ISO_single_shift_2_7:
3012 case ISO_single_shift_2:
df7492f9
KH		 3013 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3014 goto invalid_code;
4ed46869
KH		 3015 /* SS2 is handled as an escape sequence of ESC 'N' */
3016 c1 = 'N';
3017 goto label_escape_sequence;
3018
3019 case ISO_single_shift_3:
df7492f9
KH		 3020 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3021 goto invalid_code;
4ed46869
KH		 3022 /* SS2 is handled as an escape sequence of ESC 'O' */
3023 c1 = 'O';
3024 goto label_escape_sequence;
3025
3026 case ISO_control_sequence_introducer:
3027 /* CSI is handled as an escape sequence of ESC '[' ... */
3028 c1 = '[';
3029 goto label_escape_sequence;
3030
3031 case ISO_escape:
3032 ONE_MORE_BYTE (c1);
3033 label_escape_sequence:
df7492f9 3034 /* Escape sequences handled here are invocation,
4ed46869
KH		 3035 designation, direction specification, and character
3036 composition specification. */
3037 switch (c1)
3038 {
3039 case '&': /* revision of following character set */
3040 ONE_MORE_BYTE (c1);
3041 if (!(c1 >= '@' && c1 <= '~'))
df7492f9 3042 goto invalid_code;
4ed46869
KH		 3043 ONE_MORE_BYTE (c1);
3044 if (c1 != ISO_CODE_ESC)
df7492f9 3045 goto invalid_code;
4ed46869
KH		 3046 ONE_MORE_BYTE (c1);
3047 goto label_escape_sequence;
3048
3049 case '$': /* designation of 2-byte character set */
df7492f9
KH		 3050 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3051 goto invalid_code;
134b9549
KH		 3052 {
3053 int reg, chars96;
3054
3055 ONE_MORE_BYTE (c1);
3056 if (c1 >= '@' && c1 <= 'B')
3057 { /* designation of JISX0208.1978, GB2312.1980,
88993dfd 3058 or JISX0208.1980 */
134b9549
KH		 3059 reg = 0, chars96 = 0;
3060 }
3061 else if (c1 >= 0x28 && c1 <= 0x2B)
3062 { /* designation of DIMENSION2_CHARS94 character set */
3063 reg = c1 - 0x28, chars96 = 0;
3064 ONE_MORE_BYTE (c1);
3065 }
3066 else if (c1 >= 0x2C && c1 <= 0x2F)
3067 { /* designation of DIMENSION2_CHARS96 character set */
3068 reg = c1 - 0x2C, chars96 = 1;
3069 ONE_MORE_BYTE (c1);
3070 }
3071 else
3072 goto invalid_code;
3073 DECODE_DESIGNATION (reg, 2, chars96, c1);
3074 /* We must update these variables now. */
3075 if (reg == 0)
3076 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3077 else if (reg == 1)
3078 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3079 if (chars96 < 0)
3080 goto invalid_code;
3081 }
b73bfc1c 3082 continue;
4ed46869
KH		 3083
3084 case 'n': /* invocation of locking-shift-2 */
df7492f9
KH		 3085 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3086 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3087 goto invalid_code;
3088 CODING_ISO_INVOCATION (coding, 0) = 2;
3089 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3090 continue;
4ed46869
KH		 3091
3092 case 'o': /* invocation of locking-shift-3 */
df7492f9
KH		 3093 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3094 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3095 goto invalid_code;
3096 CODING_ISO_INVOCATION (coding, 0) = 3;
3097 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
b73bfc1c 3098 continue;
4ed46869
KH		 3099
3100 case 'N': /* invocation of single-shift-2 */
df7492f9
KH		 3101 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3102 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3103 goto invalid_code;
134b9549
KH		 3104 charset_id_2 = CODING_ISO_DESIGNATION (coding, 2);
3105 if (charset_id_2 < 0)
3106 charset = CHARSET_FROM_ID (charset_ascii);
3107 else
3108 charset = CHARSET_FROM_ID (charset_id_2);
b73bfc1c 3109 ONE_MORE_BYTE (c1);
e7046a18 3110 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3111 goto invalid_code;
4ed46869
KH		 3112 break;
3113
3114 case 'O': /* invocation of single-shift-3 */
df7492f9
KH		 3115 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3116 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3117 goto invalid_code;
134b9549
KH		 3118 charset_id_3 = CODING_ISO_DESIGNATION (coding, 3);
3119 if (charset_id_3 < 0)
3120 charset = CHARSET_FROM_ID (charset_ascii);
3121 else
3122 charset = CHARSET_FROM_ID (charset_id_3);
b73bfc1c 3123 ONE_MORE_BYTE (c1);
e7046a18 3124 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
df7492f9 3125 goto invalid_code;
4ed46869
KH		 3126 break;
3127
ec6d2bb8 3128 case '0': case '2': case '3': case '4': /* start composition */
df7492f9
KH		 3129 if (! (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK))
3130 goto invalid_code;
ec6d2bb8 3131 DECODE_COMPOSITION_START (c1);
b73bfc1c 3132 continue;
4ed46869 3133
ec6d2bb8 3134 case '1': /* end composition */
df7492f9
KH		 3135 if (composition_state == COMPOSING_NO)
3136 goto invalid_code;
3137 DECODE_COMPOSITION_END ();
b73bfc1c 3138 continue;
4ed46869
KH		 3139
3140 case '[': /* specification of direction */
df7492f9
KH		 3141 if (! CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DIRECTION)
3142 goto invalid_code;
4ed46869 3143 /* For the moment, nested direction is not supported.
d46c5b12 3144 So, `coding->mode & CODING_MODE_DIRECTION' zero means
df7492f9 3145 left-to-right, and nozero means right-to-left. */
4ed46869
KH		 3146 ONE_MORE_BYTE (c1);
3147 switch (c1)
3148 {
3149 case ']': /* end of the current direction */
d46c5b12 3150 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869
KH		 3151
3152 case '0': /* end of the current direction */
3153 case '1': /* start of left-to-right direction */
3154 ONE_MORE_BYTE (c1);
3155 if (c1 == ']')
d46c5b12 3156 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869 3157 else
df7492f9 3158 goto invalid_code;
4ed46869
KH		 3159 break;
3160
3161 case '2': /* start of right-to-left direction */
3162 ONE_MORE_BYTE (c1);
3163 if (c1 == ']')
d46c5b12 3164 coding->mode |= CODING_MODE_DIRECTION;
4ed46869 3165 else
df7492f9 3166 goto invalid_code;
4ed46869
KH		 3167 break;
3168
3169 default:
df7492f9 3170 goto invalid_code;
4ed46869 3171 }
b73bfc1c 3172 continue;
4ed46869 3173
103e0180 3174 case '%':
103e0180
KH		 3175 ONE_MORE_BYTE (c1);
3176 if (c1 == '/')
3177 {
3178 /* CTEXT extended segment:
3179 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
3180 We keep these bytes as is for the moment.
3181 They may be decoded by post-read-conversion. */
3182 int dim, M, L;
4776e638 3183 int size;
8f924df7 3184
103e0180
KH		 3185 ONE_MORE_BYTE (dim);
3186 ONE_MORE_BYTE (M);
3187 ONE_MORE_BYTE (L);
3188 size = ((M - 128) * 128) + (L - 128);
4776e638
KH		 3189 if (charbuf + 8 + size > charbuf_end)
3190 goto break_loop;
3191 *charbuf++ = ISO_CODE_ESC;
3192 *charbuf++ = '%';
3193 *charbuf++ = '/';
3194 *charbuf++ = dim;
3195 *charbuf++ = BYTE8_TO_CHAR (M);
3196 *charbuf++ = BYTE8_TO_CHAR (L);
103e0180
KH		 3197 while (size-- > 0)
3198 {
3199 ONE_MORE_BYTE (c1);
4776e638 3200 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
103e0180 3201 }
103e0180
KH		 3202 }
3203 else if (c1 == 'G')
3204 {
103e0180
KH		 3205 /* XFree86 extension for embedding UTF-8 in CTEXT:
3206 ESC % G --UTF-8-BYTES-- ESC % @
3207 We keep these bytes as is for the moment.
3208 They may be decoded by post-read-conversion. */
4776e638
KH		 3209 int *p = charbuf;
3210
3211 if (p + 6 > charbuf_end)
3212 goto break_loop;
3213 *p++ = ISO_CODE_ESC;
3214 *p++ = '%';
3215 *p++ = 'G';
3216 while (p < charbuf_end)
103e0180
KH		 3217 {
3218 ONE_MORE_BYTE (c1);
3219 if (c1 == ISO_CODE_ESC
3220 && src + 1 < src_end
3221 && src[0] == '%'
3222 && src[1] == '@')
9ffd559c
KH		 3223 {
3224 src += 2;
3225 break;
3226 }
4776e638 3227 *p++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
103e0180 3228 }
4776e638
KH		 3229 if (p + 3 > charbuf_end)
3230 goto break_loop;
3231 *p++ = ISO_CODE_ESC;
3232 *p++ = '%';
3233 *p++ = '@';
3234 charbuf = p;
103e0180
KH		 3235 }
3236 else
4776e638 3237 goto invalid_code;
103e0180 3238 continue;
4776e638 3239 break;
103e0180 3240
4ed46869 3241 default:
df7492f9
KH		 3242 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3243 goto invalid_code;
134b9549
KH		 3244 {
3245 int reg, chars96;
3246
3247 if (c1 >= 0x28 && c1 <= 0x2B)
3248 { /* designation of DIMENSION1_CHARS94 character set */
3249 reg = c1 - 0x28, chars96 = 0;
3250 ONE_MORE_BYTE (c1);
3251 }
3252 else if (c1 >= 0x2C && c1 <= 0x2F)
3253 { /* designation of DIMENSION1_CHARS96 character set */
3254 reg = c1 - 0x2C, chars96 = 1;
3255 ONE_MORE_BYTE (c1);
3256 }
3257 else
3258 goto invalid_code;
3259 DECODE_DESIGNATION (reg, 1, chars96, c1);
3260 /* We must update these variables now. */
3261 if (reg == 0)
3262 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3263 else if (reg == 1)
3264 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3265 if (chars96 < 0)
3266 goto invalid_code;
3267 }
b73bfc1c 3268 continue;
4ed46869 3269 }
b73bfc1c 3270 }
4ed46869 3271
ff0dacd7
KH		 3272 if (charset->id != charset_ascii
3273 && last_id != charset->id)
3274 {
3275 if (last_id != charset_ascii)
69a80ea3 3276 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
ff0dacd7
KH		 3277 last_id = charset->id;
3278 last_offset = char_offset;
3279 }
3280
b73bfc1c 3281 /* Now we know CHARSET and 1st position code C1 of a character.
df7492f9
KH		 3282 Produce a decoded character while getting 2nd position code
3283 C2 if necessary. */
3284 c1 &= 0x7F;
3285 if (CHARSET_DIMENSION (charset) > 1)
b73bfc1c
KH		 3286 {
3287 ONE_MORE_BYTE (c2);
df7492f9 3288 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
b73bfc1c 3289 /* C2 is not in a valid range. */
df7492f9
KH		 3290 goto invalid_code;
3291 c1 = (c1 << 8) | (c2 & 0x7F);
3292 if (CHARSET_DIMENSION (charset) > 2)
3293 {
3294 ONE_MORE_BYTE (c2);
3295 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
3296 /* C2 is not in a valid range. */
3297 goto invalid_code;
3298 c1 = (c1 << 8) | (c2 & 0x7F);
3299 }
3300 }
3301
3302 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c1, c);
3303 if (c < 0)
3304 {
3305 MAYBE_FINISH_COMPOSITION ();
3306 for (; src_base < src; src_base++, char_offset++)
3307 {
3308 if (ASCII_BYTE_P (*src_base))
3309 *charbuf++ = *src_base;
3310 else
3311 *charbuf++ = BYTE8_TO_CHAR (*src_base);
3312 }
3313 }
3314 else if (composition_state == COMPOSING_NO)
3315 {
3316 *charbuf++ = c;
3317 char_offset++;
4ed46869 3318 }
df7492f9 3319 else
781d7a48
KH		 3320 {
3321 components[component_idx++] = c;
3322 if (method == COMPOSITION_WITH_RULE
3323 || (method == COMPOSITION_WITH_RULE_ALTCHARS
3324 && composition_state == COMPOSING_COMPONENT_CHAR))
3325 composition_state++;
4ed46869
KH		 3326 }
3327 continue;
3328
df7492f9
KH		 3329 invalid_code:
3330 MAYBE_FINISH_COMPOSITION ();
4ed46869 3331 src = src_base;
df7492f9
KH		 3332 consumed_chars = consumed_chars_base;
3333 ONE_MORE_BYTE (c);
065e3595 3334 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 3335 char_offset++;
df7492f9 3336 coding->errors++;
4776e638
KH		 3337 continue;
3338
3339 break_loop:
3340 break;
4ed46869 3341 }
fb88bf2d 3342
df7492f9 3343 no_more_source:
ff0dacd7 3344 if (last_id != charset_ascii)
69a80ea3 3345 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 3346 coding->consumed_char += consumed_chars_base;
3347 coding->consumed = src_base - coding->source;
3348 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 3349}
3350
b73bfc1c 3351
f4dee582 3352/* ISO2022 encoding stuff. */
4ed46869
KH		 3353
3354/*
f4dee582 3355 It is not enough to say just "ISO2022" on encoding, we have to
df7492f9 3356 specify more details. In Emacs, each coding system of ISO2022
4ed46869 3357 variant has the following specifications:
df7492f9 3358 1. Initial designation to G0 thru G3.
4ed46869
KH		 3359 2. Allows short-form designation?
3360 3. ASCII should be designated to G0 before control characters?
3361 4. ASCII should be designated to G0 at end of line?
3362 5. 7-bit environment or 8-bit environment?
3363 6. Use locking-shift?
3364 7. Use Single-shift?
3365 And the following two are only for Japanese:
3366 8. Use ASCII in place of JIS0201-1976-Roman?
3367 9. Use JISX0208-1983 in place of JISX0208-1978?
df7492f9
KH		 3368 These specifications are encoded in CODING_ISO_FLAGS (coding) as flag bits
3369 defined by macros CODING_ISO_FLAG_XXX. See `coding.h' for more
f4dee582 3370 details.
4ed46869
KH		 3371*/
3372
3373/* Produce codes (escape sequence) for designating CHARSET to graphic
b73bfc1c
KH		 3374 register REG at DST, and increment DST. If <final-char> of CHARSET is
3375 '@', 'A', or 'B' and the coding system CODING allows, produce
3376 designation sequence of short-form. */
4ed46869
KH		 3377
3378#define ENCODE_DESIGNATION(charset, reg, coding) \
3379 do { \
df7492f9 3380 unsigned char final_char = CHARSET_ISO_FINAL (charset); \
4ed46869
KH		 3381 char *intermediate_char_94 = "()*+"; \
3382 char *intermediate_char_96 = ",-./"; \
df7492f9
KH		 3383 int revision = -1; \
3384 int c; \
3385 \
3386 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_REVISION) \
c197f191 3387 revision = CHARSET_ISO_REVISION (charset); \
df7492f9
KH		 3388 \
3389 if (revision >= 0) \
70c22245 3390 { \
df7492f9
KH		 3391 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '&'); \
3392 EMIT_ONE_BYTE ('@' + revision); \
4ed46869 3393 } \
df7492f9 3394 EMIT_ONE_ASCII_BYTE (ISO_CODE_ESC); \
4ed46869
KH		 3395 if (CHARSET_DIMENSION (charset) == 1) \
3396 { \
df7492f9
KH		 3397 if (! CHARSET_ISO_CHARS_96 (charset)) \
3398 c = intermediate_char_94[reg]; \
4ed46869 3399 else \
df7492f9
KH		 3400 c = intermediate_char_96[reg]; \
3401 EMIT_ONE_ASCII_BYTE (c); \
4ed46869
KH		 3402 } \
3403 else \
3404 { \
df7492f9
KH		 3405 EMIT_ONE_ASCII_BYTE ('$'); \
3406 if (! CHARSET_ISO_CHARS_96 (charset)) \
4ed46869 3407 { \
df7492f9 3408 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LONG_FORM \
b73bfc1c
KH		 3409 || reg != 0 \
3410 || final_char < '@' || final_char > 'B') \
df7492f9 3411 EMIT_ONE_ASCII_BYTE (intermediate_char_94[reg]); \
4ed46869
KH		 3412 } \
3413 else \
df7492f9 3414 EMIT_ONE_ASCII_BYTE (intermediate_char_96[reg]); \
4ed46869 3415 } \
df7492f9
KH		 3416 EMIT_ONE_ASCII_BYTE (final_char); \
3417 \
3418 CODING_ISO_DESIGNATION (coding, reg) = CHARSET_ID (charset); \
4ed46869
KH		 3419 } while (0)
3420
df7492f9 3421
4ed46869
KH		 3422/* The following two macros produce codes (control character or escape
3423 sequence) for ISO2022 single-shift functions (single-shift-2 and
3424 single-shift-3). */
3425
df7492f9
KH		 3426#define ENCODE_SINGLE_SHIFT_2 \
3427 do { \
3428 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3429 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'N'); \
3430 else \
3431 EMIT_ONE_BYTE (ISO_CODE_SS2); \
3432 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 3433 } while (0)
3434
df7492f9
KH		 3435
3436#define ENCODE_SINGLE_SHIFT_3 \
3437 do { \
3438 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3439 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'O'); \
3440 else \
3441 EMIT_ONE_BYTE (ISO_CODE_SS3); \
3442 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
4ed46869
KH		 3443 } while (0)
3444
df7492f9 3445
4ed46869
KH		 3446/* The following four macros produce codes (control character or
3447 escape sequence) for ISO2022 locking-shift functions (shift-in,
3448 shift-out, locking-shift-2, and locking-shift-3). */
3449
df7492f9
KH		 3450#define ENCODE_SHIFT_IN \
3451 do { \
3452 EMIT_ONE_ASCII_BYTE (ISO_CODE_SI); \
3453 CODING_ISO_INVOCATION (coding, 0) = 0; \
4ed46869
KH		 3454 } while (0)
3455
df7492f9
KH		 3456
3457#define ENCODE_SHIFT_OUT \
3458 do { \
3459 EMIT_ONE_ASCII_BYTE (ISO_CODE_SO); \
3460 CODING_ISO_INVOCATION (coding, 0) = 1; \
4ed46869
KH		 3461 } while (0)
3462
df7492f9
KH		 3463
3464#define ENCODE_LOCKING_SHIFT_2 \
3465 do { \
3466 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3467 CODING_ISO_INVOCATION (coding, 0) = 2; \
4ed46869
KH		 3468 } while (0)
3469
df7492f9
KH		 3470
3471#define ENCODE_LOCKING_SHIFT_3 \
3472 do { \
3473 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3474 CODING_ISO_INVOCATION (coding, 0) = 3; \
4ed46869
KH		 3475 } while (0)
3476
df7492f9 3477
f4dee582
RS		 3478/* Produce codes for a DIMENSION1 character whose character set is
3479 CHARSET and whose position-code is C1. Designation and invocation
4ed46869
KH		 3480 sequences are also produced in advance if necessary. */
3481
6e85d753
KH		 3482#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
3483 do { \
df7492f9 3484 int id = CHARSET_ID (charset); \
bf16eb23
KH		 3485 \
3486 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
3487 && id == charset_ascii) \
3488 { \
3489 id = charset_jisx0201_roman; \
3490 charset = CHARSET_FROM_ID (id); \
3491 } \
3492 \
df7492f9 3493 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 3494 { \
df7492f9
KH		 3495 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3496 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753 3497 else \
df7492f9
KH		 3498 EMIT_ONE_BYTE (c1 | 0x80); \
3499 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 3500 break; \
3501 } \
df7492f9 3502 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 3503 { \
df7492f9 3504 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
6e85d753
KH		 3505 break; \
3506 } \
df7492f9 3507 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 3508 { \
df7492f9 3509 EMIT_ONE_BYTE (c1 | 0x80); \
6e85d753
KH		 3510 break; \
3511 } \
6e85d753
KH		 3512 else \
3513 /* Since CHARSET is not yet invoked to any graphic planes, we \
3514 must invoke it, or, at first, designate it to some graphic \
3515 register. Then repeat the loop to actually produce the \
3516 character. */ \
df7492f9
KH		 3517 dst = encode_invocation_designation (charset, coding, dst, \
3518 &produced_chars); \
4ed46869
KH		 3519 } while (1)
3520
df7492f9 3521
f4dee582
RS		 3522/* Produce codes for a DIMENSION2 character whose character set is
3523 CHARSET and whose position-codes are C1 and C2. Designation and
4ed46869
KH		 3524 invocation codes are also produced in advance if necessary. */
3525
6e85d753
KH		 3526#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
3527 do { \
df7492f9 3528 int id = CHARSET_ID (charset); \
bf16eb23
KH		 3529 \
3530 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
3531 && id == charset_jisx0208) \
3532 { \
3533 id = charset_jisx0208_1978; \
3534 charset = CHARSET_FROM_ID (id); \
3535 } \
3536 \
df7492f9 3537 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
6e85d753 3538 { \
df7492f9
KH		 3539 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3540 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753 3541 else \
df7492f9
KH		 3542 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
3543 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
6e85d753
KH		 3544 break; \
3545 } \
df7492f9 3546 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
6e85d753 3547 { \
df7492f9 3548 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
6e85d753
KH		 3549 break; \
3550 } \
df7492f9 3551 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
6e85d753 3552 { \
df7492f9 3553 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
6e85d753
KH		 3554 break; \
3555 } \
6e85d753
KH		 3556 else \
3557 /* Since CHARSET is not yet invoked to any graphic planes, we \
3558 must invoke it, or, at first, designate it to some graphic \
3559 register. Then repeat the loop to actually produce the \
3560 character. */ \
df7492f9
KH		 3561 dst = encode_invocation_designation (charset, coding, dst, \
3562 &produced_chars); \
4ed46869
KH		 3563 } while (1)
3564
05e6f5dc 3565
df7492f9
KH		 3566#define ENCODE_ISO_CHARACTER(charset, c) \
3567 do { \
3568 int code = ENCODE_CHAR ((charset),(c)); \
3569 \
3570 if (CHARSET_DIMENSION (charset) == 1) \
3571 ENCODE_ISO_CHARACTER_DIMENSION1 ((charset), code); \
3572 else \
3573 ENCODE_ISO_CHARACTER_DIMENSION2 ((charset), code >> 8, code & 0xFF); \
84fbb8a0 3574 } while (0)
bdd9fb48 3575
05e6f5dc 3576
4ed46869 3577/* Produce designation and invocation codes at a place pointed by DST
df7492f9 3578 to use CHARSET. The element `spec.iso_2022' of *CODING is updated.
4ed46869
KH		 3579 Return new DST. */
3580
3581unsigned char *
df7492f9
KH		 3582encode_invocation_designation (charset, coding, dst, p_nchars)
3583 struct charset *charset;
4ed46869
KH		 3584 struct coding_system *coding;
3585 unsigned char *dst;
df7492f9 3586 int *p_nchars;
4ed46869 3587{
df7492f9
KH		 3588 int multibytep = coding->dst_multibyte;
3589 int produced_chars = *p_nchars;
4ed46869 3590 int reg; /* graphic register number */
df7492f9 3591 int id = CHARSET_ID (charset);
4ed46869
KH		 3592
3593 /* At first, check designations. */
3594 for (reg = 0; reg < 4; reg++)
df7492f9 3595 if (id == CODING_ISO_DESIGNATION (coding, reg))
4ed46869
KH		 3596 break;
3597
3598 if (reg >= 4)
3599 {
3600 /* CHARSET is not yet designated to any graphic registers. */
3601 /* At first check the requested designation. */
df7492f9
KH		 3602 reg = CODING_ISO_REQUEST (coding, id);
3603 if (reg < 0)
1ba9e4ab
KH		 3604 /* Since CHARSET requests no special designation, designate it
3605 to graphic register 0. */
4ed46869
KH		 3606 reg = 0;
3607
3608 ENCODE_DESIGNATION (charset, reg, coding);
3609 }
3610
df7492f9
KH		 3611 if (CODING_ISO_INVOCATION (coding, 0) != reg
3612 && CODING_ISO_INVOCATION (coding, 1) != reg)
4ed46869
KH		 3613 {
3614 /* Since the graphic register REG is not invoked to any graphic
3615 planes, invoke it to graphic plane 0. */
3616 switch (reg)
3617 {
3618 case 0: /* graphic register 0 */
3619 ENCODE_SHIFT_IN;
3620 break;
3621
3622 case 1: /* graphic register 1 */
3623 ENCODE_SHIFT_OUT;
3624 break;
3625
3626 case 2: /* graphic register 2 */
df7492f9 3627 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 3628 ENCODE_SINGLE_SHIFT_2;
3629 else
3630 ENCODE_LOCKING_SHIFT_2;
3631 break;
3632
3633 case 3: /* graphic register 3 */
df7492f9 3634 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
4ed46869
KH		 3635 ENCODE_SINGLE_SHIFT_3;
3636 else
3637 ENCODE_LOCKING_SHIFT_3;
3638 break;
3639 }
3640 }
b73bfc1c 3641
df7492f9 3642 *p_nchars = produced_chars;
4ed46869
KH		 3643 return dst;
3644}
3645
df7492f9
KH		 3646/* The following three macros produce codes for indicating direction
3647 of text. */
3648#define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
ec6d2bb8 3649 do { \
df7492f9
KH		 3650 if (CODING_ISO_FLAGS (coding) == CODING_ISO_FLAG_SEVEN_BITS) \
3651 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '['); \
ec6d2bb8 3652 else \
df7492f9 3653 EMIT_ONE_BYTE (ISO_CODE_CSI); \
ec6d2bb8
KH		 3654 } while (0)
3655
ec6d2bb8 3656
df7492f9
KH		 3657#define ENCODE_DIRECTION_R2L() \
3658 do { \
3659 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
3660 EMIT_TWO_ASCII_BYTES ('2', ']'); \
ec6d2bb8
KH		 3661 } while (0)
3662
ec6d2bb8 3663
df7492f9 3664#define ENCODE_DIRECTION_L2R() \
ec6d2bb8 3665 do { \
df7492f9
KH		 3666 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
3667 EMIT_TWO_ASCII_BYTES ('0', ']'); \
ec6d2bb8 3668 } while (0)
4ed46869 3669
4ed46869
KH		 3670
3671/* Produce codes for designation and invocation to reset the graphic
3672 planes and registers to initial state. */
df7492f9
KH		 3673#define ENCODE_RESET_PLANE_AND_REGISTER() \
3674 do { \
3675 int reg; \
3676 struct charset *charset; \
3677 \
3678 if (CODING_ISO_INVOCATION (coding, 0) != 0) \
3679 ENCODE_SHIFT_IN; \
3680 for (reg = 0; reg < 4; reg++) \
3681 if (CODING_ISO_INITIAL (coding, reg) >= 0 \
3682 && (CODING_ISO_DESIGNATION (coding, reg) \
3683 != CODING_ISO_INITIAL (coding, reg))) \
3684 { \
3685 charset = CHARSET_FROM_ID (CODING_ISO_INITIAL (coding, reg)); \
3686 ENCODE_DESIGNATION (charset, reg, coding); \
3687 } \
4ed46869
KH		 3688 } while (0)
3689
df7492f9 3690
bdd9fb48 3691/* Produce designation sequences of charsets in the line started from
b73bfc1c 3692 SRC to a place pointed by DST, and return updated DST.
bdd9fb48
KH		 3693
3694 If the current block ends before any end-of-line, we may fail to
d46c5b12
KH		 3695 find all the necessary designations. */
3696
b73bfc1c 3697static unsigned char *
df7492f9 3698encode_designation_at_bol (coding, charbuf, charbuf_end, dst)
e0e989f6 3699 struct coding_system *coding;
df7492f9
KH		 3700 int *charbuf, *charbuf_end;
3701 unsigned char *dst;
e0e989f6 3702{
df7492f9 3703 struct charset *charset;
bdd9fb48
KH		 3704 /* Table of charsets to be designated to each graphic register. */
3705 int r[4];
df7492f9
KH		 3706 int c, found = 0, reg;
3707 int produced_chars = 0;
3708 int multibytep = coding->dst_multibyte;
3709 Lisp_Object attrs;
3710 Lisp_Object charset_list;
3711
3712 attrs = CODING_ID_ATTRS (coding->id);
3713 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
3714 if (EQ (charset_list, Qiso_2022))
3715 charset_list = Viso_2022_charset_list;
bdd9fb48
KH		 3716
3717 for (reg = 0; reg < 4; reg++)
3718 r[reg] = -1;
3719
b73bfc1c 3720 while (found < 4)
e0e989f6 3721 {
df7492f9
KH		 3722 int id;
3723
3724 c = *charbuf++;
b73bfc1c
KH		 3725 if (c == '\n')
3726 break;
df7492f9
KH		 3727 charset = char_charset (c, charset_list, NULL);
3728 id = CHARSET_ID (charset);
3729 reg = CODING_ISO_REQUEST (coding, id);
3730 if (reg >= 0 && r[reg] < 0)
bdd9fb48
KH		 3731 {
3732 found++;
df7492f9 3733 r[reg] = id;
bdd9fb48 3734 }
bdd9fb48
KH		 3735 }
3736
3737 if (found)
3738 {
3739 for (reg = 0; reg < 4; reg++)
3740 if (r[reg] >= 0
df7492f9
KH		 3741 && CODING_ISO_DESIGNATION (coding, reg) != r[reg])
3742 ENCODE_DESIGNATION (CHARSET_FROM_ID (r[reg]), reg, coding);
e0e989f6 3743 }
b73bfc1c
KH		 3744
3745 return dst;
e0e989f6
KH		 3746}
3747
4ed46869
KH		 3748/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
3749
df7492f9
KH		 3750static int
3751encode_coding_iso_2022 (coding)
4ed46869 3752 struct coding_system *coding;
4ed46869 3753{
df7492f9
KH		 3754 int multibytep = coding->dst_multibyte;
3755 int *charbuf = coding->charbuf;
3756 int *charbuf_end = charbuf + coding->charbuf_used;
3757 unsigned char *dst = coding->destination + coding->produced;
3758 unsigned char *dst_end = coding->destination + coding->dst_bytes;
3759 int safe_room = 16;
3760 int bol_designation
3761 = (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL
3762 && CODING_ISO_BOL (coding));
3763 int produced_chars = 0;
3764 Lisp_Object attrs, eol_type, charset_list;
3765 int ascii_compatible;
b73bfc1c 3766 int c;
ff0dacd7 3767 int preferred_charset_id = -1;
05e6f5dc 3768
24a73b0a
KH		 3769 CODING_GET_INFO (coding, attrs, charset_list);
3770 eol_type = CODING_ID_EOL_TYPE (coding->id);
3771 if (VECTORP (eol_type))
3772 eol_type = Qunix;
3773
004068e4 3774 setup_iso_safe_charsets (attrs);
ff0dacd7
KH		 3775 /* Charset list may have been changed. */
3776 charset_list = CODING_ATTR_CHARSET_LIST (attrs); \
8f924df7 3777 coding->safe_charsets = (char *) SDATA (CODING_ATTR_SAFE_CHARSETS(attrs));
0eecad43 3778
df7492f9 3779 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
bdd9fb48 3780
df7492f9 3781 while (charbuf < charbuf_end)
4ed46869 3782 {
df7492f9 3783 ASSURE_DESTINATION (safe_room);
b73bfc1c 3784
df7492f9 3785 if (bol_designation)
b73bfc1c 3786 {
df7492f9 3787 unsigned char *dst_prev = dst;
4ed46869 3788
bdd9fb48 3789 /* We have to produce designation sequences if any now. */
df7492f9
KH		 3790 dst = encode_designation_at_bol (coding, charbuf, charbuf_end, dst);
3791 bol_designation = 0;
3792 /* We are sure that designation sequences are all ASCII bytes. */
3793 produced_chars += dst - dst_prev;
e0e989f6
KH		 3794 }
3795
df7492f9 3796 c = *charbuf++;
ec6d2bb8 3797
ff0dacd7
KH		 3798 if (c < 0)
3799 {
3800 /* Handle an annotation. */
3801 switch (*charbuf)
ec6d2bb8 3802 {
ff0dacd7
KH		 3803 case CODING_ANNOTATE_COMPOSITION_MASK:
3804 /* Not yet implemented. */
3805 break;
3806 case CODING_ANNOTATE_CHARSET_MASK:
cf7dfdf5 3807 preferred_charset_id = charbuf[2];
ff0dacd7
KH		 3808 if (preferred_charset_id >= 0
3809 && NILP (Fmemq (make_number (preferred_charset_id),
3810 charset_list)))
3811 preferred_charset_id = -1;
3812 break;
3813 default:
3814 abort ();
4ed46869 3815 }
ff0dacd7
KH		 3816 charbuf += -c - 1;
3817 continue;
4ed46869 3818 }
ec6d2bb8 3819
b73bfc1c
KH		 3820 /* Now encode the character C. */
3821 if (c < 0x20 || c == 0x7F)
3822 {
df7492f9
KH		 3823 if (c == '\n'
3824 || (c == '\r' && EQ (eol_type, Qmac)))
19a8d9e0 3825 {
df7492f9
KH		 3826 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
3827 ENCODE_RESET_PLANE_AND_REGISTER ();
3828 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_INIT_AT_BOL)
b73bfc1c 3829 {
df7492f9
KH		 3830 int i;
3831
3832 for (i = 0; i < 4; i++)
3833 CODING_ISO_DESIGNATION (coding, i)
3834 = CODING_ISO_INITIAL (coding, i);
b73bfc1c 3835 }
df7492f9
KH		 3836 bol_designation
3837 = CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL;
19a8d9e0 3838 }
df7492f9
KH		 3839 else if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_CNTL)
3840 ENCODE_RESET_PLANE_AND_REGISTER ();
3841 EMIT_ONE_ASCII_BYTE (c);
4ed46869 3842 }
df7492f9 3843 else if (ASCII_CHAR_P (c))
88993dfd 3844 {
df7492f9
KH		 3845 if (ascii_compatible)
3846 EMIT_ONE_ASCII_BYTE (c);
93dec019 3847 else
19a8d9e0 3848 {
bf16eb23
KH		 3849 struct charset *charset = CHARSET_FROM_ID (charset_ascii);
3850 ENCODE_ISO_CHARACTER (charset, c);
19a8d9e0 3851 }
4ed46869 3852 }
16eafb5d 3853 else if (CHAR_BYTE8_P (c))
88993dfd 3854 {
16eafb5d
KH		 3855 c = CHAR_TO_BYTE8 (c);
3856 EMIT_ONE_BYTE (c);
88993dfd 3857 }
b73bfc1c 3858 else
df7492f9 3859 {
ff0dacd7 3860 struct charset *charset;
b73bfc1c 3861
ff0dacd7
KH		 3862 if (preferred_charset_id >= 0)
3863 {
3864 charset = CHARSET_FROM_ID (preferred_charset_id);
3865 if (! CHAR_CHARSET_P (c, charset))
3866 charset = char_charset (c, charset_list, NULL);
3867 }
3868 else
3869 charset = char_charset (c, charset_list, NULL);
df7492f9
KH		 3870 if (!charset)
3871 {
41cbe562
KH		 3872 if (coding->mode & CODING_MODE_SAFE_ENCODING)
3873 {
3874 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
3875 charset = CHARSET_FROM_ID (charset_ascii);
3876 }
3877 else
3878 {
3879 c = coding->default_char;
3880 charset = char_charset (c, charset_list, NULL);
3881 }
df7492f9
KH		 3882 }
3883 ENCODE_ISO_CHARACTER (charset, c);
3884 }
84fbb8a0 3885 }
b73bfc1c 3886
df7492f9
KH		 3887 if (coding->mode & CODING_MODE_LAST_BLOCK
3888 && CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
3889 {
3890 ASSURE_DESTINATION (safe_room);
3891 ENCODE_RESET_PLANE_AND_REGISTER ();
3892 }
065e3595 3893 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 3894 CODING_ISO_BOL (coding) = bol_designation;
3895 coding->produced_char += produced_chars;
3896 coding->produced = dst - coding->destination;
3897 return 0;
4ed46869
KH		 3898}
3899
3900\f
df7492f9 3901/*** 8,9. SJIS and BIG5 handlers ***/
4ed46869 3902
df7492f9 3903/* Although SJIS and BIG5 are not ISO's coding system, they are used
4ed46869
KH		 3904 quite widely. So, for the moment, Emacs supports them in the bare
3905 C code. But, in the future, they may be supported only by CCL. */
3906
3907/* SJIS is a coding system encoding three character sets: ASCII, right
3908 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
3909 as is. A character of charset katakana-jisx0201 is encoded by
3910 "position-code + 0x80". A character of charset japanese-jisx0208
3911 is encoded in 2-byte but two position-codes are divided and shifted
df7492f9 3912 so that it fit in the range below.
4ed46869
KH		 3913
3914 --- CODE RANGE of SJIS ---
3915 (character set) (range)
3916 ASCII 0x00 .. 0x7F
df7492f9 3917 KATAKANA-JISX0201 0xA0 .. 0xDF
c28a9453 3918 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
d14d03ac 3919 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
4ed46869
KH		 3920 -------------------------------
3921
3922*/
3923
3924/* BIG5 is a coding system encoding two character sets: ASCII and
3925 Big5. An ASCII character is encoded as is. Big5 is a two-byte
df7492f9 3926 character set and is encoded in two-byte.
4ed46869
KH		 3927
3928 --- CODE RANGE of BIG5 ---
3929 (character set) (range)
3930 ASCII 0x00 .. 0x7F
3931 Big5 (1st byte) 0xA1 .. 0xFE
3932 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
3933 --------------------------
3934
df7492f9 3935 */
4ed46869
KH		 3936
3937/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3938 Check if a text is encoded in SJIS. If it is, return
df7492f9 3939 CATEGORY_MASK_SJIS, else return 0. */
4ed46869 3940
0a28aafb 3941static int
ff0dacd7 3942detect_coding_sjis (coding, detect_info)
df7492f9 3943 struct coding_system *coding;
ff0dacd7 3944 struct coding_detection_info *detect_info;
4ed46869 3945{
065e3595 3946 const unsigned char *src = coding->source, *src_base;
8f924df7 3947 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 3948 int multibytep = coding->src_multibyte;
3949 int consumed_chars = 0;
3950 int found = 0;
b73bfc1c 3951 int c;
df7492f9 3952
ff0dacd7 3953 detect_info->checked |= CATEGORY_MASK_SJIS;
df7492f9
KH		 3954 /* A coding system of this category is always ASCII compatible. */
3955 src += coding->head_ascii;
4ed46869 3956
b73bfc1c 3957 while (1)
4ed46869 3958 {
065e3595 3959 src_base = src;
df7492f9 3960 ONE_MORE_BYTE (c);
682169fe
KH		 3961 if (c < 0x80)
3962 continue;
df7492f9 3963 if ((c >= 0x81 && c <= 0x9F) || (c >= 0xE0 && c <= 0xEF))
4ed46869 3964 {
df7492f9 3965 ONE_MORE_BYTE (c);
682169fe 3966 if (c < 0x40 || c == 0x7F || c > 0xFC)
df7492f9 3967 break;
ff0dacd7 3968 found = CATEGORY_MASK_SJIS;
4ed46869 3969 }
df7492f9 3970 else if (c >= 0xA0 && c < 0xE0)
ff0dacd7 3971 found = CATEGORY_MASK_SJIS;
df7492f9
KH		 3972 else
3973 break;
4ed46869 3974 }
ff0dacd7 3975 detect_info->rejected |= CATEGORY_MASK_SJIS;
df7492f9
KH		 3976 return 0;
3977
3978 no_more_source:
065e3595 3979 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 3980 {
ff0dacd7 3981 detect_info->rejected |= CATEGORY_MASK_SJIS;
89528eb3 3982 return 0;
4ed46869 3983 }
ff0dacd7
KH		 3984 detect_info->found |= found;
3985 return 1;
4ed46869
KH		 3986}
3987
3988/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3989 Check if a text is encoded in BIG5. If it is, return
df7492f9 3990 CATEGORY_MASK_BIG5, else return 0. */
4ed46869 3991
0a28aafb 3992static int
ff0dacd7 3993detect_coding_big5 (coding, detect_info)
df7492f9 3994 struct coding_system *coding;
ff0dacd7 3995 struct coding_detection_info *detect_info;
4ed46869 3996{
065e3595 3997 const unsigned char *src = coding->source, *src_base;
8f924df7 3998 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 3999 int multibytep = coding->src_multibyte;
4000 int consumed_chars = 0;
4001 int found = 0;
b73bfc1c 4002 int c;
fa42c37f 4003
ff0dacd7 4004 detect_info->checked |= CATEGORY_MASK_BIG5;
df7492f9
KH		 4005 /* A coding system of this category is always ASCII compatible. */
4006 src += coding->head_ascii;
fa42c37f 4007
b73bfc1c 4008 while (1)
fa42c37f 4009 {
065e3595 4010 src_base = src;
df7492f9
KH		 4011 ONE_MORE_BYTE (c);
4012 if (c < 0x80)
fa42c37f 4013 continue;
df7492f9 4014 if (c >= 0xA1)
fa42c37f 4015 {
df7492f9
KH		 4016 ONE_MORE_BYTE (c);
4017 if (c < 0x40 || (c >= 0x7F && c <= 0xA0))
fa42c37f 4018 return 0;
ff0dacd7 4019 found = CATEGORY_MASK_BIG5;
fa42c37f 4020 }
df7492f9
KH		 4021 else
4022 break;
fa42c37f 4023 }
ff0dacd7 4024 detect_info->rejected |= CATEGORY_MASK_BIG5;
fa42c37f 4025 return 0;
fa42c37f 4026
df7492f9 4027 no_more_source:
065e3595 4028 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
89528eb3 4029 {
ff0dacd7 4030 detect_info->rejected |= CATEGORY_MASK_BIG5;
89528eb3
KH		 4031 return 0;
4032 }
ff0dacd7
KH		 4033 detect_info->found |= found;
4034 return 1;
fa42c37f
KH		 4035}
4036
4ed46869
KH		 4037/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
4038 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
fa42c37f 4039
b73bfc1c 4040static void
df7492f9 4041decode_coding_sjis (coding)
4ed46869 4042 struct coding_system *coding;
4ed46869 4043{
8f924df7
KH		 4044 const unsigned char *src = coding->source + coding->consumed;
4045 const unsigned char *src_end = coding->source + coding->src_bytes;
4046 const unsigned char *src_base;
69a80ea3
KH		 4047 int *charbuf = coding->charbuf + coding->charbuf_used;
4048 int *charbuf_end
4049 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 4050 int consumed_chars = 0, consumed_chars_base;
4051 int multibytep = coding->src_multibyte;
4052 struct charset *charset_roman, *charset_kanji, *charset_kana;
57a47f8a 4053 struct charset *charset_kanji2;
24a73b0a 4054 Lisp_Object attrs, charset_list, val;
ff0dacd7
KH		 4055 int char_offset = coding->produced_char;
4056 int last_offset = char_offset;
4057 int last_id = charset_ascii;
a5d301df 4058
24a73b0a 4059 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4060
4061 val = charset_list;
4062 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
89528eb3 4063 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4064 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4065 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 4066
b73bfc1c 4067 while (1)
4ed46869 4068 {
df7492f9 4069 int c, c1;
24a73b0a 4070 struct charset *charset;
fa42c37f 4071
b73bfc1c 4072 src_base = src;
df7492f9 4073 consumed_chars_base = consumed_chars;
fa42c37f 4074
df7492f9
KH		 4075 if (charbuf >= charbuf_end)
4076 break;
4077
4078 ONE_MORE_BYTE (c);
065e3595
KH		 4079 if (c < 0)
4080 goto invalid_code;
24a73b0a
KH		 4081 if (c < 0x80)
4082 charset = charset_roman;
57a47f8a 4083 else if (c == 0x80 || c == 0xA0)
8e921c4b 4084 goto invalid_code;
57a47f8a
KH		 4085 else if (c >= 0xA1 && c <= 0xDF)
4086 {
4087 /* SJIS -> JISX0201-Kana */
4088 c &= 0x7F;
4089 charset = charset_kana;
4090 }
4091 else if (c <= 0xEF)
df7492f9 4092 {
57a47f8a
KH		 4093 /* SJIS -> JISX0208 */
4094 ONE_MORE_BYTE (c1);
4095 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4096 goto invalid_code;
57a47f8a
KH		 4097 c = (c << 8) | c1;
4098 SJIS_TO_JIS (c);
4099 charset = charset_kanji;
4100 }
4101 else if (c <= 0xFC && charset_kanji2)
4102 {
c6876370 4103 /* SJIS -> JISX0213-2 */
57a47f8a
KH		 4104 ONE_MORE_BYTE (c1);
4105 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
24a73b0a 4106 goto invalid_code;
57a47f8a
KH		 4107 c = (c << 8) | c1;
4108 SJIS_TO_JIS2 (c);
4109 charset = charset_kanji2;
df7492f9 4110 }
57a47f8a
KH		 4111 else
4112 goto invalid_code;
24a73b0a
KH		 4113 if (charset->id != charset_ascii
4114 && last_id != charset->id)
4115 {
4116 if (last_id != charset_ascii)
69a80ea3 4117 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4118 last_id = charset->id;
4119 last_offset = char_offset;
4120 }
4121 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4122 *charbuf++ = c;
ff0dacd7 4123 char_offset++;
df7492f9 4124 continue;
b73bfc1c 4125
df7492f9
KH		 4126 invalid_code:
4127 src = src_base;
4128 consumed_chars = consumed_chars_base;
4129 ONE_MORE_BYTE (c);
065e3595 4130 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4131 char_offset++;
df7492f9
KH		 4132 coding->errors++;
4133 }
fa42c37f 4134
df7492f9 4135 no_more_source:
ff0dacd7 4136 if (last_id != charset_ascii)
69a80ea3 4137 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4138 coding->consumed_char += consumed_chars_base;
4139 coding->consumed = src_base - coding->source;
4140 coding->charbuf_used = charbuf - coding->charbuf;
fa42c37f
KH		 4141}
4142
b73bfc1c 4143static void
df7492f9 4144decode_coding_big5 (coding)
4ed46869 4145 struct coding_system *coding;
4ed46869 4146{
8f924df7
KH		 4147 const unsigned char *src = coding->source + coding->consumed;
4148 const unsigned char *src_end = coding->source + coding->src_bytes;
4149 const unsigned char *src_base;
69a80ea3
KH		 4150 int *charbuf = coding->charbuf + coding->charbuf_used;
4151 int *charbuf_end
4152 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 4153 int consumed_chars = 0, consumed_chars_base;
4154 int multibytep = coding->src_multibyte;
4155 struct charset *charset_roman, *charset_big5;
24a73b0a 4156 Lisp_Object attrs, charset_list, val;
ff0dacd7
KH		 4157 int char_offset = coding->produced_char;
4158 int last_offset = char_offset;
4159 int last_id = charset_ascii;
df7492f9 4160
24a73b0a 4161 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4162 val = charset_list;
4163 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4164 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4165
b73bfc1c 4166 while (1)
4ed46869 4167 {
df7492f9 4168 int c, c1;
24a73b0a 4169 struct charset *charset;
b73bfc1c
KH		 4170
4171 src_base = src;
df7492f9
KH		 4172 consumed_chars_base = consumed_chars;
4173
4174 if (charbuf >= charbuf_end)
4175 break;
4176
4177 ONE_MORE_BYTE (c);
b73bfc1c 4178
065e3595
KH		 4179 if (c < 0)
4180 goto invalid_code;
24a73b0a
KH		 4181 if (c < 0x80)
4182 charset = charset_roman;
4183 else
4ed46869 4184 {
24a73b0a
KH		 4185 /* BIG5 -> Big5 */
4186 if (c < 0xA1 || c > 0xFE)
4187 goto invalid_code;
4188 ONE_MORE_BYTE (c1);
4189 if (c1 < 0x40 || (c1 > 0x7E && c1 < 0xA1) || c1 > 0xFE)
4190 goto invalid_code;
4191 c = c << 8 | c1;
4192 charset = charset_big5;
4ed46869 4193 }
24a73b0a
KH		 4194 if (charset->id != charset_ascii
4195 && last_id != charset->id)
df7492f9 4196 {
24a73b0a 4197 if (last_id != charset_ascii)
69a80ea3 4198 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4199 last_id = charset->id;
4200 last_offset = char_offset;
4ed46869 4201 }
24a73b0a 4202 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
df7492f9 4203 *charbuf++ = c;
ff0dacd7 4204 char_offset++;
fb88bf2d
KH		 4205 continue;
4206
df7492f9 4207 invalid_code:
4ed46869 4208 src = src_base;
df7492f9
KH		 4209 consumed_chars = consumed_chars_base;
4210 ONE_MORE_BYTE (c);
065e3595 4211 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
ff0dacd7 4212 char_offset++;
df7492f9 4213 coding->errors++;
fb88bf2d 4214 }
d46c5b12 4215
df7492f9 4216 no_more_source:
ff0dacd7 4217 if (last_id != charset_ascii)
69a80ea3 4218 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4219 coding->consumed_char += consumed_chars_base;
4220 coding->consumed = src_base - coding->source;
4221 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 4222}
4223
4224/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
b73bfc1c
KH		 4225 This function can encode charsets `ascii', `katakana-jisx0201',
4226 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
4227 are sure that all these charsets are registered as official charset
4ed46869
KH		 4228 (i.e. do not have extended leading-codes). Characters of other
4229 charsets are produced without any encoding. If SJIS_P is 1, encode
4230 SJIS text, else encode BIG5 text. */
4231
df7492f9
KH		 4232static int
4233encode_coding_sjis (coding)
4ed46869 4234 struct coding_system *coding;
4ed46869 4235{
df7492f9
KH		 4236 int multibytep = coding->dst_multibyte;
4237 int *charbuf = coding->charbuf;
4238 int *charbuf_end = charbuf + coding->charbuf_used;
4239 unsigned char *dst = coding->destination + coding->produced;
4240 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4241 int safe_room = 4;
4242 int produced_chars = 0;
24a73b0a 4243 Lisp_Object attrs, charset_list, val;
df7492f9
KH		 4244 int ascii_compatible;
4245 struct charset *charset_roman, *charset_kanji, *charset_kana;
57a47f8a 4246 struct charset *charset_kanji2;
df7492f9 4247 int c;
a5d301df 4248
24a73b0a 4249 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4250 val = charset_list;
4251 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4252 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
57a47f8a
KH		 4253 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4254 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
4ed46869 4255
df7492f9 4256 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
93dec019 4257
df7492f9
KH		 4258 while (charbuf < charbuf_end)
4259 {
4260 ASSURE_DESTINATION (safe_room);
4261 c = *charbuf++;
b73bfc1c 4262 /* Now encode the character C. */
df7492f9
KH		 4263 if (ASCII_CHAR_P (c) && ascii_compatible)
4264 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4265 else if (CHAR_BYTE8_P (c))
4266 {
4267 c = CHAR_TO_BYTE8 (c);
4268 EMIT_ONE_BYTE (c);
4269 }
df7492f9 4270 else
b73bfc1c 4271 {
df7492f9
KH		 4272 unsigned code;
4273 struct charset *charset = char_charset (c, charset_list, &code);
4274
4275 if (!charset)
4ed46869 4276 {
41cbe562 4277 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4278 {
41cbe562
KH		 4279 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4280 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4281 }
41cbe562 4282 else
b73bfc1c 4283 {
41cbe562
KH		 4284 c = coding->default_char;
4285 charset = char_charset (c, charset_list, &code);
b73bfc1c 4286 }
b73bfc1c 4287 }
df7492f9
KH		 4288 if (code == CHARSET_INVALID_CODE (charset))
4289 abort ();
4290 if (charset == charset_kanji)
4291 {
4292 int c1, c2;
4293 JIS_TO_SJIS (code);
4294 c1 = code >> 8, c2 = code & 0xFF;
4295 EMIT_TWO_BYTES (c1, c2);
4296 }
4297 else if (charset == charset_kana)
4298 EMIT_ONE_BYTE (code | 0x80);
57a47f8a
KH		 4299 else if (charset_kanji2 && charset == charset_kanji2)
4300 {
4301 int c1, c2;
4302
4303 c1 = code >> 8;
4304 if (c1 == 0x21 || (c1 >= 0x23 && c1 < 0x25)
4305 || (c1 >= 0x2C && c1 <= 0x2F) || c1 >= 0x6E)
4306 {
4307 JIS_TO_SJIS2 (code);
4308 c1 = code >> 8, c2 = code & 0xFF;
4309 EMIT_TWO_BYTES (c1, c2);
4310 }
4311 else
4312 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4313 }
df7492f9
KH		 4314 else
4315 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4316 }
4317 }
065e3595 4318 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4319 coding->produced_char += produced_chars;
4320 coding->produced = dst - coding->destination;
4321 return 0;
4322}
4323
4324static int
4325encode_coding_big5 (coding)
4326 struct coding_system *coding;
4327{
4328 int multibytep = coding->dst_multibyte;
4329 int *charbuf = coding->charbuf;
4330 int *charbuf_end = charbuf + coding->charbuf_used;
4331 unsigned char *dst = coding->destination + coding->produced;
4332 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4333 int safe_room = 4;
4334 int produced_chars = 0;
24a73b0a 4335 Lisp_Object attrs, charset_list, val;
df7492f9
KH		 4336 int ascii_compatible;
4337 struct charset *charset_roman, *charset_big5;
4338 int c;
4339
24a73b0a 4340 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4341 val = charset_list;
4342 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4343 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4344 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4345
4346 while (charbuf < charbuf_end)
4347 {
4348 ASSURE_DESTINATION (safe_room);
4349 c = *charbuf++;
4350 /* Now encode the character C. */
4351 if (ASCII_CHAR_P (c) && ascii_compatible)
4352 EMIT_ONE_ASCII_BYTE (c);
16eafb5d
KH		 4353 else if (CHAR_BYTE8_P (c))
4354 {
4355 c = CHAR_TO_BYTE8 (c);
4356 EMIT_ONE_BYTE (c);
b73bfc1c
KH		 4357 }
4358 else
4359 {
df7492f9
KH		 4360 unsigned code;
4361 struct charset *charset = char_charset (c, charset_list, &code);
4362
4363 if (! charset)
b73bfc1c 4364 {
41cbe562 4365 if (coding->mode & CODING_MODE_SAFE_ENCODING)
b73bfc1c 4366 {
41cbe562
KH		 4367 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4368 charset = CHARSET_FROM_ID (charset_ascii);
b73bfc1c 4369 }
41cbe562 4370 else
0eecad43 4371 {
41cbe562
KH		 4372 c = coding->default_char;
4373 charset = char_charset (c, charset_list, &code);
0eecad43 4374 }
4ed46869 4375 }
df7492f9
KH		 4376 if (code == CHARSET_INVALID_CODE (charset))
4377 abort ();
4378 if (charset == charset_big5)
b73bfc1c 4379 {
df7492f9
KH		 4380 int c1, c2;
4381
4382 c1 = code >> 8, c2 = code & 0xFF;
4383 EMIT_TWO_BYTES (c1, c2);
b73bfc1c 4384 }
df7492f9
KH		 4385 else
4386 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4ed46869 4387 }
4ed46869 4388 }
065e3595 4389 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4390 coding->produced_char += produced_chars;
4391 coding->produced = dst - coding->destination;
4392 return 0;
4ed46869
KH		 4393}
4394
4395\f
df7492f9 4396/*** 10. CCL handlers ***/
1397dc18
KH		 4397
4398/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4399 Check if a text is encoded in a coding system of which
4400 encoder/decoder are written in CCL program. If it is, return
df7492f9 4401 CATEGORY_MASK_CCL, else return 0. */
1397dc18 4402
0a28aafb 4403static int
ff0dacd7 4404detect_coding_ccl (coding, detect_info)
df7492f9 4405 struct coding_system *coding;
ff0dacd7 4406 struct coding_detection_info *detect_info;
1397dc18 4407{
065e3595 4408 const unsigned char *src = coding->source, *src_base;
8f924df7 4409 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4410 int multibytep = coding->src_multibyte;
4411 int consumed_chars = 0;
4412 int found = 0;
0e219d54 4413 unsigned char *valids;
df7492f9
KH		 4414 int head_ascii = coding->head_ascii;
4415 Lisp_Object attrs;
4416
ff0dacd7
KH		 4417 detect_info->checked |= CATEGORY_MASK_CCL;
4418
df7492f9 4419 coding = &coding_categories[coding_category_ccl];
0e219d54 4420 valids = CODING_CCL_VALIDS (coding);
df7492f9
KH		 4421 attrs = CODING_ID_ATTRS (coding->id);
4422 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
4423 src += head_ascii;
1397dc18 4424
b73bfc1c 4425 while (1)
1397dc18 4426 {
df7492f9 4427 int c;
065e3595
KH		 4428
4429 src_base = src;
df7492f9 4430 ONE_MORE_BYTE (c);
065e3595 4431 if (c < 0 || ! valids[c])
df7492f9 4432 break;
ff0dacd7
KH		 4433 if ((valids[c] > 1))
4434 found = CATEGORY_MASK_CCL;
df7492f9 4435 }
ff0dacd7 4436 detect_info->rejected |= CATEGORY_MASK_CCL;
df7492f9
KH		 4437 return 0;
4438
4439 no_more_source:
ff0dacd7
KH		 4440 detect_info->found |= found;
4441 return 1;
df7492f9
KH		 4442}
4443
4444static void
4445decode_coding_ccl (coding)
4446 struct coding_system *coding;
4447{
7c78e542 4448 const unsigned char *src = coding->source + coding->consumed;
8f924df7 4449 const unsigned char *src_end = coding->source + coding->src_bytes;
69a80ea3
KH		 4450 int *charbuf = coding->charbuf + coding->charbuf_used;
4451 int *charbuf_end = coding->charbuf + coding->charbuf_size;
df7492f9
KH		 4452 int consumed_chars = 0;
4453 int multibytep = coding->src_multibyte;
4454 struct ccl_program ccl;
4455 int source_charbuf[1024];
4456 int source_byteidx[1024];
24a73b0a 4457 Lisp_Object attrs, charset_list;
df7492f9 4458
24a73b0a 4459 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4460 setup_ccl_program (&ccl, CODING_CCL_DECODER (coding));
4461
4462 while (src < src_end)
4463 {
7c78e542 4464 const unsigned char *p = src;
df7492f9
KH		 4465 int *source, *source_end;
4466 int i = 0;
4467
4468 if (multibytep)
4469 while (i < 1024 && p < src_end)
4470 {
4471 source_byteidx[i] = p - src;
4472 source_charbuf[i++] = STRING_CHAR_ADVANCE (p);
4473 }
4474 else
4475 while (i < 1024 && p < src_end)
4476 source_charbuf[i++] = *p++;
8f924df7 4477
df7492f9
KH		 4478 if (p == src_end && coding->mode & CODING_MODE_LAST_BLOCK)
4479 ccl.last_block = 1;
4480
4481 source = source_charbuf;
4482 source_end = source + i;
4483 while (source < source_end)
4484 {
4485 ccl_driver (&ccl, source, charbuf,
8dcbea82
KH		 4486 source_end - source, charbuf_end - charbuf,
4487 charset_list);
df7492f9
KH		 4488 source += ccl.consumed;
4489 charbuf += ccl.produced;
4490 if (ccl.status != CCL_STAT_SUSPEND_BY_DST)
4491 break;
4492 }
4493 if (source < source_end)
4494 src += source_byteidx[source - source_charbuf];
4495 else
4496 src = p;
4497 consumed_chars += source - source_charbuf;
4498
4499 if (ccl.status != CCL_STAT_SUSPEND_BY_SRC
4500 && ccl.status != CODING_RESULT_INSUFFICIENT_SRC)
4501 break;
4502 }
4503
4504 switch (ccl.status)
4505 {
4506 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 4507 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 4508 break;
4509 case CCL_STAT_SUSPEND_BY_DST:
4510 break;
4511 case CCL_STAT_QUIT:
4512 case CCL_STAT_INVALID_CMD:
065e3595 4513 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 4514 break;
4515 default:
065e3595 4516 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4517 break;
4518 }
4519 coding->consumed_char += consumed_chars;
4520 coding->consumed = src - coding->source;
4521 coding->charbuf_used = charbuf - coding->charbuf;
4522}
4523
4524static int
4525encode_coding_ccl (coding)
4526 struct coding_system *coding;
4527{
4528 struct ccl_program ccl;
4529 int multibytep = coding->dst_multibyte;
4530 int *charbuf = coding->charbuf;
4531 int *charbuf_end = charbuf + coding->charbuf_used;
4532 unsigned char *dst = coding->destination + coding->produced;
4533 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4534 unsigned char *adjusted_dst_end = dst_end - 1;
4535 int destination_charbuf[1024];
4536 int i, produced_chars = 0;
24a73b0a 4537 Lisp_Object attrs, charset_list;
df7492f9 4538
24a73b0a 4539 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9
KH		 4540 setup_ccl_program (&ccl, CODING_CCL_ENCODER (coding));
4541
4542 ccl.last_block = coding->mode & CODING_MODE_LAST_BLOCK;
4543 ccl.dst_multibyte = coding->dst_multibyte;
4544
4545 while (charbuf < charbuf_end && dst < adjusted_dst_end)
4546 {
4547 int dst_bytes = dst_end - dst;
4548 if (dst_bytes > 1024)
4549 dst_bytes = 1024;
4550
4551 ccl_driver (&ccl, charbuf, destination_charbuf,
8dcbea82 4552 charbuf_end - charbuf, dst_bytes, charset_list);
df7492f9
KH		 4553 charbuf += ccl.consumed;
4554 if (multibytep)
4555 for (i = 0; i < ccl.produced; i++)
4556 EMIT_ONE_BYTE (destination_charbuf[i] & 0xFF);
4557 else
4558 {
4559 for (i = 0; i < ccl.produced; i++)
4560 *dst++ = destination_charbuf[i] & 0xFF;
4561 produced_chars += ccl.produced;
4562 }
4563 }
4564
4565 switch (ccl.status)
4566 {
4567 case CCL_STAT_SUSPEND_BY_SRC:
065e3595 4568 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
df7492f9
KH		 4569 break;
4570 case CCL_STAT_SUSPEND_BY_DST:
065e3595 4571 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 4572 break;
4573 case CCL_STAT_QUIT:
4574 case CCL_STAT_INVALID_CMD:
065e3595 4575 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
df7492f9
KH		 4576 break;
4577 default:
065e3595 4578 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 4579 break;
1397dc18 4580 }
df7492f9
KH		 4581
4582 coding->produced_char += produced_chars;
4583 coding->produced = dst - coding->destination;
4584 return 0;
1397dc18
KH		 4585}
4586
df7492f9 4587
1397dc18 4588\f
df7492f9 4589/*** 10, 11. no-conversion handlers ***/
4ed46869 4590
b73bfc1c 4591/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
4ed46869 4592
b73bfc1c 4593static void
df7492f9 4594decode_coding_raw_text (coding)
4ed46869 4595 struct coding_system *coding;
4ed46869 4596{
df7492f9 4597 coding->chars_at_source = 1;
2c78b7e1
KH		 4598 coding->consumed_char = 0;
4599 coding->consumed = 0;
065e3595 4600 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 4601}
4ed46869 4602
df7492f9
KH		 4603static int
4604encode_coding_raw_text (coding)
4605 struct coding_system *coding;
4606{
4607 int multibytep = coding->dst_multibyte;
4608 int *charbuf = coding->charbuf;
4609 int *charbuf_end = coding->charbuf + coding->charbuf_used;
4610 unsigned char *dst = coding->destination + coding->produced;
4611 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4612 int produced_chars = 0;
b73bfc1c
KH		 4613 int c;
4614
df7492f9 4615 if (multibytep)
b73bfc1c 4616 {
df7492f9 4617 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
4ed46869 4618
df7492f9
KH		 4619 if (coding->src_multibyte)
4620 while (charbuf < charbuf_end)
4621 {
4622 ASSURE_DESTINATION (safe_room);
4623 c = *charbuf++;
4624 if (ASCII_CHAR_P (c))
4625 EMIT_ONE_ASCII_BYTE (c);
4626 else if (CHAR_BYTE8_P (c))
4627 {
4628 c = CHAR_TO_BYTE8 (c);
4629 EMIT_ONE_BYTE (c);
4630 }
4631 else
4632 {
4633 unsigned char str[MAX_MULTIBYTE_LENGTH], *p0 = str, *p1 = str;
93dec019 4634
df7492f9
KH		 4635 CHAR_STRING_ADVANCE (c, p1);
4636 while (p0 < p1)
9d123124
KH		 4637 {
4638 EMIT_ONE_BYTE (*p0);
4639 p0++;
4640 }
df7492f9
KH		 4641 }
4642 }
b73bfc1c 4643 else
df7492f9
KH		 4644 while (charbuf < charbuf_end)
4645 {
4646 ASSURE_DESTINATION (safe_room);
4647 c = *charbuf++;
4648 EMIT_ONE_BYTE (c);
4649 }
4650 }
4651 else
4ed46869 4652 {
df7492f9 4653 if (coding->src_multibyte)
d46c5b12 4654 {
df7492f9
KH		 4655 int safe_room = MAX_MULTIBYTE_LENGTH;
4656
4657 while (charbuf < charbuf_end)
d46c5b12 4658 {
df7492f9
KH		 4659 ASSURE_DESTINATION (safe_room);
4660 c = *charbuf++;
4661 if (ASCII_CHAR_P (c))
4662 *dst++ = c;
4663 else if (CHAR_BYTE8_P (c))
4664 *dst++ = CHAR_TO_BYTE8 (c);
b73bfc1c 4665 else
df7492f9
KH		 4666 CHAR_STRING_ADVANCE (c, dst);
4667 produced_chars++;
d46c5b12
KH		 4668 }
4669 }
df7492f9
KH		 4670 else
4671 {
4672 ASSURE_DESTINATION (charbuf_end - charbuf);
4673 while (charbuf < charbuf_end && dst < dst_end)
4674 *dst++ = *charbuf++;
4675 produced_chars = dst - (coding->destination + coding->dst_bytes);
8f924df7 4676 }
4ed46869 4677 }
065e3595 4678 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4679 coding->produced_char += produced_chars;
4680 coding->produced = dst - coding->destination;
4681 return 0;
4ed46869
KH		 4682}
4683
ff0dacd7
KH		 4684/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4685 Check if a text is encoded in a charset-based coding system. If it
4686 is, return 1, else return 0. */
4687
0a28aafb 4688static int
ff0dacd7 4689detect_coding_charset (coding, detect_info)
df7492f9 4690 struct coding_system *coding;
ff0dacd7 4691 struct coding_detection_info *detect_info;
1397dc18 4692{
065e3595 4693 const unsigned char *src = coding->source, *src_base;
8f924df7 4694 const unsigned char *src_end = coding->source + coding->src_bytes;
df7492f9
KH		 4695 int multibytep = coding->src_multibyte;
4696 int consumed_chars = 0;
4697 Lisp_Object attrs, valids;
584948ac 4698 int found = 0;
1397dc18 4699
ff0dacd7
KH		 4700 detect_info->checked |= CATEGORY_MASK_CHARSET;
4701
df7492f9
KH		 4702 coding = &coding_categories[coding_category_charset];
4703 attrs = CODING_ID_ATTRS (coding->id);
4704 valids = AREF (attrs, coding_attr_charset_valids);
4705
4706 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
4707 src += coding->head_ascii;
1397dc18 4708
b73bfc1c 4709 while (1)
1397dc18 4710 {
df7492f9 4711 int c;
1397dc18 4712
065e3595 4713 src_base = src;
df7492f9 4714 ONE_MORE_BYTE (c);
065e3595
KH		 4715 if (c < 0)
4716 continue;
df7492f9
KH		 4717 if (NILP (AREF (valids, c)))
4718 break;
584948ac 4719 if (c >= 0x80)
ff0dacd7 4720 found = CATEGORY_MASK_CHARSET;
df7492f9 4721 }
ff0dacd7 4722 detect_info->rejected |= CATEGORY_MASK_CHARSET;
df7492f9 4723 return 0;
4ed46869 4724
df7492f9 4725 no_more_source:
ff0dacd7
KH		 4726 detect_info->found |= found;
4727 return 1;
df7492f9 4728}
b73bfc1c 4729
b73bfc1c 4730static void
df7492f9 4731decode_coding_charset (coding)
4ed46869 4732 struct coding_system *coding;
4ed46869 4733{
8f924df7
KH		 4734 const unsigned char *src = coding->source + coding->consumed;
4735 const unsigned char *src_end = coding->source + coding->src_bytes;
4736 const unsigned char *src_base;
69a80ea3
KH		 4737 int *charbuf = coding->charbuf + coding->charbuf_used;
4738 int *charbuf_end
4739 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
df7492f9
KH		 4740 int consumed_chars = 0, consumed_chars_base;
4741 int multibytep = coding->src_multibyte;
24a73b0a 4742 Lisp_Object attrs, charset_list, valids;
ff0dacd7
KH		 4743 int char_offset = coding->produced_char;
4744 int last_offset = char_offset;
4745 int last_id = charset_ascii;
df7492f9 4746
24a73b0a 4747 CODING_GET_INFO (coding, attrs, charset_list);
4eb6d3f1 4748 valids = AREF (attrs, coding_attr_charset_valids);
b73bfc1c 4749
df7492f9 4750 while (1)
4ed46869 4751 {
4eb6d3f1 4752 int c;
24a73b0a
KH		 4753 Lisp_Object val;
4754 struct charset *charset;
4755 int dim;
4756 int len = 1;
4757 unsigned code;
df7492f9
KH		 4758
4759 src_base = src;
4760 consumed_chars_base = consumed_chars;
b73bfc1c 4761
df7492f9
KH		 4762 if (charbuf >= charbuf_end)
4763 break;
4764
4eb6d3f1 4765 ONE_MORE_BYTE (c);
065e3595
KH		 4766 if (c < 0)
4767 goto invalid_code;
24a73b0a
KH		 4768 code = c;
4769
4770 val = AREF (valids, c);
4771 if (NILP (val))
4772 goto invalid_code;
4773 if (INTEGERP (val))
d46c5b12 4774 {
24a73b0a
KH		 4775 charset = CHARSET_FROM_ID (XFASTINT (val));
4776 dim = CHARSET_DIMENSION (charset);
4777 while (len < dim)
b73bfc1c 4778 {
24a73b0a
KH		 4779 ONE_MORE_BYTE (c);
4780 code = (code << 8) | c;
4781 len++;
b73bfc1c 4782 }
24a73b0a
KH		 4783 CODING_DECODE_CHAR (coding, src, src_base, src_end,
4784 charset, code, c);
d46c5b12 4785 }
df7492f9 4786 else
d46c5b12 4787 {
24a73b0a
KH		 4788 /* VAL is a list of charset IDs. It is assured that the
4789 list is sorted by charset dimensions (smaller one
4790 comes first). */
4791 while (CONSP (val))
4eb6d3f1 4792 {
24a73b0a 4793 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
c7c66a95 4794 dim = CHARSET_DIMENSION (charset);
f9d71dcd 4795 while (len < dim)
4eb6d3f1 4796 {
acb2a965
KH		 4797 ONE_MORE_BYTE (c);
4798 code = (code << 8) | c;
f9d71dcd 4799 len++;
4eb6d3f1 4800 }
24a73b0a
KH		 4801 CODING_DECODE_CHAR (coding, src, src_base,
4802 src_end, charset, code, c);
4803 if (c >= 0)
4804 break;
4805 val = XCDR (val);
ff0dacd7 4806 }
d46c5b12 4807 }
24a73b0a
KH		 4808 if (c < 0)
4809 goto invalid_code;
4810 if (charset->id != charset_ascii
4811 && last_id != charset->id)
4812 {
4813 if (last_id != charset_ascii)
69a80ea3 4814 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
24a73b0a
KH		 4815 last_id = charset->id;
4816 last_offset = char_offset;
4817 }
4818
df7492f9 4819 *charbuf++ = c;
ff0dacd7 4820 char_offset++;
df7492f9
KH		 4821 continue;
4822
4823 invalid_code:
4824 src = src_base;
4825 consumed_chars = consumed_chars_base;
4826 ONE_MORE_BYTE (c);
065e3595 4827 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
ff0dacd7 4828 char_offset++;
df7492f9 4829 coding->errors++;
4ed46869
KH		 4830 }
4831
df7492f9 4832 no_more_source:
ff0dacd7 4833 if (last_id != charset_ascii)
69a80ea3 4834 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
df7492f9
KH		 4835 coding->consumed_char += consumed_chars_base;
4836 coding->consumed = src_base - coding->source;
4837 coding->charbuf_used = charbuf - coding->charbuf;
4ed46869
KH		 4838}
4839
df7492f9
KH		 4840static int
4841encode_coding_charset (coding)
4ed46869 4842 struct coding_system *coding;
4ed46869 4843{
df7492f9
KH		 4844 int multibytep = coding->dst_multibyte;
4845 int *charbuf = coding->charbuf;
4846 int *charbuf_end = charbuf + coding->charbuf_used;
4847 unsigned char *dst = coding->destination + coding->produced;
4848 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4849 int safe_room = MAX_MULTIBYTE_LENGTH;
4850 int produced_chars = 0;
24a73b0a 4851 Lisp_Object attrs, charset_list;
df7492f9 4852 int ascii_compatible;
b73bfc1c 4853 int c;
b73bfc1c 4854
24a73b0a 4855 CODING_GET_INFO (coding, attrs, charset_list);
df7492f9 4856 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
fb88bf2d 4857
df7492f9 4858 while (charbuf < charbuf_end)
4ed46869 4859 {
4eb6d3f1 4860 struct charset *charset;
df7492f9 4861 unsigned code;
8f924df7 4862
df7492f9
KH		 4863 ASSURE_DESTINATION (safe_room);
4864 c = *charbuf++;
4865 if (ascii_compatible && ASCII_CHAR_P (c))
4866 EMIT_ONE_ASCII_BYTE (c);
16eafb5d 4867 else if (CHAR_BYTE8_P (c))
4ed46869 4868 {
16eafb5d
KH		 4869 c = CHAR_TO_BYTE8 (c);
4870 EMIT_ONE_BYTE (c);
d46c5b12 4871 }
d46c5b12 4872 else
b73bfc1c 4873 {
4eb6d3f1
KH		 4874 charset = char_charset (c, charset_list, &code);
4875 if (charset)
4876 {
4877 if (CHARSET_DIMENSION (charset) == 1)
4878 EMIT_ONE_BYTE (code);
4879 else if (CHARSET_DIMENSION (charset) == 2)
4880 EMIT_TWO_BYTES (code >> 8, code & 0xFF);
4881 else if (CHARSET_DIMENSION (charset) == 3)
4882 EMIT_THREE_BYTES (code >> 16, (code >> 8) & 0xFF, code & 0xFF);
4883 else
4884 EMIT_FOUR_BYTES (code >> 24, (code >> 16) & 0xFF,
4885 (code >> 8) & 0xFF, code & 0xFF);
4886 }
4887 else
41cbe562
KH		 4888 {
4889 if (coding->mode & CODING_MODE_SAFE_ENCODING)
4890 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4891 else
4892 c = coding->default_char;
4893 EMIT_ONE_BYTE (c);
4894 }
4ed46869 4895 }
4ed46869
KH		 4896 }
4897
065e3595 4898 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9
KH		 4899 coding->produced_char += produced_chars;
4900 coding->produced = dst - coding->destination;
4901 return 0;
4ed46869
KH		 4902}
4903
4904\f
1397dc18 4905/*** 7. C library functions ***/
4ed46869 4906
df7492f9
KH		 4907/* Setup coding context CODING from information about CODING_SYSTEM.
4908 If CODING_SYSTEM is nil, `no-conversion' is assumed. If
4909 CODING_SYSTEM is invalid, signal an error. */
4ed46869 4910
ec6d2bb8 4911void
e0e989f6
KH		 4912setup_coding_system (coding_system, coding)
4913 Lisp_Object coding_system;
4ed46869
KH		 4914 struct coding_system *coding;
4915{
df7492f9
KH		 4916 Lisp_Object attrs;
4917 Lisp_Object eol_type;
4918 Lisp_Object coding_type;
4608c386 4919 Lisp_Object val;
4ed46869 4920
df7492f9 4921 if (NILP (coding_system))
ae6f73fa 4922 coding_system = Qundecided;
c07c8e12 4923
df7492f9 4924 CHECK_CODING_SYSTEM_GET_ID (coding_system, coding->id);
1f5dbf34 4925
df7492f9
KH		 4926 attrs = CODING_ID_ATTRS (coding->id);
4927 eol_type = CODING_ID_EOL_TYPE (coding->id);
1f5dbf34 4928
df7492f9
KH		 4929 coding->mode = 0;
4930 coding->head_ascii = -1;
4931 coding->common_flags
4932 = (VECTORP (eol_type) ? CODING_REQUIRE_DETECTION_MASK : 0);
5e5c78be
KH		 4933 if (! NILP (CODING_ATTR_POST_READ (attrs)))
4934 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
4935 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
4936 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
8f924df7
KH		 4937 if (! NILP (CODING_ATTR_FOR_UNIBYTE (attrs)))
4938 coding->common_flags |= CODING_FOR_UNIBYTE_MASK;
1f5dbf34 4939
df7492f9 4940 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 4941 coding->max_charset_id = SCHARS (val) - 1;
4942 coding->safe_charsets = (char *) SDATA (val);
df7492f9 4943 coding->default_char = XINT (CODING_ATTR_DEFAULT_CHAR (attrs));
4608c386 4944
df7492f9
KH		 4945 coding_type = CODING_ATTR_TYPE (attrs);
4946 if (EQ (coding_type, Qundecided))
d46c5b12 4947 {
df7492f9
KH		 4948 coding->detector = NULL;
4949 coding->decoder = decode_coding_raw_text;
4950 coding->encoder = encode_coding_raw_text;
4951 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 4952 }
df7492f9 4953 else if (EQ (coding_type, Qiso_2022))
d46c5b12 4954 {
df7492f9
KH		 4955 int i;
4956 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
4957
4958 /* Invoke graphic register 0 to plane 0. */
4959 CODING_ISO_INVOCATION (coding, 0) = 0;
4960 /* Invoke graphic register 1 to plane 1 if we can use 8-bit. */
4961 CODING_ISO_INVOCATION (coding, 1)
4962 = (flags & CODING_ISO_FLAG_SEVEN_BITS ? -1 : 1);
4963 /* Setup the initial status of designation. */
4964 for (i = 0; i < 4; i++)
4965 CODING_ISO_DESIGNATION (coding, i) = CODING_ISO_INITIAL (coding, i);
4966 /* Not single shifting initially. */
4967 CODING_ISO_SINGLE_SHIFTING (coding) = 0;
4968 /* Beginning of buffer should also be regarded as bol. */
4969 CODING_ISO_BOL (coding) = 1;
4970 coding->detector = detect_coding_iso_2022;
4971 coding->decoder = decode_coding_iso_2022;
4972 coding->encoder = encode_coding_iso_2022;
4973 if (flags & CODING_ISO_FLAG_SAFE)
4974 coding->mode |= CODING_MODE_SAFE_ENCODING;
d46c5b12 4975 coding->common_flags
df7492f9
KH		 4976 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
4977 | CODING_REQUIRE_FLUSHING_MASK);
4978 if (flags & CODING_ISO_FLAG_COMPOSITION)
4979 coding->common_flags |= CODING_ANNOTATE_COMPOSITION_MASK;
ff0dacd7
KH		 4980 if (flags & CODING_ISO_FLAG_DESIGNATION)
4981 coding->common_flags |= CODING_ANNOTATE_CHARSET_MASK;
df7492f9
KH		 4982 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
4983 {
4984 setup_iso_safe_charsets (attrs);
4985 val = CODING_ATTR_SAFE_CHARSETS (attrs);
8f924df7
KH		 4986 coding->max_charset_id = SCHARS (val) - 1;
4987 coding->safe_charsets = (char *) SDATA (val);
df7492f9
KH		 4988 }
4989 CODING_ISO_FLAGS (coding) = flags;
d46c5b12 4990 }
df7492f9 4991 else if (EQ (coding_type, Qcharset))
d46c5b12 4992 {
df7492f9
KH		 4993 coding->detector = detect_coding_charset;
4994 coding->decoder = decode_coding_charset;
4995 coding->encoder = encode_coding_charset;
d46c5b12 4996 coding->common_flags
df7492f9 4997 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
d46c5b12 4998 }
df7492f9 4999 else if (EQ (coding_type, Qutf_8))
d46c5b12 5000 {
df7492f9
KH		 5001 coding->detector = detect_coding_utf_8;
5002 coding->decoder = decode_coding_utf_8;
5003 coding->encoder = encode_coding_utf_8;
5004 coding->common_flags
5005 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5006 }
5007 else if (EQ (coding_type, Qutf_16))
5008 {
5009 val = AREF (attrs, coding_attr_utf_16_bom);
5010 CODING_UTF_16_BOM (coding) = (CONSP (val) ? utf_16_detect_bom
5011 : EQ (val, Qt) ? utf_16_with_bom
5012 : utf_16_without_bom);
5013 val = AREF (attrs, coding_attr_utf_16_endian);
b49a1807 5014 CODING_UTF_16_ENDIAN (coding) = (EQ (val, Qbig) ? utf_16_big_endian
df7492f9 5015 : utf_16_little_endian);
e19c3639 5016 CODING_UTF_16_SURROGATE (coding) = 0;
df7492f9
KH		 5017 coding->detector = detect_coding_utf_16;
5018 coding->decoder = decode_coding_utf_16;
5019 coding->encoder = encode_coding_utf_16;
5020 coding->common_flags
5021 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
b49a1807
KH		 5022 if (CODING_UTF_16_BOM (coding) == utf_16_detect_bom)
5023 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
d46c5b12 5024 }
df7492f9 5025 else if (EQ (coding_type, Qccl))
4ed46869 5026 {
df7492f9
KH		 5027 coding->detector = detect_coding_ccl;
5028 coding->decoder = decode_coding_ccl;
5029 coding->encoder = encode_coding_ccl;
c952af22 5030 coding->common_flags
df7492f9
KH		 5031 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5032 | CODING_REQUIRE_FLUSHING_MASK);
5033 }
5034 else if (EQ (coding_type, Qemacs_mule))
5035 {
5036 coding->detector = detect_coding_emacs_mule;
5037 coding->decoder = decode_coding_emacs_mule;
5038 coding->encoder = encode_coding_emacs_mule;
c952af22 5039 coding->common_flags
df7492f9
KH		 5040 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5041 if (! NILP (AREF (attrs, coding_attr_emacs_mule_full))
5042 && ! EQ (CODING_ATTR_CHARSET_LIST (attrs), Vemacs_mule_charset_list))
5043 {
5044 Lisp_Object tail, safe_charsets;
5045 int max_charset_id = 0;
5046
5047 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5048 tail = XCDR (tail))
5049 if (max_charset_id < XFASTINT (XCAR (tail)))
5050 max_charset_id = XFASTINT (XCAR (tail));
5051 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
5052 make_number (255));
5053 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5054 tail = XCDR (tail))
8f924df7 5055 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 5056 coding->max_charset_id = max_charset_id;
8f924df7 5057 coding->safe_charsets = (char *) SDATA (safe_charsets);
df7492f9
KH		 5058 }
5059 }
5060 else if (EQ (coding_type, Qshift_jis))
5061 {
5062 coding->detector = detect_coding_sjis;
5063 coding->decoder = decode_coding_sjis;
5064 coding->encoder = encode_coding_sjis;
c952af22 5065 coding->common_flags
df7492f9
KH		 5066 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5067 }
5068 else if (EQ (coding_type, Qbig5))
5069 {
5070 coding->detector = detect_coding_big5;
5071 coding->decoder = decode_coding_big5;
5072 coding->encoder = encode_coding_big5;
c952af22 5073 coding->common_flags
df7492f9
KH		 5074 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5075 }
5076 else /* EQ (coding_type, Qraw_text) */
ec6d2bb8 5077 {
df7492f9
KH		 5078 coding->detector = NULL;
5079 coding->decoder = decode_coding_raw_text;
5080 coding->encoder = encode_coding_raw_text;
ea29edf2
KH		 5081 if (! EQ (eol_type, Qunix))
5082 {
5083 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5084 if (! VECTORP (eol_type))
5085 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
5086 }
5087
4ed46869 5088 }
4ed46869 5089
df7492f9 5090 return;
4ed46869
KH		 5091}
5092
0ff61e78
KH		 5093/* Return a list of charsets supported by CODING. */
5094
5095Lisp_Object
5096coding_charset_list (coding)
5097 struct coding_system *coding;
5098{
35befdaa 5099 Lisp_Object attrs, charset_list;
0ff61e78
KH		 5100
5101 CODING_GET_INFO (coding, attrs, charset_list);
5102 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5103 {
5104 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5105
5106 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5107 charset_list = Viso_2022_charset_list;
5108 }
5109 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5110 {
5111 charset_list = Vemacs_mule_charset_list;
5112 }
5113 return charset_list;
5114}
5115
5116
df7492f9
KH		 5117/* Return raw-text or one of its subsidiaries that has the same
5118 eol_type as CODING-SYSTEM. */
ec6d2bb8 5119
df7492f9
KH		 5120Lisp_Object
5121raw_text_coding_system (coding_system)
5122 Lisp_Object coding_system;
ec6d2bb8 5123{
0be8721c 5124 Lisp_Object spec, attrs;
df7492f9 5125 Lisp_Object eol_type, raw_text_eol_type;
ec6d2bb8 5126
d3e4cb56
KH		 5127 if (NILP (coding_system))
5128 return Qraw_text;
df7492f9
KH		 5129 spec = CODING_SYSTEM_SPEC (coding_system);
5130 attrs = AREF (spec, 0);
ec6d2bb8 5131
df7492f9
KH		 5132 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
5133 return coding_system;
ec6d2bb8 5134
df7492f9
KH		 5135 eol_type = AREF (spec, 2);
5136 if (VECTORP (eol_type))
5137 return Qraw_text;
5138 spec = CODING_SYSTEM_SPEC (Qraw_text);
5139 raw_text_eol_type = AREF (spec, 2);
5140 return (EQ (eol_type, Qunix) ? AREF (raw_text_eol_type, 0)
5141 : EQ (eol_type, Qdos) ? AREF (raw_text_eol_type, 1)
5142 : AREF (raw_text_eol_type, 2));
ec6d2bb8
KH		 5143}
5144
54f78171 5145
df7492f9
KH		 5146/* If CODING_SYSTEM doesn't specify end-of-line format but PARENT
5147 does, return one of the subsidiary that has the same eol-spec as
5148 PARENT. Otherwise, return CODING_SYSTEM. */
5149
5150Lisp_Object
5151coding_inherit_eol_type (coding_system, parent)
b74e4686 5152 Lisp_Object coding_system, parent;
54f78171 5153{
3e139625 5154 Lisp_Object spec, eol_type;
54f78171 5155
d3e4cb56
KH		 5156 if (NILP (coding_system))
5157 coding_system = Qraw_text;
df7492f9 5158 spec = CODING_SYSTEM_SPEC (coding_system);
df7492f9 5159 eol_type = AREF (spec, 2);
d3e4cb56
KH		 5160 if (VECTORP (eol_type)
5161 && ! NILP (parent))
df7492f9
KH		 5162 {
5163 Lisp_Object parent_spec;
df7492f9
KH		 5164 Lisp_Object parent_eol_type;
5165
5166 parent_spec
5167 = CODING_SYSTEM_SPEC (buffer_defaults.buffer_file_coding_system);
5168 parent_eol_type = AREF (parent_spec, 2);
5169 if (EQ (parent_eol_type, Qunix))
5170 coding_system = AREF (eol_type, 0);
5171 else if (EQ (parent_eol_type, Qdos))
5172 coding_system = AREF (eol_type, 1);
5173 else if (EQ (parent_eol_type, Qmac))
5174 coding_system = AREF (eol_type, 2);
54f78171 5175 }
df7492f9 5176 return coding_system;
54f78171
KH		 5177}
5178
4ed46869
KH		 5179/* Emacs has a mechanism to automatically detect a coding system if it
5180 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
5181 it's impossible to distinguish some coding systems accurately
5182 because they use the same range of codes. So, at first, coding
5183 systems are categorized into 7, those are:
5184
0ef69138 5185 o coding-category-emacs-mule
4ed46869
KH		 5186
5187 The category for a coding system which has the same code range
5188 as Emacs' internal format. Assigned the coding-system (Lisp
0ef69138 5189 symbol) `emacs-mule' by default.
4ed46869
KH		 5190
5191 o coding-category-sjis
5192
5193 The category for a coding system which has the same code range
5194 as SJIS. Assigned the coding-system (Lisp
7717c392 5195 symbol) `japanese-shift-jis' by default.
4ed46869
KH		 5196
5197 o coding-category-iso-7
5198
5199 The category for a coding system which has the same code range
7717c392 5200 as ISO2022 of 7-bit environment. This doesn't use any locking
d46c5b12
KH		 5201 shift and single shift functions. This can encode/decode all
5202 charsets. Assigned the coding-system (Lisp symbol)
5203 `iso-2022-7bit' by default.
5204
5205 o coding-category-iso-7-tight
5206
5207 Same as coding-category-iso-7 except that this can
5208 encode/decode only the specified charsets.
4ed46869
KH		 5209
5210 o coding-category-iso-8-1
5211
5212 The category for a coding system which has the same code range
5213 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 5214 for DIMENSION1 charset. This doesn't use any locking shift
5215 and single shift functions. Assigned the coding-system (Lisp
5216 symbol) `iso-latin-1' by default.
4ed46869
KH		 5217
5218 o coding-category-iso-8-2
5219
5220 The category for a coding system which has the same code range
5221 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH		 5222 for DIMENSION2 charset. This doesn't use any locking shift
5223 and single shift functions. Assigned the coding-system (Lisp
5224 symbol) `japanese-iso-8bit' by default.
4ed46869 5225
7717c392 5226 o coding-category-iso-7-else
4ed46869
KH		 5227
5228 The category for a coding system which has the same code range
df7492f9 5229 as ISO2022 of 7-bit environemnt but uses locking shift or
7717c392
KH		 5230 single shift functions. Assigned the coding-system (Lisp
5231 symbol) `iso-2022-7bit-lock' by default.
5232
5233 o coding-category-iso-8-else
5234
5235 The category for a coding system which has the same code range
df7492f9 5236 as ISO2022 of 8-bit environemnt but uses locking shift or
7717c392
KH		 5237 single shift functions. Assigned the coding-system (Lisp
5238 symbol) `iso-2022-8bit-ss2' by default.
4ed46869
KH		 5239
5240 o coding-category-big5
5241
5242 The category for a coding system which has the same code range
5243 as BIG5. Assigned the coding-system (Lisp symbol)
e0e989f6 5244 `cn-big5' by default.
4ed46869 5245
fa42c37f
KH		 5246 o coding-category-utf-8
5247
5248 The category for a coding system which has the same code range
6e76ae91 5249 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
fa42c37f
KH		 5250 symbol) `utf-8' by default.
5251
5252 o coding-category-utf-16-be
5253
5254 The category for a coding system in which a text has an
5255 Unicode signature (cf. Unicode Standard) in the order of BIG
5256 endian at the head. Assigned the coding-system (Lisp symbol)
5257 `utf-16-be' by default.
5258
5259 o coding-category-utf-16-le
5260
5261 The category for a coding system in which a text has an
5262 Unicode signature (cf. Unicode Standard) in the order of
5263 LITTLE endian at the head. Assigned the coding-system (Lisp
5264 symbol) `utf-16-le' by default.
5265
1397dc18
KH		 5266 o coding-category-ccl
5267
5268 The category for a coding system of which encoder/decoder is
5269 written in CCL programs. The default value is nil, i.e., no
5270 coding system is assigned.
5271
4ed46869
KH		 5272 o coding-category-binary
5273
5274 The category for a coding system not categorized in any of the
5275 above. Assigned the coding-system (Lisp symbol)
e0e989f6 5276 `no-conversion' by default.
4ed46869
KH		 5277
5278 Each of them is a Lisp symbol and the value is an actual
df7492f9 5279 `coding-system's (this is also a Lisp symbol) assigned by a user.
4ed46869
KH		 5280 What Emacs does actually is to detect a category of coding system.
5281 Then, it uses a `coding-system' assigned to it. If Emacs can't
df7492f9 5282 decide only one possible category, it selects a category of the
4ed46869
KH		 5283 highest priority. Priorities of categories are also specified by a
5284 user in a Lisp variable `coding-category-list'.
5285
5286*/
5287
df7492f9
KH		 5288#define EOL_SEEN_NONE 0
5289#define EOL_SEEN_LF 1
5290#define EOL_SEEN_CR 2
5291#define EOL_SEEN_CRLF 4
66cfb530 5292
ff0dacd7
KH		 5293/* Detect how end-of-line of a text of length SRC_BYTES pointed by
5294 SOURCE is encoded. If CATEGORY is one of
5295 coding_category_utf_16_XXXX, assume that CR and LF are encoded by
5296 two-byte, else they are encoded by one-byte.
5297
5298 Return one of EOL_SEEN_XXX. */
4ed46869 5299
bc4bc72a 5300#define MAX_EOL_CHECK_COUNT 3
d46c5b12
KH		 5301
5302static int
89528eb3 5303detect_eol (source, src_bytes, category)
f6cbaf43 5304 const unsigned char *source;
df7492f9 5305 EMACS_INT src_bytes;
89528eb3 5306 enum coding_category category;
4ed46869 5307{
f6cbaf43 5308 const unsigned char *src = source, *src_end = src + src_bytes;
4ed46869 5309 unsigned char c;
df7492f9
KH		 5310 int total = 0;
5311 int eol_seen = EOL_SEEN_NONE;
4ed46869 5312
89528eb3 5313 if ((1 << category) & CATEGORY_MASK_UTF_16)
4ed46869 5314 {
df7492f9 5315 int msb, lsb;
fa42c37f 5316
89528eb3
KH		 5317 msb = category == (coding_category_utf_16_le
5318 | coding_category_utf_16_le_nosig);
df7492f9 5319 lsb = 1 - msb;
fa42c37f 5320
df7492f9 5321 while (src + 1 < src_end)
fa42c37f 5322 {
df7492f9
KH		 5323 c = src[lsb];
5324 if (src[msb] == 0 && (c == '\n' || c == '\r'))
fa42c37f 5325 {
df7492f9
KH		 5326 int this_eol;
5327
5328 if (c == '\n')
5329 this_eol = EOL_SEEN_LF;
5330 else if (src + 3 >= src_end
5331 || src[msb + 2] != 0
5332 || src[lsb + 2] != '\n')
5333 this_eol = EOL_SEEN_CR;
fa42c37f 5334 else
8f924df7 5335 this_eol = EOL_SEEN_CRLF;
df7492f9
KH		 5336
5337 if (eol_seen == EOL_SEEN_NONE)
5338 /* This is the first end-of-line. */
5339 eol_seen = this_eol;
5340 else if (eol_seen != this_eol)
fa42c37f 5341 {
df7492f9
KH		 5342 /* The found type is different from what found before. */
5343 eol_seen = EOL_SEEN_LF;
5344 break;
fa42c37f 5345 }
df7492f9
KH		 5346 if (++total == MAX_EOL_CHECK_COUNT)
5347 break;
fa42c37f 5348 }
df7492f9 5349 src += 2;
fa42c37f 5350 }
bcf26d6a 5351 }
d46c5b12 5352 else
c4825358 5353 {
df7492f9 5354 while (src < src_end)
27901516 5355 {
df7492f9
KH		 5356 c = *src++;
5357 if (c == '\n' || c == '\r')
5358 {
5359 int this_eol;
d46c5b12 5360
df7492f9
KH		 5361 if (c == '\n')
5362 this_eol = EOL_SEEN_LF;
5363 else if (src >= src_end || *src != '\n')
5364 this_eol = EOL_SEEN_CR;
5365 else
5366 this_eol = EOL_SEEN_CRLF, src++;
d46c5b12 5367
df7492f9
KH		 5368 if (eol_seen == EOL_SEEN_NONE)
5369 /* This is the first end-of-line. */
5370 eol_seen = this_eol;
5371 else if (eol_seen != this_eol)
5372 {
5373 /* The found type is different from what found before. */
5374 eol_seen = EOL_SEEN_LF;
5375 break;
5376 }
5377 if (++total == MAX_EOL_CHECK_COUNT)
5378 break;
5379 }
5380 }
73be902c 5381 }
df7492f9 5382 return eol_seen;
73be902c
KH		 5383}
5384
df7492f9 5385
24a73b0a 5386static Lisp_Object
df7492f9
KH		 5387adjust_coding_eol_type (coding, eol_seen)
5388 struct coding_system *coding;
5389 int eol_seen;
73be902c 5390{
0be8721c 5391 Lisp_Object eol_type;
8f924df7 5392
df7492f9
KH		 5393 eol_type = CODING_ID_EOL_TYPE (coding->id);
5394 if (eol_seen & EOL_SEEN_LF)
24a73b0a
KH		 5395 {
5396 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 0));
5397 eol_type = Qunix;
5398 }
6f197c07 5399 else if (eol_seen & EOL_SEEN_CRLF)
24a73b0a
KH		 5400 {
5401 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 1));
5402 eol_type = Qdos;
5403 }
6f197c07 5404 else if (eol_seen & EOL_SEEN_CR)
24a73b0a
KH		 5405 {
5406 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 2));
5407 eol_type = Qmac;
5408 }
5409 return eol_type;
d46c5b12 5410}
4ed46869 5411
df7492f9
KH		 5412/* Detect how a text specified in CODING is encoded. If a coding
5413 system is detected, update fields of CODING by the detected coding
5414 system. */
0a28aafb 5415
df7492f9
KH		 5416void
5417detect_coding (coding)
d46c5b12 5418 struct coding_system *coding;
d46c5b12 5419{
8f924df7 5420 const unsigned char *src, *src_end;
d46c5b12 5421
df7492f9
KH		 5422 coding->consumed = coding->consumed_char = 0;
5423 coding->produced = coding->produced_char = 0;
5424 coding_set_source (coding);
1c3478b0 5425
df7492f9 5426 src_end = coding->source + coding->src_bytes;
1c3478b0 5427
df7492f9
KH		 5428 /* If we have not yet decided the text encoding type, detect it
5429 now. */
5430 if (EQ (CODING_ATTR_TYPE (CODING_ID_ATTRS (coding->id)), Qundecided))
b73bfc1c 5431 {
df7492f9 5432 int c, i;
6cb21a4f 5433 struct coding_detection_info detect_info;
df7492f9 5434
6cb21a4f 5435 detect_info.checked = detect_info.found = detect_info.rejected = 0;
24a73b0a 5436 for (i = 0, src = coding->source; src < src_end; i++, src++)
d46c5b12 5437 {
df7492f9 5438 c = *src;
6cb21a4f 5439 if (c & 0x80)
df7492f9 5440 break;
6cb21a4f
KH		 5441 if (c < 0x20
5442 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
5443 && ! inhibit_iso_escape_detection
5444 && ! detect_info.checked)
5445 {
5446 coding->head_ascii = src - (coding->source + coding->consumed);
5447 if (detect_coding_iso_2022 (coding, &detect_info))
5448 {
5449 /* We have scanned the whole data. */
5450 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
5451 /* We didn't find an 8-bit code. */
5452 src = src_end;
5453 break;
5454 }
5455 }
d46c5b12 5456 }
df7492f9
KH		 5457 coding->head_ascii = src - (coding->source + coding->consumed);
5458
3aef54f3 5459 if (coding->head_ascii < coding->src_bytes
6cb21a4f 5460 || detect_info.found)
d46c5b12 5461 {
ff0dacd7
KH		 5462 enum coding_category category;
5463 struct coding_system *this;
df7492f9 5464
6cb21a4f
KH		 5465 if (coding->head_ascii == coding->src_bytes)
5466 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
5467 for (i = 0; i < coding_category_raw_text; i++)
5468 {
5469 category = coding_priorities[i];
5470 this = coding_categories + category;
5471 if (detect_info.found & (1 << category))
24a73b0a 5472 break;
6cb21a4f
KH		 5473 }
5474 else
5475 for (i = 0; i < coding_category_raw_text; i++)
5476 {
5477 category = coding_priorities[i];
5478 this = coding_categories + category;
5479 if (this->id < 0)
5480 {
5481 /* No coding system of this category is defined. */
5482 detect_info.rejected |= (1 << category);
5483 }
5484 else if (category >= coding_category_raw_text)
5485 continue;
5486 else if (detect_info.checked & (1 << category))
5487 {
5488 if (detect_info.found & (1 << category))
5489 break;
5490 }
5491 else if ((*(this->detector)) (coding, &detect_info)
5492 && detect_info.found & (1 << category))
5493 {
5494 if (category == coding_category_utf_16_auto)
5495 {
5496 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
5497 category = coding_category_utf_16_le;
5498 else
5499 category = coding_category_utf_16_be;
5500 }
5501 break;
5502 }
5503 }
5504
ff0dacd7
KH		 5505 if (i < coding_category_raw_text)
5506 setup_coding_system (CODING_ID_NAME (this->id), coding);
5507 else if (detect_info.rejected == CATEGORY_MASK_ANY)
df7492f9 5508 setup_coding_system (Qraw_text, coding);
ff0dacd7 5509 else if (detect_info.rejected)
df7492f9 5510 for (i = 0; i < coding_category_raw_text; i++)
ff0dacd7
KH		 5511 if (! (detect_info.rejected & (1 << coding_priorities[i])))
5512 {
5513 this = coding_categories + coding_priorities[i];
5514 setup_coding_system (CODING_ID_NAME (this->id), coding);
5515 break;
5516 }
d46c5b12 5517 }
b73bfc1c 5518 }
24a73b0a
KH		 5519 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
5520 == coding_category_utf_16_auto)
b49a1807
KH		 5521 {
5522 Lisp_Object coding_systems;
5523 struct coding_detection_info detect_info;
5524
5525 coding_systems
5526 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_16_bom);
5527 detect_info.found = detect_info.rejected = 0;
5528 if (CONSP (coding_systems)
24a73b0a 5529 && detect_coding_utf_16 (coding, &detect_info))
b49a1807
KH		 5530 {
5531 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
5532 setup_coding_system (XCAR (coding_systems), coding);
24a73b0a 5533 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
b49a1807
KH		 5534 setup_coding_system (XCDR (coding_systems), coding);
5535 }
5536 }
4ed46869 5537}
4ed46869 5538
d46c5b12 5539
aaaf0b1e 5540static void
df7492f9 5541decode_eol (coding)
aaaf0b1e 5542 struct coding_system *coding;
aaaf0b1e 5543{
24a73b0a
KH		 5544 Lisp_Object eol_type;
5545 unsigned char *p, *pbeg, *pend;
5546
5547 eol_type = CODING_ID_EOL_TYPE (coding->id);
5548 if (EQ (eol_type, Qunix))
5549 return;
5550
5551 if (NILP (coding->dst_object))
5552 pbeg = coding->destination;
5553 else
5554 pbeg = BYTE_POS_ADDR (coding->dst_pos_byte);
5555 pend = pbeg + coding->produced;
5556
5557 if (VECTORP (eol_type))
aaaf0b1e 5558 {
df7492f9 5559 int eol_seen = EOL_SEEN_NONE;
4ed46869 5560
24a73b0a 5561 for (p = pbeg; p < pend; p++)
aaaf0b1e 5562 {
df7492f9
KH		 5563 if (*p == '\n')
5564 eol_seen |= EOL_SEEN_LF;
5565 else if (*p == '\r')
aaaf0b1e 5566 {
df7492f9 5567 if (p + 1 < pend && *(p + 1) == '\n')
aaaf0b1e 5568 {
df7492f9
KH		 5569 eol_seen |= EOL_SEEN_CRLF;
5570 p++;
aaaf0b1e 5571 }
aaaf0b1e 5572 else
df7492f9 5573 eol_seen |= EOL_SEEN_CR;
aaaf0b1e 5574 }
aaaf0b1e 5575 }
24a73b0a
KH		 5576 if (eol_seen != EOL_SEEN_NONE
5577 && eol_seen != EOL_SEEN_LF
5578 && eol_seen != EOL_SEEN_CRLF
5579 && eol_seen != EOL_SEEN_CR)
5580 eol_seen = EOL_SEEN_LF;
df7492f9 5581 if (eol_seen != EOL_SEEN_NONE)
24a73b0a 5582 eol_type = adjust_coding_eol_type (coding, eol_seen);
aaaf0b1e 5583 }
d46c5b12 5584
24a73b0a 5585 if (EQ (eol_type, Qmac))
27901516 5586 {
24a73b0a 5587 for (p = pbeg; p < pend; p++)
df7492f9
KH		 5588 if (*p == '\r')
5589 *p = '\n';
4ed46869 5590 }
24a73b0a 5591 else if (EQ (eol_type, Qdos))
df7492f9 5592 {
24a73b0a 5593 int n = 0;
b73bfc1c 5594
24a73b0a
KH		 5595 if (NILP (coding->dst_object))
5596 {
5597 for (p = pend - 2; p >= pbeg; p--)
5598 if (*p == '\r')
5599 {
5600 safe_bcopy ((char *) (p + 1), (char *) p, pend-- - p - 1);
5601 n++;
5602 }
5603 }
5604 else
5605 {
5606 for (p = pend - 2; p >= pbeg; p--)
5607 if (*p == '\r')
5608 {
5609 int pos_byte = coding->dst_pos_byte + (p - pbeg);
5610 int pos = BYTE_TO_CHAR (pos_byte);
5611
5612 del_range_2 (pos, pos_byte, pos + 1, pos_byte + 1, 0);
5613 n++;
5614 }
5615 }
5616 coding->produced -= n;
5617 coding->produced_char -= n;
aaaf0b1e 5618 }
4ed46869
KH		 5619}
5620
7d64c6ad 5621
a6f87d34
KH		 5622/* Return a translation table (or list of them) from coding system
5623 attribute vector ATTRS for encoding (ENCODEP is nonzero) or
5624 decoding (ENCODEP is zero). */
7d64c6ad 5625
e6a54062 5626static Lisp_Object
09ee6fdd
KH		 5627get_translation_table (attrs, encodep, max_lookup)
5628 Lisp_Object attrs;
5629 int encodep, *max_lookup;
7d64c6ad
KH		 5630{
5631 Lisp_Object standard, translation_table;
09ee6fdd 5632 Lisp_Object val;
7d64c6ad
KH		 5633
5634 if (encodep)
5635 translation_table = CODING_ATTR_ENCODE_TBL (attrs),
5636 standard = Vstandard_translation_table_for_encode;
5637 else
5638 translation_table = CODING_ATTR_DECODE_TBL (attrs),
5639 standard = Vstandard_translation_table_for_decode;
7d64c6ad 5640 if (NILP (translation_table))
09ee6fdd
KH		 5641 translation_table = standard;
5642 else
a6f87d34 5643 {
09ee6fdd
KH		 5644 if (SYMBOLP (translation_table))
5645 translation_table = Fget (translation_table, Qtranslation_table);
5646 else if (CONSP (translation_table))
5647 {
5648 translation_table = Fcopy_sequence (translation_table);
5649 for (val = translation_table; CONSP (val); val = XCDR (val))
5650 if (SYMBOLP (XCAR (val)))
5651 XSETCAR (val, Fget (XCAR (val), Qtranslation_table));
5652 }
5653 if (CHAR_TABLE_P (standard))
5654 {
5655 if (CONSP (translation_table))
5656 translation_table = nconc2 (translation_table,
5657 Fcons (standard, Qnil));
5658 else
5659 translation_table = Fcons (translation_table,
5660 Fcons (standard, Qnil));
5661 }
a6f87d34 5662 }
2170c8f0
KH		 5663
5664 if (max_lookup)
09ee6fdd 5665 {
2170c8f0
KH		 5666 *max_lookup = 1;
5667 if (CHAR_TABLE_P (translation_table)
5668 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (translation_table)) > 1)
5669 {
5670 val = XCHAR_TABLE (translation_table)->extras[1];
5671 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
5672 *max_lookup = XFASTINT (val);
5673 }
5674 else if (CONSP (translation_table))
5675 {
5676 Lisp_Object tail, val;
09ee6fdd 5677
2170c8f0
KH		 5678 for (tail = translation_table; CONSP (tail); tail = XCDR (tail))
5679 if (CHAR_TABLE_P (XCAR (tail))
5680 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (XCAR (tail))) > 1)
5681 {
5682 val = XCHAR_TABLE (XCAR (tail))->extras[1];
5683 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
5684 *max_lookup = XFASTINT (val);
5685 }
5686 }
a6f87d34 5687 }
7d64c6ad
KH		 5688 return translation_table;
5689}
5690
09ee6fdd
KH		 5691#define LOOKUP_TRANSLATION_TABLE(table, c, trans) \
5692 do { \
5693 trans = Qnil; \
5694 if (CHAR_TABLE_P (table)) \
5695 { \
5696 trans = CHAR_TABLE_REF (table, c); \
5697 if (CHARACTERP (trans)) \
5698 c = XFASTINT (trans), trans = Qnil; \
5699 } \
5700 else if (CONSP (table)) \
5701 { \
5702 Lisp_Object tail; \
5703 \
5704 for (tail = table; CONSP (tail); tail = XCDR (tail)) \
5705 if (CHAR_TABLE_P (XCAR (tail))) \
5706 { \
5707 trans = CHAR_TABLE_REF (XCAR (tail), c); \
5708 if (CHARACTERP (trans)) \
5709 c = XFASTINT (trans), trans = Qnil; \
5710 else if (! NILP (trans)) \
5711 break; \
5712 } \
5713 } \
e6a54062
KH		 5714 } while (0)
5715
7d64c6ad 5716
69a80ea3
KH		 5717static Lisp_Object
5718get_translation (val, buf, buf_end, last_block, from_nchars, to_nchars)
5719 Lisp_Object val;
5720 int *buf, *buf_end;
5721 int last_block;
5722 int *from_nchars, *to_nchars;
5723{
433f7f87
KH		 5724 /* VAL is TO or (([FROM-CHAR ...] . TO) ...) where TO is TO-CHAR or
5725 [TO-CHAR ...]. */
69a80ea3
KH		 5726 if (CONSP (val))
5727 {
433f7f87 5728 Lisp_Object from, tail;
69a80ea3
KH		 5729 int i, len;
5730
433f7f87 5731 for (tail = val; CONSP (tail); tail = XCDR (tail))
69a80ea3 5732 {
433f7f87
KH		 5733 val = XCAR (tail);
5734 from = XCAR (val);
5735 len = ASIZE (from);
5736 for (i = 0; i < len; i++)
5737 {
5738 if (buf + i == buf_end)
5739 {
5740 if (! last_block)
5741 return Qt;
5742 break;
5743 }
5744 if (XINT (AREF (from, i)) != buf[i])
5745 break;
5746 }
5747 if (i == len)
5748 {
5749 val = XCDR (val);
5750 *from_nchars = len;
5751 break;
5752 }
69a80ea3 5753 }
433f7f87
KH		 5754 if (! CONSP (tail))
5755 return Qnil;
69a80ea3
KH		 5756 }
5757 if (VECTORP (val))
5758 *buf = XINT (AREF (val, 0)), *to_nchars = ASIZE (val);
5759 else
5760 *buf = XINT (val);
5761 return val;
5762}
5763
5764
d46c5b12 5765static int
69a80ea3 5766produce_chars (coding, translation_table, last_block)
df7492f9 5767 struct coding_system *coding;
69a80ea3
KH		 5768 Lisp_Object translation_table;
5769 int last_block;
4ed46869 5770{
df7492f9
KH		 5771 unsigned char *dst = coding->destination + coding->produced;
5772 unsigned char *dst_end = coding->destination + coding->dst_bytes;
5773 int produced;
5774 int produced_chars = 0;
69a80ea3 5775 int carryover = 0;
4ed46869 5776
df7492f9 5777 if (! coding->chars_at_source)
4ed46869 5778 {
df7492f9 5779 /* Characters are in coding->charbuf. */
fba4576f
AS		 5780 int *buf = coding->charbuf;
5781 int *buf_end = buf + coding->charbuf_used;
4ed46869 5782
df7492f9
KH		 5783 if (BUFFERP (coding->src_object)
5784 && EQ (coding->src_object, coding->dst_object))
8f924df7 5785 dst_end = ((unsigned char *) coding->source) + coding->consumed;
4ed46869 5786
df7492f9 5787 while (buf < buf_end)
4ed46869 5788 {
69a80ea3 5789 int c = *buf, i;
bc4bc72a 5790
df7492f9
KH		 5791 if (c >= 0)
5792 {
69a80ea3
KH		 5793 int from_nchars = 1, to_nchars = 1;
5794 Lisp_Object trans = Qnil;
5795
09ee6fdd 5796 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 5797 if (! NILP (trans))
69a80ea3
KH		 5798 {
5799 trans = get_translation (trans, buf, buf_end, last_block,
5800 &from_nchars, &to_nchars);
5801 if (EQ (trans, Qt))
5802 break;
5803 c = *buf;
5804 }
5805
5806 if (dst + MAX_MULTIBYTE_LENGTH * to_nchars > dst_end)
5807 {
5808 dst = alloc_destination (coding,
5809 buf_end - buf
5810 + MAX_MULTIBYTE_LENGTH * to_nchars,
5811 dst);
5812 dst_end = coding->destination + coding->dst_bytes;
5813 }
5814
433f7f87 5815 for (i = 0; i < to_nchars; i++)
69a80ea3 5816 {
433f7f87
KH		 5817 if (i > 0)
5818 c = XINT (AREF (trans, i));
69a80ea3
KH		 5819 if (coding->dst_multibyte
5820 || ! CHAR_BYTE8_P (c))
5821 CHAR_STRING_ADVANCE (c, dst);
5822 else
5823 *dst++ = CHAR_TO_BYTE8 (c);
5824 }
5825 produced_chars += to_nchars;
5826 *buf++ = to_nchars;
5827 while (--from_nchars > 0)
5828 *buf++ = 0;
d46c5b12 5829 }
df7492f9 5830 else
69a80ea3
KH		 5831 /* This is an annotation datum. (-C) is the length. */
5832 buf += -c;
4ed46869 5833 }
69a80ea3 5834 carryover = buf_end - buf;
4ed46869 5835 }
fa42c37f 5836 else
fa42c37f 5837 {
8f924df7
KH		 5838 const unsigned char *src = coding->source;
5839 const unsigned char *src_end = src + coding->src_bytes;
df7492f9 5840 Lisp_Object eol_type;
fa42c37f 5841
df7492f9 5842 eol_type = CODING_ID_EOL_TYPE (coding->id);
4ed46869 5843
df7492f9 5844 if (coding->src_multibyte != coding->dst_multibyte)
fa42c37f 5845 {
df7492f9 5846 if (coding->src_multibyte)
fa42c37f 5847 {
71c81426 5848 int multibytep = 1;
df7492f9 5849 int consumed_chars;
d46c5b12 5850
df7492f9
KH		 5851 while (1)
5852 {
8f924df7 5853 const unsigned char *src_base = src;
df7492f9 5854 int c;
b73bfc1c 5855
df7492f9
KH		 5856 ONE_MORE_BYTE (c);
5857 if (c == '\r')
5858 {
5859 if (EQ (eol_type, Qdos))
5860 {
98725083
KH		 5861 if (src == src_end)
5862 {
065e3595
KH		 5863 record_conversion_result
5864 (coding, CODING_RESULT_INSUFFICIENT_SRC);
98725083
KH		 5865 goto no_more_source;
5866 }
5867 if (*src == '\n')
df7492f9
KH		 5868 c = *src++;
5869 }
5870 else if (EQ (eol_type, Qmac))
5871 c = '\n';
5872 }
5873 if (dst == dst_end)
5874 {
2c78b7e1 5875 coding->consumed = src - coding->source;
b73bfc1c 5876
2c78b7e1 5877 if (EQ (coding->src_object, coding->dst_object))
8f924df7 5878 dst_end = (unsigned char *) src;
2c78b7e1
KH		 5879 if (dst == dst_end)
5880 {
5881 dst = alloc_destination (coding, src_end - src + 1,
5882 dst);
5883 dst_end = coding->destination + coding->dst_bytes;
5884 coding_set_source (coding);
5885 src = coding->source + coding->consumed;
5886 src_end = coding->source + coding->src_bytes;
5887 }
df7492f9
KH		 5888 }
5889 *dst++ = c;
5890 produced_chars++;
5891 }
5892 no_more_source:
5893 ;
fa42c37f
KH		 5894 }
5895 else
df7492f9
KH		 5896 while (src < src_end)
5897 {
71c81426 5898 int multibytep = 1;
df7492f9 5899 int c = *src++;
b73bfc1c 5900
df7492f9
KH		 5901 if (c == '\r')
5902 {
5903 if (EQ (eol_type, Qdos))
5904 {
5905 if (src < src_end
5906 && *src == '\n')
5907 c = *src++;
5908 }
5909 else if (EQ (eol_type, Qmac))
5910 c = '\n';
5911 }
5912 if (dst >= dst_end - 1)
5913 {
2c78b7e1 5914 coding->consumed = src - coding->source;
df7492f9 5915
2c78b7e1 5916 if (EQ (coding->src_object, coding->dst_object))
8f924df7 5917 dst_end = (unsigned char *) src;
2c78b7e1
KH		 5918 if (dst >= dst_end - 1)
5919 {
5920 dst = alloc_destination (coding, src_end - src + 2,
5921 dst);
5922 dst_end = coding->destination + coding->dst_bytes;
5923 coding_set_source (coding);
5924 src = coding->source + coding->consumed;
5925 src_end = coding->source + coding->src_bytes;
5926 }
df7492f9
KH		 5927 }
5928 EMIT_ONE_BYTE (c);
5929 }
d46c5b12 5930 }
df7492f9
KH		 5931 else
5932 {
5933 if (!EQ (coding->src_object, coding->dst_object))
fa42c37f 5934 {
df7492f9 5935 int require = coding->src_bytes - coding->dst_bytes;
4ed46869 5936
df7492f9 5937 if (require > 0)
fa42c37f 5938 {
df7492f9
KH		 5939 EMACS_INT offset = src - coding->source;
5940
5941 dst = alloc_destination (coding, require, dst);
5942 coding_set_source (coding);
5943 src = coding->source + offset;
5944 src_end = coding->source + coding->src_bytes;
fa42c37f
KH		 5945 }
5946 }
df7492f9
KH		 5947 produced_chars = coding->src_chars;
5948 while (src < src_end)
fa42c37f 5949 {
df7492f9
KH		 5950 int c = *src++;
5951
5952 if (c == '\r')
5953 {
5954 if (EQ (eol_type, Qdos))
5955 {
5956 if (src < src_end
5957 && *src == '\n')
5958 c = *src++;
5959 produced_chars--;
5960 }
5961 else if (EQ (eol_type, Qmac))
5962 c = '\n';
5963 }
5964 *dst++ = c;
fa42c37f
KH		 5965 }
5966 }
2c78b7e1
KH		 5967 coding->consumed = coding->src_bytes;
5968 coding->consumed_char = coding->src_chars;
fa42c37f
KH		 5969 }
5970
df7492f9
KH		 5971 produced = dst - (coding->destination + coding->produced);
5972 if (BUFFERP (coding->dst_object))
5973 insert_from_gap (produced_chars, produced);
5974 coding->produced += produced;
5975 coding->produced_char += produced_chars;
69a80ea3 5976 return carryover;
fa42c37f
KH		 5977}
5978
ff0dacd7
KH		 5979/* Compose text in CODING->object according to the annotation data at
5980 CHARBUF. CHARBUF is an array:
5981 [ -LENGTH ANNOTATION_MASK FROM TO METHOD COMP_LEN [ COMPONENTS... ] ]
df7492f9 5982 */
4ed46869 5983
df7492f9 5984static INLINE void
69a80ea3 5985produce_composition (coding, charbuf, pos)
4ed46869 5986 struct coding_system *coding;
df7492f9 5987 int *charbuf;
69a80ea3 5988 EMACS_INT pos;
4ed46869 5989{
df7492f9 5990 int len;
69a80ea3 5991 EMACS_INT to;
df7492f9 5992 enum composition_method method;
df7492f9 5993 Lisp_Object components;
fa42c37f 5994
df7492f9 5995 len = -charbuf[0];
69a80ea3 5996 to = pos + charbuf[2];
9ffd559c
KH		 5997 if (to <= pos)
5998 return;
69a80ea3 5999 method = (enum composition_method) (charbuf[3]);
d46c5b12 6000
df7492f9
KH		 6001 if (method == COMPOSITION_RELATIVE)
6002 components = Qnil;
9ffd559c
KH		 6003 else if (method >= COMPOSITION_WITH_RULE
6004 && method <= COMPOSITION_WITH_RULE_ALTCHARS)
d46c5b12 6005 {
df7492f9
KH		 6006 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
6007 int i;
b73bfc1c 6008
69a80ea3
KH		 6009 len -= 4;
6010 charbuf += 4;
df7492f9 6011 for (i = 0; i < len; i++)
9ffd559c
KH		 6012 {
6013 args[i] = make_number (charbuf[i]);
6014 if (args[i] < 0)
6015 return;
6016 }
df7492f9
KH		 6017 components = (method == COMPOSITION_WITH_ALTCHARS
6018 ? Fstring (len, args) : Fvector (len, args));
d46c5b12 6019 }
9ffd559c
KH		 6020 else
6021 return;
69a80ea3 6022 compose_text (pos, to, components, Qnil, coding->dst_object);
d46c5b12
KH		 6023}
6024
d46c5b12 6025
ff0dacd7
KH		 6026/* Put `charset' property on text in CODING->object according to
6027 the annotation data at CHARBUF. CHARBUF is an array:
69a80ea3 6028 [ -LENGTH ANNOTATION_MASK NCHARS CHARSET-ID ]
ff0dacd7 6029 */
d46c5b12 6030
ff0dacd7 6031static INLINE void
69a80ea3 6032produce_charset (coding, charbuf, pos)
d46c5b12 6033 struct coding_system *coding;
ff0dacd7 6034 int *charbuf;
69a80ea3 6035 EMACS_INT pos;
d46c5b12 6036{
69a80ea3
KH		 6037 EMACS_INT from = pos - charbuf[2];
6038 struct charset *charset = CHARSET_FROM_ID (charbuf[3]);
b73bfc1c 6039
69a80ea3 6040 Fput_text_property (make_number (from), make_number (pos),
ff0dacd7
KH		 6041 Qcharset, CHARSET_NAME (charset),
6042 coding->dst_object);
d46c5b12
KH		 6043}
6044
d46c5b12 6045
df7492f9
KH		 6046#define CHARBUF_SIZE 0x4000
6047
6048#define ALLOC_CONVERSION_WORK_AREA(coding) \
6049 do { \
6050 int size = CHARBUF_SIZE;; \
6051 \
6052 coding->charbuf = NULL; \
6053 while (size > 1024) \
6054 { \
6055 coding->charbuf = (int *) alloca (sizeof (int) * size); \
6056 if (coding->charbuf) \
6057 break; \
6058 size >>= 1; \
6059 } \
6060 if (! coding->charbuf) \
6061 { \
065e3595 6062 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_MEM); \
df7492f9
KH		 6063 return coding->result; \
6064 } \
6065 coding->charbuf_size = size; \
6066 } while (0)
4ed46869 6067
d46c5b12
KH		 6068
6069static void
69a80ea3 6070produce_annotation (coding, pos)
d46c5b12 6071 struct coding_system *coding;
69a80ea3 6072 EMACS_INT pos;
d46c5b12 6073{
df7492f9
KH		 6074 int *charbuf = coding->charbuf;
6075 int *charbuf_end = charbuf + coding->charbuf_used;
d46c5b12 6076
ff0dacd7
KH		 6077 if (NILP (coding->dst_object))
6078 return;
d46c5b12 6079
df7492f9 6080 while (charbuf < charbuf_end)
a84f1519 6081 {
df7492f9 6082 if (*charbuf >= 0)
69a80ea3 6083 pos += *charbuf++;
d46c5b12 6084 else
d46c5b12 6085 {
df7492f9 6086 int len = -*charbuf;
ff0dacd7 6087 switch (charbuf[1])
df7492f9
KH		 6088 {
6089 case CODING_ANNOTATE_COMPOSITION_MASK:
69a80ea3 6090 produce_composition (coding, charbuf, pos);
df7492f9 6091 break;
ff0dacd7 6092 case CODING_ANNOTATE_CHARSET_MASK:
69a80ea3 6093 produce_charset (coding, charbuf, pos);
ff0dacd7 6094 break;
df7492f9
KH		 6095 default:
6096 abort ();
6097 }
6098 charbuf += len;
d46c5b12 6099 }
a84f1519 6100 }
d46c5b12
KH		 6101}
6102
df7492f9
KH		 6103/* Decode the data at CODING->src_object into CODING->dst_object.
6104 CODING->src_object is a buffer, a string, or nil.
6105 CODING->dst_object is a buffer.
d46c5b12 6106
df7492f9
KH		 6107 If CODING->src_object is a buffer, it must be the current buffer.
6108 In this case, if CODING->src_pos is positive, it is a position of
6109 the source text in the buffer, otherwise, the source text is in the
6110 gap area of the buffer, and CODING->src_pos specifies the offset of
6111 the text from GPT (which must be the same as PT). If this is the
6112 same buffer as CODING->dst_object, CODING->src_pos must be
6113 negative.
d46c5b12 6114
df7492f9
KH		 6115 If CODING->src_object is a string, CODING->src_pos in an index to
6116 that string.
d46c5b12 6117
df7492f9
KH		 6118 If CODING->src_object is nil, CODING->source must already point to
6119 the non-relocatable memory area. In this case, CODING->src_pos is
6120 an offset from CODING->source.
73be902c 6121
df7492f9
KH		 6122 The decoded data is inserted at the current point of the buffer
6123 CODING->dst_object.
6124*/
d46c5b12 6125
df7492f9
KH		 6126static int
6127decode_coding (coding)
d46c5b12 6128 struct coding_system *coding;
d46c5b12 6129{
df7492f9 6130 Lisp_Object attrs;
24a73b0a 6131 Lisp_Object undo_list;
7d64c6ad 6132 Lisp_Object translation_table;
69a80ea3
KH		 6133 int carryover;
6134 int i;
d46c5b12 6135
df7492f9
KH		 6136 if (BUFFERP (coding->src_object)
6137 && coding->src_pos > 0
6138 && coding->src_pos < GPT
6139 && coding->src_pos + coding->src_chars > GPT)
6140 move_gap_both (coding->src_pos, coding->src_pos_byte);
d46c5b12 6141
24a73b0a 6142 undo_list = Qt;
df7492f9 6143 if (BUFFERP (coding->dst_object))
1c3478b0 6144 {
df7492f9
KH		 6145 if (current_buffer != XBUFFER (coding->dst_object))
6146 set_buffer_internal (XBUFFER (coding->dst_object));
6147 if (GPT != PT)
6148 move_gap_both (PT, PT_BYTE);
24a73b0a
KH		 6149 undo_list = current_buffer->undo_list;
6150 current_buffer->undo_list = Qt;
1c3478b0
KH		 6151 }
6152
df7492f9
KH		 6153 coding->consumed = coding->consumed_char = 0;
6154 coding->produced = coding->produced_char = 0;
6155 coding->chars_at_source = 0;
065e3595 6156 record_conversion_result (coding, CODING_RESULT_SUCCESS);
df7492f9 6157 coding->errors = 0;
1c3478b0 6158
df7492f9
KH		 6159 ALLOC_CONVERSION_WORK_AREA (coding);
6160
6161 attrs = CODING_ID_ATTRS (coding->id);
2170c8f0 6162 translation_table = get_translation_table (attrs, 0, NULL);
df7492f9 6163
69a80ea3 6164 carryover = 0;
df7492f9 6165 do
b73bfc1c 6166 {
69a80ea3
KH		 6167 EMACS_INT pos = coding->dst_pos + coding->produced_char;
6168
df7492f9
KH		 6169 coding_set_source (coding);
6170 coding->annotated = 0;
69a80ea3 6171 coding->charbuf_used = carryover;
df7492f9 6172 (*(coding->decoder)) (coding);
df7492f9 6173 coding_set_destination (coding);
69a80ea3 6174 carryover = produce_chars (coding, translation_table, 0);
df7492f9 6175 if (coding->annotated)
69a80ea3
KH		 6176 produce_annotation (coding, pos);
6177 for (i = 0; i < carryover; i++)
6178 coding->charbuf[i]
6179 = coding->charbuf[coding->charbuf_used - carryover + i];
d46c5b12 6180 }
df7492f9
KH		 6181 while (coding->consumed < coding->src_bytes
6182 && ! coding->result);
d46c5b12 6183
69a80ea3
KH		 6184 if (carryover > 0)
6185 {
6186 coding_set_destination (coding);
6187 coding->charbuf_used = carryover;
6188 produce_chars (coding, translation_table, 1);
6189 }
6190
df7492f9
KH		 6191 coding->carryover_bytes = 0;
6192 if (coding->consumed < coding->src_bytes)
d46c5b12 6193 {
df7492f9 6194 int nbytes = coding->src_bytes - coding->consumed;
8f924df7 6195 const unsigned char *src;
df7492f9
KH		 6196
6197 coding_set_source (coding);
6198 coding_set_destination (coding);
6199 src = coding->source + coding->consumed;
6200
6201 if (coding->mode & CODING_MODE_LAST_BLOCK)
1c3478b0 6202 {
df7492f9
KH		 6203 /* Flush out unprocessed data as binary chars. We are sure
6204 that the number of data is less than the size of
6205 coding->charbuf. */
065e3595 6206 coding->charbuf_used = 0;
df7492f9 6207 while (nbytes-- > 0)
1c3478b0 6208 {
df7492f9 6209 int c = *src++;
98725083 6210
1c91457d
KH		 6211 if (c & 0x80)
6212 c = BYTE8_TO_CHAR (c);
6213 coding->charbuf[coding->charbuf_used++] = c;
1c3478b0 6214 }
f6cbaf43 6215 produce_chars (coding, Qnil, 1);
d46c5b12 6216 }
d46c5b12 6217 else
df7492f9
KH		 6218 {
6219 /* Record unprocessed bytes in coding->carryover. We are
6220 sure that the number of data is less than the size of
6221 coding->carryover. */
6222 unsigned char *p = coding->carryover;
6223
6224 coding->carryover_bytes = nbytes;
6225 while (nbytes-- > 0)
6226 *p++ = *src++;
1c3478b0 6227 }
df7492f9 6228 coding->consumed = coding->src_bytes;
b73bfc1c 6229 }
69f76525 6230
24a73b0a
KH		 6231 if (BUFFERP (coding->dst_object))
6232 {
6233 current_buffer->undo_list = undo_list;
6234 record_insert (coding->dst_pos, coding->produced_char);
6235 }
6236 if (! EQ (CODING_ID_EOL_TYPE (coding->id), Qunix))
6237 decode_eol (coding);
73be902c 6238 return coding->result;
4ed46869
KH		 6239}
6240
aaaf0b1e 6241
e1c23804 6242/* Extract an annotation datum from a composition starting at POS and
ff0dacd7
KH		 6243 ending before LIMIT of CODING->src_object (buffer or string), store
6244 the data in BUF, set *STOP to a starting position of the next
6245 composition (if any) or to LIMIT, and return the address of the
6246 next element of BUF.
6247
6248 If such an annotation is not found, set *STOP to a starting
6249 position of a composition after POS (if any) or to LIMIT, and
6250 return BUF. */
6251
6252static INLINE int *
6253handle_composition_annotation (pos, limit, coding, buf, stop)
6254 EMACS_INT pos, limit;
aaaf0b1e 6255 struct coding_system *coding;
ff0dacd7
KH		 6256 int *buf;
6257 EMACS_INT *stop;
aaaf0b1e 6258{
ff0dacd7
KH		 6259 EMACS_INT start, end;
6260 Lisp_Object prop;
aaaf0b1e 6261
ff0dacd7
KH		 6262 if (! find_composition (pos, limit, &start, &end, &prop, coding->src_object)
6263 || end > limit)
6264 *stop = limit;
6265 else if (start > pos)
6266 *stop = start;
6267 else
aaaf0b1e 6268 {
ff0dacd7 6269 if (start == pos)
aaaf0b1e 6270 {
ff0dacd7
KH		 6271 /* We found a composition. Store the corresponding
6272 annotation data in BUF. */
6273 int *head = buf;
6274 enum composition_method method = COMPOSITION_METHOD (prop);
6275 int nchars = COMPOSITION_LENGTH (prop);
6276
69a80ea3 6277 ADD_COMPOSITION_DATA (buf, nchars, method);
ff0dacd7 6278 if (method != COMPOSITION_RELATIVE)
aaaf0b1e 6279 {
ff0dacd7
KH		 6280 Lisp_Object components;
6281 int len, i, i_byte;
6282
6283 components = COMPOSITION_COMPONENTS (prop);
6284 if (VECTORP (components))
aaaf0b1e 6285 {
ff0dacd7
KH		 6286 len = XVECTOR (components)->size;
6287 for (i = 0; i < len; i++)
6288 *buf++ = XINT (AREF (components, i));
aaaf0b1e 6289 }
ff0dacd7 6290 else if (STRINGP (components))
aaaf0b1e 6291 {
8f924df7 6292 len = SCHARS (components);
ff0dacd7
KH		 6293 i = i_byte = 0;
6294 while (i < len)
6295 {
6296 FETCH_STRING_CHAR_ADVANCE (*buf, components, i, i_byte);
6297 buf++;
6298 }
6299 }
6300 else if (INTEGERP (components))
6301 {
6302 len = 1;
6303 *buf++ = XINT (components);
6304 }
6305 else if (CONSP (components))
6306 {
6307 for (len = 0; CONSP (components);
6308 len++, components = XCDR (components))
6309 *buf++ = XINT (XCAR (components));
aaaf0b1e 6310 }
aaaf0b1e 6311 else
ff0dacd7
KH		 6312 abort ();
6313 *head -= len;
aaaf0b1e 6314 }
aaaf0b1e 6315 }
ff0dacd7
KH		 6316
6317 if (find_composition (end, limit, &start, &end, &prop,
6318 coding->src_object)
6319 && end <= limit)
6320 *stop = start;
6321 else
6322 *stop = limit;
aaaf0b1e 6323 }
ff0dacd7
KH		 6324 return buf;
6325}
6326
6327
e1c23804 6328/* Extract an annotation datum from a text property `charset' at POS of
ff0dacd7
KH		 6329 CODING->src_object (buffer of string), store the data in BUF, set
6330 *STOP to the position where the value of `charset' property changes
6331 (limiting by LIMIT), and return the address of the next element of
6332 BUF.
6333
6334 If the property value is nil, set *STOP to the position where the
6335 property value is non-nil (limiting by LIMIT), and return BUF. */
6336
6337static INLINE int *
6338handle_charset_annotation (pos, limit, coding, buf, stop)
6339 EMACS_INT pos, limit;
6340 struct coding_system *coding;
6341 int *buf;
6342 EMACS_INT *stop;
6343{
6344 Lisp_Object val, next;
6345 int id;
6346
6347 val = Fget_text_property (make_number (pos), Qcharset, coding->src_object);
6348 if (! NILP (val) && CHARSETP (val))
6349 id = XINT (CHARSET_SYMBOL_ID (val));
6350 else
6351 id = -1;
69a80ea3 6352 ADD_CHARSET_DATA (buf, 0, id);
ff0dacd7
KH		 6353 next = Fnext_single_property_change (make_number (pos), Qcharset,
6354 coding->src_object,
6355 make_number (limit));
6356 *stop = XINT (next);
6357 return buf;
6358}
6359
6360
df7492f9 6361static void
09ee6fdd 6362consume_chars (coding, translation_table, max_lookup)
df7492f9 6363 struct coding_system *coding;
433f7f87 6364 Lisp_Object translation_table;
09ee6fdd 6365 int max_lookup;
df7492f9
KH		 6366{
6367 int *buf = coding->charbuf;
ff0dacd7 6368 int *buf_end = coding->charbuf + coding->charbuf_size;
7c78e542 6369 const unsigned char *src = coding->source + coding->consumed;
4776e638 6370 const unsigned char *src_end = coding->source + coding->src_bytes;
ff0dacd7
KH		 6371 EMACS_INT pos = coding->src_pos + coding->consumed_char;
6372 EMACS_INT end_pos = coding->src_pos + coding->src_chars;
df7492f9
KH		 6373 int multibytep = coding->src_multibyte;
6374 Lisp_Object eol_type;
6375 int c;
ff0dacd7 6376 EMACS_INT stop, stop_composition, stop_charset;
09ee6fdd 6377 int *lookup_buf = NULL;
433f7f87
KH		 6378
6379 if (! NILP (translation_table))
09ee6fdd 6380 lookup_buf = alloca (sizeof (int) * max_lookup);
88993dfd 6381
df7492f9
KH		 6382 eol_type = CODING_ID_EOL_TYPE (coding->id);
6383 if (VECTORP (eol_type))
6384 eol_type = Qunix;
88993dfd 6385
df7492f9
KH		 6386 /* Note: composition handling is not yet implemented. */
6387 coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
ec6d2bb8 6388
0b5670c9
KH		 6389 if (NILP (coding->src_object))
6390 stop = stop_composition = stop_charset = end_pos;
ff0dacd7 6391 else
0b5670c9
KH		 6392 {
6393 if (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK)
6394 stop = stop_composition = pos;
6395 else
6396 stop = stop_composition = end_pos;
6397 if (coding->common_flags & CODING_ANNOTATE_CHARSET_MASK)
6398 stop = stop_charset = pos;
6399 else
6400 stop_charset = end_pos;
6401 }
ec6d2bb8 6402
24a73b0a 6403 /* Compensate for CRLF and conversion. */
ff0dacd7 6404 buf_end -= 1 + MAX_ANNOTATION_LENGTH;
df7492f9 6405 while (buf < buf_end)
aaaf0b1e 6406 {
433f7f87
KH		 6407 Lisp_Object trans;
6408
df7492f9 6409 if (pos == stop)
ec6d2bb8 6410 {
df7492f9
KH		 6411 if (pos == end_pos)
6412 break;
ff0dacd7
KH		 6413 if (pos == stop_composition)
6414 buf = handle_composition_annotation (pos, end_pos, coding,
6415 buf, &stop_composition);
6416 if (pos == stop_charset)
6417 buf = handle_charset_annotation (pos, end_pos, coding,
6418 buf, &stop_charset);
6419 stop = (stop_composition < stop_charset
6420 ? stop_composition : stop_charset);
df7492f9
KH		 6421 }
6422
6423 if (! multibytep)
4776e638 6424 {
d3e4cb56 6425 EMACS_INT bytes;
aaaf0b1e 6426
ea29edf2
KH		 6427 if (coding->encoder == encode_coding_raw_text)
6428 c = *src++, pos++;
6429 else if ((bytes = MULTIBYTE_LENGTH (src, src_end)) > 0)
4776e638
KH		 6430 c = STRING_CHAR_ADVANCE (src), pos += bytes;
6431 else
f03caae0 6432 c = BYTE8_TO_CHAR (*src), src++, pos++;
4776e638 6433 }
df7492f9 6434 else
4776e638 6435 c = STRING_CHAR_ADVANCE (src), pos++;
df7492f9
KH		 6436 if ((c == '\r') && (coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
6437 c = '\n';
6438 if (! EQ (eol_type, Qunix))
aaaf0b1e 6439 {
df7492f9 6440 if (c == '\n')
aaaf0b1e 6441 {
df7492f9
KH		 6442 if (EQ (eol_type, Qdos))
6443 *buf++ = '\r';
6444 else
6445 c = '\r';
aaaf0b1e
KH		 6446 }
6447 }
433f7f87 6448
e6a54062 6449 trans = Qnil;
09ee6fdd 6450 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
e6a54062 6451 if (NILP (trans))
433f7f87
KH		 6452 *buf++ = c;
6453 else
6454 {
6455 int from_nchars = 1, to_nchars = 1;
6456 int *lookup_buf_end;
6457 const unsigned char *p = src;
6458 int i;
6459
6460 lookup_buf[0] = c;
6461 for (i = 1; i < max_lookup && p < src_end; i++)
6462 lookup_buf[i] = STRING_CHAR_ADVANCE (p);
6463 lookup_buf_end = lookup_buf + i;
6464 trans = get_translation (trans, lookup_buf, lookup_buf_end, 1,
6465 &from_nchars, &to_nchars);
6466 if (EQ (trans, Qt)
6467 || buf + to_nchars > buf_end)
6468 break;
6469 *buf++ = *lookup_buf;
6470 for (i = 1; i < to_nchars; i++)
6471 *buf++ = XINT (AREF (trans, i));
6472 for (i = 1; i < from_nchars; i++, pos++)
6473 src += MULTIBYTE_LENGTH_NO_CHECK (src);
6474 }
aaaf0b1e 6475 }
ec6d2bb8 6476
df7492f9
KH		 6477 coding->consumed = src - coding->source;
6478 coding->consumed_char = pos - coding->src_pos;
6479 coding->charbuf_used = buf - coding->charbuf;
6480 coding->chars_at_source = 0;
aaaf0b1e
KH		 6481}
6482
4ed46869 6483
df7492f9
KH		 6484/* Encode the text at CODING->src_object into CODING->dst_object.
6485 CODING->src_object is a buffer or a string.
6486 CODING->dst_object is a buffer or nil.
6487
6488 If CODING->src_object is a buffer, it must be the current buffer.
6489 In this case, if CODING->src_pos is positive, it is a position of
6490 the source text in the buffer, otherwise. the source text is in the
6491 gap area of the buffer, and coding->src_pos specifies the offset of
6492 the text from GPT (which must be the same as PT). If this is the
6493 same buffer as CODING->dst_object, CODING->src_pos must be
6494 negative and CODING should not have `pre-write-conversion'.
6495
6496 If CODING->src_object is a string, CODING should not have
6497 `pre-write-conversion'.
6498
6499 If CODING->dst_object is a buffer, the encoded data is inserted at
6500 the current point of that buffer.
6501
6502 If CODING->dst_object is nil, the encoded data is placed at the
6503 memory area specified by CODING->destination. */
6504
6505static int
6506encode_coding (coding)
4ed46869 6507 struct coding_system *coding;
4ed46869 6508{
df7492f9 6509 Lisp_Object attrs;
7d64c6ad 6510 Lisp_Object translation_table;
09ee6fdd 6511 int max_lookup;
9861e777 6512
df7492f9 6513 attrs = CODING_ID_ATTRS (coding->id);
ea29edf2
KH		 6514 if (coding->encoder == encode_coding_raw_text)
6515 translation_table = Qnil, max_lookup = 0;
6516 else
6517 translation_table = get_translation_table (attrs, 1, &max_lookup);
4ed46869 6518
df7492f9 6519 if (BUFFERP (coding->dst_object))
8844fa83 6520 {
df7492f9
KH		 6521 set_buffer_internal (XBUFFER (coding->dst_object));
6522 coding->dst_multibyte
6523 = ! NILP (current_buffer->enable_multibyte_characters);
8844fa83 6524 }
4ed46869 6525
b73bfc1c 6526 coding->consumed = coding->consumed_char = 0;
df7492f9 6527 coding->produced = coding->produced_char = 0;
065e3595 6528 record_conversion_result (coding, CODING_RESULT_SUCCESS);
b73bfc1c 6529 coding->errors = 0;
b73bfc1c 6530
df7492f9 6531 ALLOC_CONVERSION_WORK_AREA (coding);
4ed46869 6532
df7492f9
KH		 6533 do {
6534 coding_set_source (coding);
09ee6fdd 6535 consume_chars (coding, translation_table, max_lookup);
df7492f9
KH		 6536 coding_set_destination (coding);
6537 (*(coding->encoder)) (coding);
6538 } while (coding->consumed_char < coding->src_chars);
6539
6540 if (BUFFERP (coding->dst_object))
6541 insert_from_gap (coding->produced_char, coding->produced);
6542
6543 return (coding->result);
ec6d2bb8
KH		 6544}
6545
fb88bf2d 6546
24a73b0a
KH		 6547/* Name (or base name) of work buffer for code conversion. */
6548static Lisp_Object Vcode_conversion_workbuf_name;
d46c5b12 6549
24a73b0a
KH		 6550/* A working buffer used by the top level conversion. Once it is
6551 created, it is never destroyed. It has the name
6552 Vcode_conversion_workbuf_name. The other working buffers are
6553 destroyed after the use is finished, and their names are modified
6554 versions of Vcode_conversion_workbuf_name. */
6555static Lisp_Object Vcode_conversion_reused_workbuf;
b73bfc1c 6556
24a73b0a
KH		 6557/* 1 iff Vcode_conversion_reused_workbuf is already in use. */
6558static int reused_workbuf_in_use;
4ed46869 6559
24a73b0a
KH		 6560
6561/* Return a working buffer of code convesion. MULTIBYTE specifies the
6562 multibyteness of returning buffer. */
b73bfc1c 6563
f6cbaf43 6564static Lisp_Object
24a73b0a 6565make_conversion_work_buffer (multibyte)
f6cbaf43 6566 int multibyte;
df7492f9 6567{
24a73b0a
KH		 6568 Lisp_Object name, workbuf;
6569 struct buffer *current;
4ed46869 6570
24a73b0a 6571 if (reused_workbuf_in_use++)
065e3595
KH		 6572 {
6573 name = Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name, Qnil);
6574 workbuf = Fget_buffer_create (name);
6575 }
df7492f9 6576 else
065e3595
KH		 6577 {
6578 name = Vcode_conversion_workbuf_name;
6579 workbuf = Fget_buffer_create (name);
6580 if (NILP (Vcode_conversion_reused_workbuf))
6581 Vcode_conversion_reused_workbuf = workbuf;
6582 }
24a73b0a
KH		 6583 current = current_buffer;
6584 set_buffer_internal (XBUFFER (workbuf));
6585 Ferase_buffer ();
df7492f9 6586 current_buffer->undo_list = Qt;
24a73b0a 6587 current_buffer->enable_multibyte_characters = multibyte ? Qt : Qnil;
df7492f9 6588 set_buffer_internal (current);
24a73b0a 6589 return workbuf;
df7492f9 6590}
d46c5b12 6591
24a73b0a 6592
4776e638 6593static Lisp_Object
24a73b0a
KH		 6594code_conversion_restore (arg)
6595 Lisp_Object arg;
4776e638 6596{
24a73b0a
KH		 6597 Lisp_Object current, workbuf;
6598
6599 current = XCAR (arg);
6600 workbuf = XCDR (arg);
6601 if (! NILP (workbuf))
6602 {
6603 if (EQ (workbuf, Vcode_conversion_reused_workbuf))
6604 reused_workbuf_in_use = 0;
6605 else if (! NILP (Fbuffer_live_p (workbuf)))
6606 Fkill_buffer (workbuf);
6607 }
6608 set_buffer_internal (XBUFFER (current));
4776e638
KH		 6609 return Qnil;
6610}
b73bfc1c 6611
24a73b0a
KH		 6612Lisp_Object
6613code_conversion_save (with_work_buf, multibyte)
4776e638 6614 int with_work_buf, multibyte;
df7492f9 6615{
24a73b0a 6616 Lisp_Object workbuf = Qnil;
b73bfc1c 6617
4776e638 6618 if (with_work_buf)
24a73b0a
KH		 6619 workbuf = make_conversion_work_buffer (multibyte);
6620 record_unwind_protect (code_conversion_restore,
6621 Fcons (Fcurrent_buffer (), workbuf));
4776e638 6622 return workbuf;
df7492f9 6623}
d46c5b12 6624
df7492f9
KH		 6625int
6626decode_coding_gap (coding, chars, bytes)
6627 struct coding_system *coding;
6628 EMACS_INT chars, bytes;
6629{
6630 int count = specpdl_ptr - specpdl;
5e5c78be 6631 Lisp_Object attrs;
fb88bf2d 6632
24a73b0a 6633 code_conversion_save (0, 0);
ec6d2bb8 6634
24a73b0a 6635 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 6636 coding->src_chars = chars;
6637 coding->src_bytes = bytes;
6638 coding->src_pos = -chars;
6639 coding->src_pos_byte = -bytes;
6640 coding->src_multibyte = chars < bytes;
24a73b0a 6641 coding->dst_object = coding->src_object;
df7492f9
KH		 6642 coding->dst_pos = PT;
6643 coding->dst_pos_byte = PT_BYTE;
71c81426 6644 coding->dst_multibyte = ! NILP (current_buffer->enable_multibyte_characters);
98725083 6645 coding->mode |= CODING_MODE_LAST_BLOCK;
4ed46869 6646
df7492f9
KH		 6647 if (CODING_REQUIRE_DETECTION (coding))
6648 detect_coding (coding);
8f924df7 6649
df7492f9 6650 decode_coding (coding);
d46c5b12 6651
5e5c78be
KH		 6652 attrs = CODING_ID_ATTRS (coding->id);
6653 if (! NILP (CODING_ATTR_POST_READ (attrs)))
b73bfc1c 6654 {
5e5c78be
KH		 6655 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
6656 Lisp_Object val;
6657
6658 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
5e5c78be
KH		 6659 val = call1 (CODING_ATTR_POST_READ (attrs),
6660 make_number (coding->produced_char));
5e5c78be
KH		 6661 CHECK_NATNUM (val);
6662 coding->produced_char += Z - prev_Z;
6663 coding->produced += Z_BYTE - prev_Z_BYTE;
b73bfc1c 6664 }
4ed46869 6665
df7492f9 6666 unbind_to (count, Qnil);
b73bfc1c
KH		 6667 return coding->result;
6668}
52d41803 6669
4ed46869 6670int
df7492f9 6671encode_coding_gap (coding, chars, bytes)
4ed46869 6672 struct coding_system *coding;
df7492f9 6673 EMACS_INT chars, bytes;
4ed46869 6674{
df7492f9 6675 int count = specpdl_ptr - specpdl;
4ed46869 6676
24a73b0a 6677 code_conversion_save (0, 0);
4ed46869 6678
24a73b0a 6679 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 6680 coding->src_chars = chars;
6681 coding->src_bytes = bytes;
6682 coding->src_pos = -chars;
6683 coding->src_pos_byte = -bytes;
6684 coding->src_multibyte = chars < bytes;
6685 coding->dst_object = coding->src_object;
6686 coding->dst_pos = PT;
6687 coding->dst_pos_byte = PT_BYTE;
4ed46869 6688
df7492f9 6689 encode_coding (coding);
b73bfc1c 6690
df7492f9
KH		 6691 unbind_to (count, Qnil);
6692 return coding->result;
6693}
4ed46869 6694
d46c5b12 6695
df7492f9
KH		 6696/* Decode the text in the range FROM/FROM_BYTE and TO/TO_BYTE in
6697 SRC_OBJECT into DST_OBJECT by coding context CODING.
b73bfc1c 6698
df7492f9 6699 SRC_OBJECT is a buffer, a string, or Qnil.
b73bfc1c 6700
df7492f9
KH		 6701 If it is a buffer, the text is at point of the buffer. FROM and TO
6702 are positions in the buffer.
b73bfc1c 6703
df7492f9
KH		 6704 If it is a string, the text is at the beginning of the string.
6705 FROM and TO are indices to the string.
4ed46869 6706
df7492f9
KH		 6707 If it is nil, the text is at coding->source. FROM and TO are
6708 indices to coding->source.
bb10be8b 6709
df7492f9 6710 DST_OBJECT is a buffer, Qt, or Qnil.
4ed46869 6711
df7492f9
KH		 6712 If it is a buffer, the decoded text is inserted at point of the
6713 buffer. If the buffer is the same as SRC_OBJECT, the source text
6714 is deleted.
4ed46869 6715
df7492f9
KH		 6716 If it is Qt, a string is made from the decoded text, and
6717 set in CODING->dst_object.
d46c5b12 6718
df7492f9 6719 If it is Qnil, the decoded text is stored at CODING->destination.
2cb26057 6720 The caller must allocate CODING->dst_bytes bytes at
df7492f9
KH		 6721 CODING->destination by xmalloc. If the decoded text is longer than
6722 CODING->dst_bytes, CODING->destination is relocated by xrealloc.
6723 */
d46c5b12 6724
df7492f9
KH		 6725void
6726decode_coding_object (coding, src_object, from, from_byte, to, to_byte,
6727 dst_object)
d46c5b12 6728 struct coding_system *coding;
df7492f9
KH		 6729 Lisp_Object src_object;
6730 EMACS_INT from, from_byte, to, to_byte;
6731 Lisp_Object dst_object;
d46c5b12 6732{
df7492f9
KH		 6733 int count = specpdl_ptr - specpdl;
6734 unsigned char *destination;
6735 EMACS_INT dst_bytes;
6736 EMACS_INT chars = to - from;
6737 EMACS_INT bytes = to_byte - from_byte;
6738 Lisp_Object attrs;
4776e638
KH		 6739 Lisp_Object buffer;
6740 int saved_pt = -1, saved_pt_byte;
d46c5b12 6741
4776e638 6742 buffer = Fcurrent_buffer ();
93dec019 6743
df7492f9 6744 if (NILP (dst_object))
d46c5b12 6745 {
df7492f9
KH		 6746 destination = coding->destination;
6747 dst_bytes = coding->dst_bytes;
d46c5b12 6748 }
93dec019 6749
df7492f9
KH		 6750 coding->src_object = src_object;
6751 coding->src_chars = chars;
6752 coding->src_bytes = bytes;
6753 coding->src_multibyte = chars < bytes;
70ad9fc4 6754
df7492f9 6755 if (STRINGP (src_object))
d46c5b12 6756 {
df7492f9
KH		 6757 coding->src_pos = from;
6758 coding->src_pos_byte = from_byte;
d46c5b12 6759 }
df7492f9 6760 else if (BUFFERP (src_object))
88993dfd 6761 {
df7492f9
KH		 6762 set_buffer_internal (XBUFFER (src_object));
6763 if (from != GPT)
6764 move_gap_both (from, from_byte);
6765 if (EQ (src_object, dst_object))
fb88bf2d 6766 {
4776e638 6767 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9
KH		 6768 TEMP_SET_PT_BOTH (from, from_byte);
6769 del_range_both (from, from_byte, to, to_byte, 1);
6770 coding->src_pos = -chars;
6771 coding->src_pos_byte = -bytes;
fb88bf2d 6772 }
df7492f9 6773 else
fb88bf2d 6774 {
df7492f9
KH		 6775 coding->src_pos = from;
6776 coding->src_pos_byte = from_byte;
fb88bf2d 6777 }
88993dfd
KH		 6778 }
6779
df7492f9
KH		 6780 if (CODING_REQUIRE_DETECTION (coding))
6781 detect_coding (coding);
6782 attrs = CODING_ID_ATTRS (coding->id);
d46c5b12 6783
2cb26057
KH		 6784 if (EQ (dst_object, Qt)
6785 || (! NILP (CODING_ATTR_POST_READ (attrs))
6786 && NILP (dst_object)))
b73bfc1c 6787 {
24a73b0a 6788 coding->dst_object = code_conversion_save (1, 1);
df7492f9
KH		 6789 coding->dst_pos = BEG;
6790 coding->dst_pos_byte = BEG_BYTE;
6791 coding->dst_multibyte = 1;
b73bfc1c 6792 }
df7492f9 6793 else if (BUFFERP (dst_object))
d46c5b12 6794 {
24a73b0a 6795 code_conversion_save (0, 0);
df7492f9
KH		 6796 coding->dst_object = dst_object;
6797 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
6798 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
6799 coding->dst_multibyte
6800 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
d46c5b12
KH		 6801 }
6802 else
6803 {
24a73b0a 6804 code_conversion_save (0, 0);
df7492f9
KH		 6805 coding->dst_object = Qnil;
6806 coding->dst_multibyte = 1;
d46c5b12
KH		 6807 }
6808
df7492f9 6809 decode_coding (coding);
fa46990e 6810
df7492f9
KH		 6811 if (BUFFERP (coding->dst_object))
6812 set_buffer_internal (XBUFFER (coding->dst_object));
d46c5b12 6813
df7492f9 6814 if (! NILP (CODING_ATTR_POST_READ (attrs)))
d46c5b12 6815 {
df7492f9
KH		 6816 struct gcpro gcpro1, gcpro2;
6817 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
2b4f9037 6818 Lisp_Object val;
d46c5b12 6819
c0cc7f7f 6820 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
df7492f9 6821 GCPRO2 (coding->src_object, coding->dst_object);
d4850d67
KH		 6822 val = safe_call1 (CODING_ATTR_POST_READ (attrs),
6823 make_number (coding->produced_char));
df7492f9
KH		 6824 UNGCPRO;
6825 CHECK_NATNUM (val);
6826 coding->produced_char += Z - prev_Z;
6827 coding->produced += Z_BYTE - prev_Z_BYTE;
d46c5b12 6828 }
de79a6a5 6829
df7492f9 6830 if (EQ (dst_object, Qt))
ec6d2bb8 6831 {
df7492f9
KH		 6832 coding->dst_object = Fbuffer_string ();
6833 }
6834 else if (NILP (dst_object) && BUFFERP (coding->dst_object))
6835 {
6836 set_buffer_internal (XBUFFER (coding->dst_object));
6837 if (dst_bytes < coding->produced)
6838 {
6839 destination
6840 = (unsigned char *) xrealloc (destination, coding->produced);
6841 if (! destination)
6842 {
065e3595
KH		 6843 record_conversion_result (coding,
6844 CODING_RESULT_INSUFFICIENT_DST);
df7492f9
KH		 6845 unbind_to (count, Qnil);
6846 return;
6847 }
6848 if (BEGV < GPT && GPT < BEGV + coding->produced_char)
6849 move_gap_both (BEGV, BEGV_BYTE);
6850 bcopy (BEGV_ADDR, destination, coding->produced);
6851 coding->destination = destination;
d46c5b12 6852 }
ec6d2bb8 6853 }
b73bfc1c 6854
4776e638
KH		 6855 if (saved_pt >= 0)
6856 {
6857 /* This is the case of:
6858 (BUFFERP (src_object) && EQ (src_object, dst_object))
6859 As we have moved PT while replacing the original buffer
6860 contents, we must recover it now. */
6861 set_buffer_internal (XBUFFER (src_object));
6862 if (saved_pt < from)
6863 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
6864 else if (saved_pt < from + chars)
6865 TEMP_SET_PT_BOTH (from, from_byte);
6866 else if (! NILP (current_buffer->enable_multibyte_characters))
6867 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
6868 saved_pt_byte + (coding->produced - bytes));
6869 else
6870 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
6871 saved_pt_byte + (coding->produced - bytes));
d46c5b12 6872 }
4776e638 6873
065e3595 6874 unbind_to (count, coding->dst_object);
d46c5b12
KH		 6875}
6876
d46c5b12 6877
df7492f9
KH		 6878void
6879encode_coding_object (coding, src_object, from, from_byte, to, to_byte,
6880 dst_object)
d46c5b12 6881 struct coding_system *coding;
df7492f9
KH		 6882 Lisp_Object src_object;
6883 EMACS_INT from, from_byte, to, to_byte;
6884 Lisp_Object dst_object;
d46c5b12 6885{
b73bfc1c 6886 int count = specpdl_ptr - specpdl;
df7492f9
KH		 6887 EMACS_INT chars = to - from;
6888 EMACS_INT bytes = to_byte - from_byte;
6889 Lisp_Object attrs;
4776e638
KH		 6890 Lisp_Object buffer;
6891 int saved_pt = -1, saved_pt_byte;
c02d943b 6892 int kill_src_buffer = 0;
df7492f9 6893
4776e638 6894 buffer = Fcurrent_buffer ();
df7492f9
KH		 6895
6896 coding->src_object = src_object;
6897 coding->src_chars = chars;
6898 coding->src_bytes = bytes;
6899 coding->src_multibyte = chars < bytes;
6900
6901 attrs = CODING_ID_ATTRS (coding->id);
6902
6903 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
6bac5b12 6904 {
24a73b0a 6905 coding->src_object = code_conversion_save (1, coding->src_multibyte);
df7492f9
KH		 6906 set_buffer_internal (XBUFFER (coding->src_object));
6907 if (STRINGP (src_object))
6908 insert_from_string (src_object, from, from_byte, chars, bytes, 0);
6909 else if (BUFFERP (src_object))
6910 insert_from_buffer (XBUFFER (src_object), from, chars, 0);
6911 else
6912 insert_1_both (coding->source + from, chars, bytes, 0, 0, 0);
d46c5b12 6913
df7492f9
KH		 6914 if (EQ (src_object, dst_object))
6915 {
6916 set_buffer_internal (XBUFFER (src_object));
4776e638 6917 saved_pt = PT, saved_pt_byte = PT_BYTE;
df7492f9
KH		 6918 del_range_both (from, from_byte, to, to_byte, 1);
6919 set_buffer_internal (XBUFFER (coding->src_object));
6920 }
6921
d4850d67
KH		 6922 {
6923 Lisp_Object args[3];
6924
6925 args[0] = CODING_ATTR_PRE_WRITE (attrs);
6926 args[1] = make_number (BEG);
6927 args[2] = make_number (Z);
6928 safe_call (3, args);
6929 }
c02d943b
KH		 6930 if (XBUFFER (coding->src_object) != current_buffer)
6931 kill_src_buffer = 1;
ac87bbef 6932 coding->src_object = Fcurrent_buffer ();
df7492f9
KH		 6933 if (BEG != GPT)
6934 move_gap_both (BEG, BEG_BYTE);
6935 coding->src_chars = Z - BEG;
6936 coding->src_bytes = Z_BYTE - BEG_BYTE;
6937 coding->src_pos = BEG;
6938 coding->src_pos_byte = BEG_BYTE;
6939 coding->src_multibyte = Z < Z_BYTE;
6940 }
6941 else if (STRINGP (src_object))
d46c5b12 6942 {
24a73b0a 6943 code_conversion_save (0, 0);
df7492f9
KH		 6944 coding->src_pos = from;
6945 coding->src_pos_byte = from_byte;
b73bfc1c 6946 }
df7492f9 6947 else if (BUFFERP (src_object))
b73bfc1c 6948 {
24a73b0a 6949 code_conversion_save (0, 0);
df7492f9 6950 set_buffer_internal (XBUFFER (src_object));
df7492f9 6951 if (EQ (src_object, dst_object))
d46c5b12 6952 {
4776e638 6953 saved_pt = PT, saved_pt_byte = PT_BYTE;
ff0dacd7
KH		 6954 coding->src_object = del_range_1 (from, to, 1, 1);
6955 coding->src_pos = 0;
6956 coding->src_pos_byte = 0;
d46c5b12 6957 }
df7492f9 6958 else
d46c5b12 6959 {
ff0dacd7
KH		 6960 if (from < GPT && to >= GPT)
6961 move_gap_both (from, from_byte);
df7492f9
KH		 6962 coding->src_pos = from;
6963 coding->src_pos_byte = from_byte;
d46c5b12 6964 }
d46c5b12 6965 }
4776e638 6966 else
24a73b0a 6967 code_conversion_save (0, 0);
d46c5b12 6968
df7492f9 6969 if (BUFFERP (dst_object))
88993dfd 6970 {
df7492f9 6971 coding->dst_object = dst_object;
28f67a95
KH		 6972 if (EQ (src_object, dst_object))
6973 {
6974 coding->dst_pos = from;
6975 coding->dst_pos_byte = from_byte;
6976 }
6977 else
6978 {
6979 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
6980 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
6981 }
df7492f9
KH		 6982 coding->dst_multibyte
6983 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
88993dfd 6984 }
df7492f9 6985 else if (EQ (dst_object, Qt))
d46c5b12 6986 {
df7492f9 6987 coding->dst_object = Qnil;
df7492f9 6988 coding->dst_bytes = coding->src_chars;
ac87bbef
KH		 6989 if (coding->dst_bytes == 0)
6990 coding->dst_bytes = 1;
6991 coding->destination = (unsigned char *) xmalloc (coding->dst_bytes);
df7492f9 6992 coding->dst_multibyte = 0;
d46c5b12
KH		 6993 }
6994 else
6995 {
df7492f9
KH		 6996 coding->dst_object = Qnil;
6997 coding->dst_multibyte = 0;
d46c5b12
KH		 6998 }
6999
df7492f9 7000 encode_coding (coding);
d46c5b12 7001
df7492f9 7002 if (EQ (dst_object, Qt))
d46c5b12 7003 {
df7492f9
KH		 7004 if (BUFFERP (coding->dst_object))
7005 coding->dst_object = Fbuffer_string ();
7006 else
d46c5b12 7007 {
df7492f9
KH		 7008 coding->dst_object
7009 = make_unibyte_string ((char *) coding->destination,
7010 coding->produced);
7011 xfree (coding->destination);
d46c5b12 7012 }
4ed46869 7013 }
d46c5b12 7014
4776e638
KH		 7015 if (saved_pt >= 0)
7016 {
7017 /* This is the case of:
7018 (BUFFERP (src_object) && EQ (src_object, dst_object))
7019 As we have moved PT while replacing the original buffer
7020 contents, we must recover it now. */
7021 set_buffer_internal (XBUFFER (src_object));
7022 if (saved_pt < from)
7023 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7024 else if (saved_pt < from + chars)
7025 TEMP_SET_PT_BOTH (from, from_byte);
7026 else if (! NILP (current_buffer->enable_multibyte_characters))
7027 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7028 saved_pt_byte + (coding->produced - bytes));
d46c5b12 7029 else
4776e638
KH		 7030 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7031 saved_pt_byte + (coding->produced - bytes));
7032 }
7033
c02d943b
KH		 7034 if (kill_src_buffer)
7035 Fkill_buffer (coding->src_object);
df7492f9 7036 unbind_to (count, Qnil);
b73bfc1c
KH		 7037}
7038
df7492f9 7039
b73bfc1c 7040Lisp_Object
df7492f9 7041preferred_coding_system ()
b73bfc1c 7042{
df7492f9 7043 int id = coding_categories[coding_priorities[0]].id;
2391eaa4 7044
df7492f9 7045 return CODING_ID_NAME (id);
4ed46869
KH		 7046}
7047
7048\f
7049#ifdef emacs
1397dc18 7050/*** 8. Emacs Lisp library functions ***/
4ed46869 7051
4ed46869 7052DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0,
48b0f3ae 7053 doc: /* Return t if OBJECT is nil or a coding-system.
df7492f9 7054See the documentation of `define-coding-system' for information
48b0f3ae
PJ		 7055about coding-system objects. */)
7056 (obj)
4ed46869
KH		 7057 Lisp_Object obj;
7058{
df7492f9 7059 return ((NILP (obj) || CODING_SYSTEM_P (obj)) ? Qt : Qnil);
4ed46869
KH		 7060}
7061
9d991de8
RS		 7062DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system,
7063 Sread_non_nil_coding_system, 1, 1, 0,
48b0f3ae
PJ		 7064 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
7065 (prompt)
4ed46869
KH		 7066 Lisp_Object prompt;
7067{
e0e989f6 7068 Lisp_Object val;
9d991de8
RS		 7069 do
7070 {
4608c386
KH		 7071 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
7072 Qt, Qnil, Qcoding_system_history, Qnil, Qnil);
9d991de8 7073 }
8f924df7 7074 while (SCHARS (val) == 0);
e0e989f6 7075 return (Fintern (val, Qnil));
4ed46869
KH		 7076}
7077
9b787f3e 7078DEFUN ("read-coding-system", Fread_coding_system, Sread_coding_system, 1, 2, 0,
48b0f3ae
PJ		 7079 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT.
7080If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
7081 (prompt, default_coding_system)
9b787f3e 7082 Lisp_Object prompt, default_coding_system;
4ed46869 7083{
f44d27ce 7084 Lisp_Object val;
9b787f3e 7085 if (SYMBOLP (default_coding_system))
a3181084 7086 XSETSTRING (default_coding_system, XPNTR (SYMBOL_NAME (default_coding_system)));
4608c386 7087 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
9b787f3e
RS		 7088 Qt, Qnil, Qcoding_system_history,
7089 default_coding_system, Qnil);
8f924df7 7090 return (SCHARS (val) == 0 ? Qnil : Fintern (val, Qnil));
4ed46869
KH		 7091}
7092
7093DEFUN ("check-coding-system", Fcheck_coding_system, Scheck_coding_system,
7094 1, 1, 0,
48b0f3ae 7095 doc: /* Check validity of CODING-SYSTEM.
9ffd559c
KH		 7096If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
7097It is valid if it is nil or a symbol defined as a coding system by the
7098function `define-coding-system'. */)
df7492f9 7099 (coding_system)
4ed46869
KH		 7100 Lisp_Object coding_system;
7101{
b7826503 7102 CHECK_SYMBOL (coding_system);
4ed46869
KH		 7103 if (!NILP (Fcoding_system_p (coding_system)))
7104 return coding_system;
7105 while (1)
02ba4723 7106 Fsignal (Qcoding_system_error, Fcons (coding_system, Qnil));
4ed46869 7107}
df7492f9 7108
3a73fa5d 7109\f
89528eb3
KH		 7110/* Detect how the bytes at SRC of length SRC_BYTES are encoded. If
7111 HIGHEST is nonzero, return the coding system of the highest
7112 priority among the detected coding systems. Otherwize return a
7113 list of detected coding systems sorted by their priorities. If
7114 MULTIBYTEP is nonzero, it is assumed that the bytes are in correct
7115 multibyte form but contains only ASCII and eight-bit chars.
7116 Otherwise, the bytes are raw bytes.
7117
7118 CODING-SYSTEM controls the detection as below:
7119
7120 If it is nil, detect both text-format and eol-format. If the
7121 text-format part of CODING-SYSTEM is already specified
7122 (e.g. `iso-latin-1'), detect only eol-format. If the eol-format
7123 part of CODING-SYSTEM is already specified (e.g. `undecided-unix'),
7124 detect only text-format. */
7125
d46c5b12 7126Lisp_Object
24a73b0a
KH		 7127detect_coding_system (src, src_chars, src_bytes, highest, multibytep,
7128 coding_system)
8f924df7 7129 const unsigned char *src;
24a73b0a 7130 int src_chars, src_bytes, highest;
0a28aafb 7131 int multibytep;
df7492f9 7132 Lisp_Object coding_system;
4ed46869 7133{
8f924df7 7134 const unsigned char *src_end = src + src_bytes;
df7492f9
KH		 7135 Lisp_Object attrs, eol_type;
7136 Lisp_Object val;
7137 struct coding_system coding;
89528eb3 7138 int id;
ff0dacd7 7139 struct coding_detection_info detect_info;
24a73b0a 7140 enum coding_category base_category;
b73bfc1c 7141
df7492f9
KH		 7142 if (NILP (coding_system))
7143 coding_system = Qundecided;
7144 setup_coding_system (coding_system, &coding);
7145 attrs = CODING_ID_ATTRS (coding.id);
7146 eol_type = CODING_ID_EOL_TYPE (coding.id);
89528eb3 7147 coding_system = CODING_ATTR_BASE_NAME (attrs);
4ed46869 7148
df7492f9 7149 coding.source = src;
24a73b0a 7150 coding.src_chars = src_chars;
df7492f9
KH		 7151 coding.src_bytes = src_bytes;
7152 coding.src_multibyte = multibytep;
7153 coding.consumed = 0;
89528eb3 7154 coding.mode |= CODING_MODE_LAST_BLOCK;
d46c5b12 7155
ff0dacd7 7156 detect_info.checked = detect_info.found = detect_info.rejected = 0;
d46c5b12 7157
89528eb3 7158 /* At first, detect text-format if necessary. */
24a73b0a
KH		 7159 base_category = XINT (CODING_ATTR_CATEGORY (attrs));
7160 if (base_category == coding_category_undecided)
4ed46869 7161 {
ff0dacd7
KH		 7162 enum coding_category category;
7163 struct coding_system *this;
7164 int c, i;
88993dfd 7165
24a73b0a
KH		 7166 /* Skip all ASCII bytes except for a few ISO2022 controls. */
7167 for (i = 0; src < src_end; i++, src++)
4ed46869 7168 {
df7492f9 7169 c = *src;
6cb21a4f 7170 if (c & 0x80)
d46c5b12 7171 break;
6cb21a4f
KH		 7172 if (c < 0x20
7173 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
7174 && inhibit_iso_escape_detection)
7175 {
7176 coding.head_ascii = src - coding.source;
7177 if (detect_coding_iso_2022 (&coding, &detect_info))
7178 {
7179 /* We have scanned the whole data. */
7180 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
7181 /* We didn't find an 8-bit code. */
7182 src = src_end;
7183 break;
7184 }
7185 }
4ed46869 7186 }
df7492f9 7187 coding.head_ascii = src - coding.source;
88993dfd 7188
6cb21a4f
KH		 7189 if (src < src_end
7190 || detect_info.found)
7191 {
7192 if (src == src_end)
7193 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
7194 for (i = 0; i < coding_category_raw_text; i++)
ff0dacd7 7195 {
6cb21a4f
KH		 7196 category = coding_priorities[i];
7197 if (detect_info.found & (1 << category))
ff0dacd7
KH		 7198 break;
7199 }
6cb21a4f
KH		 7200 else
7201 for (i = 0; i < coding_category_raw_text; i++)
df7492f9 7202 {
6cb21a4f
KH		 7203 category = coding_priorities[i];
7204 this = coding_categories + category;
7205
7206 if (this->id < 0)
24a73b0a 7207 {
6cb21a4f
KH		 7208 /* No coding system of this category is defined. */
7209 detect_info.rejected |= (1 << category);
7210 }
7211 else if (category >= coding_category_raw_text)
7212 continue;
7213 else if (detect_info.checked & (1 << category))
7214 {
7215 if (highest
7216 && (detect_info.found & (1 << category)))
7217 break;
7218 }
7219 else
7220 {
7221 if ((*(this->detector)) (&coding, &detect_info)
7222 && highest
7223 && (detect_info.found & (1 << category)))
24a73b0a 7224 {
6cb21a4f
KH		 7225 if (category == coding_category_utf_16_auto)
7226 {
7227 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
7228 category = coding_category_utf_16_le;
7229 else
7230 category = coding_category_utf_16_be;
7231 }
7232 break;
24a73b0a 7233 }
24a73b0a 7234 }
df7492f9 7235 }
6cb21a4f 7236 }
ec6d2bb8 7237
ff0dacd7 7238 if (detect_info.rejected == CATEGORY_MASK_ANY)
ec6d2bb8 7239 {
ff0dacd7 7240 detect_info.found = CATEGORY_MASK_RAW_TEXT;
89528eb3
KH		 7241 id = coding_categories[coding_category_raw_text].id;
7242 val = Fcons (make_number (id), Qnil);
7243 }
ff0dacd7 7244 else if (! detect_info.rejected && ! detect_info.found)
89528eb3 7245 {
ff0dacd7 7246 detect_info.found = CATEGORY_MASK_ANY;
89528eb3
KH		 7247 id = coding_categories[coding_category_undecided].id;
7248 val = Fcons (make_number (id), Qnil);
7249 }
7250 else if (highest)
7251 {
ff0dacd7 7252 if (detect_info.found)
ec6d2bb8 7253 {
ff0dacd7
KH		 7254 detect_info.found = 1 << category;
7255 val = Fcons (make_number (this->id), Qnil);
7256 }
7257 else
7258 for (i = 0; i < coding_category_raw_text; i++)
7259 if (! (detect_info.rejected & (1 << coding_priorities[i])))
7260 {
7261 detect_info.found = 1 << coding_priorities[i];
7262 id = coding_categories[coding_priorities[i]].id;
7263 val = Fcons (make_number (id), Qnil);
7264 break;
7265 }
7266 }
89528eb3
KH		 7267 else
7268 {
ff0dacd7
KH		 7269 int mask = detect_info.rejected | detect_info.found;
7270 int found = 0;
89528eb3 7271 val = Qnil;
ec6d2bb8 7272
89528eb3 7273 for (i = coding_category_raw_text - 1; i >= 0; i--)
ff0dacd7
KH		 7274 {
7275 category = coding_priorities[i];
7276 if (! (mask & (1 << category)))
ec6d2bb8 7277 {
ff0dacd7
KH		 7278 found |= 1 << category;
7279 id = coding_categories[category].id;
7280 val = Fcons (make_number (id), val);
7281 }
7282 }
7283 for (i = coding_category_raw_text - 1; i >= 0; i--)
7284 {
7285 category = coding_priorities[i];
7286 if (detect_info.found & (1 << category))
7287 {
7288 id = coding_categories[category].id;
7289 val = Fcons (make_number (id), val);
ec6d2bb8 7290 }
ec6d2bb8 7291 }
ff0dacd7 7292 detect_info.found |= found;
ec6d2bb8 7293 }
ec6d2bb8 7294 }
24a73b0a
KH		 7295 else if (base_category == coding_category_utf_16_auto)
7296 {
7297 if (detect_coding_utf_16 (&coding, &detect_info))
7298 {
24a73b0a
KH		 7299 struct coding_system *this;
7300
7301 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
7302 this = coding_categories + coding_category_utf_16_le;
7303 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
7304 this = coding_categories + coding_category_utf_16_be;
7305 else if (detect_info.rejected & CATEGORY_MASK_UTF_16_LE_NOSIG)
7306 this = coding_categories + coding_category_utf_16_be_nosig;
7307 else
7308 this = coding_categories + coding_category_utf_16_le_nosig;
7309 val = Fcons (make_number (this->id), Qnil);
7310 }
7311 }
df7492f9
KH		 7312 else
7313 {
ff0dacd7 7314 detect_info.found = 1 << XINT (CODING_ATTR_CATEGORY (attrs));
89528eb3 7315 val = Fcons (make_number (coding.id), Qnil);
4ed46869 7316 }
df7492f9 7317
89528eb3 7318 /* Then, detect eol-format if necessary. */
df7492f9 7319 {
89528eb3 7320 int normal_eol = -1, utf_16_be_eol = -1, utf_16_le_eol;
df7492f9
KH		 7321 Lisp_Object tail;
7322
89528eb3
KH		 7323 if (VECTORP (eol_type))
7324 {
ff0dacd7 7325 if (detect_info.found & ~CATEGORY_MASK_UTF_16)
89528eb3
KH		 7326 normal_eol = detect_eol (coding.source, src_bytes,
7327 coding_category_raw_text);
ff0dacd7
KH		 7328 if (detect_info.found & (CATEGORY_MASK_UTF_16_BE
7329 | CATEGORY_MASK_UTF_16_BE_NOSIG))
89528eb3
KH		 7330 utf_16_be_eol = detect_eol (coding.source, src_bytes,
7331 coding_category_utf_16_be);
ff0dacd7
KH		 7332 if (detect_info.found & (CATEGORY_MASK_UTF_16_LE
7333 | CATEGORY_MASK_UTF_16_LE_NOSIG))
89528eb3
KH		 7334 utf_16_le_eol = detect_eol (coding.source, src_bytes,
7335 coding_category_utf_16_le);
7336 }
7337 else
7338 {
7339 if (EQ (eol_type, Qunix))
7340 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_LF;
7341 else if (EQ (eol_type, Qdos))
7342 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CRLF;
7343 else
7344 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CR;
7345 }
7346
df7492f9
KH		 7347 for (tail = val; CONSP (tail); tail = XCDR (tail))
7348 {
89528eb3 7349 enum coding_category category;
df7492f9 7350 int this_eol;
89528eb3
KH		 7351
7352 id = XINT (XCAR (tail));
7353 attrs = CODING_ID_ATTRS (id);
7354 category = XINT (CODING_ATTR_CATEGORY (attrs));
7355 eol_type = CODING_ID_EOL_TYPE (id);
df7492f9
KH		 7356 if (VECTORP (eol_type))
7357 {
89528eb3
KH		 7358 if (category == coding_category_utf_16_be
7359 || category == coding_category_utf_16_be_nosig)
7360 this_eol = utf_16_be_eol;
7361 else if (category == coding_category_utf_16_le
7362 || category == coding_category_utf_16_le_nosig)
7363 this_eol = utf_16_le_eol;
df7492f9 7364 else
89528eb3
KH		 7365 this_eol = normal_eol;
7366
df7492f9
KH		 7367 if (this_eol == EOL_SEEN_LF)
7368 XSETCAR (tail, AREF (eol_type, 0));
7369 else if (this_eol == EOL_SEEN_CRLF)
7370 XSETCAR (tail, AREF (eol_type, 1));
7371 else if (this_eol == EOL_SEEN_CR)
7372 XSETCAR (tail, AREF (eol_type, 2));
89528eb3
KH		 7373 else
7374 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9 7375 }
89528eb3
KH		 7376 else
7377 XSETCAR (tail, CODING_ID_NAME (id));
df7492f9
KH		 7378 }
7379 }
ec6d2bb8 7380
03699b14 7381 return (highest ? XCAR (val) : val);
ec6d2bb8
KH		 7382}
7383
ec6d2bb8 7384
d46c5b12
KH		 7385DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region,
7386 2, 3, 0,
48b0f3ae
PJ		 7387 doc: /* Detect coding system of the text in the region between START and END.
7388Return a list of possible coding systems ordered by priority.
ec6d2bb8 7389
48b0f3ae
PJ		 7390If only ASCII characters are found, it returns a list of single element
7391`undecided' or its subsidiary coding system according to a detected
7392end-of-line format.
ec6d2bb8 7393
48b0f3ae
PJ		 7394If optional argument HIGHEST is non-nil, return the coding system of
7395highest priority. */)
7396 (start, end, highest)
d46c5b12
KH		 7397 Lisp_Object start, end, highest;
7398{
7399 int from, to;
7400 int from_byte, to_byte;
ec6d2bb8 7401
b7826503
PJ		 7402 CHECK_NUMBER_COERCE_MARKER (start);
7403 CHECK_NUMBER_COERCE_MARKER (end);
ec6d2bb8 7404
d46c5b12
KH		 7405 validate_region (&start, &end);
7406 from = XINT (start), to = XINT (end);
7407 from_byte = CHAR_TO_BYTE (from);
7408 to_byte = CHAR_TO_BYTE (to);
ec6d2bb8 7409
d46c5b12
KH		 7410 if (from < GPT && to >= GPT)
7411 move_gap_both (to, to_byte);
c210f766 7412
d46c5b12 7413 return detect_coding_system (BYTE_POS_ADDR (from_byte),
24a73b0a 7414 to - from, to_byte - from_byte,
0a28aafb
KH		 7415 !NILP (highest),
7416 !NILP (current_buffer
df7492f9
KH		 7417 ->enable_multibyte_characters),
7418 Qnil);
ec6d2bb8
KH		 7419}
7420
d46c5b12
KH		 7421DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string,
7422 1, 2, 0,
48b0f3ae
PJ		 7423 doc: /* Detect coding system of the text in STRING.
7424Return a list of possible coding systems ordered by priority.
fb88bf2d 7425
48b0f3ae
PJ		 7426If only ASCII characters are found, it returns a list of single element
7427`undecided' or its subsidiary coding system according to a detected
7428end-of-line format.
d46c5b12 7429
48b0f3ae
PJ		 7430If optional argument HIGHEST is non-nil, return the coding system of
7431highest priority. */)
7432 (string, highest)
d46c5b12
KH		 7433 Lisp_Object string, highest;
7434{
b7826503 7435 CHECK_STRING (string);
b73bfc1c 7436
24a73b0a
KH		 7437 return detect_coding_system (SDATA (string),
7438 SCHARS (string), SBYTES (string),
8f924df7 7439 !NILP (highest), STRING_MULTIBYTE (string),
df7492f9 7440 Qnil);
4ed46869 7441}
4ed46869 7442
b73bfc1c 7443
df7492f9
KH		 7444static INLINE int
7445char_encodable_p (c, attrs)
7446 int c;
7447 Lisp_Object attrs;
05e6f5dc 7448{
df7492f9 7449 Lisp_Object tail;
df7492f9 7450 struct charset *charset;
7d64c6ad 7451 Lisp_Object translation_table;
d46c5b12 7452
7d64c6ad 7453 translation_table = CODING_ATTR_TRANS_TBL (attrs);
a6f87d34 7454 if (! NILP (translation_table))
7d64c6ad 7455 c = translate_char (translation_table, c);
df7492f9
KH		 7456 for (tail = CODING_ATTR_CHARSET_LIST (attrs);
7457 CONSP (tail); tail = XCDR (tail))
e133c8fa 7458 {
df7492f9
KH		 7459 charset = CHARSET_FROM_ID (XINT (XCAR (tail)));
7460 if (CHAR_CHARSET_P (c, charset))
7461 break;
e133c8fa 7462 }
df7492f9 7463 return (! NILP (tail));
05e6f5dc 7464}
83fa074f 7465
fb88bf2d 7466
df7492f9
KH		 7467/* Return a list of coding systems that safely encode the text between
7468 START and END. If EXCLUDE is non-nil, it is a list of coding
7469 systems not to check. The returned list doesn't contain any such
48468dac 7470 coding systems. In any case, if the text contains only ASCII or is
df7492f9 7471 unibyte, return t. */
e077cc80 7472
df7492f9
KH		 7473DEFUN ("find-coding-systems-region-internal",
7474 Ffind_coding_systems_region_internal,
7475 Sfind_coding_systems_region_internal, 2, 3, 0,
7476 doc: /* Internal use only. */)
7477 (start, end, exclude)
7478 Lisp_Object start, end, exclude;
7479{
7480 Lisp_Object coding_attrs_list, safe_codings;
7481 EMACS_INT start_byte, end_byte;
7c78e542 7482 const unsigned char *p, *pbeg, *pend;
df7492f9
KH		 7483 int c;
7484 Lisp_Object tail, elt;
d46c5b12 7485
df7492f9
KH		 7486 if (STRINGP (start))
7487 {
7488 if (!STRING_MULTIBYTE (start)
8f924df7 7489 || SCHARS (start) == SBYTES (start))
df7492f9
KH		 7490 return Qt;
7491 start_byte = 0;
8f924df7 7492 end_byte = SBYTES (start);
df7492f9
KH		 7493 }
7494 else
d46c5b12 7495 {
df7492f9
KH		 7496 CHECK_NUMBER_COERCE_MARKER (start);
7497 CHECK_NUMBER_COERCE_MARKER (end);
7498 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
7499 args_out_of_range (start, end);
7500 if (NILP (current_buffer->enable_multibyte_characters))
7501 return Qt;
7502 start_byte = CHAR_TO_BYTE (XINT (start));
7503 end_byte = CHAR_TO_BYTE (XINT (end));
7504 if (XINT (end) - XINT (start) == end_byte - start_byte)
7505 return Qt;
d46c5b12 7506
e1c23804 7507 if (XINT (start) < GPT && XINT (end) > GPT)
d46c5b12 7508 {
e1c23804
DL		 7509 if ((GPT - XINT (start)) < (XINT (end) - GPT))
7510 move_gap_both (XINT (start), start_byte);
df7492f9 7511 else
e1c23804 7512 move_gap_both (XINT (end), end_byte);
d46c5b12
KH		 7513 }
7514 }
7515
df7492f9
KH		 7516 coding_attrs_list = Qnil;
7517 for (tail = Vcoding_system_list; CONSP (tail); tail = XCDR (tail))
7518 if (NILP (exclude)
7519 || NILP (Fmemq (XCAR (tail), exclude)))
7520 {
7521 Lisp_Object attrs;
d46c5b12 7522
df7492f9
KH		 7523 attrs = AREF (CODING_SYSTEM_SPEC (XCAR (tail)), 0);
7524 if (EQ (XCAR (tail), CODING_ATTR_BASE_NAME (attrs))
7525 && ! EQ (CODING_ATTR_TYPE (attrs), Qundecided))
7d64c6ad
KH		 7526 {
7527 ASET (attrs, coding_attr_trans_tbl,
2170c8f0 7528 get_translation_table (attrs, 1, NULL));
7d64c6ad
KH		 7529 coding_attrs_list = Fcons (attrs, coding_attrs_list);
7530 }
df7492f9 7531 }
d46c5b12 7532
df7492f9 7533 if (STRINGP (start))
8f924df7 7534 p = pbeg = SDATA (start);
df7492f9
KH		 7535 else
7536 p = pbeg = BYTE_POS_ADDR (start_byte);
7537 pend = p + (end_byte - start_byte);
b843d1ae 7538
df7492f9
KH		 7539 while (p < pend && ASCII_BYTE_P (*p)) p++;
7540 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
d46c5b12 7541
05e6f5dc 7542 while (p < pend)
72d1a715 7543 {
df7492f9
KH		 7544 if (ASCII_BYTE_P (*p))
7545 p++;
72d1a715
RS		 7546 else
7547 {
df7492f9 7548 c = STRING_CHAR_ADVANCE (p);
12410ef1 7549
df7492f9
KH		 7550 charset_map_loaded = 0;
7551 for (tail = coding_attrs_list; CONSP (tail);)
7552 {
7553 elt = XCAR (tail);
7554 if (NILP (elt))
7555 tail = XCDR (tail);
7556 else if (char_encodable_p (c, elt))
7557 tail = XCDR (tail);
7558 else if (CONSP (XCDR (tail)))
7559 {
7560 XSETCAR (tail, XCAR (XCDR (tail)));
7561 XSETCDR (tail, XCDR (XCDR (tail)));
7562 }
7563 else
7564 {
7565 XSETCAR (tail, Qnil);
7566 tail = XCDR (tail);
7567 }
7568 }
7569 if (charset_map_loaded)
7570 {
7571 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
05e6f5dc 7572
df7492f9 7573 if (STRINGP (start))
8f924df7 7574 pbeg = SDATA (start);
df7492f9
KH		 7575 else
7576 pbeg = BYTE_POS_ADDR (start_byte);
7577 p = pbeg + p_offset;
7578 pend = pbeg + pend_offset;
7579 }
7580 }
ec6d2bb8 7581 }
fb88bf2d 7582
988b3759 7583 safe_codings = list2 (Qraw_text, Qno_conversion);
df7492f9
KH		 7584 for (tail = coding_attrs_list; CONSP (tail); tail = XCDR (tail))
7585 if (! NILP (XCAR (tail)))
7586 safe_codings = Fcons (CODING_ATTR_BASE_NAME (XCAR (tail)), safe_codings);
ec6d2bb8 7587
05e6f5dc
KH		 7588 return safe_codings;
7589}
4956c225 7590
d46c5b12 7591
8f924df7
KH		 7592DEFUN ("unencodable-char-position", Funencodable_char_position,
7593 Sunencodable_char_position, 3, 5, 0,
7594 doc: /*
7595Return position of first un-encodable character in a region.
7596START and END specfiy the region and CODING-SYSTEM specifies the
7597encoding to check. Return nil if CODING-SYSTEM does encode the region.
d46c5b12 7598
8f924df7
KH		 7599If optional 4th argument COUNT is non-nil, it specifies at most how
7600many un-encodable characters to search. In this case, the value is a
7601list of positions.
d46c5b12 7602
8f924df7
KH		 7603If optional 5th argument STRING is non-nil, it is a string to search
7604for un-encodable characters. In that case, START and END are indexes
7605to the string. */)
7606 (start, end, coding_system, count, string)
7607 Lisp_Object start, end, coding_system, count, string;
7608{
7609 int n;
7610 struct coding_system coding;
7d64c6ad 7611 Lisp_Object attrs, charset_list, translation_table;
8f924df7
KH		 7612 Lisp_Object positions;
7613 int from, to;
7614 const unsigned char *p, *stop, *pend;
7615 int ascii_compatible;
fb88bf2d 7616
8f924df7
KH		 7617 setup_coding_system (Fcheck_coding_system (coding_system), &coding);
7618 attrs = CODING_ID_ATTRS (coding.id);
7619 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
7620 return Qnil;
7621 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
7622 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2170c8f0 7623 translation_table = get_translation_table (attrs, 1, NULL);
fb88bf2d 7624
8f924df7
KH		 7625 if (NILP (string))
7626 {
7627 validate_region (&start, &end);
7628 from = XINT (start);
7629 to = XINT (end);
7630 if (NILP (current_buffer->enable_multibyte_characters)
7631 || (ascii_compatible
7632 && (to - from) == (CHAR_TO_BYTE (to) - (CHAR_TO_BYTE (from)))))
7633 return Qnil;
7634 p = CHAR_POS_ADDR (from);
7635 pend = CHAR_POS_ADDR (to);
7636 if (from < GPT && to >= GPT)
7637 stop = GPT_ADDR;
7638 else
7639 stop = pend;
7640 }
7641 else
7642 {
7643 CHECK_STRING (string);
7644 CHECK_NATNUM (start);
7645 CHECK_NATNUM (end);
7646 from = XINT (start);
7647 to = XINT (end);
7648 if (from > to
7649 || to > SCHARS (string))
7650 args_out_of_range_3 (string, start, end);
7651 if (! STRING_MULTIBYTE (string))
7652 return Qnil;
7653 p = SDATA (string) + string_char_to_byte (string, from);
7654 stop = pend = SDATA (string) + string_char_to_byte (string, to);
7655 if (ascii_compatible && (to - from) == (pend - p))
7656 return Qnil;
7657 }
f2558efd 7658
8f924df7
KH		 7659 if (NILP (count))
7660 n = 1;
7661 else
b73bfc1c 7662 {
8f924df7
KH		 7663 CHECK_NATNUM (count);
7664 n = XINT (count);
b73bfc1c
KH		 7665 }
7666
8f924df7
KH		 7667 positions = Qnil;
7668 while (1)
d46c5b12 7669 {
8f924df7 7670 int c;
ec6d2bb8 7671
8f924df7
KH		 7672 if (ascii_compatible)
7673 while (p < stop && ASCII_BYTE_P (*p))
7674 p++, from++;
7675 if (p >= stop)
0e79d667 7676 {
8f924df7
KH		 7677 if (p >= pend)
7678 break;
7679 stop = pend;
7680 p = GAP_END_ADDR;
0e79d667 7681 }
ec6d2bb8 7682
8f924df7
KH		 7683 c = STRING_CHAR_ADVANCE (p);
7684 if (! (ASCII_CHAR_P (c) && ascii_compatible)
7d64c6ad
KH		 7685 && ! char_charset (translate_char (translation_table, c),
7686 charset_list, NULL))
ec6d2bb8 7687 {
8f924df7
KH		 7688 positions = Fcons (make_number (from), positions);
7689 n--;
7690 if (n == 0)
7691 break;
ec6d2bb8
KH		 7692 }
7693
8f924df7
KH		 7694 from++;
7695 }
d46c5b12 7696
8f924df7
KH		 7697 return (NILP (count) ? Fcar (positions) : Fnreverse (positions));
7698}
d46c5b12 7699
d46c5b12 7700
df7492f9
KH		 7701DEFUN ("check-coding-systems-region", Fcheck_coding_systems_region,
7702 Scheck_coding_systems_region, 3, 3, 0,
7703 doc: /* Check if the region is encodable by coding systems.
d46c5b12 7704
df7492f9
KH		 7705START and END are buffer positions specifying the region.
7706CODING-SYSTEM-LIST is a list of coding systems to check.
d46c5b12 7707
df7492f9
KH		 7708The value is an alist ((CODING-SYSTEM POS0 POS1 ...) ...), where
7709CODING-SYSTEM is a member of CODING-SYSTEM-LIst and can't encode the
7710whole region, POS0, POS1, ... are buffer positions where non-encodable
7711characters are found.
93dec019 7712
df7492f9
KH		 7713If all coding systems in CODING-SYSTEM-LIST can encode the region, the
7714value is nil.
93dec019 7715
df7492f9
KH		 7716START may be a string. In that case, check if the string is
7717encodable, and the value contains indices to the string instead of
7718buffer positions. END is ignored. */)
7719 (start, end, coding_system_list)
7720 Lisp_Object start, end, coding_system_list;
05e6f5dc 7721{
df7492f9
KH		 7722 Lisp_Object list;
7723 EMACS_INT start_byte, end_byte;
7724 int pos;
7c78e542 7725 const unsigned char *p, *pbeg, *pend;
df7492f9 7726 int c;
7d64c6ad 7727 Lisp_Object tail, elt, attrs;
70ad9fc4 7728
05e6f5dc
KH		 7729 if (STRINGP (start))
7730 {
df7492f9 7731 if (!STRING_MULTIBYTE (start)
8f924df7 7732 && SCHARS (start) != SBYTES (start))
df7492f9
KH		 7733 return Qnil;
7734 start_byte = 0;
8f924df7 7735 end_byte = SBYTES (start);
df7492f9 7736 pos = 0;
d46c5b12 7737 }
05e6f5dc 7738 else
b73bfc1c 7739 {
b7826503
PJ		 7740 CHECK_NUMBER_COERCE_MARKER (start);
7741 CHECK_NUMBER_COERCE_MARKER (end);
05e6f5dc
KH		 7742 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
7743 args_out_of_range (start, end);
7744 if (NILP (current_buffer->enable_multibyte_characters))
df7492f9
KH		 7745 return Qnil;
7746 start_byte = CHAR_TO_BYTE (XINT (start));
7747 end_byte = CHAR_TO_BYTE (XINT (end));
7748 if (XINT (end) - XINT (start) == end_byte - start_byte)
05e6f5dc 7749 return Qt;
df7492f9 7750
e1c23804 7751 if (XINT (start) < GPT && XINT (end) > GPT)
b73bfc1c 7752 {
e1c23804
DL		 7753 if ((GPT - XINT (start)) < (XINT (end) - GPT))
7754 move_gap_both (XINT (start), start_byte);
df7492f9 7755 else
e1c23804 7756 move_gap_both (XINT (end), end_byte);
b73bfc1c 7757 }
e1c23804 7758 pos = XINT (start);
b73bfc1c 7759 }
7553d0e1 7760
df7492f9
KH		 7761 list = Qnil;
7762 for (tail = coding_system_list; CONSP (tail); tail = XCDR (tail))
12410ef1 7763 {
df7492f9 7764 elt = XCAR (tail);
7d64c6ad 7765 attrs = AREF (CODING_SYSTEM_SPEC (elt), 0);
2170c8f0
KH		 7766 ASET (attrs, coding_attr_trans_tbl,
7767 get_translation_table (attrs, 1, NULL));
7d64c6ad 7768 list = Fcons (Fcons (elt, Fcons (attrs, Qnil)), list);
12410ef1
KH		 7769 }
7770
df7492f9 7771 if (STRINGP (start))
8f924df7 7772 p = pbeg = SDATA (start);
72d1a715 7773 else
df7492f9
KH		 7774 p = pbeg = BYTE_POS_ADDR (start_byte);
7775 pend = p + (end_byte - start_byte);
4ed46869 7776
df7492f9
KH		 7777 while (p < pend && ASCII_BYTE_P (*p)) p++, pos++;
7778 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
ec6d2bb8 7779
df7492f9 7780 while (p < pend)
d46c5b12 7781 {
df7492f9
KH		 7782 if (ASCII_BYTE_P (*p))
7783 p++;
e133c8fa 7784 else
05e6f5dc 7785 {
df7492f9
KH		 7786 c = STRING_CHAR_ADVANCE (p);
7787
7788 charset_map_loaded = 0;
7789 for (tail = list; CONSP (tail); tail = XCDR (tail))
7790 {
7791 elt = XCDR (XCAR (tail));
7792 if (! char_encodable_p (c, XCAR (elt)))
7793 XSETCDR (elt, Fcons (make_number (pos), XCDR (elt)));
7794 }
7795 if (charset_map_loaded)
7796 {
7797 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
7798
7799 if (STRINGP (start))
8f924df7 7800 pbeg = SDATA (start);
df7492f9
KH		 7801 else
7802 pbeg = BYTE_POS_ADDR (start_byte);
7803 p = pbeg + p_offset;
7804 pend = pbeg + pend_offset;
7805 }
05e6f5dc 7806 }
df7492f9 7807 pos++;
d46c5b12 7808 }
4ed46869 7809
df7492f9
KH		 7810 tail = list;
7811 list = Qnil;
7812 for (; CONSP (tail); tail = XCDR (tail))
ec6d2bb8 7813 {
df7492f9
KH		 7814 elt = XCAR (tail);
7815 if (CONSP (XCDR (XCDR (elt))))
7816 list = Fcons (Fcons (XCAR (elt), Fnreverse (XCDR (XCDR (elt)))),
7817 list);
ec6d2bb8 7818 }
2b4f9037 7819
df7492f9 7820 return list;
d46c5b12
KH		 7821}
7822
3fd9494b 7823
b73bfc1c 7824Lisp_Object
df7492f9
KH		 7825code_convert_region (start, end, coding_system, dst_object, encodep, norecord)
7826 Lisp_Object start, end, coding_system, dst_object;
7827 int encodep, norecord;
4ed46869 7828{
3a73fa5d 7829 struct coding_system coding;
df7492f9
KH		 7830 EMACS_INT from, from_byte, to, to_byte;
7831 Lisp_Object src_object;
4ed46869 7832
b7826503
PJ		 7833 CHECK_NUMBER_COERCE_MARKER (start);
7834 CHECK_NUMBER_COERCE_MARKER (end);
df7492f9
KH		 7835 if (NILP (coding_system))
7836 coding_system = Qno_conversion;
7837 else
7838 CHECK_CODING_SYSTEM (coding_system);
7839 src_object = Fcurrent_buffer ();
7840 if (NILP (dst_object))
7841 dst_object = src_object;
7842 else if (! EQ (dst_object, Qt))
7843 CHECK_BUFFER (dst_object);
3a73fa5d 7844
d46c5b12
KH		 7845 validate_region (&start, &end);
7846 from = XFASTINT (start);
df7492f9 7847 from_byte = CHAR_TO_BYTE (from);
d46c5b12 7848 to = XFASTINT (end);
df7492f9 7849 to_byte = CHAR_TO_BYTE (to);
764ca8da 7850
df7492f9
KH		 7851 setup_coding_system (coding_system, &coding);
7852 coding.mode |= CODING_MODE_LAST_BLOCK;
ec6d2bb8 7853
df7492f9
KH		 7854 if (encodep)
7855 encode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
7856 dst_object);
7857 else
7858 decode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
7859 dst_object);
7860 if (! norecord)
7861 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
ec6d2bb8 7862
df7492f9
KH		 7863 return (BUFFERP (dst_object)
7864 ? make_number (coding.produced_char)
7865 : coding.dst_object);
4031e2bf 7866}
78108bcd 7867
4ed46869 7868
4031e2bf 7869DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region,
df7492f9 7870 3, 4, "r\nzCoding system: ",
48b0f3ae 7871 doc: /* Decode the current region from the specified coding system.
df7492f9
KH		 7872When called from a program, takes four arguments:
7873 START, END, CODING-SYSTEM, and DESTINATION.
7874START and END are buffer positions.
8844fa83 7875
df7492f9
KH		 7876Optional 4th arguments DESTINATION specifies where the decoded text goes.
7877If nil, the region between START and END is replace by the decoded text.
7878If buffer, the decoded text is inserted in the buffer.
7879If t, the decoded text is returned.
8844fa83 7880
48b0f3ae
PJ		 7881This function sets `last-coding-system-used' to the precise coding system
7882used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7883not fully specified.)
7884It returns the length of the decoded text. */)
df7492f9
KH		 7885 (start, end, coding_system, destination)
7886 Lisp_Object start, end, coding_system, destination;
4031e2bf 7887{
df7492f9 7888 return code_convert_region (start, end, coding_system, destination, 0, 0);
3a73fa5d 7889}
8844fa83 7890
3a73fa5d 7891DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region,
df7492f9
KH		 7892 3, 4, "r\nzCoding system: ",
7893 doc: /* Encode the current region by specified coding system.
48b0f3ae
PJ		 7894When called from a program, takes three arguments:
7895START, END, and CODING-SYSTEM. START and END are buffer positions.
d46c5b12 7896
df7492f9
KH		 7897Optional 4th arguments DESTINATION specifies where the encoded text goes.
7898If nil, the region between START and END is replace by the encoded text.
7899If buffer, the encoded text is inserted in the buffer.
7900If t, the encoded text is returned.
2391eaa4 7901
48b0f3ae
PJ		 7902This function sets `last-coding-system-used' to the precise coding system
7903used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7904not fully specified.)
7905It returns the length of the encoded text. */)
df7492f9
KH		 7906 (start, end, coding_system, destination)
7907 Lisp_Object start, end, coding_system, destination;
3a73fa5d 7908{
df7492f9 7909 return code_convert_region (start, end, coding_system, destination, 1, 0);
b73bfc1c
KH		 7910}
7911
7912Lisp_Object
df7492f9
KH		 7913code_convert_string (string, coding_system, dst_object,
7914 encodep, nocopy, norecord)
7915 Lisp_Object string, coding_system, dst_object;
7916 int encodep, nocopy, norecord;
b73bfc1c 7917{
4031e2bf 7918 struct coding_system coding;
df7492f9 7919 EMACS_INT chars, bytes;
ec6d2bb8 7920
b7826503 7921 CHECK_STRING (string);
d46c5b12 7922 if (NILP (coding_system))
4956c225 7923 {
df7492f9
KH		 7924 if (! norecord)
7925 Vlast_coding_system_used = Qno_conversion;
7926 if (NILP (dst_object))
7927 return (nocopy ? Fcopy_sequence (string) : string);
4956c225 7928 }
b73bfc1c 7929
df7492f9
KH		 7930 if (NILP (coding_system))
7931 coding_system = Qno_conversion;
7932 else
7933 CHECK_CODING_SYSTEM (coding_system);
7934 if (NILP (dst_object))
7935 dst_object = Qt;
7936 else if (! EQ (dst_object, Qt))
7937 CHECK_BUFFER (dst_object);
73be902c 7938
df7492f9 7939 setup_coding_system (coding_system, &coding);
d46c5b12 7940 coding.mode |= CODING_MODE_LAST_BLOCK;
8f924df7
KH		 7941 chars = SCHARS (string);
7942 bytes = SBYTES (string);
df7492f9
KH		 7943 if (encodep)
7944 encode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
7945 else
7946 decode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
7947 if (! norecord)
7948 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
73be902c 7949
df7492f9
KH		 7950 return (BUFFERP (dst_object)
7951 ? make_number (coding.produced_char)
7952 : coding.dst_object);
4ed46869 7953}
73be902c 7954
b73bfc1c 7955
ecec61c1 7956/* Encode or decode STRING according to CODING_SYSTEM.
ec6d2bb8 7957 Do not set Vlast_coding_system_used.
4ed46869 7958
ec6d2bb8
KH		 7959 This function is called only from macros DECODE_FILE and
7960 ENCODE_FILE, thus we ignore character composition. */
4ed46869 7961
ecec61c1
KH		 7962Lisp_Object
7963code_convert_string_norecord (string, coding_system, encodep)
7964 Lisp_Object string, coding_system;
7965 int encodep;
4ed46869 7966{
0be8721c 7967 return code_convert_string (string, coding_system, Qt, encodep, 0, 1);
4ed46869
KH		 7968}
7969
4ed46869 7970
df7492f9
KH		 7971DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string,
7972 2, 4, 0,
7973 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7974
7975Optional third arg NOCOPY non-nil means it is OK to return STRING itself
7976if the decoding operation is trivial.
ecec61c1 7977
df7492f9 7978Optional fourth arg BUFFER non-nil meant that the decoded text is
a3f6ee6d 7979inserted in BUFFER instead of returned as a string. In this case,
df7492f9 7980the return value is BUFFER.
ecec61c1 7981
df7492f9
KH		 7982This function sets `last-coding-system-used' to the precise coding system
7983used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7984not fully specified. */)
7985 (string, coding_system, nocopy, buffer)
7986 Lisp_Object string, coding_system, nocopy, buffer;
4ed46869 7987{
df7492f9
KH		 7988 return code_convert_string (string, coding_system, buffer,
7989 0, ! NILP (nocopy), 0);
4ed46869
KH		 7990}
7991
df7492f9
KH		 7992DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
7993 2, 4, 0,
7994 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
7995
7996Optional third arg NOCOPY non-nil means it is OK to return STRING
7997itself if the encoding operation is trivial.
7998
7999Optional fourth arg BUFFER non-nil meant that the encoded text is
a3f6ee6d 8000inserted in BUFFER instead of returned as a string. In this case,
df7492f9
KH		 8001the return value is BUFFER.
8002
8003This function sets `last-coding-system-used' to the precise coding system
8004used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
8005not fully specified.) */)
8006 (string, coding_system, nocopy, buffer)
8007 Lisp_Object string, coding_system, nocopy, buffer;
4ed46869 8008{
df7492f9 8009 return code_convert_string (string, coding_system, buffer,
c197f191 8010 1, ! NILP (nocopy), 1);
4ed46869 8011}
df7492f9 8012
3a73fa5d 8013\f
4ed46869 8014DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0,
48b0f3ae
PJ		 8015 doc: /* Decode a Japanese character which has CODE in shift_jis encoding.
8016Return the corresponding character. */)
8017 (code)
4ed46869 8018 Lisp_Object code;
4ed46869 8019{
df7492f9
KH		 8020 Lisp_Object spec, attrs, val;
8021 struct charset *charset_roman, *charset_kanji, *charset_kana, *charset;
8022 int c;
4ed46869 8023
df7492f9
KH		 8024 CHECK_NATNUM (code);
8025 c = XFASTINT (code);
8026 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8027 attrs = AREF (spec, 0);
4ed46869 8028
df7492f9
KH		 8029 if (ASCII_BYTE_P (c)
8030 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8031 return code;
4ed46869 8032
df7492f9
KH		 8033 val = CODING_ATTR_CHARSET_LIST (attrs);
8034 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
004068e4
KH		 8035 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8036 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val)));
fa42c37f 8037
df7492f9
KH		 8038 if (c <= 0x7F)
8039 charset = charset_roman;
8040 else if (c >= 0xA0 && c < 0xDF)
55ab7be3 8041 {
df7492f9
KH		 8042 charset = charset_kana;
8043 c -= 0x80;
4ed46869 8044 }
55ab7be3 8045 else
4ed46869 8046 {
004068e4 8047 int s1 = c >> 8, s2 = c & 0xFF;
df7492f9
KH		 8048
8049 if (s1 < 0x81 || (s1 > 0x9F && s1 < 0xE0) || s1 > 0xEF
8050 || s2 < 0x40 || s2 == 0x7F || s2 > 0xFC)
8051 error ("Invalid code: %d", code);
8052 SJIS_TO_JIS (c);
8053 charset = charset_kanji;
4ed46869 8054 }
df7492f9
KH		 8055 c = DECODE_CHAR (charset, c);
8056 if (c < 0)
8057 error ("Invalid code: %d", code);
8058 return make_number (c);
93dec019 8059}
4ed46869 8060
48b0f3ae 8061
4ed46869 8062DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0,
48b0f3ae
PJ		 8063 doc: /* Encode a Japanese character CHAR to shift_jis encoding.
8064Return the corresponding code in SJIS. */)
8065 (ch)
df7492f9 8066 Lisp_Object ch;
4ed46869 8067{
df7492f9
KH		 8068 Lisp_Object spec, attrs, charset_list;
8069 int c;
8070 struct charset *charset;
8071 unsigned code;
48b0f3ae 8072
df7492f9
KH		 8073 CHECK_CHARACTER (ch);
8074 c = XFASTINT (ch);
8075 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8076 attrs = AREF (spec, 0);
8077
8078 if (ASCII_CHAR_P (c)
8079 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8080 return ch;
8081
8082 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8083 charset = char_charset (c, charset_list, &code);
8084 if (code == CHARSET_INVALID_CODE (charset))
8085 error ("Can't encode by shift_jis encoding: %d", c);
8086 JIS_TO_SJIS (code);
8087
8088 return make_number (code);
4ed46869
KH		 8089}
8090
8091DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0,
48b0f3ae
PJ		 8092 doc: /* Decode a Big5 character which has CODE in BIG5 coding system.
8093Return the corresponding character. */)
8094 (code)
4ed46869 8095 Lisp_Object code;
d46c5b12 8096{
df7492f9
KH		 8097 Lisp_Object spec, attrs, val;
8098 struct charset *charset_roman, *charset_big5, *charset;
8099 int c;
6289dd10 8100
df7492f9
KH		 8101 CHECK_NATNUM (code);
8102 c = XFASTINT (code);
8103 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8104 attrs = AREF (spec, 0);
4ed46869 8105
df7492f9
KH		 8106 if (ASCII_BYTE_P (c)
8107 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8108 return code;
6289dd10 8109
df7492f9
KH		 8110 val = CODING_ATTR_CHARSET_LIST (attrs);
8111 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8112 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
c210f766 8113
df7492f9
KH		 8114 if (c <= 0x7F)
8115 charset = charset_roman;
c28a9453
KH		 8116 else
8117 {
df7492f9
KH		 8118 int b1 = c >> 8, b2 = c & 0x7F;
8119 if (b1 < 0xA1 || b1 > 0xFE
8120 || b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE)
8121 error ("Invalid code: %d", code);
8122 charset = charset_big5;
c28a9453 8123 }
df7492f9
KH		 8124 c = DECODE_CHAR (charset, (unsigned )c);
8125 if (c < 0)
8126 error ("Invalid code: %d", code);
8127 return make_number (c);
d46c5b12 8128}
6289dd10 8129
4ed46869 8130DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0,
48b0f3ae
PJ		 8131 doc: /* Encode the Big5 character CHAR to BIG5 coding system.
8132Return the corresponding character code in Big5. */)
8133 (ch)
4ed46869
KH		 8134 Lisp_Object ch;
8135{
df7492f9
KH		 8136 Lisp_Object spec, attrs, charset_list;
8137 struct charset *charset;
8138 int c;
8139 unsigned code;
8140
8141 CHECK_CHARACTER (ch);
8142 c = XFASTINT (ch);
8143 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8144 attrs = AREF (spec, 0);
8145 if (ASCII_CHAR_P (c)
8146 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8147 return ch;
8148
8149 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8150 charset = char_charset (c, charset_list, &code);
8151 if (code == CHARSET_INVALID_CODE (charset))
8152 error ("Can't encode by Big5 encoding: %d", c);
8153
8154 return make_number (code);
4ed46869 8155}
48b0f3ae 8156
3a73fa5d 8157\f
1ba9e4ab
KH		 8158DEFUN ("set-terminal-coding-system-internal",
8159 Fset_terminal_coding_system_internal,
48b0f3ae
PJ		 8160 Sset_terminal_coding_system_internal, 1, 1, 0,
8161 doc: /* Internal use only. */)
8162 (coding_system)
b74e4686 8163 Lisp_Object coding_system;
4ed46869 8164{
b7826503 8165 CHECK_SYMBOL (coding_system);
df7492f9
KH		 8166 setup_coding_system (Fcheck_coding_system (coding_system),
8167 &terminal_coding);
48b0f3ae 8168
70c22245 8169 /* We had better not send unsafe characters to terminal. */
df7492f9
KH		 8170 terminal_coding.mode |= CODING_MODE_SAFE_ENCODING;
8171 /* Characer composition should be disabled. */
8172 terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b73bfc1c
KH		 8173 terminal_coding.src_multibyte = 1;
8174 terminal_coding.dst_multibyte = 0;
4ed46869
KH		 8175 return Qnil;
8176}
8177
c4825358
KH		 8178DEFUN ("set-safe-terminal-coding-system-internal",
8179 Fset_safe_terminal_coding_system_internal,
48b0f3ae 8180 Sset_safe_terminal_coding_system_internal, 1, 1, 0,
ddb67bdc 8181 doc: /* Internal use only. */)
48b0f3ae 8182 (coding_system)
b74e4686 8183 Lisp_Object coding_system;
d46c5b12 8184{
b7826503 8185 CHECK_SYMBOL (coding_system);
c4825358
KH		 8186 setup_coding_system (Fcheck_coding_system (coding_system),
8187 &safe_terminal_coding);
df7492f9
KH		 8188 /* Characer composition should be disabled. */
8189 safe_terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
b73bfc1c
KH		 8190 safe_terminal_coding.src_multibyte = 1;
8191 safe_terminal_coding.dst_multibyte = 0;
c4825358
KH		 8192 return Qnil;
8193}
4ed46869 8194
4ed46869
KH		 8195DEFUN ("terminal-coding-system",
8196 Fterminal_coding_system, Sterminal_coding_system, 0, 0, 0,
48b0f3ae
PJ		 8197 doc: /* Return coding system specified for terminal output. */)
8198 ()
4ed46869 8199{
ae6f73fa
KH		 8200 Lisp_Object coding_system;
8201
8202 coding_system = CODING_ID_NAME (terminal_coding.id);
8203 /* For backward compatibility, return nil if it is `undecided'. */
8204 return (coding_system != Qundecided ? coding_system : Qnil);
4ed46869
KH		 8205}
8206
1ba9e4ab
KH		 8207DEFUN ("set-keyboard-coding-system-internal",
8208 Fset_keyboard_coding_system_internal,
48b0f3ae
PJ		 8209 Sset_keyboard_coding_system_internal, 1, 1, 0,
8210 doc: /* Internal use only. */)
8211 (coding_system)
4ed46869
KH		 8212 Lisp_Object coding_system;
8213{
b7826503 8214 CHECK_SYMBOL (coding_system);
df7492f9
KH		 8215 setup_coding_system (Fcheck_coding_system (coding_system),
8216 &keyboard_coding);
8217 /* Characer composition should be disabled. */
8218 keyboard_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
4ed46869
KH		 8219 return Qnil;
8220}
8221
8222DEFUN ("keyboard-coding-system",
8223 Fkeyboard_coding_system, Skeyboard_coding_system, 0, 0, 0,
48b0f3ae
PJ		 8224 doc: /* Return coding system specified for decoding keyboard input. */)
8225 ()
4ed46869 8226{
df7492f9 8227 return CODING_ID_NAME (keyboard_coding.id);
4ed46869
KH		 8228}
8229
4ed46869 8230\f
a5d301df
KH		 8231DEFUN ("find-operation-coding-system", Ffind_operation_coding_system,
8232 Sfind_operation_coding_system, 1, MANY, 0,
48b0f3ae
PJ		 8233 doc: /* Choose a coding system for an operation based on the target name.
8234The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
8235DECODING-SYSTEM is the coding system to use for decoding
8236\(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
8237for encoding (in case OPERATION does encoding).
05e6f5dc 8238
48b0f3ae
PJ		 8239The first argument OPERATION specifies an I/O primitive:
8240 For file I/O, `insert-file-contents' or `write-region'.
8241 For process I/O, `call-process', `call-process-region', or `start-process'.
8242 For network I/O, `open-network-stream'.
05e6f5dc 8243
48b0f3ae
PJ		 8244The remaining arguments should be the same arguments that were passed
8245to the primitive. Depending on which primitive, one of those arguments
8246is selected as the TARGET. For example, if OPERATION does file I/O,
8247whichever argument specifies the file name is TARGET.
05e6f5dc 8248
48b0f3ae
PJ		 8249TARGET has a meaning which depends on OPERATION:
8250 For file I/O, TARGET is a file name.
8251 For process I/O, TARGET is a process name.
8252 For network I/O, TARGET is a service name or a port number
05e6f5dc 8253
48b0f3ae
PJ		 8254This function looks up what specified for TARGET in,
8255`file-coding-system-alist', `process-coding-system-alist',
8256or `network-coding-system-alist' depending on OPERATION.
8257They may specify a coding system, a cons of coding systems,
8258or a function symbol to call.
8259In the last case, we call the function with one argument,
8260which is a list of all the arguments given to this function.
8261
8262usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
8263 (nargs, args)
4ed46869
KH		 8264 int nargs;
8265 Lisp_Object *args;
6b89e3aa 8266{
4ed46869
KH		 8267 Lisp_Object operation, target_idx, target, val;
8268 register Lisp_Object chain;
177c0ea7 8269
4ed46869
KH		 8270 if (nargs < 2)
8271 error ("Too few arguments");
8272 operation = args[0];
8273 if (!SYMBOLP (operation)
8274 || !INTEGERP (target_idx = Fget (operation, Qtarget_idx)))
df7492f9 8275 error ("Invalid first arguement");
4ed46869
KH		 8276 if (nargs < 1 + XINT (target_idx))
8277 error ("Too few arguments for operation: %s",
8f924df7 8278 SDATA (SYMBOL_NAME (operation)));
4ed46869
KH		 8279 target = args[XINT (target_idx) + 1];
8280 if (!(STRINGP (target)
8281 || (EQ (operation, Qopen_network_stream) && INTEGERP (target))))
df7492f9 8282 error ("Invalid %dth argument", XINT (target_idx) + 1);
4ed46869 8283
2e34157c
RS		 8284 chain = ((EQ (operation, Qinsert_file_contents)
8285 || EQ (operation, Qwrite_region))
02ba4723 8286 ? Vfile_coding_system_alist
2e34157c 8287 : (EQ (operation, Qopen_network_stream)
02ba4723
KH		 8288 ? Vnetwork_coding_system_alist
8289 : Vprocess_coding_system_alist));
4ed46869
KH		 8290 if (NILP (chain))
8291 return Qnil;
8292
03699b14 8293 for (; CONSP (chain); chain = XCDR (chain))
6b89e3aa 8294 {
f44d27ce 8295 Lisp_Object elt;
6b89e3aa 8296
df7492f9 8297 elt = XCAR (chain);
4ed46869
KH		 8298 if (CONSP (elt)
8299 && ((STRINGP (target)
03699b14
KR		 8300 && STRINGP (XCAR (elt))
8301 && fast_string_match (XCAR (elt), target) >= 0)
8302 || (INTEGERP (target) && EQ (target, XCAR (elt)))))
6b89e3aa 8303 {
03699b14 8304 val = XCDR (elt);
b19fd4c5
KH		 8305 /* Here, if VAL is both a valid coding system and a valid
8306 function symbol, we return VAL as a coding system. */
02ba4723
KH		 8307 if (CONSP (val))
8308 return val;
8309 if (! SYMBOLP (val))
8310 return Qnil;
8311 if (! NILP (Fcoding_system_p (val)))
8312 return Fcons (val, val);
b19fd4c5 8313 if (! NILP (Ffboundp (val)))
6b89e3aa 8314 {
b19fd4c5
KH		 8315 val = call1 (val, Flist (nargs, args));
8316 if (CONSP (val))
8317 return val;
8318 if (SYMBOLP (val) && ! NILP (Fcoding_system_p (val)))
8319 return Fcons (val, val);
6b89e3aa 8320 }
02ba4723 8321 return Qnil;
6b89e3aa
KH		 8322 }
8323 }
4ed46869 8324 return Qnil;
6b89e3aa
KH		 8325}
8326
df7492f9 8327DEFUN ("set-coding-system-priority", Fset_coding_system_priority,
a3f6ee6d 8328 Sset_coding_system_priority, 0, MANY, 0,
da7db224 8329 doc: /* Assign higher priority to the coding systems given as arguments.
ff563fce 8330If multiple coding systems belongs to the same category,
a3181084
DL		 8331all but the first one are ignored.
8332
8333usage: (set-coding-system-priority ...) */)
df7492f9
KH		 8334 (nargs, args)
8335 int nargs;
8336 Lisp_Object *args;
8337{
8338 int i, j;
8339 int changed[coding_category_max];
8340 enum coding_category priorities[coding_category_max];
8341
8342 bzero (changed, sizeof changed);
6b89e3aa 8343
df7492f9 8344 for (i = j = 0; i < nargs; i++)
6b89e3aa 8345 {
df7492f9
KH		 8346 enum coding_category category;
8347 Lisp_Object spec, attrs;
6b89e3aa 8348
df7492f9
KH		 8349 CHECK_CODING_SYSTEM_GET_SPEC (args[i], spec);
8350 attrs = AREF (spec, 0);
8351 category = XINT (CODING_ATTR_CATEGORY (attrs));
8352 if (changed[category])
8353 /* Ignore this coding system because a coding system of the
8354 same category already had a higher priority. */
8355 continue;
8356 changed[category] = 1;
8357 priorities[j++] = category;
8358 if (coding_categories[category].id >= 0
8359 && ! EQ (args[i], CODING_ID_NAME (coding_categories[category].id)))
8360 setup_coding_system (args[i], &coding_categories[category]);
ff563fce 8361 Fset (AREF (Vcoding_category_table, category), args[i]);
df7492f9 8362 }
6b89e3aa 8363
df7492f9
KH		 8364 /* Now we have decided top J priorities. Reflect the order of the
8365 original priorities to the remaining priorities. */
6b89e3aa 8366
df7492f9 8367 for (i = j, j = 0; i < coding_category_max; i++, j++)
6b89e3aa 8368 {
df7492f9
KH		 8369 while (j < coding_category_max
8370 && changed[coding_priorities[j]])
8371 j++;
8372 if (j == coding_category_max)
8373 abort ();
8374 priorities[i] = coding_priorities[j];
8375 }
6b89e3aa 8376
df7492f9 8377 bcopy (priorities, coding_priorities, sizeof priorities);
177c0ea7 8378
ff563fce
KH		 8379 /* Update `coding-category-list'. */
8380 Vcoding_category_list = Qnil;
8381 for (i = coding_category_max - 1; i >= 0; i--)
8382 Vcoding_category_list
8383 = Fcons (AREF (Vcoding_category_table, priorities[i]),
8384 Vcoding_category_list);
6b89e3aa 8385
df7492f9 8386 return Qnil;
6b89e3aa
KH		 8387}
8388
df7492f9
KH		 8389DEFUN ("coding-system-priority-list", Fcoding_system_priority_list,
8390 Scoding_system_priority_list, 0, 1, 0,
da7db224
DL		 8391 doc: /* Return a list of coding systems ordered by their priorities.
8392HIGHESTP non-nil means just return the highest priority one. */)
df7492f9
KH		 8393 (highestp)
8394 Lisp_Object highestp;
d46c5b12
KH		 8395{
8396 int i;
df7492f9 8397 Lisp_Object val;
6b89e3aa 8398
df7492f9 8399 for (i = 0, val = Qnil; i < coding_category_max; i++)
d46c5b12 8400 {
df7492f9
KH		 8401 enum coding_category category = coding_priorities[i];
8402 int id = coding_categories[category].id;
8403 Lisp_Object attrs;
068a9dbd 8404
df7492f9
KH		 8405 if (id < 0)
8406 continue;
8407 attrs = CODING_ID_ATTRS (id);
8408 if (! NILP (highestp))
8409 return CODING_ATTR_BASE_NAME (attrs);
8410 val = Fcons (CODING_ATTR_BASE_NAME (attrs), val);
8411 }
8412 return Fnreverse (val);
8413}
068a9dbd 8414
f0064e1f 8415static char *suffixes[] = { "-unix", "-dos", "-mac" };
068a9dbd
KH		 8416
8417static Lisp_Object
df7492f9
KH		 8418make_subsidiaries (base)
8419 Lisp_Object base;
068a9dbd 8420{
df7492f9 8421 Lisp_Object subsidiaries;
8f924df7 8422 int base_name_len = SBYTES (SYMBOL_NAME (base));
df7492f9
KH		 8423 char *buf = (char *) alloca (base_name_len + 6);
8424 int i;
068a9dbd 8425
8f924df7 8426 bcopy (SDATA (SYMBOL_NAME (base)), buf, base_name_len);
df7492f9
KH		 8427 subsidiaries = Fmake_vector (make_number (3), Qnil);
8428 for (i = 0; i < 3; i++)
068a9dbd 8429 {
df7492f9
KH		 8430 bcopy (suffixes[i], buf + base_name_len, strlen (suffixes[i]) + 1);
8431 ASET (subsidiaries, i, intern (buf));
068a9dbd 8432 }
df7492f9 8433 return subsidiaries;
068a9dbd
KH		 8434}
8435
8436
df7492f9
KH		 8437DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal,
8438 Sdefine_coding_system_internal, coding_arg_max, MANY, 0,
1fcd6c8b
DL		 8439 doc: /* For internal use only.
8440usage: (define-coding-system-internal ...) */)
df7492f9
KH		 8441 (nargs, args)
8442 int nargs;
8443 Lisp_Object *args;
068a9dbd 8444{
df7492f9
KH		 8445 Lisp_Object name;
8446 Lisp_Object spec_vec; /* [ ATTRS ALIASE EOL_TYPE ] */
8447 Lisp_Object attrs; /* Vector of attributes. */
8448 Lisp_Object eol_type;
8449 Lisp_Object aliases;
8450 Lisp_Object coding_type, charset_list, safe_charsets;
8451 enum coding_category category;
8452 Lisp_Object tail, val;
8453 int max_charset_id = 0;
8454 int i;
068a9dbd 8455
df7492f9
KH		 8456 if (nargs < coding_arg_max)
8457 goto short_args;
068a9dbd 8458
df7492f9 8459 attrs = Fmake_vector (make_number (coding_attr_last_index), Qnil);
068a9dbd 8460
df7492f9
KH		 8461 name = args[coding_arg_name];
8462 CHECK_SYMBOL (name);
8463 CODING_ATTR_BASE_NAME (attrs) = name;
068a9dbd 8464
df7492f9
KH		 8465 val = args[coding_arg_mnemonic];
8466 if (! STRINGP (val))
8467 CHECK_CHARACTER (val);
8468 CODING_ATTR_MNEMONIC (attrs) = val;
068a9dbd 8469
df7492f9
KH		 8470 coding_type = args[coding_arg_coding_type];
8471 CHECK_SYMBOL (coding_type);
8472 CODING_ATTR_TYPE (attrs) = coding_type;
068a9dbd 8473
df7492f9
KH		 8474 charset_list = args[coding_arg_charset_list];
8475 if (SYMBOLP (charset_list))
8476 {
8477 if (EQ (charset_list, Qiso_2022))
8478 {
8479 if (! EQ (coding_type, Qiso_2022))
8480 error ("Invalid charset-list");
8481 charset_list = Viso_2022_charset_list;
8482 }
8483 else if (EQ (charset_list, Qemacs_mule))
8484 {
8485 if (! EQ (coding_type, Qemacs_mule))
8486 error ("Invalid charset-list");
8487 charset_list = Vemacs_mule_charset_list;
8488 }
8489 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8490 if (max_charset_id < XFASTINT (XCAR (tail)))
8491 max_charset_id = XFASTINT (XCAR (tail));
8492 }
068a9dbd
KH		 8493 else
8494 {
df7492f9
KH		 8495 charset_list = Fcopy_sequence (charset_list);
8496 for (tail = charset_list; !NILP (tail); tail = Fcdr (tail))
068a9dbd 8497 {
df7492f9
KH		 8498 struct charset *charset;
8499
8500 val = Fcar (tail);
8501 CHECK_CHARSET_GET_CHARSET (val, charset);
8502 if (EQ (coding_type, Qiso_2022)
8503 ? CHARSET_ISO_FINAL (charset) < 0
8504 : EQ (coding_type, Qemacs_mule)
8505 ? CHARSET_EMACS_MULE_ID (charset) < 0
8506 : 0)
8507 error ("Can't handle charset `%s'",
8f924df7 8508 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9 8509
8f924df7 8510 XSETCAR (tail, make_number (charset->id));
df7492f9
KH		 8511 if (max_charset_id < charset->id)
8512 max_charset_id = charset->id;
068a9dbd
KH		 8513 }
8514 }
df7492f9 8515 CODING_ATTR_CHARSET_LIST (attrs) = charset_list;
068a9dbd 8516
df7492f9
KH		 8517 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
8518 make_number (255));
8519 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8f924df7 8520 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
df7492f9 8521 CODING_ATTR_SAFE_CHARSETS (attrs) = safe_charsets;
068a9dbd 8522
584948ac 8523 CODING_ATTR_ASCII_COMPAT (attrs) = args[coding_arg_ascii_compatible_p];
3a73fa5d 8524
df7492f9 8525 val = args[coding_arg_decode_translation_table];
a6f87d34 8526 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 8527 CHECK_SYMBOL (val);
df7492f9 8528 CODING_ATTR_DECODE_TBL (attrs) = val;
3a73fa5d 8529
df7492f9 8530 val = args[coding_arg_encode_translation_table];
a6f87d34 8531 if (! CHAR_TABLE_P (val) && ! CONSP (val))
7d64c6ad 8532 CHECK_SYMBOL (val);
df7492f9 8533 CODING_ATTR_ENCODE_TBL (attrs) = val;
d46c5b12 8534
df7492f9
KH		 8535 val = args[coding_arg_post_read_conversion];
8536 CHECK_SYMBOL (val);
8537 CODING_ATTR_POST_READ (attrs) = val;
d46c5b12 8538
df7492f9
KH		 8539 val = args[coding_arg_pre_write_conversion];
8540 CHECK_SYMBOL (val);
8541 CODING_ATTR_PRE_WRITE (attrs) = val;
3a73fa5d 8542
df7492f9
KH		 8543 val = args[coding_arg_default_char];
8544 if (NILP (val))
8545 CODING_ATTR_DEFAULT_CHAR (attrs) = make_number (' ');
8546 else
8547 {
8f924df7 8548 CHECK_CHARACTER (val);
df7492f9
KH		 8549 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
8550 }
4031e2bf 8551
8f924df7
KH		 8552 val = args[coding_arg_for_unibyte];
8553 CODING_ATTR_FOR_UNIBYTE (attrs) = NILP (val) ? Qnil : Qt;
3a73fa5d 8554
df7492f9
KH		 8555 val = args[coding_arg_plist];
8556 CHECK_LIST (val);
8557 CODING_ATTR_PLIST (attrs) = val;
3a73fa5d 8558
df7492f9
KH		 8559 if (EQ (coding_type, Qcharset))
8560 {
c7c66a95
KH		 8561 /* Generate a lisp vector of 256 elements. Each element is nil,
8562 integer, or a list of charset IDs.
3a73fa5d 8563
c7c66a95
KH		 8564 If Nth element is nil, the byte code N is invalid in this
8565 coding system.
4ed46869 8566
c7c66a95
KH		 8567 If Nth element is a number NUM, N is the first byte of a
8568 charset whose ID is NUM.
4ed46869 8569
c7c66a95
KH		 8570 If Nth element is a list of charset IDs, N is the first byte
8571 of one of them. The list is sorted by dimensions of the
2bc515e4 8572 charsets. A charset of smaller dimension comes firtst. */
df7492f9 8573 val = Fmake_vector (make_number (256), Qnil);
4ed46869 8574
5c99c2e6 8575 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
df7492f9 8576 {
c7c66a95
KH		 8577 struct charset *charset = CHARSET_FROM_ID (XFASTINT (XCAR (tail)));
8578 int dim = CHARSET_DIMENSION (charset);
8579 int idx = (dim - 1) * 4;
4ed46869 8580
5c99c2e6 8581 if (CHARSET_ASCII_COMPATIBLE_P (charset))
584948ac 8582 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
4031e2bf 8583
15d143f7
KH		 8584 for (i = charset->code_space[idx];
8585 i <= charset->code_space[idx + 1]; i++)
8586 {
c7c66a95
KH		 8587 Lisp_Object tmp, tmp2;
8588 int dim2;
ec6d2bb8 8589
c7c66a95
KH		 8590 tmp = AREF (val, i);
8591 if (NILP (tmp))
8592 tmp = XCAR (tail);
8593 else if (NUMBERP (tmp))
8594 {
8595 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (tmp)));
8596 if (dim < dim2)
c7c66a95 8597 tmp = Fcons (XCAR (tail), Fcons (tmp, Qnil));
f9d71dcd
KH		 8598 else
8599 tmp = Fcons (tmp, Fcons (XCAR (tail), Qnil));
c7c66a95 8600 }
15d143f7 8601 else
c7c66a95
KH		 8602 {
8603 for (tmp2 = tmp; CONSP (tmp2); tmp2 = XCDR (tmp2))
8604 {
8605 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (XCAR (tmp2))));
8606 if (dim < dim2)
8607 break;
8608 }
8609 if (NILP (tmp2))
8610 tmp = nconc2 (tmp, Fcons (XCAR (tail), Qnil));
8611 else
8612 {
8613 XSETCDR (tmp2, Fcons (XCAR (tmp2), XCDR (tmp2)));
8614 XSETCAR (tmp2, XCAR (tail));
8615 }
8616 }
8617 ASET (val, i, tmp);
15d143f7 8618 }
df7492f9
KH		 8619 }
8620 ASET (attrs, coding_attr_charset_valids, val);
8621 category = coding_category_charset;
8622 }
8623 else if (EQ (coding_type, Qccl))
8624 {
8625 Lisp_Object valids;
ecec61c1 8626
df7492f9
KH		 8627 if (nargs < coding_arg_ccl_max)
8628 goto short_args;
ecec61c1 8629
df7492f9
KH		 8630 val = args[coding_arg_ccl_decoder];
8631 CHECK_CCL_PROGRAM (val);
8632 if (VECTORP (val))
8633 val = Fcopy_sequence (val);
8634 ASET (attrs, coding_attr_ccl_decoder, val);
ecec61c1 8635
df7492f9
KH		 8636 val = args[coding_arg_ccl_encoder];
8637 CHECK_CCL_PROGRAM (val);
8638 if (VECTORP (val))
8639 val = Fcopy_sequence (val);
8640 ASET (attrs, coding_attr_ccl_encoder, val);
ecec61c1 8641
df7492f9
KH		 8642 val = args[coding_arg_ccl_valids];
8643 valids = Fmake_string (make_number (256), make_number (0));
8644 for (tail = val; !NILP (tail); tail = Fcdr (tail))
8645 {
8dcbea82 8646 int from, to;
ecec61c1 8647
df7492f9
KH		 8648 val = Fcar (tail);
8649 if (INTEGERP (val))
8dcbea82
KH		 8650 {
8651 from = to = XINT (val);
8652 if (from < 0 || from > 255)
8653 args_out_of_range_3 (val, make_number (0), make_number (255));
8654 }
df7492f9
KH		 8655 else
8656 {
df7492f9 8657 CHECK_CONS (val);
8f924df7
KH		 8658 CHECK_NATNUM_CAR (val);
8659 CHECK_NATNUM_CDR (val);
df7492f9 8660 from = XINT (XCAR (val));
8f924df7 8661 if (from > 255)
8dcbea82
KH		 8662 args_out_of_range_3 (XCAR (val),
8663 make_number (0), make_number (255));
df7492f9 8664 to = XINT (XCDR (val));
8dcbea82
KH		 8665 if (to < from || to > 255)
8666 args_out_of_range_3 (XCDR (val),
8667 XCAR (val), make_number (255));
df7492f9 8668 }
8dcbea82 8669 for (i = from; i <= to; i++)
8f924df7 8670 SSET (valids, i, 1);
df7492f9
KH		 8671 }
8672 ASET (attrs, coding_attr_ccl_valids, valids);
4ed46869 8673
df7492f9 8674 category = coding_category_ccl;
55ab7be3 8675 }
df7492f9 8676 else if (EQ (coding_type, Qutf_16))
55ab7be3 8677 {
df7492f9 8678 Lisp_Object bom, endian;
4ed46869 8679
584948ac 8680 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
4ed46869 8681
df7492f9
KH		 8682 if (nargs < coding_arg_utf16_max)
8683 goto short_args;
4ed46869 8684
df7492f9
KH		 8685 bom = args[coding_arg_utf16_bom];
8686 if (! NILP (bom) && ! EQ (bom, Qt))
8687 {
8688 CHECK_CONS (bom);
8f924df7
KH		 8689 val = XCAR (bom);
8690 CHECK_CODING_SYSTEM (val);
8691 val = XCDR (bom);
8692 CHECK_CODING_SYSTEM (val);
df7492f9
KH		 8693 }
8694 ASET (attrs, coding_attr_utf_16_bom, bom);
8695
8696 endian = args[coding_arg_utf16_endian];
b49a1807
KH		 8697 CHECK_SYMBOL (endian);
8698 if (NILP (endian))
8699 endian = Qbig;
8700 else if (! EQ (endian, Qbig) && ! EQ (endian, Qlittle))
8f924df7 8701 error ("Invalid endian: %s", SDATA (SYMBOL_NAME (endian)));
df7492f9
KH		 8702 ASET (attrs, coding_attr_utf_16_endian, endian);
8703
8704 category = (CONSP (bom)
8705 ? coding_category_utf_16_auto
8706 : NILP (bom)
b49a1807 8707 ? (EQ (endian, Qbig)
df7492f9
KH		 8708 ? coding_category_utf_16_be_nosig
8709 : coding_category_utf_16_le_nosig)
b49a1807 8710 : (EQ (endian, Qbig)
df7492f9
KH		 8711 ? coding_category_utf_16_be
8712 : coding_category_utf_16_le));
8713 }
8714 else if (EQ (coding_type, Qiso_2022))
8715 {
8716 Lisp_Object initial, reg_usage, request, flags;
4776e638 8717 int i;
1397dc18 8718
df7492f9
KH		 8719 if (nargs < coding_arg_iso2022_max)
8720 goto short_args;
8721
8722 initial = Fcopy_sequence (args[coding_arg_iso2022_initial]);
8723 CHECK_VECTOR (initial);
8724 for (i = 0; i < 4; i++)
8725 {
8726 val = Faref (initial, make_number (i));
8727 if (! NILP (val))
8728 {
584948ac
KH		 8729 struct charset *charset;
8730
8731 CHECK_CHARSET_GET_CHARSET (val, charset);
8732 ASET (initial, i, make_number (CHARSET_ID (charset)));
8733 if (i == 0 && CHARSET_ASCII_COMPATIBLE_P (charset))
8734 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 8735 }
8736 else
8737 ASET (initial, i, make_number (-1));
8738 }
8739
8740 reg_usage = args[coding_arg_iso2022_reg_usage];
8741 CHECK_CONS (reg_usage);
8f924df7
KH		 8742 CHECK_NUMBER_CAR (reg_usage);
8743 CHECK_NUMBER_CDR (reg_usage);
df7492f9
KH		 8744
8745 request = Fcopy_sequence (args[coding_arg_iso2022_request]);
8746 for (tail = request; ! NILP (tail); tail = Fcdr (tail))
1397dc18 8747 {
df7492f9 8748 int id;
8f924df7 8749 Lisp_Object tmp;
df7492f9
KH		 8750
8751 val = Fcar (tail);
8752 CHECK_CONS (val);
8f924df7
KH		 8753 tmp = XCAR (val);
8754 CHECK_CHARSET_GET_ID (tmp, id);
8755 CHECK_NATNUM_CDR (val);
df7492f9
KH		 8756 if (XINT (XCDR (val)) >= 4)
8757 error ("Invalid graphic register number: %d", XINT (XCDR (val)));
8f924df7 8758 XSETCAR (val, make_number (id));
1397dc18 8759 }
4ed46869 8760
df7492f9
KH		 8761 flags = args[coding_arg_iso2022_flags];
8762 CHECK_NATNUM (flags);
8763 i = XINT (flags);
8764 if (EQ (args[coding_arg_charset_list], Qiso_2022))
8765 flags = make_number (i | CODING_ISO_FLAG_FULL_SUPPORT);
8766
8767 ASET (attrs, coding_attr_iso_initial, initial);
8768 ASET (attrs, coding_attr_iso_usage, reg_usage);
8769 ASET (attrs, coding_attr_iso_request, request);
8770 ASET (attrs, coding_attr_iso_flags, flags);
8771 setup_iso_safe_charsets (attrs);
8772
8773 if (i & CODING_ISO_FLAG_SEVEN_BITS)
8774 category = ((i & (CODING_ISO_FLAG_LOCKING_SHIFT
8775 | CODING_ISO_FLAG_SINGLE_SHIFT))
8776 ? coding_category_iso_7_else
8777 : EQ (args[coding_arg_charset_list], Qiso_2022)
8778 ? coding_category_iso_7
8779 : coding_category_iso_7_tight);
8780 else
8781 {
8782 int id = XINT (AREF (initial, 1));
8783
c6fb6e98 8784 category = (((i & CODING_ISO_FLAG_LOCKING_SHIFT)
df7492f9
KH		 8785 || EQ (args[coding_arg_charset_list], Qiso_2022)
8786 || id < 0)
8787 ? coding_category_iso_8_else
8788 : (CHARSET_DIMENSION (CHARSET_FROM_ID (id)) == 1)
8789 ? coding_category_iso_8_1
8790 : coding_category_iso_8_2);
8791 }
0ce7886f
KH		 8792 if (category != coding_category_iso_8_1
8793 && category != coding_category_iso_8_2)
8794 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
df7492f9
KH		 8795 }
8796 else if (EQ (coding_type, Qemacs_mule))
c28a9453 8797 {
df7492f9
KH		 8798 if (EQ (args[coding_arg_charset_list], Qemacs_mule))
8799 ASET (attrs, coding_attr_emacs_mule_full, Qt);
584948ac 8800 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9 8801 category = coding_category_emacs_mule;
c28a9453 8802 }
df7492f9 8803 else if (EQ (coding_type, Qshift_jis))
c28a9453 8804 {
df7492f9
KH		 8805
8806 struct charset *charset;
8807
7d64c6ad 8808 if (XINT (Flength (charset_list)) != 3
6e07c25f 8809 && XINT (Flength (charset_list)) != 4)
7d64c6ad 8810 error ("There should be three or four charsets");
df7492f9
KH		 8811
8812 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8813 if (CHARSET_DIMENSION (charset) != 1)
8814 error ("Dimension of charset %s is not one",
8f924df7 8815 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 8816 if (CHARSET_ASCII_COMPATIBLE_P (charset))
8817 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 8818
8819 charset_list = XCDR (charset_list);
8820 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8821 if (CHARSET_DIMENSION (charset) != 1)
8822 error ("Dimension of charset %s is not one",
8f924df7 8823 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
df7492f9
KH		 8824
8825 charset_list = XCDR (charset_list);
8826 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8827 if (CHARSET_DIMENSION (charset) != 2)
7d64c6ad
KH		 8828 error ("Dimension of charset %s is not two",
8829 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
8830
8831 charset_list = XCDR (charset_list);
2b917a06
KH		 8832 if (! NILP (charset_list))
8833 {
8834 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8835 if (CHARSET_DIMENSION (charset) != 2)
8836 error ("Dimension of charset %s is not two",
8837 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
8838 }
df7492f9
KH		 8839
8840 category = coding_category_sjis;
8841 Vsjis_coding_system = name;
c28a9453 8842 }
df7492f9
KH		 8843 else if (EQ (coding_type, Qbig5))
8844 {
8845 struct charset *charset;
4ed46869 8846
df7492f9
KH		 8847 if (XINT (Flength (charset_list)) != 2)
8848 error ("There should be just two charsets");
8849
8850 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8851 if (CHARSET_DIMENSION (charset) != 1)
8852 error ("Dimension of charset %s is not one",
8f924df7 8853 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
584948ac
KH		 8854 if (CHARSET_ASCII_COMPATIBLE_P (charset))
8855 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
df7492f9
KH		 8856
8857 charset_list = XCDR (charset_list);
8858 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8859 if (CHARSET_DIMENSION (charset) != 2)
8860 error ("Dimension of charset %s is not two",
8f924df7 8861 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
4ed46869 8862
df7492f9
KH		 8863 category = coding_category_big5;
8864 Vbig5_coding_system = name;
8865 }
8866 else if (EQ (coding_type, Qraw_text))
c28a9453 8867 {
584948ac
KH		 8868 category = coding_category_raw_text;
8869 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
c28a9453 8870 }
df7492f9 8871 else if (EQ (coding_type, Qutf_8))
4ed46869 8872 {
584948ac
KH		 8873 category = coding_category_utf_8;
8874 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
4ed46869 8875 }
df7492f9
KH		 8876 else if (EQ (coding_type, Qundecided))
8877 category = coding_category_undecided;
4ed46869 8878 else
df7492f9 8879 error ("Invalid coding system type: %s",
8f924df7 8880 SDATA (SYMBOL_NAME (coding_type)));
4ed46869 8881
df7492f9 8882 CODING_ATTR_CATEGORY (attrs) = make_number (category);
01378f49
KH		 8883 CODING_ATTR_PLIST (attrs)
8884 = Fcons (QCcategory, Fcons (AREF (Vcoding_category_table, category),
8885 CODING_ATTR_PLIST (attrs)));
35befdaa
KH		 8886 CODING_ATTR_PLIST (attrs)
8887 = Fcons (QCascii_compatible_p,
8888 Fcons (CODING_ATTR_ASCII_COMPAT (attrs),
8889 CODING_ATTR_PLIST (attrs)));
c4825358 8890
df7492f9
KH		 8891 eol_type = args[coding_arg_eol_type];
8892 if (! NILP (eol_type)
8893 && ! EQ (eol_type, Qunix)
8894 && ! EQ (eol_type, Qdos)
8895 && ! EQ (eol_type, Qmac))
8896 error ("Invalid eol-type");
4ed46869 8897
df7492f9 8898 aliases = Fcons (name, Qnil);
4ed46869 8899
df7492f9
KH		 8900 if (NILP (eol_type))
8901 {
8902 eol_type = make_subsidiaries (name);
8903 for (i = 0; i < 3; i++)
1397dc18 8904 {
df7492f9
KH		 8905 Lisp_Object this_spec, this_name, this_aliases, this_eol_type;
8906
8907 this_name = AREF (eol_type, i);
8908 this_aliases = Fcons (this_name, Qnil);
8909 this_eol_type = (i == 0 ? Qunix : i == 1 ? Qdos : Qmac);
8910 this_spec = Fmake_vector (make_number (3), attrs);
8911 ASET (this_spec, 1, this_aliases);
8912 ASET (this_spec, 2, this_eol_type);
8913 Fputhash (this_name, this_spec, Vcoding_system_hash_table);
8914 Vcoding_system_list = Fcons (this_name, Vcoding_system_list);
8915 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (this_name), Qnil),
8916 Vcoding_system_alist);
1397dc18 8917 }
d46c5b12 8918 }
4ed46869 8919
df7492f9
KH		 8920 spec_vec = Fmake_vector (make_number (3), attrs);
8921 ASET (spec_vec, 1, aliases);
8922 ASET (spec_vec, 2, eol_type);
48b0f3ae 8923
df7492f9
KH		 8924 Fputhash (name, spec_vec, Vcoding_system_hash_table);
8925 Vcoding_system_list = Fcons (name, Vcoding_system_list);
8926 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (name), Qnil),
8927 Vcoding_system_alist);
48b0f3ae 8928
df7492f9
KH		 8929 {
8930 int id = coding_categories[category].id;
48b0f3ae 8931
df7492f9
KH		 8932 if (id < 0 || EQ (name, CODING_ID_NAME (id)))
8933 setup_coding_system (name, &coding_categories[category]);
8934 }
48b0f3ae 8935
d46c5b12 8936 return Qnil;
48b0f3ae 8937
df7492f9
KH		 8938 short_args:
8939 return Fsignal (Qwrong_number_of_arguments,
8940 Fcons (intern ("define-coding-system-internal"),
8941 make_number (nargs)));
d46c5b12 8942}
4ed46869 8943
d6925f38 8944
a6f87d34
KH		 8945DEFUN ("coding-system-put", Fcoding_system_put, Scoding_system_put,
8946 3, 3, 0,
8947 doc: /* Change value in CODING-SYSTEM's property list PROP to VAL. */)
8948 (coding_system, prop, val)
8949 Lisp_Object coding_system, prop, val;
8950{
3dbe7859 8951 Lisp_Object spec, attrs;
a6f87d34
KH		 8952
8953 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
8954 attrs = AREF (spec, 0);
8955 if (EQ (prop, QCmnemonic))
8956 {
8957 if (! STRINGP (val))
8958 CHECK_CHARACTER (val);
8959 CODING_ATTR_MNEMONIC (attrs) = val;
8960 }
8961 else if (EQ (prop, QCdefalut_char))
8962 {
8963 if (NILP (val))
8964 val = make_number (' ');
8965 else
8966 CHECK_CHARACTER (val);
8967 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
8968 }
8969 else if (EQ (prop, QCdecode_translation_table))
8970 {
8971 if (! CHAR_TABLE_P (val) && ! CONSP (val))
8972 CHECK_SYMBOL (val);
8973 CODING_ATTR_DECODE_TBL (attrs) = val;
8974 }
8975 else if (EQ (prop, QCencode_translation_table))
8976 {
8977 if (! CHAR_TABLE_P (val) && ! CONSP (val))
8978 CHECK_SYMBOL (val);
8979 CODING_ATTR_ENCODE_TBL (attrs) = val;
8980 }
8981 else if (EQ (prop, QCpost_read_conversion))
8982 {
8983 CHECK_SYMBOL (val);
8984 CODING_ATTR_POST_READ (attrs) = val;
8985 }
8986 else if (EQ (prop, QCpre_write_conversion))
8987 {
8988 CHECK_SYMBOL (val);
8989 CODING_ATTR_PRE_WRITE (attrs) = val;
8990 }
35befdaa
KH		 8991 else if (EQ (prop, QCascii_compatible_p))
8992 {
8993 CODING_ATTR_ASCII_COMPAT (attrs) = val;
8994 }
a6f87d34
KH		 8995
8996 CODING_ATTR_PLIST (attrs)
8997 = Fplist_put (CODING_ATTR_PLIST (attrs), prop, val);
8998 return val;
8999}
9000
9001
df7492f9
KH		 9002DEFUN ("define-coding-system-alias", Fdefine_coding_system_alias,
9003 Sdefine_coding_system_alias, 2, 2, 0,
9004 doc: /* Define ALIAS as an alias for CODING-SYSTEM. */)
9005 (alias, coding_system)
9006 Lisp_Object alias, coding_system;
66cfb530 9007{
df7492f9 9008 Lisp_Object spec, aliases, eol_type;
4ed46869 9009
df7492f9
KH		 9010 CHECK_SYMBOL (alias);
9011 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9012 aliases = AREF (spec, 1);
d6925f38
KH		 9013 /* ALISES should be a list of length more than zero, and the first
9014 element is a base coding system. Append ALIAS at the tail of the
9015 list. */
df7492f9
KH		 9016 while (!NILP (XCDR (aliases)))
9017 aliases = XCDR (aliases);
8f924df7 9018 XSETCDR (aliases, Fcons (alias, Qnil));
4ed46869 9019
df7492f9
KH		 9020 eol_type = AREF (spec, 2);
9021 if (VECTORP (eol_type))
4ed46869 9022 {
df7492f9
KH		 9023 Lisp_Object subsidiaries;
9024 int i;
4ed46869 9025
df7492f9
KH		 9026 subsidiaries = make_subsidiaries (alias);
9027 for (i = 0; i < 3; i++)
9028 Fdefine_coding_system_alias (AREF (subsidiaries, i),
9029 AREF (eol_type, i));
4ed46869 9030 }
df7492f9
KH		 9031
9032 Fputhash (alias, spec, Vcoding_system_hash_table);
d6925f38 9033 Vcoding_system_list = Fcons (alias, Vcoding_system_list);
5bad0796
DL		 9034 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (alias), Qnil),
9035 Vcoding_system_alist);
66cfb530 9036
4ed46869
KH		 9037 return Qnil;
9038}
9039
df7492f9
KH		 9040DEFUN ("coding-system-base", Fcoding_system_base, Scoding_system_base,
9041 1, 1, 0,
9042 doc: /* Return the base of CODING-SYSTEM.
da7db224 9043Any alias or subsidiary coding system is not a base coding system. */)
df7492f9
KH		 9044 (coding_system)
9045 Lisp_Object coding_system;
d46c5b12 9046{
df7492f9 9047 Lisp_Object spec, attrs;
d46c5b12 9048
df7492f9
KH		 9049 if (NILP (coding_system))
9050 return (Qno_conversion);
9051 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9052 attrs = AREF (spec, 0);
9053 return CODING_ATTR_BASE_NAME (attrs);
9054}
1397dc18 9055
df7492f9
KH		 9056DEFUN ("coding-system-plist", Fcoding_system_plist, Scoding_system_plist,
9057 1, 1, 0,
9058 doc: "Return the property list of CODING-SYSTEM.")
9059 (coding_system)
9060 Lisp_Object coding_system;
9061{
9062 Lisp_Object spec, attrs;
1397dc18 9063
df7492f9
KH		 9064 if (NILP (coding_system))
9065 coding_system = Qno_conversion;
9066 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9067 attrs = AREF (spec, 0);
9068 return CODING_ATTR_PLIST (attrs);
d46c5b12
KH		 9069}
9070
df7492f9
KH		 9071
9072DEFUN ("coding-system-aliases", Fcoding_system_aliases, Scoding_system_aliases,
9073 1, 1, 0,
da7db224 9074 doc: /* Return the list of aliases of CODING-SYSTEM. */)
df7492f9
KH		 9075 (coding_system)
9076 Lisp_Object coding_system;
66cfb530 9077{
df7492f9 9078 Lisp_Object spec;
84d60297 9079
df7492f9
KH		 9080 if (NILP (coding_system))
9081 coding_system = Qno_conversion;
9082 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
da7db224 9083 return AREF (spec, 1);
df7492f9 9084}
66cfb530 9085
df7492f9
KH		 9086DEFUN ("coding-system-eol-type", Fcoding_system_eol_type,
9087 Scoding_system_eol_type, 1, 1, 0,
9088 doc: /* Return eol-type of CODING-SYSTEM.
9089An eol-type is integer 0, 1, 2, or a vector of coding systems.
66cfb530 9090
df7492f9
KH		 9091Integer values 0, 1, and 2 indicate a format of end-of-line; LF, CRLF,
9092and CR respectively.
66cfb530 9093
df7492f9
KH		 9094A vector value indicates that a format of end-of-line should be
9095detected automatically. Nth element of the vector is the subsidiary
9096coding system whose eol-type is N. */)
6b89e3aa
KH		 9097 (coding_system)
9098 Lisp_Object coding_system;
9099{
df7492f9
KH		 9100 Lisp_Object spec, eol_type;
9101 int n;
6b89e3aa 9102
df7492f9
KH		 9103 if (NILP (coding_system))
9104 coding_system = Qno_conversion;
9105 if (! CODING_SYSTEM_P (coding_system))
9106 return Qnil;
9107 spec = CODING_SYSTEM_SPEC (coding_system);
9108 eol_type = AREF (spec, 2);
9109 if (VECTORP (eol_type))
9110 return Fcopy_sequence (eol_type);
9111 n = EQ (eol_type, Qunix) ? 0 : EQ (eol_type, Qdos) ? 1 : 2;
9112 return make_number (n);
6b89e3aa
KH		 9113}
9114
4ed46869
KH		 9115#endif /* emacs */
9116
9117\f
1397dc18 9118/*** 9. Post-amble ***/
4ed46869 9119
dfcf069d 9120void
4ed46869
KH		 9121init_coding_once ()
9122{
9123 int i;
9124
df7492f9
KH		 9125 for (i = 0; i < coding_category_max; i++)
9126 {
9127 coding_categories[i].id = -1;
9128 coding_priorities[i] = i;
9129 }
4ed46869
KH		 9130
9131 /* ISO2022 specific initialize routine. */
9132 for (i = 0; i < 0x20; i++)
b73bfc1c 9133 iso_code_class[i] = ISO_control_0;
4ed46869
KH		 9134 for (i = 0x21; i < 0x7F; i++)
9135 iso_code_class[i] = ISO_graphic_plane_0;
9136 for (i = 0x80; i < 0xA0; i++)
b73bfc1c 9137 iso_code_class[i] = ISO_control_1;
4ed46869
KH		 9138 for (i = 0xA1; i < 0xFF; i++)
9139 iso_code_class[i] = ISO_graphic_plane_1;
9140 iso_code_class[0x20] = iso_code_class[0x7F] = ISO_0x20_or_0x7F;
9141 iso_code_class[0xA0] = iso_code_class[0xFF] = ISO_0xA0_or_0xFF;
4ed46869
KH		 9142 iso_code_class[ISO_CODE_SO] = ISO_shift_out;
9143 iso_code_class[ISO_CODE_SI] = ISO_shift_in;
9144 iso_code_class[ISO_CODE_SS2_7] = ISO_single_shift_2_7;
9145 iso_code_class[ISO_CODE_ESC] = ISO_escape;
9146 iso_code_class[ISO_CODE_SS2] = ISO_single_shift_2;
9147 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3;
9148 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer;
9149
df7492f9
KH		 9150 for (i = 0; i < 256; i++)
9151 {
9152 emacs_mule_bytes[i] = 1;
9153 }
7c78e542
KH		 9154 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_11] = 3;
9155 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_12] = 3;
9156 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_21] = 4;
9157 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_22] = 4;
e0e989f6
KH		 9158}
9159
9160#ifdef emacs
9161
dfcf069d 9162void
e0e989f6
KH		 9163syms_of_coding ()
9164{
df7492f9 9165 staticpro (&Vcoding_system_hash_table);
8f924df7
KH		 9166 {
9167 Lisp_Object args[2];
9168 args[0] = QCtest;
9169 args[1] = Qeq;
9170 Vcoding_system_hash_table = Fmake_hash_table (2, args);
9171 }
df7492f9
KH		 9172
9173 staticpro (&Vsjis_coding_system);
9174 Vsjis_coding_system = Qnil;
e0e989f6 9175
df7492f9
KH		 9176 staticpro (&Vbig5_coding_system);
9177 Vbig5_coding_system = Qnil;
9178
24a73b0a
KH		 9179 staticpro (&Vcode_conversion_reused_workbuf);
9180 Vcode_conversion_reused_workbuf = Qnil;
9181
9182 staticpro (&Vcode_conversion_workbuf_name);
9183 Vcode_conversion_workbuf_name = build_string (" *code-conversion-work*");
e0e989f6 9184
24a73b0a 9185 reused_workbuf_in_use = 0;
df7492f9
KH		 9186
9187 DEFSYM (Qcharset, "charset");
9188 DEFSYM (Qtarget_idx, "target-idx");
9189 DEFSYM (Qcoding_system_history, "coding-system-history");
bb0115a2
RS		 9190 Fset (Qcoding_system_history, Qnil);
9191
9ce27fde 9192 /* Target FILENAME is the first argument. */
e0e989f6 9193 Fput (Qinsert_file_contents, Qtarget_idx, make_number (0));
9ce27fde 9194 /* Target FILENAME is the third argument. */
e0e989f6
KH		 9195 Fput (Qwrite_region, Qtarget_idx, make_number (2));
9196
df7492f9 9197 DEFSYM (Qcall_process, "call-process");
9ce27fde 9198 /* Target PROGRAM is the first argument. */
e0e989f6
KH		 9199 Fput (Qcall_process, Qtarget_idx, make_number (0));
9200
df7492f9 9201 DEFSYM (Qcall_process_region, "call-process-region");
9ce27fde 9202 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 9203 Fput (Qcall_process_region, Qtarget_idx, make_number (2));
9204
df7492f9 9205 DEFSYM (Qstart_process, "start-process");
9ce27fde 9206 /* Target PROGRAM is the third argument. */
e0e989f6
KH		 9207 Fput (Qstart_process, Qtarget_idx, make_number (2));
9208
df7492f9 9209 DEFSYM (Qopen_network_stream, "open-network-stream");
9ce27fde 9210 /* Target SERVICE is the fourth argument. */
e0e989f6
KH		 9211 Fput (Qopen_network_stream, Qtarget_idx, make_number (3));
9212
df7492f9
KH		 9213 DEFSYM (Qcoding_system, "coding-system");
9214 DEFSYM (Qcoding_aliases, "coding-aliases");
4ed46869 9215
df7492f9
KH		 9216 DEFSYM (Qeol_type, "eol-type");
9217 DEFSYM (Qunix, "unix");
9218 DEFSYM (Qdos, "dos");
4ed46869 9219
df7492f9
KH		 9220 DEFSYM (Qbuffer_file_coding_system, "buffer-file-coding-system");
9221 DEFSYM (Qpost_read_conversion, "post-read-conversion");
9222 DEFSYM (Qpre_write_conversion, "pre-write-conversion");
9223 DEFSYM (Qdefault_char, "default-char");
9224 DEFSYM (Qundecided, "undecided");
9225 DEFSYM (Qno_conversion, "no-conversion");
9226 DEFSYM (Qraw_text, "raw-text");
4ed46869 9227
df7492f9 9228 DEFSYM (Qiso_2022, "iso-2022");
4ed46869 9229
df7492f9 9230 DEFSYM (Qutf_8, "utf-8");
8f924df7 9231 DEFSYM (Qutf_8_emacs, "utf-8-emacs");
27901516 9232
df7492f9 9233 DEFSYM (Qutf_16, "utf-16");
df7492f9
KH		 9234 DEFSYM (Qbig, "big");
9235 DEFSYM (Qlittle, "little");
27901516 9236
df7492f9
KH		 9237 DEFSYM (Qshift_jis, "shift-jis");
9238 DEFSYM (Qbig5, "big5");
4ed46869 9239
df7492f9 9240 DEFSYM (Qcoding_system_p, "coding-system-p");
4ed46869 9241
df7492f9 9242 DEFSYM (Qcoding_system_error, "coding-system-error");
4ed46869
KH		 9243 Fput (Qcoding_system_error, Qerror_conditions,
9244 Fcons (Qcoding_system_error, Fcons (Qerror, Qnil)));
9245 Fput (Qcoding_system_error, Qerror_message,
9ce27fde 9246 build_string ("Invalid coding system"));
4ed46869 9247
05e6f5dc
KH		 9248 /* Intern this now in case it isn't already done.
9249 Setting this variable twice is harmless.
9250 But don't staticpro it here--that is done in alloc.c. */
9251 Qchar_table_extra_slots = intern ("char-table-extra-slots");
70c22245 9252
df7492f9 9253 DEFSYM (Qtranslation_table, "translation-table");
433f7f87 9254 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (2));
df7492f9
KH		 9255 DEFSYM (Qtranslation_table_id, "translation-table-id");
9256 DEFSYM (Qtranslation_table_for_decode, "translation-table-for-decode");
9257 DEFSYM (Qtranslation_table_for_encode, "translation-table-for-encode");
1397dc18 9258
df7492f9 9259 DEFSYM (Qvalid_codes, "valid-codes");
9ce27fde 9260
df7492f9 9261 DEFSYM (Qemacs_mule, "emacs-mule");
d46c5b12 9262
01378f49 9263 DEFSYM (QCcategory, ":category");
a6f87d34
KH		 9264 DEFSYM (QCmnemonic, ":mnemonic");
9265 DEFSYM (QCdefalut_char, ":default-char");
9266 DEFSYM (QCdecode_translation_table, ":decode-translation-table");
9267 DEFSYM (QCencode_translation_table, ":encode-translation-table");
9268 DEFSYM (QCpost_read_conversion, ":post-read-conversion");
9269 DEFSYM (QCpre_write_conversion, ":pre-write-conversion");
35befdaa 9270 DEFSYM (QCascii_compatible_p, ":ascii-compatible-p");
01378f49 9271
df7492f9
KH		 9272 Vcoding_category_table
9273 = Fmake_vector (make_number (coding_category_max), Qnil);
9274 staticpro (&Vcoding_category_table);
9275 /* Followings are target of code detection. */
9276 ASET (Vcoding_category_table, coding_category_iso_7,
9277 intern ("coding-category-iso-7"));
9278 ASET (Vcoding_category_table, coding_category_iso_7_tight,
9279 intern ("coding-category-iso-7-tight"));
9280 ASET (Vcoding_category_table, coding_category_iso_8_1,
9281 intern ("coding-category-iso-8-1"));
9282 ASET (Vcoding_category_table, coding_category_iso_8_2,
9283 intern ("coding-category-iso-8-2"));
9284 ASET (Vcoding_category_table, coding_category_iso_7_else,
9285 intern ("coding-category-iso-7-else"));
9286 ASET (Vcoding_category_table, coding_category_iso_8_else,
9287 intern ("coding-category-iso-8-else"));
9288 ASET (Vcoding_category_table, coding_category_utf_8,
9289 intern ("coding-category-utf-8"));
9290 ASET (Vcoding_category_table, coding_category_utf_16_be,
9291 intern ("coding-category-utf-16-be"));
ff563fce
KH		 9292 ASET (Vcoding_category_table, coding_category_utf_16_auto,
9293 intern ("coding-category-utf-16-auto"));
df7492f9
KH		 9294 ASET (Vcoding_category_table, coding_category_utf_16_le,
9295 intern ("coding-category-utf-16-le"));
9296 ASET (Vcoding_category_table, coding_category_utf_16_be_nosig,
9297 intern ("coding-category-utf-16-be-nosig"));
9298 ASET (Vcoding_category_table, coding_category_utf_16_le_nosig,
9299 intern ("coding-category-utf-16-le-nosig"));
9300 ASET (Vcoding_category_table, coding_category_charset,
9301 intern ("coding-category-charset"));
9302 ASET (Vcoding_category_table, coding_category_sjis,
9303 intern ("coding-category-sjis"));
9304 ASET (Vcoding_category_table, coding_category_big5,
9305 intern ("coding-category-big5"));
9306 ASET (Vcoding_category_table, coding_category_ccl,
9307 intern ("coding-category-ccl"));
9308 ASET (Vcoding_category_table, coding_category_emacs_mule,
9309 intern ("coding-category-emacs-mule"));
9310 /* Followings are NOT target of code detection. */
9311 ASET (Vcoding_category_table, coding_category_raw_text,
9312 intern ("coding-category-raw-text"));
9313 ASET (Vcoding_category_table, coding_category_undecided,
9314 intern ("coding-category-undecided"));
ecf488bc 9315
065e3595
KH		 9316 DEFSYM (Qinsufficient_source, "insufficient-source");
9317 DEFSYM (Qinconsistent_eol, "inconsistent-eol");
9318 DEFSYM (Qinvalid_source, "invalid-source");
9319 DEFSYM (Qinterrupted, "interrupted");
9320 DEFSYM (Qinsufficient_memory, "insufficient-memory");
9321
4ed46869
KH		 9322 defsubr (&Scoding_system_p);
9323 defsubr (&Sread_coding_system);
9324 defsubr (&Sread_non_nil_coding_system);
9325 defsubr (&Scheck_coding_system);
9326 defsubr (&Sdetect_coding_region);
d46c5b12 9327 defsubr (&Sdetect_coding_string);
05e6f5dc 9328 defsubr (&Sfind_coding_systems_region_internal);
068a9dbd 9329 defsubr (&Sunencodable_char_position);
df7492f9 9330 defsubr (&Scheck_coding_systems_region);
4ed46869
KH		 9331 defsubr (&Sdecode_coding_region);
9332 defsubr (&Sencode_coding_region);
9333 defsubr (&Sdecode_coding_string);
9334 defsubr (&Sencode_coding_string);
9335 defsubr (&Sdecode_sjis_char);
9336 defsubr (&Sencode_sjis_char);
9337 defsubr (&Sdecode_big5_char);
9338 defsubr (&Sencode_big5_char);
1ba9e4ab 9339 defsubr (&Sset_terminal_coding_system_internal);
c4825358 9340 defsubr (&Sset_safe_terminal_coding_system_internal);
4ed46869 9341 defsubr (&Sterminal_coding_system);
1ba9e4ab 9342 defsubr (&Sset_keyboard_coding_system_internal);
4ed46869 9343 defsubr (&Skeyboard_coding_system);
a5d301df 9344 defsubr (&Sfind_operation_coding_system);
df7492f9 9345 defsubr (&Sset_coding_system_priority);
6b89e3aa 9346 defsubr (&Sdefine_coding_system_internal);
df7492f9 9347 defsubr (&Sdefine_coding_system_alias);
a6f87d34 9348 defsubr (&Scoding_system_put);
df7492f9
KH		 9349 defsubr (&Scoding_system_base);
9350 defsubr (&Scoding_system_plist);
9351 defsubr (&Scoding_system_aliases);
9352 defsubr (&Scoding_system_eol_type);
9353 defsubr (&Scoding_system_priority_list);
4ed46869 9354
4608c386 9355 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list,
48b0f3ae
PJ		 9356 doc: /* List of coding systems.
9357
9358Do not alter the value of this variable manually. This variable should be
df7492f9 9359updated by the functions `define-coding-system' and
48b0f3ae 9360`define-coding-system-alias'. */);
4608c386
KH		 9361 Vcoding_system_list = Qnil;
9362
9363 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist,
48b0f3ae
PJ		 9364 doc: /* Alist of coding system names.
9365Each element is one element list of coding system name.
9366This variable is given to `completing-read' as TABLE argument.
9367
9368Do not alter the value of this variable manually. This variable should be
9369updated by the functions `make-coding-system' and
9370`define-coding-system-alias'. */);
4608c386
KH		 9371 Vcoding_system_alist = Qnil;
9372
4ed46869 9373 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list,
48b0f3ae
PJ		 9374 doc: /* List of coding-categories (symbols) ordered by priority.
9375
9376On detecting a coding system, Emacs tries code detection algorithms
9377associated with each coding-category one by one in this order. When
9378one algorithm agrees with a byte sequence of source text, the coding
0ec31faf
KH		 9379system bound to the corresponding coding-category is selected.
9380
42205607 9381Don't modify this variable directly, but use `set-coding-priority'. */);
4ed46869
KH		 9382 {
9383 int i;
9384
9385 Vcoding_category_list = Qnil;
df7492f9 9386 for (i = coding_category_max - 1; i >= 0; i--)
4ed46869 9387 Vcoding_category_list
d46c5b12
KH		 9388 = Fcons (XVECTOR (Vcoding_category_table)->contents[i],
9389 Vcoding_category_list);
4ed46869
KH		 9390 }
9391
9392 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read,
48b0f3ae
PJ		 9393 doc: /* Specify the coding system for read operations.
9394It is useful to bind this variable with `let', but do not set it globally.
9395If the value is a coding system, it is used for decoding on read operation.
9396If not, an appropriate element is used from one of the coding system alists:
9397There are three such tables, `file-coding-system-alist',
9398`process-coding-system-alist', and `network-coding-system-alist'. */);
4ed46869
KH		 9399 Vcoding_system_for_read = Qnil;
9400
9401 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write,
48b0f3ae
PJ		 9402 doc: /* Specify the coding system for write operations.
9403Programs bind this variable with `let', but you should not set it globally.
9404If the value is a coding system, it is used for encoding of output,
9405when writing it to a file and when sending it to a file or subprocess.
9406
9407If this does not specify a coding system, an appropriate element
9408is used from one of the coding system alists:
9409There are three such tables, `file-coding-system-alist',
9410`process-coding-system-alist', and `network-coding-system-alist'.
9411For output to files, if the above procedure does not specify a coding system,
9412the value of `buffer-file-coding-system' is used. */);
4ed46869
KH		 9413 Vcoding_system_for_write = Qnil;
9414
9415 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used,
df7492f9
KH		 9416 doc: /*
9417Coding system used in the latest file or process I/O. */);
4ed46869
KH		 9418 Vlast_coding_system_used = Qnil;
9419
065e3595
KH		 9420 DEFVAR_LISP ("last-code-conversion-error", &Vlast_code_conversion_error,
9421 doc: /*
9422Error status of the last code conversion.
9423
9424When an error was detected in the last code conversion, this variable
9425is set to one of the following symbols.
9426 `insufficient-source'
9427 `inconsistent-eol'
9428 `invalid-source'
9429 `interrupted'
9430 `insufficient-memory'
9431When no error was detected, the value doesn't change. So, to check
9432the error status of a code conversion by this variable, you must
9433explicitly set this variable to nil before performing code
9434conversion. */);
9435 Vlast_code_conversion_error = Qnil;
9436
9ce27fde 9437 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion,
df7492f9
KH		 9438 doc: /*
9439*Non-nil means always inhibit code conversion of end-of-line format.
48b0f3ae
PJ		 9440See info node `Coding Systems' and info node `Text and Binary' concerning
9441such conversion. */);
9ce27fde
KH		 9442 inhibit_eol_conversion = 0;
9443
ed29121d 9444 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system,
df7492f9
KH		 9445 doc: /*
9446Non-nil means process buffer inherits coding system of process output.
48b0f3ae
PJ		 9447Bind it to t if the process output is to be treated as if it were a file
9448read from some filesystem. */);
ed29121d
EZ		 9449 inherit_process_coding_system = 0;
9450
02ba4723 9451 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist,
df7492f9
KH		 9452 doc: /*
9453Alist to decide a coding system to use for a file I/O operation.
48b0f3ae
PJ		 9454The format is ((PATTERN . VAL) ...),
9455where PATTERN is a regular expression matching a file name,
9456VAL is a coding system, a cons of coding systems, or a function symbol.
9457If VAL is a coding system, it is used for both decoding and encoding
9458the file contents.
9459If VAL is a cons of coding systems, the car part is used for decoding,
9460and the cdr part is used for encoding.
9461If VAL is a function symbol, the function must return a coding system
0192762c
DL		 9462or a cons of coding systems which are used as above. The function gets
9463the arguments with which `find-operation-coding-systems' was called.
48b0f3ae
PJ		 9464
9465See also the function `find-operation-coding-system'
9466and the variable `auto-coding-alist'. */);
02ba4723
KH		 9467 Vfile_coding_system_alist = Qnil;
9468
9469 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist,
df7492f9
KH		 9470 doc: /*
9471Alist to decide a coding system to use for a process I/O operation.
48b0f3ae
PJ		 9472The format is ((PATTERN . VAL) ...),
9473where PATTERN is a regular expression matching a program name,
9474VAL is a coding system, a cons of coding systems, or a function symbol.
9475If VAL is a coding system, it is used for both decoding what received
9476from the program and encoding what sent to the program.
9477If VAL is a cons of coding systems, the car part is used for decoding,
9478and the cdr part is used for encoding.
9479If VAL is a function symbol, the function must return a coding system
9480or a cons of coding systems which are used as above.
9481
9482See also the function `find-operation-coding-system'. */);
02ba4723
KH		 9483 Vprocess_coding_system_alist = Qnil;
9484
9485 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist,
df7492f9
KH		 9486 doc: /*
9487Alist to decide a coding system to use for a network I/O operation.
48b0f3ae
PJ		 9488The format is ((PATTERN . VAL) ...),
9489where PATTERN is a regular expression matching a network service name
9490or is a port number to connect to,
9491VAL is a coding system, a cons of coding systems, or a function symbol.
9492If VAL is a coding system, it is used for both decoding what received
9493from the network stream and encoding what sent to the network stream.
9494If VAL is a cons of coding systems, the car part is used for decoding,
9495and the cdr part is used for encoding.
9496If VAL is a function symbol, the function must return a coding system
9497or a cons of coding systems which are used as above.
9498
9499See also the function `find-operation-coding-system'. */);
02ba4723 9500 Vnetwork_coding_system_alist = Qnil;
4ed46869 9501
68c45bf0 9502 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system,
75205970
RS		 9503 doc: /* Coding system to use with system messages.
9504Also used for decoding keyboard input on X Window system. */);
68c45bf0
PE		 9505 Vlocale_coding_system = Qnil;
9506
005f0d35 9507 /* The eol mnemonics are reset in startup.el system-dependently. */
7722baf9 9508 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix,
df7492f9
KH		 9509 doc: /*
9510*String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7722baf9 9511 eol_mnemonic_unix = build_string (":");
4ed46869 9512
7722baf9 9513 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos,
df7492f9
KH		 9514 doc: /*
9515*String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7722baf9 9516 eol_mnemonic_dos = build_string ("\\");
4ed46869 9517
7722baf9 9518 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac,
df7492f9
KH		 9519 doc: /*
9520*String displayed in mode line for MAC-like (CR) end-of-line format. */);
7722baf9 9521 eol_mnemonic_mac = build_string ("/");
4ed46869 9522
7722baf9 9523 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided,
df7492f9
KH		 9524 doc: /*
9525*String displayed in mode line when end-of-line format is not yet determined. */);
7722baf9 9526 eol_mnemonic_undecided = build_string (":");
4ed46869 9527
84fbb8a0 9528 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation,
df7492f9
KH		 9529 doc: /*
9530*Non-nil enables character translation while encoding and decoding. */);
84fbb8a0 9531 Venable_character_translation = Qt;
bdd9fb48 9532
f967223b 9533 DEFVAR_LISP ("standard-translation-table-for-decode",
48b0f3ae
PJ		 9534 &Vstandard_translation_table_for_decode,
9535 doc: /* Table for translating characters while decoding. */);
f967223b 9536 Vstandard_translation_table_for_decode = Qnil;
bdd9fb48 9537
f967223b 9538 DEFVAR_LISP ("standard-translation-table-for-encode",
48b0f3ae
PJ		 9539 &Vstandard_translation_table_for_encode,
9540 doc: /* Table for translating characters while encoding. */);
f967223b 9541 Vstandard_translation_table_for_encode = Qnil;
4ed46869 9542
df7492f9 9543 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_table,
48b0f3ae
PJ		 9544 doc: /* Alist of charsets vs revision numbers.
9545While encoding, if a charset (car part of an element) is found,
df7492f9
KH		 9546designate it with the escape sequence identifying revision (cdr part
9547of the element). */);
9548 Vcharset_revision_table = Qnil;
02ba4723
KH		 9549
9550 DEFVAR_LISP ("default-process-coding-system",
9551 &Vdefault_process_coding_system,
48b0f3ae
PJ		 9552 doc: /* Cons of coding systems used for process I/O by default.
9553The car part is used for decoding a process output,
9554the cdr part is used for encoding a text to be sent to a process. */);
02ba4723 9555 Vdefault_process_coding_system = Qnil;
c4825358 9556
3f003981 9557 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table,
df7492f9
KH		 9558 doc: /*
9559Table of extra Latin codes in the range 128..159 (inclusive).
48b0f3ae
PJ		 9560This is a vector of length 256.
9561If Nth element is non-nil, the existence of code N in a file
9562\(or output of subprocess) doesn't prevent it to be detected as
9563a coding system of ISO 2022 variant which has a flag
9564`accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
9565or reading output of a subprocess.
9566Only 128th through 159th elements has a meaning. */);
3f003981 9567 Vlatin_extra_code_table = Fmake_vector (make_number (256), Qnil);
d46c5b12
KH		 9568
9569 DEFVAR_LISP ("select-safe-coding-system-function",
9570 &Vselect_safe_coding_system_function,
df7492f9
KH		 9571 doc: /*
9572Function to call to select safe coding system for encoding a text.
48b0f3ae
PJ		 9573
9574If set, this function is called to force a user to select a proper
9575coding system which can encode the text in the case that a default
9576coding system used in each operation can't encode the text.
9577
9578The default value is `select-safe-coding-system' (which see). */);
d46c5b12
KH		 9579 Vselect_safe_coding_system_function = Qnil;
9580
5d5bf4d8
KH		 9581 DEFVAR_BOOL ("coding-system-require-warning",
9582 &coding_system_require_warning,
9583 doc: /* Internal use only.
6b89e3aa
KH		 9584If non-nil, on writing a file, `select-safe-coding-system-function' is
9585called even if `coding-system-for-write' is non-nil. The command
9586`universal-coding-system-argument' binds this variable to t temporarily. */);
5d5bf4d8
KH		 9587 coding_system_require_warning = 0;
9588
9589
22ab2303 9590 DEFVAR_BOOL ("inhibit-iso-escape-detection",
74383408 9591 &inhibit_iso_escape_detection,
df7492f9
KH		 9592 doc: /*
9593If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
48b0f3ae
PJ		 9594
9595By default, on reading a file, Emacs tries to detect how the text is
9596encoded. This code detection is sensitive to escape sequences. If
9597the sequence is valid as ISO2022, the code is determined as one of
9598the ISO2022 encodings, and the file is decoded by the corresponding
9599coding system (e.g. `iso-2022-7bit').
9600
9601However, there may be a case that you want to read escape sequences in
9602a file as is. In such a case, you can set this variable to non-nil.
9603Then, as the code detection ignores any escape sequences, no file is
9604detected as encoded in some ISO2022 encoding. The result is that all
9605escape sequences become visible in a buffer.
9606
9607The default value is nil, and it is strongly recommended not to change
9608it. That is because many Emacs Lisp source files that contain
9609non-ASCII characters are encoded by the coding system `iso-2022-7bit'
9610in Emacs's distribution, and they won't be decoded correctly on
9611reading if you suppress escape sequence detection.
9612
9613The other way to read escape sequences in a file without decoding is
9614to explicitly specify some coding system that doesn't use ISO2022's
9615escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
74383408 9616 inhibit_iso_escape_detection = 0;
002fdb44
DL		 9617
9618 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input,
15c8f9d1
DL		 9619 doc: /* Char table for translating self-inserting characters.
9620This is applied to the result of input methods, not their input. See also
9621`keyboard-translate-table'. */);
002fdb44 9622 Vtranslation_table_for_input = Qnil;
8f924df7 9623
2c78b7e1
KH		 9624 {
9625 Lisp_Object args[coding_arg_max];
8f924df7 9626 Lisp_Object plist[16];
2c78b7e1
KH		 9627 int i;
9628
9629 for (i = 0; i < coding_arg_max; i++)
9630 args[i] = Qnil;
9631
9632 plist[0] = intern (":name");
9633 plist[1] = args[coding_arg_name] = Qno_conversion;
9634 plist[2] = intern (":mnemonic");
9635 plist[3] = args[coding_arg_mnemonic] = make_number ('=');
9636 plist[4] = intern (":coding-type");
9637 plist[5] = args[coding_arg_coding_type] = Qraw_text;
9638 plist[6] = intern (":ascii-compatible-p");
9639 plist[7] = args[coding_arg_ascii_compatible_p] = Qt;
9640 plist[8] = intern (":default-char");
9641 plist[9] = args[coding_arg_default_char] = make_number (0);
8f924df7
KH		 9642 plist[10] = intern (":for-unibyte");
9643 plist[11] = args[coding_arg_for_unibyte] = Qt;
9644 plist[12] = intern (":docstring");
9645 plist[13] = build_string ("Do no conversion.\n\
2c78b7e1
KH		 9646\n\
9647When you visit a file with this coding, the file is read into a\n\
9648unibyte buffer as is, thus each byte of a file is treated as a\n\
9649character.");
8f924df7
KH		 9650 plist[14] = intern (":eol-type");
9651 plist[15] = args[coding_arg_eol_type] = Qunix;
9652 args[coding_arg_plist] = Flist (16, plist);
2c78b7e1 9653 Fdefine_coding_system_internal (coding_arg_max, args);
ae6f73fa
KH		 9654
9655 plist[1] = args[coding_arg_name] = Qundecided;
9656 plist[3] = args[coding_arg_mnemonic] = make_number ('-');
9657 plist[5] = args[coding_arg_coding_type] = Qundecided;
9658 /* This is already set.
35befdaa 9659 plist[7] = args[coding_arg_ascii_compatible_p] = Qt; */
ae6f73fa
KH		 9660 plist[8] = intern (":charset-list");
9661 plist[9] = args[coding_arg_charset_list] = Fcons (Qascii, Qnil);
9662 plist[11] = args[coding_arg_for_unibyte] = Qnil;
9663 plist[13] = build_string ("No conversion on encoding, automatic conversion on decoding.");
9664 plist[15] = args[coding_arg_eol_type] = Qnil;
9665 args[coding_arg_plist] = Flist (16, plist);
9666 Fdefine_coding_system_internal (coding_arg_max, args);
2c78b7e1
KH		 9667 }
9668
9669 setup_coding_system (Qno_conversion, &keyboard_coding);
ae6f73fa 9670 setup_coding_system (Qundecided, &terminal_coding);
2c78b7e1 9671 setup_coding_system (Qno_conversion, &safe_terminal_coding);
ff563fce
KH		 9672
9673 {
9674 int i;
9675
9676 for (i = 0; i < coding_category_max; i++)
9677 Fset (AREF (Vcoding_category_table, i), Qno_conversion);
9678 }
4ed46869
KH		 9679}
9680
68c45bf0
PE		 9681char *
9682emacs_strerror (error_number)
9683 int error_number;
9684{
9685 char *str;
9686
ca9c0567 9687 synchronize_system_messages_locale ();
68c45bf0
PE		 9688 str = strerror (error_number);
9689
9690 if (! NILP (Vlocale_coding_system))
9691 {
9692 Lisp_Object dec = code_convert_string_norecord (build_string (str),
9693 Vlocale_coding_system,
9694 0);
d5db4077 9695 str = (char *) SDATA (dec);
68c45bf0
PE		 9696 }
9697
9698 return str;
9699}
9700
4ed46869 9701#endif /* emacs */
9ffd559c
KH		 9702
9703/* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
9704 (do not change this comment) */
Unnamed repository; edit this file 'description' to name the repository.