(merge_named_face): GCPRO the face_name in the
[bpt/emacs.git] / src / coding.c
CommitLineData
4ed46869 1/* Coding system handler (conversion, detection, and etc).
f1ce3dcf 2 Copyright (C) 1995,97,1998,2002,2003 Electrotechnical Laboratory, JAPAN.
203cb916 3 Licensed to the Free Software Foundation.
58f99379 4 Copyright (C) 2001,2002,2003 Free Software Foundation, Inc.
4ed46869 5
369314dc
KH
6This file is part of GNU Emacs.
7
8GNU Emacs is free software; you can redistribute it and/or modify
9it under the terms of the GNU General Public License as published by
10the Free Software Foundation; either version 2, or (at your option)
11any later version.
4ed46869 12
369314dc
KH
13GNU Emacs is distributed in the hope that it will be useful,
14but WITHOUT ANY WARRANTY; without even the implied warranty of
15MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16GNU General Public License for more details.
4ed46869 17
369314dc
KH
18You should have received a copy of the GNU General Public License
19along with GNU Emacs; see the file COPYING. If not, write to
20the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
21Boston, MA 02111-1307, USA. */
4ed46869
KH
22
23/*** TABLE OF CONTENTS ***
24
b73bfc1c 25 0. General comments
4ed46869 26 1. Preamble
0ef69138 27 2. Emacs' internal format (emacs-mule) handlers
4ed46869
KH
28 3. ISO2022 handlers
29 4. Shift-JIS and BIG5 handlers
1397dc18
KH
30 5. CCL handlers
31 6. End-of-line handlers
32 7. C library functions
33 8. Emacs Lisp library functions
34 9. Post-amble
4ed46869
KH
35
36*/
37
b73bfc1c
KH
38/*** 0. General comments ***/
39
40
cfb43547 41/*** GENERAL NOTE on CODING SYSTEMS ***
4ed46869 42
cfb43547 43 A coding system is an encoding mechanism for one or more character
4ed46869
KH
44 sets. Here's a list of coding systems which Emacs can handle. When
45 we say "decode", it means converting some other coding system to
cfb43547 46 Emacs' internal format (emacs-mule), and when we say "encode",
0ef69138
KH
47 it means converting the coding system emacs-mule to some other
48 coding system.
4ed46869 49
0ef69138 50 0. Emacs' internal format (emacs-mule)
4ed46869 51
cfb43547 52 Emacs itself holds a multi-lingual character in buffers and strings
f4dee582 53 in a special format. Details are described in section 2.
4ed46869
KH
54
55 1. ISO2022
56
57 The most famous coding system for multiple character sets. X's
f4dee582
RS
58 Compound Text, various EUCs (Extended Unix Code), and coding
59 systems used in Internet communication such as ISO-2022-JP are
60 all variants of ISO2022. Details are described in section 3.
4ed46869
KH
61
62 2. SJIS (or Shift-JIS or MS-Kanji-Code)
93dec019 63
4ed46869
KH
64 A coding system to encode character sets: ASCII, JISX0201, and
65 JISX0208. Widely used for PC's in Japan. Details are described in
f4dee582 66 section 4.
4ed46869
KH
67
68 3. BIG5
69
cfb43547
DL
70 A coding system to encode the character sets ASCII and Big5. Widely
71 used for Chinese (mainly in Taiwan and Hong Kong). Details are
f4dee582
RS
72 described in section 4. In this file, when we write "BIG5"
73 (all uppercase), we mean the coding system, and when we write
74 "Big5" (capitalized), we mean the character set.
4ed46869 75
27901516
KH
76 4. Raw text
77
cfb43547
DL
78 A coding system for text containing random 8-bit code. Emacs does
79 no code conversion on such text except for end-of-line format.
27901516
KH
80
81 5. Other
4ed46869 82
cfb43547
DL
83 If a user wants to read/write text encoded in a coding system not
84 listed above, he can supply a decoder and an encoder for it as CCL
4ed46869
KH
85 (Code Conversion Language) programs. Emacs executes the CCL program
86 while reading/writing.
87
d46c5b12
KH
88 Emacs represents a coding system by a Lisp symbol that has a property
89 `coding-system'. But, before actually using the coding system, the
4ed46869 90 information about it is set in a structure of type `struct
f4dee582 91 coding_system' for rapid processing. See section 6 for more details.
4ed46869
KH
92
93*/
94
95/*** GENERAL NOTES on END-OF-LINE FORMAT ***
96
cfb43547
DL
97 How end-of-line of text is encoded depends on the operating system.
98 For instance, Unix's format is just one byte of `line-feed' code,
f4dee582 99 whereas DOS's format is two-byte sequence of `carriage-return' and
d46c5b12
KH
100 `line-feed' codes. MacOS's format is usually one byte of
101 `carriage-return'.
4ed46869 102
cfb43547
DL
103 Since text character encoding and end-of-line encoding are
104 independent, any coding system described above can have any
105 end-of-line format. So Emacs has information about end-of-line
106 format in each coding-system. See section 6 for more details.
4ed46869
KH
107
108*/
109
110/*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
111
112 These functions check if a text between SRC and SRC_END is encoded
113 in the coding system category XXX. Each returns an integer value in
cfb43547 114 which appropriate flag bits for the category XXX are set. The flag
4ed46869 115 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
cfb43547 116 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
0a28aafb 117 of the range 0x80..0x9F are in multibyte form. */
4ed46869
KH
118#if 0
119int
0a28aafb 120detect_coding_emacs_mule (src, src_end, multibytep)
4ed46869 121 unsigned char *src, *src_end;
0a28aafb 122 int multibytep;
4ed46869
KH
123{
124 ...
125}
126#endif
127
128/*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
129
b73bfc1c
KH
130 These functions decode SRC_BYTES length of unibyte text at SOURCE
131 encoded in CODING to Emacs' internal format. The resulting
132 multibyte text goes to a place pointed to by DESTINATION, the length
133 of which should not exceed DST_BYTES.
d46c5b12 134
cfb43547
DL
135 These functions set the information about original and decoded texts
136 in the members `produced', `produced_char', `consumed', and
137 `consumed_char' of the structure *CODING. They also set the member
138 `result' to one of CODING_FINISH_XXX indicating how the decoding
139 finished.
d46c5b12 140
cfb43547 141 DST_BYTES zero means that the source area and destination area are
d46c5b12 142 overlapped, which means that we can produce a decoded text until it
cfb43547 143 reaches the head of the not-yet-decoded source text.
d46c5b12 144
cfb43547 145 Below is a template for these functions. */
4ed46869 146#if 0
b73bfc1c 147static void
d46c5b12 148decode_coding_XXX (coding, source, destination, src_bytes, dst_bytes)
4ed46869
KH
149 struct coding_system *coding;
150 unsigned char *source, *destination;
151 int src_bytes, dst_bytes;
4ed46869
KH
152{
153 ...
154}
155#endif
156
157/*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
158
cfb43547 159 These functions encode SRC_BYTES length text at SOURCE from Emacs'
b73bfc1c
KH
160 internal multibyte format to CODING. The resulting unibyte text
161 goes to a place pointed to by DESTINATION, the length of which
162 should not exceed DST_BYTES.
d46c5b12 163
cfb43547
DL
164 These functions set the information about original and encoded texts
165 in the members `produced', `produced_char', `consumed', and
166 `consumed_char' of the structure *CODING. They also set the member
167 `result' to one of CODING_FINISH_XXX indicating how the encoding
168 finished.
d46c5b12 169
cfb43547
DL
170 DST_BYTES zero means that the source area and destination area are
171 overlapped, which means that we can produce encoded text until it
172 reaches at the head of the not-yet-encoded source text.
d46c5b12 173
cfb43547 174 Below is a template for these functions. */
4ed46869 175#if 0
b73bfc1c 176static void
d46c5b12 177encode_coding_XXX (coding, source, destination, src_bytes, dst_bytes)
4ed46869
KH
178 struct coding_system *coding;
179 unsigned char *source, *destination;
180 int src_bytes, dst_bytes;
4ed46869
KH
181{
182 ...
183}
184#endif
185
186/*** COMMONLY USED MACROS ***/
187
b73bfc1c
KH
188/* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
189 get one, two, and three bytes from the source text respectively.
190 If there are not enough bytes in the source, they jump to
191 `label_end_of_loop'. The caller should set variables `coding',
192 `src' and `src_end' to appropriate pointer in advance. These
193 macros are called from decoding routines `decode_coding_XXX', thus
194 it is assumed that the source text is unibyte. */
4ed46869 195
b73bfc1c
KH
196#define ONE_MORE_BYTE(c1) \
197 do { \
198 if (src >= src_end) \
199 { \
200 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
201 goto label_end_of_loop; \
202 } \
203 c1 = *src++; \
4ed46869
KH
204 } while (0)
205
b73bfc1c
KH
206#define TWO_MORE_BYTES(c1, c2) \
207 do { \
208 if (src + 1 >= src_end) \
209 { \
210 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
211 goto label_end_of_loop; \
212 } \
213 c1 = *src++; \
214 c2 = *src++; \
4ed46869
KH
215 } while (0)
216
4ed46869 217
0a28aafb
KH
218/* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
219 form if MULTIBYTEP is nonzero. */
220
221#define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep) \
222 do { \
223 if (src >= src_end) \
224 { \
225 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
226 goto label_end_of_loop; \
227 } \
228 c1 = *src++; \
229 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
230 c1 = *src++ - 0x20; \
231 } while (0)
232
b73bfc1c
KH
233/* Set C to the next character at the source text pointed by `src'.
234 If there are not enough characters in the source, jump to
235 `label_end_of_loop'. The caller should set variables `coding'
236 `src', `src_end', and `translation_table' to appropriate pointers
237 in advance. This macro is used in encoding routines
238 `encode_coding_XXX', thus it assumes that the source text is in
239 multibyte form except for 8-bit characters. 8-bit characters are
240 in multibyte form if coding->src_multibyte is nonzero, else they
241 are represented by a single byte. */
4ed46869 242
b73bfc1c
KH
243#define ONE_MORE_CHAR(c) \
244 do { \
245 int len = src_end - src; \
246 int bytes; \
247 if (len <= 0) \
248 { \
249 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
250 goto label_end_of_loop; \
251 } \
252 if (coding->src_multibyte \
253 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
254 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
255 else \
256 c = *src, bytes = 1; \
257 if (!NILP (translation_table)) \
39658efc 258 c = translate_char (translation_table, c, -1, 0, 0); \
b73bfc1c 259 src += bytes; \
4ed46869
KH
260 } while (0)
261
4ed46869 262
8ca3766a 263/* Produce a multibyte form of character C to `dst'. Jump to
b73bfc1c
KH
264 `label_end_of_loop' if there's not enough space at `dst'.
265
cfb43547 266 If we are now in the middle of a composition sequence, the decoded
b73bfc1c
KH
267 character may be ALTCHAR (for the current composition). In that
268 case, the character goes to coding->cmp_data->data instead of
269 `dst'.
270
271 This macro is used in decoding routines. */
272
273#define EMIT_CHAR(c) \
4ed46869 274 do { \
b73bfc1c
KH
275 if (! COMPOSING_P (coding) \
276 || coding->composing == COMPOSITION_RELATIVE \
277 || coding->composing == COMPOSITION_WITH_RULE) \
278 { \
279 int bytes = CHAR_BYTES (c); \
280 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
281 { \
282 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
283 goto label_end_of_loop; \
284 } \
285 dst += CHAR_STRING (c, dst); \
286 coding->produced_char++; \
287 } \
ec6d2bb8 288 \
b73bfc1c
KH
289 if (COMPOSING_P (coding) \
290 && coding->composing != COMPOSITION_RELATIVE) \
291 { \
292 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
293 coding->composition_rule_follows \
294 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
295 } \
4ed46869
KH
296 } while (0)
297
4ed46869 298
b73bfc1c
KH
299#define EMIT_ONE_BYTE(c) \
300 do { \
301 if (dst >= (dst_bytes ? dst_end : src)) \
302 { \
303 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
304 goto label_end_of_loop; \
305 } \
306 *dst++ = c; \
307 } while (0)
308
309#define EMIT_TWO_BYTES(c1, c2) \
310 do { \
311 if (dst + 2 > (dst_bytes ? dst_end : src)) \
312 { \
313 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
314 goto label_end_of_loop; \
315 } \
316 *dst++ = c1, *dst++ = c2; \
317 } while (0)
318
319#define EMIT_BYTES(from, to) \
320 do { \
321 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
322 { \
323 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
324 goto label_end_of_loop; \
325 } \
326 while (from < to) \
327 *dst++ = *from++; \
4ed46869
KH
328 } while (0)
329
330\f
331/*** 1. Preamble ***/
332
68c45bf0
PE
333#ifdef emacs
334#include <config.h>
335#endif
336
4ed46869
KH
337#include <stdio.h>
338
339#ifdef emacs
340
4ed46869
KH
341#include "lisp.h"
342#include "buffer.h"
343#include "charset.h"
ec6d2bb8 344#include "composite.h"
4ed46869
KH
345#include "ccl.h"
346#include "coding.h"
347#include "window.h"
66638433 348#include "intervals.h"
4ed46869
KH
349
350#else /* not emacs */
351
352#include "mulelib.h"
353
354#endif /* not emacs */
355
356Lisp_Object Qcoding_system, Qeol_type;
357Lisp_Object Qbuffer_file_coding_system;
358Lisp_Object Qpost_read_conversion, Qpre_write_conversion;
27901516 359Lisp_Object Qno_conversion, Qundecided;
bb0115a2 360Lisp_Object Qcoding_system_history;
05e6f5dc 361Lisp_Object Qsafe_chars;
1397dc18 362Lisp_Object Qvalid_codes;
4ed46869
KH
363
364extern Lisp_Object Qinsert_file_contents, Qwrite_region;
365Lisp_Object Qcall_process, Qcall_process_region, Qprocess_argument;
366Lisp_Object Qstart_process, Qopen_network_stream;
367Lisp_Object Qtarget_idx;
368
a362520d
KH
369/* If a symbol has this property, evaluate the value to define the
370 symbol as a coding system. */
371Lisp_Object Qcoding_system_define_form;
372
d46c5b12
KH
373Lisp_Object Vselect_safe_coding_system_function;
374
5d5bf4d8
KH
375int coding_system_require_warning;
376
7722baf9
EZ
377/* Mnemonic string for each format of end-of-line. */
378Lisp_Object eol_mnemonic_unix, eol_mnemonic_dos, eol_mnemonic_mac;
379/* Mnemonic string to indicate format of end-of-line is not yet
4ed46869 380 decided. */
7722baf9 381Lisp_Object eol_mnemonic_undecided;
4ed46869 382
9ce27fde
KH
383/* Format of end-of-line decided by system. This is CODING_EOL_LF on
384 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
385int system_eol_type;
386
4ed46869
KH
387#ifdef emacs
388
6b89e3aa
KH
389/* Information about which coding system is safe for which chars.
390 The value has the form (GENERIC-LIST . NON-GENERIC-ALIST).
391
392 GENERIC-LIST is a list of generic coding systems which can encode
393 any characters.
394
395 NON-GENERIC-ALIST is an alist of non generic coding systems vs the
396 corresponding char table that contains safe chars. */
397Lisp_Object Vcoding_system_safe_chars;
398
4608c386
KH
399Lisp_Object Vcoding_system_list, Vcoding_system_alist;
400
401Lisp_Object Qcoding_system_p, Qcoding_system_error;
4ed46869 402
d46c5b12
KH
403/* Coding system emacs-mule and raw-text are for converting only
404 end-of-line format. */
405Lisp_Object Qemacs_mule, Qraw_text;
9ce27fde 406
ecf488bc
DL
407Lisp_Object Qutf_8;
408
4ed46869
KH
409/* Coding-systems are handed between Emacs Lisp programs and C internal
410 routines by the following three variables. */
411/* Coding-system for reading files and receiving data from process. */
412Lisp_Object Vcoding_system_for_read;
413/* Coding-system for writing files and sending data to process. */
414Lisp_Object Vcoding_system_for_write;
415/* Coding-system actually used in the latest I/O. */
416Lisp_Object Vlast_coding_system_used;
417
c4825358 418/* A vector of length 256 which contains information about special
94487c4e 419 Latin codes (especially for dealing with Microsoft codes). */
3f003981 420Lisp_Object Vlatin_extra_code_table;
c4825358 421
9ce27fde
KH
422/* Flag to inhibit code conversion of end-of-line format. */
423int inhibit_eol_conversion;
424
74383408
KH
425/* Flag to inhibit ISO2022 escape sequence detection. */
426int inhibit_iso_escape_detection;
427
ed29121d
EZ
428/* Flag to make buffer-file-coding-system inherit from process-coding. */
429int inherit_process_coding_system;
430
c4825358 431/* Coding system to be used to encode text for terminal display. */
4ed46869
KH
432struct coding_system terminal_coding;
433
c4825358
KH
434/* Coding system to be used to encode text for terminal display when
435 terminal coding system is nil. */
436struct coding_system safe_terminal_coding;
437
438/* Coding system of what is sent from terminal keyboard. */
4ed46869
KH
439struct coding_system keyboard_coding;
440
6bc51348
KH
441/* Default coding system to be used to write a file. */
442struct coding_system default_buffer_file_coding;
443
02ba4723
KH
444Lisp_Object Vfile_coding_system_alist;
445Lisp_Object Vprocess_coding_system_alist;
446Lisp_Object Vnetwork_coding_system_alist;
4ed46869 447
68c45bf0
PE
448Lisp_Object Vlocale_coding_system;
449
4ed46869
KH
450#endif /* emacs */
451
d46c5b12 452Lisp_Object Qcoding_category, Qcoding_category_index;
4ed46869
KH
453
454/* List of symbols `coding-category-xxx' ordered by priority. */
455Lisp_Object Vcoding_category_list;
456
d46c5b12
KH
457/* Table of coding categories (Lisp symbols). */
458Lisp_Object Vcoding_category_table;
4ed46869
KH
459
460/* Table of names of symbol for each coding-category. */
461char *coding_category_name[CODING_CATEGORY_IDX_MAX] = {
0ef69138 462 "coding-category-emacs-mule",
4ed46869
KH
463 "coding-category-sjis",
464 "coding-category-iso-7",
d46c5b12 465 "coding-category-iso-7-tight",
4ed46869
KH
466 "coding-category-iso-8-1",
467 "coding-category-iso-8-2",
7717c392
KH
468 "coding-category-iso-7-else",
469 "coding-category-iso-8-else",
89fa8b36 470 "coding-category-ccl",
4ed46869 471 "coding-category-big5",
fa42c37f
KH
472 "coding-category-utf-8",
473 "coding-category-utf-16-be",
474 "coding-category-utf-16-le",
27901516 475 "coding-category-raw-text",
89fa8b36 476 "coding-category-binary"
4ed46869
KH
477};
478
66cfb530 479/* Table of pointers to coding systems corresponding to each coding
d46c5b12
KH
480 categories. */
481struct coding_system *coding_system_table[CODING_CATEGORY_IDX_MAX];
482
66cfb530 483/* Table of coding category masks. Nth element is a mask for a coding
8ca3766a 484 category of which priority is Nth. */
66cfb530
KH
485static
486int coding_priorities[CODING_CATEGORY_IDX_MAX];
487
f967223b
KH
488/* Flag to tell if we look up translation table on character code
489 conversion. */
84fbb8a0 490Lisp_Object Venable_character_translation;
f967223b
KH
491/* Standard translation table to look up on decoding (reading). */
492Lisp_Object Vstandard_translation_table_for_decode;
493/* Standard translation table to look up on encoding (writing). */
494Lisp_Object Vstandard_translation_table_for_encode;
84fbb8a0 495
f967223b
KH
496Lisp_Object Qtranslation_table;
497Lisp_Object Qtranslation_table_id;
498Lisp_Object Qtranslation_table_for_decode;
499Lisp_Object Qtranslation_table_for_encode;
4ed46869
KH
500
501/* Alist of charsets vs revision number. */
502Lisp_Object Vcharset_revision_alist;
503
02ba4723
KH
504/* Default coding systems used for process I/O. */
505Lisp_Object Vdefault_process_coding_system;
506
002fdb44
DL
507/* Char table for translating Quail and self-inserting input. */
508Lisp_Object Vtranslation_table_for_input;
509
b843d1ae
KH
510/* Global flag to tell that we can't call post-read-conversion and
511 pre-write-conversion functions. Usually the value is zero, but it
512 is set to 1 temporarily while such functions are running. This is
513 to avoid infinite recursive call. */
514static int inhibit_pre_post_conversion;
515
05e6f5dc
KH
516Lisp_Object Qchar_coding_system;
517
6b89e3aa
KH
518/* Return `safe-chars' property of CODING_SYSTEM (symbol). Don't check
519 its validity. */
05e6f5dc
KH
520
521Lisp_Object
6b89e3aa
KH
522coding_safe_chars (coding_system)
523 Lisp_Object coding_system;
05e6f5dc
KH
524{
525 Lisp_Object coding_spec, plist, safe_chars;
93dec019 526
6b89e3aa 527 coding_spec = Fget (coding_system, Qcoding_system);
05e6f5dc
KH
528 plist = XVECTOR (coding_spec)->contents[3];
529 safe_chars = Fplist_get (XVECTOR (coding_spec)->contents[3], Qsafe_chars);
530 return (CHAR_TABLE_P (safe_chars) ? safe_chars : Qt);
531}
532
533#define CODING_SAFE_CHAR_P(safe_chars, c) \
534 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
535
4ed46869 536\f
0ef69138 537/*** 2. Emacs internal format (emacs-mule) handlers ***/
4ed46869 538
aa72b389
KH
539/* Emacs' internal format for representation of multiple character
540 sets is a kind of multi-byte encoding, i.e. characters are
541 represented by variable-length sequences of one-byte codes.
b73bfc1c
KH
542
543 ASCII characters and control characters (e.g. `tab', `newline') are
544 represented by one-byte sequences which are their ASCII codes, in
545 the range 0x00 through 0x7F.
546
547 8-bit characters of the range 0x80..0x9F are represented by
548 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
549 code + 0x20).
550
551 8-bit characters of the range 0xA0..0xFF are represented by
552 one-byte sequences which are their 8-bit code.
553
554 The other characters are represented by a sequence of `base
555 leading-code', optional `extended leading-code', and one or two
556 `position-code's. The length of the sequence is determined by the
aa72b389 557 base leading-code. Leading-code takes the range 0x81 through 0x9D,
b73bfc1c
KH
558 whereas extended leading-code and position-code take the range 0xA0
559 through 0xFF. See `charset.h' for more details about leading-code
560 and position-code.
f4dee582 561
4ed46869 562 --- CODE RANGE of Emacs' internal format ---
b73bfc1c
KH
563 character set range
564 ------------- -----
565 ascii 0x00..0x7F
566 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
567 eight-bit-graphic 0xA0..0xBF
aa72b389 568 ELSE 0x81..0x9D + [0xA0..0xFF]+
4ed46869
KH
569 ---------------------------------------------
570
aa72b389
KH
571 As this is the internal character representation, the format is
572 usually not used externally (i.e. in a file or in a data sent to a
573 process). But, it is possible to have a text externally in this
574 format (i.e. by encoding by the coding system `emacs-mule').
575
576 In that case, a sequence of one-byte codes has a slightly different
577 form.
578
ae5145c2 579 Firstly, all characters in eight-bit-control are represented by
aa72b389
KH
580 one-byte sequences which are their 8-bit code.
581
582 Next, character composition data are represented by the byte
583 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
584 where,
585 METHOD is 0xF0 plus one of composition method (enum
586 composition_method),
587
ae5145c2 588 BYTES is 0xA0 plus the byte length of these composition data,
aa72b389 589
ae5145c2 590 CHARS is 0xA0 plus the number of characters composed by these
aa72b389
KH
591 data,
592
8ca3766a 593 COMPONENTs are characters of multibyte form or composition
aa72b389
KH
594 rules encoded by two-byte of ASCII codes.
595
596 In addition, for backward compatibility, the following formats are
597 also recognized as composition data on decoding.
598
599 0x80 MSEQ ...
600 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
601
602 Here,
603 MSEQ is a multibyte form but in these special format:
604 ASCII: 0xA0 ASCII_CODE+0x80,
605 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
606 RULE is a one byte code of the range 0xA0..0xF0 that
607 represents a composition rule.
4ed46869
KH
608 */
609
610enum emacs_code_class_type emacs_code_class[256];
611
4ed46869
KH
612/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
613 Check if a text is encoded in Emacs' internal format. If it is,
d46c5b12 614 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
4ed46869 615
0a28aafb
KH
616static int
617detect_coding_emacs_mule (src, src_end, multibytep)
b73bfc1c 618 unsigned char *src, *src_end;
0a28aafb 619 int multibytep;
4ed46869
KH
620{
621 unsigned char c;
622 int composing = 0;
b73bfc1c
KH
623 /* Dummy for ONE_MORE_BYTE. */
624 struct coding_system dummy_coding;
625 struct coding_system *coding = &dummy_coding;
4ed46869 626
b73bfc1c 627 while (1)
4ed46869 628 {
0a28aafb 629 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
4ed46869
KH
630
631 if (composing)
632 {
633 if (c < 0xA0)
634 composing = 0;
b73bfc1c
KH
635 else if (c == 0xA0)
636 {
0a28aafb 637 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
b73bfc1c
KH
638 c &= 0x7F;
639 }
4ed46869
KH
640 else
641 c -= 0x20;
642 }
643
b73bfc1c 644 if (c < 0x20)
4ed46869 645 {
4ed46869
KH
646 if (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
647 return 0;
b73bfc1c
KH
648 }
649 else if (c >= 0x80 && c < 0xA0)
650 {
651 if (c == 0x80)
652 /* Old leading code for a composite character. */
653 composing = 1;
654 else
655 {
656 unsigned char *src_base = src - 1;
657 int bytes;
4ed46869 658
b73bfc1c
KH
659 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base, src_end - src_base,
660 bytes))
661 return 0;
662 src = src_base + bytes;
663 }
664 }
665 }
666 label_end_of_loop:
667 return CODING_CATEGORY_MASK_EMACS_MULE;
668}
4ed46869 669
4ed46869 670
aa72b389
KH
671/* Record the starting position START and METHOD of one composition. */
672
673#define CODING_ADD_COMPOSITION_START(coding, start, method) \
674 do { \
675 struct composition_data *cmp_data = coding->cmp_data; \
676 int *data = cmp_data->data + cmp_data->used; \
677 coding->cmp_data_start = cmp_data->used; \
678 data[0] = -1; \
679 data[1] = cmp_data->char_offset + start; \
680 data[3] = (int) method; \
681 cmp_data->used += 4; \
682 } while (0)
683
684/* Record the ending position END of the current composition. */
685
686#define CODING_ADD_COMPOSITION_END(coding, end) \
687 do { \
688 struct composition_data *cmp_data = coding->cmp_data; \
689 int *data = cmp_data->data + coding->cmp_data_start; \
690 data[0] = cmp_data->used - coding->cmp_data_start; \
691 data[2] = cmp_data->char_offset + end; \
692 } while (0)
693
694/* Record one COMPONENT (alternate character or composition rule). */
695
b6871cc7
KH
696#define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
697 do { \
698 coding->cmp_data->data[coding->cmp_data->used++] = component; \
699 if (coding->cmp_data->used - coding->cmp_data_start \
700 == COMPOSITION_DATA_MAX_BUNCH_LENGTH) \
701 { \
702 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
703 coding->composing = COMPOSITION_NO; \
704 } \
705 } while (0)
aa72b389
KH
706
707
708/* Get one byte from a data pointed by SRC and increment SRC. If SRC
8ca3766a 709 is not less than SRC_END, return -1 without incrementing Src. */
aa72b389
KH
710
711#define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
712
713
714/* Decode a character represented as a component of composition
715 sequence of Emacs 20 style at SRC. Set C to that character, store
716 its multibyte form sequence at P, and set P to the end of that
717 sequence. If no valid character is found, set C to -1. */
718
719#define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
720 do { \
721 int bytes; \
fd3ae0b9 722 \
aa72b389
KH
723 c = SAFE_ONE_MORE_BYTE (); \
724 if (c < 0) \
725 break; \
726 if (CHAR_HEAD_P (c)) \
727 c = -1; \
728 else if (c == 0xA0) \
729 { \
730 c = SAFE_ONE_MORE_BYTE (); \
731 if (c < 0xA0) \
732 c = -1; \
733 else \
734 { \
735 c -= 0xA0; \
736 *p++ = c; \
737 } \
738 } \
739 else if (BASE_LEADING_CODE_P (c - 0x20)) \
740 { \
741 unsigned char *p0 = p; \
742 \
743 c -= 0x20; \
744 *p++ = c; \
745 bytes = BYTES_BY_CHAR_HEAD (c); \
746 while (--bytes) \
747 { \
748 c = SAFE_ONE_MORE_BYTE (); \
749 if (c < 0) \
750 break; \
751 *p++ = c; \
752 } \
fd3ae0b9
KH
753 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes) \
754 || (coding->flags /* We are recovering a file. */ \
755 && p0[0] == LEADING_CODE_8_BIT_CONTROL \
756 && ! CHAR_HEAD_P (p0[1]))) \
aa72b389
KH
757 c = STRING_CHAR (p0, bytes); \
758 else \
759 c = -1; \
760 } \
761 else \
762 c = -1; \
763 } while (0)
764
765
766/* Decode a composition rule represented as a component of composition
767 sequence of Emacs 20 style at SRC. Set C to the rule. If not
768 valid rule is found, set C to -1. */
769
770#define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
771 do { \
772 c = SAFE_ONE_MORE_BYTE (); \
773 c -= 0xA0; \
774 if (c < 0 || c >= 81) \
775 c = -1; \
776 else \
777 { \
778 gref = c / 9, nref = c % 9; \
779 c = COMPOSITION_ENCODE_RULE (gref, nref); \
780 } \
781 } while (0)
782
783
784/* Decode composition sequence encoded by `emacs-mule' at the source
785 pointed by SRC. SRC_END is the end of source. Store information
786 of the composition in CODING->cmp_data.
787
788 For backward compatibility, decode also a composition sequence of
789 Emacs 20 style. In that case, the composition sequence contains
790 characters that should be extracted into a buffer or string. Store
791 those characters at *DESTINATION in multibyte form.
792
793 If we encounter an invalid byte sequence, return 0.
794 If we encounter an insufficient source or destination, or
795 insufficient space in CODING->cmp_data, return 1.
796 Otherwise, return consumed bytes in the source.
797
798*/
799static INLINE int
800decode_composition_emacs_mule (coding, src, src_end,
801 destination, dst_end, dst_bytes)
802 struct coding_system *coding;
803 unsigned char *src, *src_end, **destination, *dst_end;
804 int dst_bytes;
805{
806 unsigned char *dst = *destination;
807 int method, data_len, nchars;
808 unsigned char *src_base = src++;
8ca3766a 809 /* Store components of composition. */
aa72b389
KH
810 int component[COMPOSITION_DATA_MAX_BUNCH_LENGTH];
811 int ncomponent;
812 /* Store multibyte form of characters to be composed. This is for
813 Emacs 20 style composition sequence. */
814 unsigned char buf[MAX_COMPOSITION_COMPONENTS * MAX_MULTIBYTE_LENGTH];
815 unsigned char *bufp = buf;
816 int c, i, gref, nref;
817
818 if (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH
819 >= COMPOSITION_DATA_SIZE)
820 {
821 coding->result = CODING_FINISH_INSUFFICIENT_CMP;
822 return -1;
823 }
824
825 ONE_MORE_BYTE (c);
826 if (c - 0xF0 >= COMPOSITION_RELATIVE
827 && c - 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS)
828 {
829 int with_rule;
830
831 method = c - 0xF0;
832 with_rule = (method == COMPOSITION_WITH_RULE
833 || method == COMPOSITION_WITH_RULE_ALTCHARS);
834 ONE_MORE_BYTE (c);
835 data_len = c - 0xA0;
836 if (data_len < 4
837 || src_base + data_len > src_end)
838 return 0;
839 ONE_MORE_BYTE (c);
840 nchars = c - 0xA0;
841 if (c < 1)
842 return 0;
843 for (ncomponent = 0; src < src_base + data_len; ncomponent++)
844 {
b1887814
RS
845 /* If it is longer than this, it can't be valid. */
846 if (ncomponent >= COMPOSITION_DATA_MAX_BUNCH_LENGTH)
847 return 0;
848
aa72b389
KH
849 if (ncomponent % 2 && with_rule)
850 {
851 ONE_MORE_BYTE (gref);
852 gref -= 32;
853 ONE_MORE_BYTE (nref);
854 nref -= 32;
855 c = COMPOSITION_ENCODE_RULE (gref, nref);
856 }
857 else
858 {
859 int bytes;
fd3ae0b9
KH
860 if (UNIBYTE_STR_AS_MULTIBYTE_P (src, src_end - src, bytes)
861 || (coding->flags /* We are recovering a file. */
862 && src[0] == LEADING_CODE_8_BIT_CONTROL
863 && ! CHAR_HEAD_P (src[1])))
aa72b389
KH
864 c = STRING_CHAR (src, bytes);
865 else
866 c = *src, bytes = 1;
867 src += bytes;
868 }
869 component[ncomponent] = c;
870 }
871 }
872 else
873 {
874 /* This may be an old Emacs 20 style format. See the comment at
875 the section 2 of this file. */
876 while (src < src_end && !CHAR_HEAD_P (*src)) src++;
877 if (src == src_end
878 && !(coding->mode & CODING_MODE_LAST_BLOCK))
879 goto label_end_of_loop;
880
881 src_end = src;
882 src = src_base + 1;
883 if (c < 0xC0)
884 {
885 method = COMPOSITION_RELATIVE;
886 for (ncomponent = 0; ncomponent < MAX_COMPOSITION_COMPONENTS;)
887 {
888 DECODE_EMACS_MULE_COMPOSITION_CHAR (c, bufp);
889 if (c < 0)
890 break;
891 component[ncomponent++] = c;
892 }
893 if (ncomponent < 2)
894 return 0;
895 nchars = ncomponent;
896 }
897 else if (c == 0xFF)
898 {
899 method = COMPOSITION_WITH_RULE;
900 src++;
901 DECODE_EMACS_MULE_COMPOSITION_CHAR (c, bufp);
902 if (c < 0)
903 return 0;
904 component[0] = c;
905 for (ncomponent = 1;
906 ncomponent < MAX_COMPOSITION_COMPONENTS * 2 - 1;)
907 {
908 DECODE_EMACS_MULE_COMPOSITION_RULE (c);
909 if (c < 0)
910 break;
911 component[ncomponent++] = c;
912 DECODE_EMACS_MULE_COMPOSITION_CHAR (c, bufp);
913 if (c < 0)
914 break;
915 component[ncomponent++] = c;
916 }
917 if (ncomponent < 3)
918 return 0;
919 nchars = (ncomponent + 1) / 2;
920 }
921 else
922 return 0;
923 }
924
925 if (buf == bufp || dst + (bufp - buf) <= (dst_bytes ? dst_end : src))
926 {
927 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, method);
928 for (i = 0; i < ncomponent; i++)
929 CODING_ADD_COMPOSITION_COMPONENT (coding, component[i]);
93dec019 930 CODING_ADD_COMPOSITION_END (coding, coding->produced_char + nchars);
aa72b389
KH
931 if (buf < bufp)
932 {
933 unsigned char *p = buf;
934 EMIT_BYTES (p, bufp);
935 *destination += bufp - buf;
936 coding->produced_char += nchars;
937 }
938 return (src - src_base);
939 }
940 label_end_of_loop:
941 return -1;
942}
943
b73bfc1c 944/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
4ed46869 945
b73bfc1c
KH
946static void
947decode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes)
948 struct coding_system *coding;
949 unsigned char *source, *destination;
950 int src_bytes, dst_bytes;
951{
952 unsigned char *src = source;
953 unsigned char *src_end = source + src_bytes;
954 unsigned char *dst = destination;
955 unsigned char *dst_end = destination + dst_bytes;
956 /* SRC_BASE remembers the start position in source in each loop.
957 The loop will be exited when there's not enough source code, or
958 when there's not enough destination area to produce a
959 character. */
960 unsigned char *src_base;
4ed46869 961
b73bfc1c 962 coding->produced_char = 0;
8a33cf7b 963 while ((src_base = src) < src_end)
b73bfc1c
KH
964 {
965 unsigned char tmp[MAX_MULTIBYTE_LENGTH], *p;
966 int bytes;
ec6d2bb8 967
4af310db
EZ
968 if (*src == '\r')
969 {
2bcdf662 970 int c = *src++;
4af310db 971
4af310db
EZ
972 if (coding->eol_type == CODING_EOL_CR)
973 c = '\n';
974 else if (coding->eol_type == CODING_EOL_CRLF)
975 {
976 ONE_MORE_BYTE (c);
977 if (c != '\n')
978 {
4af310db
EZ
979 src--;
980 c = '\r';
981 }
982 }
983 *dst++ = c;
984 coding->produced_char++;
985 continue;
986 }
987 else if (*src == '\n')
988 {
989 if ((coding->eol_type == CODING_EOL_CR
990 || coding->eol_type == CODING_EOL_CRLF)
991 && coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
992 {
993 coding->result = CODING_FINISH_INCONSISTENT_EOL;
994 goto label_end_of_loop;
995 }
996 *dst++ = *src++;
997 coding->produced_char++;
998 continue;
999 }
3089d25c 1000 else if (*src == 0x80 && coding->cmp_data)
aa72b389
KH
1001 {
1002 /* Start of composition data. */
1003 int consumed = decode_composition_emacs_mule (coding, src, src_end,
1004 &dst, dst_end,
1005 dst_bytes);
1006 if (consumed < 0)
1007 goto label_end_of_loop;
1008 else if (consumed > 0)
1009 {
1010 src += consumed;
1011 continue;
1012 }
1013 bytes = CHAR_STRING (*src, tmp);
1014 p = tmp;
1015 src++;
1016 }
fd3ae0b9
KH
1017 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src, src_end - src, bytes)
1018 || (coding->flags /* We are recovering a file. */
1019 && src[0] == LEADING_CODE_8_BIT_CONTROL
1020 && ! CHAR_HEAD_P (src[1])))
b73bfc1c
KH
1021 {
1022 p = src;
1023 src += bytes;
1024 }
1025 else
1026 {
6eced09c
KH
1027 int i, c;
1028
1029 bytes = BYTES_BY_CHAR_HEAD (*src);
b73bfc1c 1030 src++;
6eced09c
KH
1031 for (i = 1; i < bytes; i++)
1032 {
1033 ONE_MORE_BYTE (c);
1034 if (CHAR_HEAD_P (c))
1035 break;
1036 }
1037 if (i < bytes)
1038 {
1039 bytes = CHAR_STRING (*src_base, tmp);
1040 p = tmp;
1041 src = src_base + 1;
1042 }
1043 else
1044 {
1045 p = src_base;
1046 }
b73bfc1c
KH
1047 }
1048 if (dst + bytes >= (dst_bytes ? dst_end : src))
1049 {
1050 coding->result = CODING_FINISH_INSUFFICIENT_DST;
4ed46869
KH
1051 break;
1052 }
b73bfc1c
KH
1053 while (bytes--) *dst++ = *p++;
1054 coding->produced_char++;
4ed46869 1055 }
4af310db 1056 label_end_of_loop:
b73bfc1c
KH
1057 coding->consumed = coding->consumed_char = src_base - source;
1058 coding->produced = dst - destination;
4ed46869
KH
1059}
1060
b73bfc1c 1061
aa72b389
KH
1062/* Encode composition data stored at DATA into a special byte sequence
1063 starting by 0x80. Update CODING->cmp_data_start and maybe
1064 CODING->cmp_data for the next call. */
1065
1066#define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1067 do { \
1068 unsigned char buf[1024], *p0 = buf, *p; \
1069 int len = data[0]; \
1070 int i; \
1071 \
1072 buf[0] = 0x80; \
1073 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1074 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1075 p = buf + 4; \
1076 if (data[3] == COMPOSITION_WITH_RULE \
1077 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1078 { \
1079 p += CHAR_STRING (data[4], p); \
1080 for (i = 5; i < len; i += 2) \
1081 { \
1082 int gref, nref; \
1083 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1084 *p++ = 0x20 + gref; \
1085 *p++ = 0x20 + nref; \
1086 p += CHAR_STRING (data[i + 1], p); \
1087 } \
1088 } \
1089 else \
1090 { \
1091 for (i = 4; i < len; i++) \
1092 p += CHAR_STRING (data[i], p); \
1093 } \
1094 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1095 \
1096 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1097 { \
1098 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1099 goto label_end_of_loop; \
1100 } \
1101 while (p0 < p) \
1102 *dst++ = *p0++; \
1103 coding->cmp_data_start += data[0]; \
1104 if (coding->cmp_data_start == coding->cmp_data->used \
1105 && coding->cmp_data->next) \
1106 { \
1107 coding->cmp_data = coding->cmp_data->next; \
1108 coding->cmp_data_start = 0; \
1109 } \
1110 } while (0)
93dec019 1111
aa72b389 1112
a4244313 1113static void encode_eol P_ ((struct coding_system *, const unsigned char *,
aa72b389
KH
1114 unsigned char *, int, int));
1115
1116static void
1117encode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes)
1118 struct coding_system *coding;
1119 unsigned char *source, *destination;
1120 int src_bytes, dst_bytes;
1121{
1122 unsigned char *src = source;
1123 unsigned char *src_end = source + src_bytes;
1124 unsigned char *dst = destination;
1125 unsigned char *dst_end = destination + dst_bytes;
1126 unsigned char *src_base;
1127 int c;
1128 int char_offset;
1129 int *data;
1130
1131 Lisp_Object translation_table;
1132
1133 translation_table = Qnil;
1134
1135 /* Optimization for the case that there's no composition. */
1136 if (!coding->cmp_data || coding->cmp_data->used == 0)
1137 {
1138 encode_eol (coding, source, destination, src_bytes, dst_bytes);
1139 return;
1140 }
1141
1142 char_offset = coding->cmp_data->char_offset;
1143 data = coding->cmp_data->data + coding->cmp_data_start;
1144 while (1)
1145 {
1146 src_base = src;
1147
1148 /* If SRC starts a composition, encode the information about the
1149 composition in advance. */
1150 if (coding->cmp_data_start < coding->cmp_data->used
1151 && char_offset + coding->consumed_char == data[1])
1152 {
1153 ENCODE_COMPOSITION_EMACS_MULE (coding, data);
1154 char_offset = coding->cmp_data->char_offset;
1155 data = coding->cmp_data->data + coding->cmp_data_start;
1156 }
1157
1158 ONE_MORE_CHAR (c);
1159 if (c == '\n' && (coding->eol_type == CODING_EOL_CRLF
1160 || coding->eol_type == CODING_EOL_CR))
1161 {
1162 if (coding->eol_type == CODING_EOL_CRLF)
1163 EMIT_TWO_BYTES ('\r', c);
1164 else
1165 EMIT_ONE_BYTE ('\r');
1166 }
1167 else if (SINGLE_BYTE_CHAR_P (c))
fd3ae0b9
KH
1168 {
1169 if (coding->flags && ! ASCII_BYTE_P (c))
1170 {
1171 /* As we are auto saving, retain the multibyte form for
1172 8-bit chars. */
1173 unsigned char buf[MAX_MULTIBYTE_LENGTH];
1174 int bytes = CHAR_STRING (c, buf);
1175
1176 if (bytes == 1)
1177 EMIT_ONE_BYTE (buf[0]);
1178 else
1179 EMIT_TWO_BYTES (buf[0], buf[1]);
1180 }
1181 else
1182 EMIT_ONE_BYTE (c);
1183 }
aa72b389
KH
1184 else
1185 EMIT_BYTES (src_base, src);
1186 coding->consumed_char++;
1187 }
1188 label_end_of_loop:
1189 coding->consumed = src_base - source;
1190 coding->produced = coding->produced_char = dst - destination;
1191 return;
1192}
b73bfc1c 1193
4ed46869
KH
1194\f
1195/*** 3. ISO2022 handlers ***/
1196
1197/* The following note describes the coding system ISO2022 briefly.
39787efd 1198 Since the intention of this note is to help understand the
cfb43547 1199 functions in this file, some parts are NOT ACCURATE or are OVERLY
39787efd 1200 SIMPLIFIED. For thorough understanding, please refer to the
cfb43547
DL
1201 original document of ISO2022. This is equivalent to the standard
1202 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
4ed46869
KH
1203
1204 ISO2022 provides many mechanisms to encode several character sets
cfb43547 1205 in 7-bit and 8-bit environments. For 7-bit environments, all text
39787efd
KH
1206 is encoded using bytes less than 128. This may make the encoded
1207 text a little bit longer, but the text passes more easily through
cfb43547 1208 several types of gateway, some of which strip off the MSB (Most
8ca3766a 1209 Significant Bit).
b73bfc1c 1210
cfb43547
DL
1211 There are two kinds of character sets: control character sets and
1212 graphic character sets. The former contain control characters such
4ed46869 1213 as `newline' and `escape' to provide control functions (control
39787efd 1214 functions are also provided by escape sequences). The latter
cfb43547 1215 contain graphic characters such as 'A' and '-'. Emacs recognizes
4ed46869
KH
1216 two control character sets and many graphic character sets.
1217
1218 Graphic character sets are classified into one of the following
39787efd
KH
1219 four classes, according to the number of bytes (DIMENSION) and
1220 number of characters in one dimension (CHARS) of the set:
1221 - DIMENSION1_CHARS94
1222 - DIMENSION1_CHARS96
1223 - DIMENSION2_CHARS94
1224 - DIMENSION2_CHARS96
1225
1226 In addition, each character set is assigned an identification tag,
cfb43547 1227 unique for each set, called the "final character" (denoted as <F>
39787efd
KH
1228 hereafter). The <F> of each character set is decided by ECMA(*)
1229 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1230 (0x30..0x3F are for private use only).
4ed46869
KH
1231
1232 Note (*): ECMA = European Computer Manufacturers Association
1233
cfb43547 1234 Here are examples of graphic character sets [NAME(<F>)]:
4ed46869
KH
1235 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1236 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1237 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1238 o DIMENSION2_CHARS96 -- none for the moment
1239
39787efd 1240 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
4ed46869
KH
1241 C0 [0x00..0x1F] -- control character plane 0
1242 GL [0x20..0x7F] -- graphic character plane 0
1243 C1 [0x80..0x9F] -- control character plane 1
1244 GR [0xA0..0xFF] -- graphic character plane 1
1245
1246 A control character set is directly designated and invoked to C0 or
39787efd
KH
1247 C1 by an escape sequence. The most common case is that:
1248 - ISO646's control character set is designated/invoked to C0, and
1249 - ISO6429's control character set is designated/invoked to C1,
1250 and usually these designations/invocations are omitted in encoded
1251 text. In a 7-bit environment, only C0 can be used, and a control
1252 character for C1 is encoded by an appropriate escape sequence to
1253 fit into the environment. All control characters for C1 are
1254 defined to have corresponding escape sequences.
4ed46869
KH
1255
1256 A graphic character set is at first designated to one of four
1257 graphic registers (G0 through G3), then these graphic registers are
1258 invoked to GL or GR. These designations and invocations can be
1259 done independently. The most common case is that G0 is invoked to
39787efd
KH
1260 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1261 these invocations and designations are omitted in encoded text.
1262 In a 7-bit environment, only GL can be used.
4ed46869 1263
39787efd
KH
1264 When a graphic character set of CHARS94 is invoked to GL, codes
1265 0x20 and 0x7F of the GL area work as control characters SPACE and
1266 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1267 be used.
4ed46869
KH
1268
1269 There are two ways of invocation: locking-shift and single-shift.
1270 With locking-shift, the invocation lasts until the next different
39787efd
KH
1271 invocation, whereas with single-shift, the invocation affects the
1272 following character only and doesn't affect the locking-shift
1273 state. Invocations are done by the following control characters or
1274 escape sequences:
4ed46869
KH
1275
1276 ----------------------------------------------------------------------
39787efd 1277 abbrev function cntrl escape seq description
4ed46869 1278 ----------------------------------------------------------------------
39787efd
KH
1279 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1280 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1281 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1282 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1283 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1284 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1285 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1286 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1287 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
4ed46869 1288 ----------------------------------------------------------------------
39787efd
KH
1289 (*) These are not used by any known coding system.
1290
1291 Control characters for these functions are defined by macros
1292 ISO_CODE_XXX in `coding.h'.
4ed46869 1293
39787efd 1294 Designations are done by the following escape sequences:
4ed46869
KH
1295 ----------------------------------------------------------------------
1296 escape sequence description
1297 ----------------------------------------------------------------------
1298 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1299 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1300 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1301 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1302 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1303 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1304 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1305 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1306 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1307 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1308 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1309 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1310 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1311 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1312 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1313 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1314 ----------------------------------------------------------------------
1315
1316 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
39787efd 1317 of dimension 1, chars 94, and final character <F>, etc...
4ed46869
KH
1318
1319 Note (*): Although these designations are not allowed in ISO2022,
1320 Emacs accepts them on decoding, and produces them on encoding
39787efd 1321 CHARS96 character sets in a coding system which is characterized as
4ed46869
KH
1322 7-bit environment, non-locking-shift, and non-single-shift.
1323
1324 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
39787efd 1325 '(' can be omitted. We refer to this as "short-form" hereafter.
4ed46869 1326
cfb43547 1327 Now you may notice that there are a lot of ways of encoding the
39787efd
KH
1328 same multilingual text in ISO2022. Actually, there exist many
1329 coding systems such as Compound Text (used in X11's inter client
8ca3766a
DL
1330 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1331 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
4ed46869
KH
1332 localized platforms), and all of these are variants of ISO2022.
1333
1334 In addition to the above, Emacs handles two more kinds of escape
1335 sequences: ISO6429's direction specification and Emacs' private
1336 sequence for specifying character composition.
1337
39787efd 1338 ISO6429's direction specification takes the following form:
4ed46869
KH
1339 o CSI ']' -- end of the current direction
1340 o CSI '0' ']' -- end of the current direction
1341 o CSI '1' ']' -- start of left-to-right text
1342 o CSI '2' ']' -- start of right-to-left text
1343 The control character CSI (0x9B: control sequence introducer) is
39787efd
KH
1344 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1345
1346 Character composition specification takes the following form:
ec6d2bb8
KH
1347 o ESC '0' -- start relative composition
1348 o ESC '1' -- end composition
1349 o ESC '2' -- start rule-base composition (*)
1350 o ESC '3' -- start relative composition with alternate chars (**)
1351 o ESC '4' -- start rule-base composition with alternate chars (**)
b73bfc1c 1352 Since these are not standard escape sequences of any ISO standard,
cfb43547 1353 the use of them with these meanings is restricted to Emacs only.
ec6d2bb8 1354
cfb43547 1355 (*) This form is used only in Emacs 20.5 and older versions,
b73bfc1c 1356 but the newer versions can safely decode it.
cfb43547 1357 (**) This form is used only in Emacs 21.1 and newer versions,
b73bfc1c 1358 and the older versions can't decode it.
ec6d2bb8 1359
cfb43547 1360 Here's a list of example usages of these composition escape
b73bfc1c 1361 sequences (categorized by `enum composition_method').
ec6d2bb8 1362
b73bfc1c 1363 COMPOSITION_RELATIVE:
ec6d2bb8 1364 ESC 0 CHAR [ CHAR ] ESC 1
8ca3766a 1365 COMPOSITION_WITH_RULE:
ec6d2bb8 1366 ESC 2 CHAR [ RULE CHAR ] ESC 1
b73bfc1c 1367 COMPOSITION_WITH_ALTCHARS:
ec6d2bb8 1368 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
b73bfc1c 1369 COMPOSITION_WITH_RULE_ALTCHARS:
ec6d2bb8 1370 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
4ed46869
KH
1371
1372enum iso_code_class_type iso_code_class[256];
1373
05e6f5dc
KH
1374#define CHARSET_OK(idx, charset, c) \
1375 (coding_system_table[idx] \
1376 && (charset == CHARSET_ASCII \
6b89e3aa 1377 || (safe_chars = coding_safe_chars (coding_system_table[idx]->symbol), \
05e6f5dc
KH
1378 CODING_SAFE_CHAR_P (safe_chars, c))) \
1379 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1380 charset) \
1381 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
d46c5b12
KH
1382
1383#define SHIFT_OUT_OK(idx) \
1384 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1385
b6871cc7
KH
1386#define COMPOSITION_OK(idx) \
1387 (coding_system_table[idx]->composing != COMPOSITION_DISABLED)
1388
4ed46869 1389/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
cfb43547 1390 Check if a text is encoded in ISO2022. If it is, return an
4ed46869
KH
1391 integer in which appropriate flag bits any of:
1392 CODING_CATEGORY_MASK_ISO_7
d46c5b12 1393 CODING_CATEGORY_MASK_ISO_7_TIGHT
4ed46869
KH
1394 CODING_CATEGORY_MASK_ISO_8_1
1395 CODING_CATEGORY_MASK_ISO_8_2
7717c392
KH
1396 CODING_CATEGORY_MASK_ISO_7_ELSE
1397 CODING_CATEGORY_MASK_ISO_8_ELSE
4ed46869
KH
1398 are set. If a code which should never appear in ISO2022 is found,
1399 returns 0. */
1400
0a28aafb
KH
1401static int
1402detect_coding_iso2022 (src, src_end, multibytep)
4ed46869 1403 unsigned char *src, *src_end;
0a28aafb 1404 int multibytep;
4ed46869 1405{
d46c5b12
KH
1406 int mask = CODING_CATEGORY_MASK_ISO;
1407 int mask_found = 0;
f46869e4 1408 int reg[4], shift_out = 0, single_shifting = 0;
da55a2b7 1409 int c, c1, charset;
b73bfc1c
KH
1410 /* Dummy for ONE_MORE_BYTE. */
1411 struct coding_system dummy_coding;
1412 struct coding_system *coding = &dummy_coding;
05e6f5dc 1413 Lisp_Object safe_chars;
3f003981 1414
d46c5b12 1415 reg[0] = CHARSET_ASCII, reg[1] = reg[2] = reg[3] = -1;
3f003981 1416 while (mask && src < src_end)
4ed46869 1417 {
0a28aafb 1418 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
8d239c89 1419 retry:
4ed46869
KH
1420 switch (c)
1421 {
1422 case ISO_CODE_ESC:
74383408
KH
1423 if (inhibit_iso_escape_detection)
1424 break;
f46869e4 1425 single_shifting = 0;
0a28aafb 1426 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
d46c5b12 1427 if (c >= '(' && c <= '/')
4ed46869 1428 {
bf9cdd4e 1429 /* Designation sequence for a charset of dimension 1. */
0a28aafb 1430 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1, multibytep);
d46c5b12
KH
1431 if (c1 < ' ' || c1 >= 0x80
1432 || (charset = iso_charset_table[0][c >= ','][c1]) < 0)
1433 /* Invalid designation sequence. Just ignore. */
1434 break;
1435 reg[(c - '(') % 4] = charset;
bf9cdd4e
KH
1436 }
1437 else if (c == '$')
1438 {
1439 /* Designation sequence for a charset of dimension 2. */
0a28aafb 1440 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
bf9cdd4e
KH
1441 if (c >= '@' && c <= 'B')
1442 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
d46c5b12 1443 reg[0] = charset = iso_charset_table[1][0][c];
bf9cdd4e 1444 else if (c >= '(' && c <= '/')
bcf26d6a 1445 {
0a28aafb 1446 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1, multibytep);
d46c5b12
KH
1447 if (c1 < ' ' || c1 >= 0x80
1448 || (charset = iso_charset_table[1][c >= ','][c1]) < 0)
1449 /* Invalid designation sequence. Just ignore. */
1450 break;
1451 reg[(c - '(') % 4] = charset;
bcf26d6a 1452 }
bf9cdd4e 1453 else
d46c5b12
KH
1454 /* Invalid designation sequence. Just ignore. */
1455 break;
1456 }
ae9ff118 1457 else if (c == 'N' || c == 'O')
d46c5b12 1458 {
ae9ff118
KH
1459 /* ESC <Fe> for SS2 or SS3. */
1460 mask &= CODING_CATEGORY_MASK_ISO_7_ELSE;
d46c5b12 1461 break;
4ed46869 1462 }
ec6d2bb8
KH
1463 else if (c >= '0' && c <= '4')
1464 {
1465 /* ESC <Fp> for start/end composition. */
b6871cc7
KH
1466 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7))
1467 mask_found |= CODING_CATEGORY_MASK_ISO_7;
1468 else
1469 mask &= ~CODING_CATEGORY_MASK_ISO_7;
1470 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT))
1471 mask_found |= CODING_CATEGORY_MASK_ISO_7_TIGHT;
1472 else
1473 mask &= ~CODING_CATEGORY_MASK_ISO_7_TIGHT;
1474 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_1))
1475 mask_found |= CODING_CATEGORY_MASK_ISO_8_1;
1476 else
1477 mask &= ~CODING_CATEGORY_MASK_ISO_8_1;
1478 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_2))
1479 mask_found |= CODING_CATEGORY_MASK_ISO_8_2;
1480 else
1481 mask &= ~CODING_CATEGORY_MASK_ISO_8_2;
1482 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_ELSE))
1483 mask_found |= CODING_CATEGORY_MASK_ISO_7_ELSE;
1484 else
1485 mask &= ~CODING_CATEGORY_MASK_ISO_7_ELSE;
1486 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_ELSE))
1487 mask_found |= CODING_CATEGORY_MASK_ISO_8_ELSE;
1488 else
1489 mask &= ~CODING_CATEGORY_MASK_ISO_8_ELSE;
ec6d2bb8
KH
1490 break;
1491 }
bf9cdd4e 1492 else
d46c5b12
KH
1493 /* Invalid escape sequence. Just ignore. */
1494 break;
1495
1496 /* We found a valid designation sequence for CHARSET. */
1497 mask &= ~CODING_CATEGORY_MASK_ISO_8BIT;
05e6f5dc
KH
1498 c = MAKE_CHAR (charset, 0, 0);
1499 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7, charset, c))
d46c5b12
KH
1500 mask_found |= CODING_CATEGORY_MASK_ISO_7;
1501 else
1502 mask &= ~CODING_CATEGORY_MASK_ISO_7;
05e6f5dc 1503 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT, charset, c))
d46c5b12
KH
1504 mask_found |= CODING_CATEGORY_MASK_ISO_7_TIGHT;
1505 else
1506 mask &= ~CODING_CATEGORY_MASK_ISO_7_TIGHT;
05e6f5dc 1507 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE, charset, c))
ae9ff118
KH
1508 mask_found |= CODING_CATEGORY_MASK_ISO_7_ELSE;
1509 else
d46c5b12 1510 mask &= ~CODING_CATEGORY_MASK_ISO_7_ELSE;
05e6f5dc 1511 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE, charset, c))
ae9ff118
KH
1512 mask_found |= CODING_CATEGORY_MASK_ISO_8_ELSE;
1513 else
d46c5b12 1514 mask &= ~CODING_CATEGORY_MASK_ISO_8_ELSE;
4ed46869
KH
1515 break;
1516
4ed46869 1517 case ISO_CODE_SO:
74383408
KH
1518 if (inhibit_iso_escape_detection)
1519 break;
f46869e4 1520 single_shifting = 0;
d46c5b12
KH
1521 if (shift_out == 0
1522 && (reg[1] >= 0
1523 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE)
1524 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE)))
1525 {
1526 /* Locking shift out. */
1527 mask &= ~CODING_CATEGORY_MASK_ISO_7BIT;
1528 mask_found |= CODING_CATEGORY_MASK_ISO_SHIFT;
1529 }
e0e989f6 1530 break;
93dec019 1531
d46c5b12 1532 case ISO_CODE_SI:
74383408
KH
1533 if (inhibit_iso_escape_detection)
1534 break;
f46869e4 1535 single_shifting = 0;
d46c5b12
KH
1536 if (shift_out == 1)
1537 {
1538 /* Locking shift in. */
1539 mask &= ~CODING_CATEGORY_MASK_ISO_7BIT;
1540 mask_found |= CODING_CATEGORY_MASK_ISO_SHIFT;
1541 }
1542 break;
1543
4ed46869 1544 case ISO_CODE_CSI:
f46869e4 1545 single_shifting = 0;
4ed46869
KH
1546 case ISO_CODE_SS2:
1547 case ISO_CODE_SS3:
3f003981
KH
1548 {
1549 int newmask = CODING_CATEGORY_MASK_ISO_8_ELSE;
1550
74383408
KH
1551 if (inhibit_iso_escape_detection)
1552 break;
70c22245
KH
1553 if (c != ISO_CODE_CSI)
1554 {
d46c5b12
KH
1555 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_1]->flags
1556 & CODING_FLAG_ISO_SINGLE_SHIFT)
70c22245 1557 newmask |= CODING_CATEGORY_MASK_ISO_8_1;
d46c5b12
KH
1558 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_2]->flags
1559 & CODING_FLAG_ISO_SINGLE_SHIFT)
70c22245 1560 newmask |= CODING_CATEGORY_MASK_ISO_8_2;
f46869e4 1561 single_shifting = 1;
70c22245 1562 }
3f003981
KH
1563 if (VECTORP (Vlatin_extra_code_table)
1564 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
1565 {
d46c5b12
KH
1566 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_1]->flags
1567 & CODING_FLAG_ISO_LATIN_EXTRA)
3f003981 1568 newmask |= CODING_CATEGORY_MASK_ISO_8_1;
d46c5b12
KH
1569 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_2]->flags
1570 & CODING_FLAG_ISO_LATIN_EXTRA)
3f003981
KH
1571 newmask |= CODING_CATEGORY_MASK_ISO_8_2;
1572 }
1573 mask &= newmask;
d46c5b12 1574 mask_found |= newmask;
3f003981
KH
1575 }
1576 break;
4ed46869
KH
1577
1578 default:
1579 if (c < 0x80)
f46869e4
KH
1580 {
1581 single_shifting = 0;
1582 break;
1583 }
4ed46869 1584 else if (c < 0xA0)
c4825358 1585 {
f46869e4 1586 single_shifting = 0;
3f003981
KH
1587 if (VECTORP (Vlatin_extra_code_table)
1588 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
c4825358 1589 {
3f003981
KH
1590 int newmask = 0;
1591
d46c5b12
KH
1592 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_1]->flags
1593 & CODING_FLAG_ISO_LATIN_EXTRA)
3f003981 1594 newmask |= CODING_CATEGORY_MASK_ISO_8_1;
d46c5b12
KH
1595 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_2]->flags
1596 & CODING_FLAG_ISO_LATIN_EXTRA)
3f003981
KH
1597 newmask |= CODING_CATEGORY_MASK_ISO_8_2;
1598 mask &= newmask;
d46c5b12 1599 mask_found |= newmask;
c4825358 1600 }
3f003981
KH
1601 else
1602 return 0;
c4825358 1603 }
4ed46869
KH
1604 else
1605 {
d46c5b12 1606 mask &= ~(CODING_CATEGORY_MASK_ISO_7BIT
7717c392 1607 | CODING_CATEGORY_MASK_ISO_7_ELSE);
d46c5b12 1608 mask_found |= CODING_CATEGORY_MASK_ISO_8_1;
f46869e4
KH
1609 /* Check the length of succeeding codes of the range
1610 0xA0..0FF. If the byte length is odd, we exclude
1611 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1612 when we are not single shifting. */
b73bfc1c
KH
1613 if (!single_shifting
1614 && mask & CODING_CATEGORY_MASK_ISO_8_2)
f46869e4 1615 {
e17de821 1616 int i = 1;
8d239c89
KH
1617
1618 c = -1;
b73bfc1c
KH
1619 while (src < src_end)
1620 {
0a28aafb 1621 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
b73bfc1c
KH
1622 if (c < 0xA0)
1623 break;
1624 i++;
1625 }
1626
1627 if (i & 1 && src < src_end)
f46869e4
KH
1628 mask &= ~CODING_CATEGORY_MASK_ISO_8_2;
1629 else
1630 mask_found |= CODING_CATEGORY_MASK_ISO_8_2;
8d239c89
KH
1631 if (c >= 0)
1632 /* This means that we have read one extra byte. */
1633 goto retry;
f46869e4 1634 }
4ed46869
KH
1635 }
1636 break;
1637 }
1638 }
b73bfc1c 1639 label_end_of_loop:
d46c5b12 1640 return (mask & mask_found);
4ed46869
KH
1641}
1642
b73bfc1c
KH
1643/* Decode a character of which charset is CHARSET, the 1st position
1644 code is C1, the 2nd position code is C2, and return the decoded
1645 character code. If the variable `translation_table' is non-nil,
1646 returned the translated code. */
ec6d2bb8 1647
b73bfc1c
KH
1648#define DECODE_ISO_CHARACTER(charset, c1, c2) \
1649 (NILP (translation_table) \
1650 ? MAKE_CHAR (charset, c1, c2) \
1651 : translate_char (translation_table, -1, charset, c1, c2))
4ed46869
KH
1652
1653/* Set designation state into CODING. */
d46c5b12
KH
1654#define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1655 do { \
05e6f5dc 1656 int charset, c; \
944bd420
KH
1657 \
1658 if (final_char < '0' || final_char >= 128) \
1659 goto label_invalid_code; \
1660 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1661 make_number (chars), \
1662 make_number (final_char)); \
05e6f5dc 1663 c = MAKE_CHAR (charset, 0, 0); \
d46c5b12 1664 if (charset >= 0 \
704c5781 1665 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
05e6f5dc 1666 || CODING_SAFE_CHAR_P (safe_chars, c))) \
d46c5b12
KH
1667 { \
1668 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1669 && reg == 0 \
1670 && charset == CHARSET_ASCII) \
1671 { \
1672 /* We should insert this designation sequence as is so \
1673 that it is surely written back to a file. */ \
1674 coding->spec.iso2022.last_invalid_designation_register = -1; \
1675 goto label_invalid_code; \
1676 } \
1677 coding->spec.iso2022.last_invalid_designation_register = -1; \
1678 if ((coding->mode & CODING_MODE_DIRECTION) \
1679 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1680 charset = CHARSET_REVERSE_CHARSET (charset); \
1681 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1682 } \
1683 else \
1684 { \
1685 coding->spec.iso2022.last_invalid_designation_register = reg; \
1686 goto label_invalid_code; \
1687 } \
4ed46869
KH
1688 } while (0)
1689
ec6d2bb8
KH
1690/* Allocate a memory block for storing information about compositions.
1691 The block is chained to the already allocated blocks. */
d46c5b12 1692
33fb63eb 1693void
ec6d2bb8 1694coding_allocate_composition_data (coding, char_offset)
d46c5b12 1695 struct coding_system *coding;
ec6d2bb8 1696 int char_offset;
d46c5b12 1697{
ec6d2bb8
KH
1698 struct composition_data *cmp_data
1699 = (struct composition_data *) xmalloc (sizeof *cmp_data);
1700
1701 cmp_data->char_offset = char_offset;
1702 cmp_data->used = 0;
1703 cmp_data->prev = coding->cmp_data;
1704 cmp_data->next = NULL;
1705 if (coding->cmp_data)
1706 coding->cmp_data->next = cmp_data;
1707 coding->cmp_data = cmp_data;
1708 coding->cmp_data_start = 0;
4307d534 1709 coding->composing = COMPOSITION_NO;
ec6d2bb8 1710}
d46c5b12 1711
aa72b389
KH
1712/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1713 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1714 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1715 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1716 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1717 */
ec6d2bb8 1718
33fb63eb
KH
1719#define DECODE_COMPOSITION_START(c1) \
1720 do { \
1721 if (coding->composing == COMPOSITION_DISABLED) \
1722 { \
1723 *dst++ = ISO_CODE_ESC; \
1724 *dst++ = c1 & 0x7f; \
1725 coding->produced_char += 2; \
1726 } \
1727 else if (!COMPOSING_P (coding)) \
1728 { \
1729 /* This is surely the start of a composition. We must be sure \
1730 that coding->cmp_data has enough space to store the \
1731 information about the composition. If not, terminate the \
1732 current decoding loop, allocate one more memory block for \
8ca3766a 1733 coding->cmp_data in the caller, then start the decoding \
33fb63eb
KH
1734 loop again. We can't allocate memory here directly because \
1735 it may cause buffer/string relocation. */ \
1736 if (!coding->cmp_data \
1737 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1738 >= COMPOSITION_DATA_SIZE)) \
1739 { \
1740 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1741 goto label_end_of_loop; \
1742 } \
1743 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1744 : c1 == '2' ? COMPOSITION_WITH_RULE \
1745 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1746 : COMPOSITION_WITH_RULE_ALTCHARS); \
1747 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1748 coding->composing); \
1749 coding->composition_rule_follows = 0; \
1750 } \
1751 else \
1752 { \
1753 /* We are already handling a composition. If the method is \
1754 the following two, the codes following the current escape \
1755 sequence are actual characters stored in a buffer. */ \
1756 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1757 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1758 { \
1759 coding->composing = COMPOSITION_RELATIVE; \
1760 coding->composition_rule_follows = 0; \
1761 } \
1762 } \
ec6d2bb8
KH
1763 } while (0)
1764
8ca3766a 1765/* Handle composition end sequence ESC 1. */
ec6d2bb8
KH
1766
1767#define DECODE_COMPOSITION_END(c1) \
1768 do { \
93dec019 1769 if (! COMPOSING_P (coding)) \
ec6d2bb8
KH
1770 { \
1771 *dst++ = ISO_CODE_ESC; \
1772 *dst++ = c1; \
1773 coding->produced_char += 2; \
1774 } \
1775 else \
1776 { \
1777 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1778 coding->composing = COMPOSITION_NO; \
1779 } \
1780 } while (0)
1781
1782/* Decode a composition rule from the byte C1 (and maybe one more byte
1783 from SRC) and store one encoded composition rule in
1784 coding->cmp_data. */
1785
1786#define DECODE_COMPOSITION_RULE(c1) \
1787 do { \
1788 int rule = 0; \
1789 (c1) -= 32; \
1790 if (c1 < 81) /* old format (before ver.21) */ \
1791 { \
1792 int gref = (c1) / 9; \
1793 int nref = (c1) % 9; \
1794 if (gref == 4) gref = 10; \
1795 if (nref == 4) nref = 10; \
1796 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1797 } \
b73bfc1c 1798 else if (c1 < 93) /* new format (after ver.21) */ \
ec6d2bb8
KH
1799 { \
1800 ONE_MORE_BYTE (c2); \
1801 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1802 } \
1803 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1804 coding->composition_rule_follows = 0; \
1805 } while (0)
88993dfd 1806
d46c5b12 1807
4ed46869
KH
1808/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1809
b73bfc1c 1810static void
d46c5b12 1811decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
4ed46869
KH
1812 struct coding_system *coding;
1813 unsigned char *source, *destination;
1814 int src_bytes, dst_bytes;
4ed46869
KH
1815{
1816 unsigned char *src = source;
1817 unsigned char *src_end = source + src_bytes;
1818 unsigned char *dst = destination;
1819 unsigned char *dst_end = destination + dst_bytes;
4ed46869
KH
1820 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1821 int charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
1822 int charset1 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 1);
b73bfc1c
KH
1823 /* SRC_BASE remembers the start position in source in each loop.
1824 The loop will be exited when there's not enough source code
1825 (within macro ONE_MORE_BYTE), or when there's not enough
1826 destination area to produce a character (within macro
1827 EMIT_CHAR). */
1828 unsigned char *src_base;
1829 int c, charset;
1830 Lisp_Object translation_table;
05e6f5dc
KH
1831 Lisp_Object safe_chars;
1832
6b89e3aa 1833 safe_chars = coding_safe_chars (coding->symbol);
bdd9fb48 1834
b73bfc1c
KH
1835 if (NILP (Venable_character_translation))
1836 translation_table = Qnil;
1837 else
1838 {
1839 translation_table = coding->translation_table_for_decode;
1840 if (NILP (translation_table))
1841 translation_table = Vstandard_translation_table_for_decode;
1842 }
4ed46869 1843
b73bfc1c
KH
1844 coding->result = CODING_FINISH_NORMAL;
1845
1846 while (1)
4ed46869 1847 {
85478bc6 1848 int c1, c2 = 0;
b73bfc1c
KH
1849
1850 src_base = src;
1851 ONE_MORE_BYTE (c1);
4ed46869 1852
ec6d2bb8 1853 /* We produce no character or one character. */
4ed46869
KH
1854 switch (iso_code_class [c1])
1855 {
1856 case ISO_0x20_or_0x7F:
ec6d2bb8
KH
1857 if (COMPOSING_P (coding) && coding->composition_rule_follows)
1858 {
1859 DECODE_COMPOSITION_RULE (c1);
b73bfc1c 1860 continue;
ec6d2bb8
KH
1861 }
1862 if (charset0 < 0 || CHARSET_CHARS (charset0) == 94)
4ed46869
KH
1863 {
1864 /* This is SPACE or DEL. */
b73bfc1c 1865 charset = CHARSET_ASCII;
4ed46869
KH
1866 break;
1867 }
1868 /* This is a graphic character, we fall down ... */
1869
1870 case ISO_graphic_plane_0:
ec6d2bb8 1871 if (COMPOSING_P (coding) && coding->composition_rule_follows)
b73bfc1c
KH
1872 {
1873 DECODE_COMPOSITION_RULE (c1);
1874 continue;
1875 }
1876 charset = charset0;
4ed46869
KH
1877 break;
1878
1879 case ISO_0xA0_or_0xFF:
d46c5b12
KH
1880 if (charset1 < 0 || CHARSET_CHARS (charset1) == 94
1881 || coding->flags & CODING_FLAG_ISO_SEVEN_BITS)
fb88bf2d 1882 goto label_invalid_code;
4ed46869
KH
1883 /* This is a graphic character, we fall down ... */
1884
1885 case ISO_graphic_plane_1:
b73bfc1c 1886 if (charset1 < 0)
fb88bf2d 1887 goto label_invalid_code;
b73bfc1c 1888 charset = charset1;
4ed46869
KH
1889 break;
1890
b73bfc1c 1891 case ISO_control_0:
ec6d2bb8
KH
1892 if (COMPOSING_P (coding))
1893 DECODE_COMPOSITION_END ('1');
1894
4ed46869
KH
1895 /* All ISO2022 control characters in this class have the
1896 same representation in Emacs internal format. */
d46c5b12
KH
1897 if (c1 == '\n'
1898 && (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
1899 && (coding->eol_type == CODING_EOL_CR
1900 || coding->eol_type == CODING_EOL_CRLF))
1901 {
b73bfc1c
KH
1902 coding->result = CODING_FINISH_INCONSISTENT_EOL;
1903 goto label_end_of_loop;
d46c5b12 1904 }
b73bfc1c 1905 charset = CHARSET_ASCII;
4ed46869
KH
1906 break;
1907
b73bfc1c
KH
1908 case ISO_control_1:
1909 if (COMPOSING_P (coding))
1910 DECODE_COMPOSITION_END ('1');
1911 goto label_invalid_code;
1912
4ed46869 1913 case ISO_carriage_return:
ec6d2bb8
KH
1914 if (COMPOSING_P (coding))
1915 DECODE_COMPOSITION_END ('1');
1916
4ed46869 1917 if (coding->eol_type == CODING_EOL_CR)
b73bfc1c 1918 c1 = '\n';
4ed46869
KH
1919 else if (coding->eol_type == CODING_EOL_CRLF)
1920 {
1921 ONE_MORE_BYTE (c1);
b73bfc1c 1922 if (c1 != ISO_CODE_LF)
4ed46869
KH
1923 {
1924 src--;
b73bfc1c 1925 c1 = '\r';
4ed46869
KH
1926 }
1927 }
b73bfc1c 1928 charset = CHARSET_ASCII;
4ed46869
KH
1929 break;
1930
1931 case ISO_shift_out:
d46c5b12
KH
1932 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT)
1933 || CODING_SPEC_ISO_DESIGNATION (coding, 1) < 0)
1934 goto label_invalid_code;
4ed46869
KH
1935 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1;
1936 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
b73bfc1c 1937 continue;
4ed46869
KH
1938
1939 case ISO_shift_in:
d46c5b12
KH
1940 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT))
1941 goto label_invalid_code;
4ed46869
KH
1942 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0;
1943 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
b73bfc1c 1944 continue;
4ed46869
KH
1945
1946 case ISO_single_shift_2_7:
1947 case ISO_single_shift_2:
d46c5b12
KH
1948 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT))
1949 goto label_invalid_code;
4ed46869
KH
1950 /* SS2 is handled as an escape sequence of ESC 'N' */
1951 c1 = 'N';
1952 goto label_escape_sequence;
1953
1954 case ISO_single_shift_3:
d46c5b12
KH
1955 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT))
1956 goto label_invalid_code;
4ed46869
KH
1957 /* SS2 is handled as an escape sequence of ESC 'O' */
1958 c1 = 'O';
1959 goto label_escape_sequence;
1960
1961 case ISO_control_sequence_introducer:
1962 /* CSI is handled as an escape sequence of ESC '[' ... */
1963 c1 = '[';
1964 goto label_escape_sequence;
1965
1966 case ISO_escape:
1967 ONE_MORE_BYTE (c1);
1968 label_escape_sequence:
1969 /* Escape sequences handled by Emacs are invocation,
1970 designation, direction specification, and character
1971 composition specification. */
1972 switch (c1)
1973 {
1974 case '&': /* revision of following character set */
1975 ONE_MORE_BYTE (c1);
1976 if (!(c1 >= '@' && c1 <= '~'))
d46c5b12 1977 goto label_invalid_code;
4ed46869
KH
1978 ONE_MORE_BYTE (c1);
1979 if (c1 != ISO_CODE_ESC)
d46c5b12 1980 goto label_invalid_code;
4ed46869
KH
1981 ONE_MORE_BYTE (c1);
1982 goto label_escape_sequence;
1983
1984 case '$': /* designation of 2-byte character set */
d46c5b12
KH
1985 if (! (coding->flags & CODING_FLAG_ISO_DESIGNATION))
1986 goto label_invalid_code;
4ed46869
KH
1987 ONE_MORE_BYTE (c1);
1988 if (c1 >= '@' && c1 <= 'B')
1989 { /* designation of JISX0208.1978, GB2312.1980,
88993dfd 1990 or JISX0208.1980 */
4ed46869
KH
1991 DECODE_DESIGNATION (0, 2, 94, c1);
1992 }
1993 else if (c1 >= 0x28 && c1 <= 0x2B)
1994 { /* designation of DIMENSION2_CHARS94 character set */
1995 ONE_MORE_BYTE (c2);
1996 DECODE_DESIGNATION (c1 - 0x28, 2, 94, c2);
1997 }
1998 else if (c1 >= 0x2C && c1 <= 0x2F)
1999 { /* designation of DIMENSION2_CHARS96 character set */
2000 ONE_MORE_BYTE (c2);
2001 DECODE_DESIGNATION (c1 - 0x2C, 2, 96, c2);
2002 }
2003 else
d46c5b12 2004 goto label_invalid_code;
b73bfc1c
KH
2005 /* We must update these variables now. */
2006 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
2007 charset1 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 1);
2008 continue;
4ed46869
KH
2009
2010 case 'n': /* invocation of locking-shift-2 */
d46c5b12
KH
2011 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT)
2012 || CODING_SPEC_ISO_DESIGNATION (coding, 2) < 0)
2013 goto label_invalid_code;
4ed46869 2014 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2;
e0e989f6 2015 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
b73bfc1c 2016 continue;
4ed46869
KH
2017
2018 case 'o': /* invocation of locking-shift-3 */
d46c5b12
KH
2019 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT)
2020 || CODING_SPEC_ISO_DESIGNATION (coding, 3) < 0)
2021 goto label_invalid_code;
4ed46869 2022 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3;
e0e989f6 2023 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
b73bfc1c 2024 continue;
4ed46869
KH
2025
2026 case 'N': /* invocation of single-shift-2 */
d46c5b12
KH
2027 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT)
2028 || CODING_SPEC_ISO_DESIGNATION (coding, 2) < 0)
2029 goto label_invalid_code;
4ed46869 2030 charset = CODING_SPEC_ISO_DESIGNATION (coding, 2);
b73bfc1c 2031 ONE_MORE_BYTE (c1);
e7046a18
KH
2032 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
2033 goto label_invalid_code;
4ed46869
KH
2034 break;
2035
2036 case 'O': /* invocation of single-shift-3 */
d46c5b12
KH
2037 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT)
2038 || CODING_SPEC_ISO_DESIGNATION (coding, 3) < 0)
2039 goto label_invalid_code;
4ed46869 2040 charset = CODING_SPEC_ISO_DESIGNATION (coding, 3);
b73bfc1c 2041 ONE_MORE_BYTE (c1);
e7046a18
KH
2042 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
2043 goto label_invalid_code;
4ed46869
KH
2044 break;
2045
ec6d2bb8
KH
2046 case '0': case '2': case '3': case '4': /* start composition */
2047 DECODE_COMPOSITION_START (c1);
b73bfc1c 2048 continue;
4ed46869 2049
ec6d2bb8
KH
2050 case '1': /* end composition */
2051 DECODE_COMPOSITION_END (c1);
b73bfc1c 2052 continue;
4ed46869
KH
2053
2054 case '[': /* specification of direction */
d46c5b12
KH
2055 if (coding->flags & CODING_FLAG_ISO_NO_DIRECTION)
2056 goto label_invalid_code;
4ed46869 2057 /* For the moment, nested direction is not supported.
d46c5b12 2058 So, `coding->mode & CODING_MODE_DIRECTION' zero means
8ca3766a 2059 left-to-right, and nonzero means right-to-left. */
4ed46869
KH
2060 ONE_MORE_BYTE (c1);
2061 switch (c1)
2062 {
2063 case ']': /* end of the current direction */
d46c5b12 2064 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869
KH
2065
2066 case '0': /* end of the current direction */
2067 case '1': /* start of left-to-right direction */
2068 ONE_MORE_BYTE (c1);
2069 if (c1 == ']')
d46c5b12 2070 coding->mode &= ~CODING_MODE_DIRECTION;
4ed46869 2071 else
d46c5b12 2072 goto label_invalid_code;
4ed46869
KH
2073 break;
2074
2075 case '2': /* start of right-to-left direction */
2076 ONE_MORE_BYTE (c1);
2077 if (c1 == ']')
d46c5b12 2078 coding->mode |= CODING_MODE_DIRECTION;
4ed46869 2079 else
d46c5b12 2080 goto label_invalid_code;
4ed46869
KH
2081 break;
2082
2083 default:
d46c5b12 2084 goto label_invalid_code;
4ed46869 2085 }
b73bfc1c 2086 continue;
4ed46869 2087
103e0180
KH
2088 case '%':
2089 if (COMPOSING_P (coding))
2090 DECODE_COMPOSITION_END ('1');
2091 ONE_MORE_BYTE (c1);
2092 if (c1 == '/')
2093 {
2094 /* CTEXT extended segment:
2095 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
2096 We keep these bytes as is for the moment.
2097 They may be decoded by post-read-conversion. */
2098 int dim, M, L;
2099 int size, required;
2100 int produced_chars;
43e4a82f 2101
103e0180
KH
2102 ONE_MORE_BYTE (dim);
2103 ONE_MORE_BYTE (M);
2104 ONE_MORE_BYTE (L);
2105 size = ((M - 128) * 128) + (L - 128);
2106 required = 8 + size * 2;
2107 if (dst + required > (dst_bytes ? dst_end : src))
2108 goto label_end_of_loop;
2109 *dst++ = ISO_CODE_ESC;
2110 *dst++ = '%';
2111 *dst++ = '/';
2112 *dst++ = dim;
2113 produced_chars = 4;
2114 dst += CHAR_STRING (M, dst), produced_chars++;
2115 dst += CHAR_STRING (L, dst), produced_chars++;
2116 while (size-- > 0)
2117 {
2118 ONE_MORE_BYTE (c1);
2119 dst += CHAR_STRING (c1, dst), produced_chars++;
2120 }
2121 coding->produced_char += produced_chars;
2122 }
2123 else if (c1 == 'G')
2124 {
2125 unsigned char *d = dst;
2126 int produced_chars;
2127
2128 /* XFree86 extension for embedding UTF-8 in CTEXT:
2129 ESC % G --UTF-8-BYTES-- ESC % @
2130 We keep these bytes as is for the moment.
2131 They may be decoded by post-read-conversion. */
2132 if (d + 6 > (dst_bytes ? dst_end : src))
2133 goto label_end_of_loop;
2134 *d++ = ISO_CODE_ESC;
2135 *d++ = '%';
2136 *d++ = 'G';
2137 produced_chars = 3;
2138 while (d + 1 < (dst_bytes ? dst_end : src))
2139 {
2140 ONE_MORE_BYTE (c1);
2141 if (c1 == ISO_CODE_ESC
2142 && src + 1 < src_end
2143 && src[0] == '%'
2144 && src[1] == '@')
47dc91ad
KH
2145 {
2146 src += 2;
2147 break;
2148 }
103e0180
KH
2149 d += CHAR_STRING (c1, d), produced_chars++;
2150 }
2151 if (d + 3 > (dst_bytes ? dst_end : src))
2152 goto label_end_of_loop;
2153 *d++ = ISO_CODE_ESC;
2154 *d++ = '%';
2155 *d++ = '@';
2156 dst = d;
2157 coding->produced_char += produced_chars + 3;
2158 }
2159 else
2160 goto label_invalid_code;
2161 continue;
2162
4ed46869 2163 default:
d46c5b12
KH
2164 if (! (coding->flags & CODING_FLAG_ISO_DESIGNATION))
2165 goto label_invalid_code;
4ed46869
KH
2166 if (c1 >= 0x28 && c1 <= 0x2B)
2167 { /* designation of DIMENSION1_CHARS94 character set */
2168 ONE_MORE_BYTE (c2);
2169 DECODE_DESIGNATION (c1 - 0x28, 1, 94, c2);
2170 }
2171 else if (c1 >= 0x2C && c1 <= 0x2F)
2172 { /* designation of DIMENSION1_CHARS96 character set */
2173 ONE_MORE_BYTE (c2);
2174 DECODE_DESIGNATION (c1 - 0x2C, 1, 96, c2);
2175 }
2176 else
b73bfc1c
KH
2177 goto label_invalid_code;
2178 /* We must update these variables now. */
2179 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
2180 charset1 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 1);
2181 continue;
4ed46869 2182 }
b73bfc1c 2183 }
4ed46869 2184
b73bfc1c
KH
2185 /* Now we know CHARSET and 1st position code C1 of a character.
2186 Produce a multibyte sequence for that character while getting
2187 2nd position code C2 if necessary. */
2188 if (CHARSET_DIMENSION (charset) == 2)
2189 {
2190 ONE_MORE_BYTE (c2);
2191 if (c1 < 0x80 ? c2 < 0x20 || c2 >= 0x80 : c2 < 0xA0)
2192 /* C2 is not in a valid range. */
2193 goto label_invalid_code;
4ed46869 2194 }
b73bfc1c
KH
2195 c = DECODE_ISO_CHARACTER (charset, c1, c2);
2196 EMIT_CHAR (c);
4ed46869
KH
2197 continue;
2198
b73bfc1c
KH
2199 label_invalid_code:
2200 coding->errors++;
2201 if (COMPOSING_P (coding))
2202 DECODE_COMPOSITION_END ('1');
4ed46869 2203 src = src_base;
b73bfc1c
KH
2204 c = *src++;
2205 EMIT_CHAR (c);
4ed46869 2206 }
fb88bf2d 2207
b73bfc1c
KH
2208 label_end_of_loop:
2209 coding->consumed = coding->consumed_char = src_base - source;
d46c5b12 2210 coding->produced = dst - destination;
b73bfc1c 2211 return;
4ed46869
KH
2212}
2213
b73bfc1c 2214
f4dee582 2215/* ISO2022 encoding stuff. */
4ed46869
KH
2216
2217/*
f4dee582 2218 It is not enough to say just "ISO2022" on encoding, we have to
cfb43547 2219 specify more details. In Emacs, each ISO2022 coding system
4ed46869 2220 variant has the following specifications:
8ca3766a 2221 1. Initial designation to G0 through G3.
4ed46869
KH
2222 2. Allows short-form designation?
2223 3. ASCII should be designated to G0 before control characters?
2224 4. ASCII should be designated to G0 at end of line?
2225 5. 7-bit environment or 8-bit environment?
2226 6. Use locking-shift?
2227 7. Use Single-shift?
2228 And the following two are only for Japanese:
2229 8. Use ASCII in place of JIS0201-1976-Roman?
2230 9. Use JISX0208-1983 in place of JISX0208-1978?
2231 These specifications are encoded in `coding->flags' as flag bits
2232 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
f4dee582 2233 details.
4ed46869
KH
2234*/
2235
2236/* Produce codes (escape sequence) for designating CHARSET to graphic
b73bfc1c
KH
2237 register REG at DST, and increment DST. If <final-char> of CHARSET is
2238 '@', 'A', or 'B' and the coding system CODING allows, produce
2239 designation sequence of short-form. */
4ed46869
KH
2240
2241#define ENCODE_DESIGNATION(charset, reg, coding) \
2242 do { \
2243 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2244 char *intermediate_char_94 = "()*+"; \
2245 char *intermediate_char_96 = ",-./"; \
70c22245 2246 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
b73bfc1c 2247 \
70c22245
KH
2248 if (revision < 255) \
2249 { \
4ed46869
KH
2250 *dst++ = ISO_CODE_ESC; \
2251 *dst++ = '&'; \
70c22245 2252 *dst++ = '@' + revision; \
4ed46869 2253 } \
b73bfc1c 2254 *dst++ = ISO_CODE_ESC; \
4ed46869
KH
2255 if (CHARSET_DIMENSION (charset) == 1) \
2256 { \
2257 if (CHARSET_CHARS (charset) == 94) \
2258 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2259 else \
2260 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2261 } \
2262 else \
2263 { \
2264 *dst++ = '$'; \
2265 if (CHARSET_CHARS (charset) == 94) \
2266 { \
b73bfc1c
KH
2267 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2268 || reg != 0 \
2269 || final_char < '@' || final_char > 'B') \
4ed46869
KH
2270 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2271 } \
2272 else \
b73bfc1c 2273 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
4ed46869 2274 } \
b73bfc1c 2275 *dst++ = final_char; \
4ed46869
KH
2276 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2277 } while (0)
2278
2279/* The following two macros produce codes (control character or escape
2280 sequence) for ISO2022 single-shift functions (single-shift-2 and
2281 single-shift-3). */
2282
2283#define ENCODE_SINGLE_SHIFT_2 \
2284 do { \
2285 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2286 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2287 else \
b73bfc1c 2288 *dst++ = ISO_CODE_SS2; \
4ed46869
KH
2289 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2290 } while (0)
2291
fb88bf2d
KH
2292#define ENCODE_SINGLE_SHIFT_3 \
2293 do { \
4ed46869 2294 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
fb88bf2d
KH
2295 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2296 else \
b73bfc1c 2297 *dst++ = ISO_CODE_SS3; \
4ed46869
KH
2298 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2299 } while (0)
2300
2301/* The following four macros produce codes (control character or
2302 escape sequence) for ISO2022 locking-shift functions (shift-in,
2303 shift-out, locking-shift-2, and locking-shift-3). */
2304
b73bfc1c
KH
2305#define ENCODE_SHIFT_IN \
2306 do { \
2307 *dst++ = ISO_CODE_SI; \
4ed46869
KH
2308 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2309 } while (0)
2310
b73bfc1c
KH
2311#define ENCODE_SHIFT_OUT \
2312 do { \
2313 *dst++ = ISO_CODE_SO; \
4ed46869
KH
2314 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2315 } while (0)
2316
2317#define ENCODE_LOCKING_SHIFT_2 \
2318 do { \
2319 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2320 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2321 } while (0)
2322
b73bfc1c
KH
2323#define ENCODE_LOCKING_SHIFT_3 \
2324 do { \
2325 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
4ed46869
KH
2326 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2327 } while (0)
2328
f4dee582
RS
2329/* Produce codes for a DIMENSION1 character whose character set is
2330 CHARSET and whose position-code is C1. Designation and invocation
4ed46869
KH
2331 sequences are also produced in advance if necessary. */
2332
6e85d753
KH
2333#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2334 do { \
2335 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2336 { \
2337 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2338 *dst++ = c1 & 0x7F; \
2339 else \
2340 *dst++ = c1 | 0x80; \
2341 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2342 break; \
2343 } \
2344 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2345 { \
2346 *dst++ = c1 & 0x7F; \
2347 break; \
2348 } \
2349 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2350 { \
2351 *dst++ = c1 | 0x80; \
2352 break; \
2353 } \
6e85d753
KH
2354 else \
2355 /* Since CHARSET is not yet invoked to any graphic planes, we \
2356 must invoke it, or, at first, designate it to some graphic \
2357 register. Then repeat the loop to actually produce the \
2358 character. */ \
2359 dst = encode_invocation_designation (charset, coding, dst); \
4ed46869
KH
2360 } while (1)
2361
f4dee582
RS
2362/* Produce codes for a DIMENSION2 character whose character set is
2363 CHARSET and whose position-codes are C1 and C2. Designation and
4ed46869
KH
2364 invocation codes are also produced in advance if necessary. */
2365
6e85d753
KH
2366#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2367 do { \
2368 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2369 { \
2370 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2371 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2372 else \
2373 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2374 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2375 break; \
2376 } \
2377 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2378 { \
2379 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2380 break; \
2381 } \
2382 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2383 { \
2384 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2385 break; \
2386 } \
6e85d753
KH
2387 else \
2388 /* Since CHARSET is not yet invoked to any graphic planes, we \
2389 must invoke it, or, at first, designate it to some graphic \
2390 register. Then repeat the loop to actually produce the \
2391 character. */ \
2392 dst = encode_invocation_designation (charset, coding, dst); \
4ed46869
KH
2393 } while (1)
2394
05e6f5dc
KH
2395#define ENCODE_ISO_CHARACTER(c) \
2396 do { \
2397 int charset, c1, c2; \
2398 \
2399 SPLIT_CHAR (c, charset, c1, c2); \
2400 if (CHARSET_DEFINED_P (charset)) \
2401 { \
2402 if (CHARSET_DIMENSION (charset) == 1) \
2403 { \
2404 if (charset == CHARSET_ASCII \
2405 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2406 charset = charset_latin_jisx0201; \
2407 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2408 } \
2409 else \
2410 { \
2411 if (charset == charset_jisx0208 \
2412 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2413 charset = charset_jisx0208_1978; \
2414 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2415 } \
2416 } \
2417 else \
2418 { \
2419 *dst++ = c1; \
2420 if (c2 >= 0) \
2421 *dst++ = c2; \
2422 } \
2423 } while (0)
2424
2425
2426/* Instead of encoding character C, produce one or two `?'s. */
2427
0eecad43
KH
2428#define ENCODE_UNSAFE_CHARACTER(c) \
2429 do { \
2430 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2431 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2432 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
84fbb8a0 2433 } while (0)
bdd9fb48 2434
05e6f5dc 2435
4ed46869
KH
2436/* Produce designation and invocation codes at a place pointed by DST
2437 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2438 Return new DST. */
2439
2440unsigned char *
2441encode_invocation_designation (charset, coding, dst)
2442 int charset;
2443 struct coding_system *coding;
2444 unsigned char *dst;
2445{
2446 int reg; /* graphic register number */
2447
2448 /* At first, check designations. */
2449 for (reg = 0; reg < 4; reg++)
2450 if (charset == CODING_SPEC_ISO_DESIGNATION (coding, reg))
2451 break;
2452
2453 if (reg >= 4)
2454 {
2455 /* CHARSET is not yet designated to any graphic registers. */
2456 /* At first check the requested designation. */
2457 reg = CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset);
1ba9e4ab
KH
2458 if (reg == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION)
2459 /* Since CHARSET requests no special designation, designate it
2460 to graphic register 0. */
4ed46869
KH
2461 reg = 0;
2462
2463 ENCODE_DESIGNATION (charset, reg, coding);
2464 }
2465
2466 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != reg
2467 && CODING_SPEC_ISO_INVOCATION (coding, 1) != reg)
2468 {
2469 /* Since the graphic register REG is not invoked to any graphic
2470 planes, invoke it to graphic plane 0. */
2471 switch (reg)
2472 {
2473 case 0: /* graphic register 0 */
2474 ENCODE_SHIFT_IN;
2475 break;
2476
2477 case 1: /* graphic register 1 */
2478 ENCODE_SHIFT_OUT;
2479 break;
2480
2481 case 2: /* graphic register 2 */
2482 if (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT)
2483 ENCODE_SINGLE_SHIFT_2;
2484 else
2485 ENCODE_LOCKING_SHIFT_2;
2486 break;
2487
2488 case 3: /* graphic register 3 */
2489 if (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT)
2490 ENCODE_SINGLE_SHIFT_3;
2491 else
2492 ENCODE_LOCKING_SHIFT_3;
2493 break;
2494 }
2495 }
b73bfc1c 2496
4ed46869
KH
2497 return dst;
2498}
2499
ec6d2bb8
KH
2500/* Produce 2-byte codes for encoded composition rule RULE. */
2501
2502#define ENCODE_COMPOSITION_RULE(rule) \
2503 do { \
2504 int gref, nref; \
2505 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2506 *dst++ = 32 + 81 + gref; \
2507 *dst++ = 32 + nref; \
2508 } while (0)
2509
2510/* Produce codes for indicating the start of a composition sequence
2511 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2512 which specify information about the composition. See the comment
2513 in coding.h for the format of DATA. */
2514
2515#define ENCODE_COMPOSITION_START(coding, data) \
2516 do { \
2517 coding->composing = data[3]; \
2518 *dst++ = ISO_CODE_ESC; \
2519 if (coding->composing == COMPOSITION_RELATIVE) \
2520 *dst++ = '0'; \
2521 else \
2522 { \
2523 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2524 ? '3' : '4'); \
2525 coding->cmp_data_index = coding->cmp_data_start + 4; \
2526 coding->composition_rule_follows = 0; \
2527 } \
2528 } while (0)
2529
2530/* Produce codes for indicating the end of the current composition. */
2531
2532#define ENCODE_COMPOSITION_END(coding, data) \
2533 do { \
2534 *dst++ = ISO_CODE_ESC; \
2535 *dst++ = '1'; \
2536 coding->cmp_data_start += data[0]; \
2537 coding->composing = COMPOSITION_NO; \
2538 if (coding->cmp_data_start == coding->cmp_data->used \
2539 && coding->cmp_data->next) \
2540 { \
2541 coding->cmp_data = coding->cmp_data->next; \
2542 coding->cmp_data_start = 0; \
2543 } \
2544 } while (0)
2545
2546/* Produce composition start sequence ESC 0. Here, this sequence
2547 doesn't mean the start of a new composition but means that we have
2548 just produced components (alternate chars and composition rules) of
2549 the composition and the actual text follows in SRC. */
2550
2551#define ENCODE_COMPOSITION_FAKE_START(coding) \
2552 do { \
2553 *dst++ = ISO_CODE_ESC; \
2554 *dst++ = '0'; \
2555 coding->composing = COMPOSITION_RELATIVE; \
2556 } while (0)
4ed46869
KH
2557
2558/* The following three macros produce codes for indicating direction
2559 of text. */
b73bfc1c
KH
2560#define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2561 do { \
4ed46869 2562 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
b73bfc1c
KH
2563 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2564 else \
2565 *dst++ = ISO_CODE_CSI; \
4ed46869
KH
2566 } while (0)
2567
2568#define ENCODE_DIRECTION_R2L \
b73bfc1c 2569 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
4ed46869
KH
2570
2571#define ENCODE_DIRECTION_L2R \
b73bfc1c 2572 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
4ed46869
KH
2573
2574/* Produce codes for designation and invocation to reset the graphic
2575 planes and registers to initial state. */
e0e989f6
KH
2576#define ENCODE_RESET_PLANE_AND_REGISTER \
2577 do { \
2578 int reg; \
2579 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2580 ENCODE_SHIFT_IN; \
2581 for (reg = 0; reg < 4; reg++) \
2582 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2583 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2584 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2585 ENCODE_DESIGNATION \
2586 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
4ed46869
KH
2587 } while (0)
2588
bdd9fb48 2589/* Produce designation sequences of charsets in the line started from
b73bfc1c 2590 SRC to a place pointed by DST, and return updated DST.
bdd9fb48
KH
2591
2592 If the current block ends before any end-of-line, we may fail to
d46c5b12
KH
2593 find all the necessary designations. */
2594
b73bfc1c
KH
2595static unsigned char *
2596encode_designation_at_bol (coding, translation_table, src, src_end, dst)
e0e989f6 2597 struct coding_system *coding;
b73bfc1c
KH
2598 Lisp_Object translation_table;
2599 unsigned char *src, *src_end, *dst;
e0e989f6 2600{
bdd9fb48
KH
2601 int charset, c, found = 0, reg;
2602 /* Table of charsets to be designated to each graphic register. */
2603 int r[4];
bdd9fb48
KH
2604
2605 for (reg = 0; reg < 4; reg++)
2606 r[reg] = -1;
2607
b73bfc1c 2608 while (found < 4)
e0e989f6 2609 {
b73bfc1c
KH
2610 ONE_MORE_CHAR (c);
2611 if (c == '\n')
2612 break;
93dec019 2613
b73bfc1c 2614 charset = CHAR_CHARSET (c);
e0e989f6 2615 reg = CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset);
d46c5b12 2616 if (reg != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION && r[reg] < 0)
bdd9fb48
KH
2617 {
2618 found++;
2619 r[reg] = charset;
2620 }
bdd9fb48
KH
2621 }
2622
b73bfc1c 2623 label_end_of_loop:
bdd9fb48
KH
2624 if (found)
2625 {
2626 for (reg = 0; reg < 4; reg++)
2627 if (r[reg] >= 0
2628 && CODING_SPEC_ISO_DESIGNATION (coding, reg) != r[reg])
2629 ENCODE_DESIGNATION (r[reg], reg, coding);
e0e989f6 2630 }
b73bfc1c
KH
2631
2632 return dst;
e0e989f6
KH
2633}
2634
4ed46869
KH
2635/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2636
b73bfc1c 2637static void
d46c5b12 2638encode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
4ed46869
KH
2639 struct coding_system *coding;
2640 unsigned char *source, *destination;
2641 int src_bytes, dst_bytes;
4ed46869
KH
2642{
2643 unsigned char *src = source;
2644 unsigned char *src_end = source + src_bytes;
2645 unsigned char *dst = destination;
2646 unsigned char *dst_end = destination + dst_bytes;
b73bfc1c 2647 /* Since the maximum bytes produced by each loop is 20, we subtract 19
4ed46869
KH
2648 from DST_END to assure overflow checking is necessary only at the
2649 head of loop. */
b73bfc1c
KH
2650 unsigned char *adjusted_dst_end = dst_end - 19;
2651 /* SRC_BASE remembers the start position in source in each loop.
2652 The loop will be exited when there's not enough source text to
2653 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2654 there's not enough destination area to produce encoded codes
2655 (within macro EMIT_BYTES). */
2656 unsigned char *src_base;
2657 int c;
2658 Lisp_Object translation_table;
05e6f5dc
KH
2659 Lisp_Object safe_chars;
2660
0eecad43
KH
2661 if (coding->flags & CODING_FLAG_ISO_SAFE)
2662 coding->mode |= CODING_MODE_INHIBIT_UNENCODABLE_CHAR;
2663
6b89e3aa 2664 safe_chars = coding_safe_chars (coding->symbol);
bdd9fb48 2665
b73bfc1c
KH
2666 if (NILP (Venable_character_translation))
2667 translation_table = Qnil;
2668 else
2669 {
2670 translation_table = coding->translation_table_for_encode;
2671 if (NILP (translation_table))
2672 translation_table = Vstandard_translation_table_for_encode;
2673 }
4ed46869 2674
d46c5b12 2675 coding->consumed_char = 0;
b73bfc1c
KH
2676 coding->errors = 0;
2677 while (1)
4ed46869 2678 {
b73bfc1c
KH
2679 src_base = src;
2680
2681 if (dst >= (dst_bytes ? adjusted_dst_end : (src - 19)))
2682 {
2683 coding->result = CODING_FINISH_INSUFFICIENT_DST;
2684 break;
2685 }
4ed46869 2686
e0e989f6
KH
2687 if (coding->flags & CODING_FLAG_ISO_DESIGNATE_AT_BOL
2688 && CODING_SPEC_ISO_BOL (coding))
2689 {
bdd9fb48 2690 /* We have to produce designation sequences if any now. */
b73bfc1c
KH
2691 dst = encode_designation_at_bol (coding, translation_table,
2692 src, src_end, dst);
e0e989f6
KH
2693 CODING_SPEC_ISO_BOL (coding) = 0;
2694 }
2695
ec6d2bb8
KH
2696 /* Check composition start and end. */
2697 if (coding->composing != COMPOSITION_DISABLED
2698 && coding->cmp_data_start < coding->cmp_data->used)
4ed46869 2699 {
ec6d2bb8
KH
2700 struct composition_data *cmp_data = coding->cmp_data;
2701 int *data = cmp_data->data + coding->cmp_data_start;
2702 int this_pos = cmp_data->char_offset + coding->consumed_char;
2703
2704 if (coding->composing == COMPOSITION_RELATIVE)
4ed46869 2705 {
ec6d2bb8
KH
2706 if (this_pos == data[2])
2707 {
2708 ENCODE_COMPOSITION_END (coding, data);
2709 cmp_data = coding->cmp_data;
2710 data = cmp_data->data + coding->cmp_data_start;
2711 }
4ed46869 2712 }
ec6d2bb8 2713 else if (COMPOSING_P (coding))
4ed46869 2714 {
ec6d2bb8
KH
2715 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2716 if (coding->cmp_data_index == coding->cmp_data_start + data[0])
2717 /* We have consumed components of the composition.
8ca3766a 2718 What follows in SRC is the composition's base
ec6d2bb8
KH
2719 text. */
2720 ENCODE_COMPOSITION_FAKE_START (coding);
2721 else
4ed46869 2722 {
ec6d2bb8
KH
2723 int c = cmp_data->data[coding->cmp_data_index++];
2724 if (coding->composition_rule_follows)
2725 {
2726 ENCODE_COMPOSITION_RULE (c);
2727 coding->composition_rule_follows = 0;
2728 }
2729 else
2730 {
0eecad43 2731 if (coding->mode & CODING_MODE_INHIBIT_UNENCODABLE_CHAR
05e6f5dc
KH
2732 && ! CODING_SAFE_CHAR_P (safe_chars, c))
2733 ENCODE_UNSAFE_CHARACTER (c);
2734 else
2735 ENCODE_ISO_CHARACTER (c);
ec6d2bb8
KH
2736 if (coding->composing == COMPOSITION_WITH_RULE_ALTCHARS)
2737 coding->composition_rule_follows = 1;
2738 }
4ed46869
KH
2739 continue;
2740 }
ec6d2bb8
KH
2741 }
2742 if (!COMPOSING_P (coding))
2743 {
2744 if (this_pos == data[1])
4ed46869 2745 {
ec6d2bb8
KH
2746 ENCODE_COMPOSITION_START (coding, data);
2747 continue;
4ed46869 2748 }
4ed46869
KH
2749 }
2750 }
ec6d2bb8 2751
b73bfc1c 2752 ONE_MORE_CHAR (c);
4ed46869 2753
b73bfc1c
KH
2754 /* Now encode the character C. */
2755 if (c < 0x20 || c == 0x7F)
2756 {
2757 if (c == '\r')
19a8d9e0 2758 {
b73bfc1c
KH
2759 if (! (coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
2760 {
2761 if (coding->flags & CODING_FLAG_ISO_RESET_AT_CNTL)
2762 ENCODE_RESET_PLANE_AND_REGISTER;
2763 *dst++ = c;
2764 continue;
2765 }
2766 /* fall down to treat '\r' as '\n' ... */
2767 c = '\n';
19a8d9e0 2768 }
b73bfc1c 2769 if (c == '\n')
19a8d9e0 2770 {
b73bfc1c
KH
2771 if (coding->flags & CODING_FLAG_ISO_RESET_AT_EOL)
2772 ENCODE_RESET_PLANE_AND_REGISTER;
2773 if (coding->flags & CODING_FLAG_ISO_INIT_AT_BOL)
2774 bcopy (coding->spec.iso2022.initial_designation,
2775 coding->spec.iso2022.current_designation,
2776 sizeof coding->spec.iso2022.initial_designation);
2777 if (coding->eol_type == CODING_EOL_LF
2778 || coding->eol_type == CODING_EOL_UNDECIDED)
2779 *dst++ = ISO_CODE_LF;
2780 else if (coding->eol_type == CODING_EOL_CRLF)
2781 *dst++ = ISO_CODE_CR, *dst++ = ISO_CODE_LF;
2782 else
2783 *dst++ = ISO_CODE_CR;
2784 CODING_SPEC_ISO_BOL (coding) = 1;
19a8d9e0 2785 }
93dec019 2786 else
19a8d9e0 2787 {
b73bfc1c
KH
2788 if (coding->flags & CODING_FLAG_ISO_RESET_AT_CNTL)
2789 ENCODE_RESET_PLANE_AND_REGISTER;
2790 *dst++ = c;
19a8d9e0 2791 }
4ed46869 2792 }
b73bfc1c 2793 else if (ASCII_BYTE_P (c))
05e6f5dc 2794 ENCODE_ISO_CHARACTER (c);
b73bfc1c 2795 else if (SINGLE_BYTE_CHAR_P (c))
88993dfd 2796 {
b73bfc1c
KH
2797 *dst++ = c;
2798 coding->errors++;
88993dfd 2799 }
0eecad43 2800 else if (coding->mode & CODING_MODE_INHIBIT_UNENCODABLE_CHAR
05e6f5dc
KH
2801 && ! CODING_SAFE_CHAR_P (safe_chars, c))
2802 ENCODE_UNSAFE_CHARACTER (c);
b73bfc1c 2803 else
05e6f5dc 2804 ENCODE_ISO_CHARACTER (c);
b73bfc1c
KH
2805
2806 coding->consumed_char++;
84fbb8a0 2807 }
b73bfc1c
KH
2808
2809 label_end_of_loop:
2810 coding->consumed = src_base - source;
d46c5b12 2811 coding->produced = coding->produced_char = dst - destination;
4ed46869
KH
2812}
2813
2814\f
2815/*** 4. SJIS and BIG5 handlers ***/
2816
cfb43547 2817/* Although SJIS and BIG5 are not ISO coding systems, they are used
4ed46869
KH
2818 quite widely. So, for the moment, Emacs supports them in the bare
2819 C code. But, in the future, they may be supported only by CCL. */
2820
2821/* SJIS is a coding system encoding three character sets: ASCII, right
2822 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2823 as is. A character of charset katakana-jisx0201 is encoded by
2824 "position-code + 0x80". A character of charset japanese-jisx0208
2825 is encoded in 2-byte but two position-codes are divided and shifted
cfb43547 2826 so that it fits in the range below.
4ed46869
KH
2827
2828 --- CODE RANGE of SJIS ---
2829 (character set) (range)
2830 ASCII 0x00 .. 0x7F
682169fe 2831 KATAKANA-JISX0201 0xA1 .. 0xDF
c28a9453 2832 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
d14d03ac 2833 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
4ed46869
KH
2834 -------------------------------
2835
2836*/
2837
2838/* BIG5 is a coding system encoding two character sets: ASCII and
2839 Big5. An ASCII character is encoded as is. Big5 is a two-byte
cfb43547 2840 character set and is encoded in two bytes.
4ed46869
KH
2841
2842 --- CODE RANGE of BIG5 ---
2843 (character set) (range)
2844 ASCII 0x00 .. 0x7F
2845 Big5 (1st byte) 0xA1 .. 0xFE
2846 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2847 --------------------------
2848
2849 Since the number of characters in Big5 is larger than maximum
2850 characters in Emacs' charset (96x96), it can't be handled as one
2851 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2852 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2853 contains frequently used characters and the latter contains less
2854 frequently used characters. */
2855
2856/* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2857 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
f458a8e0 2858 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
4ed46869
KH
2859 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2860
2861/* Number of Big5 characters which have the same code in 1st byte. */
2862#define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2863
2864#define DECODE_BIG5(b1, b2, charset, c1, c2) \
2865 do { \
2866 unsigned int temp \
2867 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2868 if (b1 < 0xC9) \
2869 charset = charset_big5_1; \
2870 else \
2871 { \
2872 charset = charset_big5_2; \
2873 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2874 } \
2875 c1 = temp / (0xFF - 0xA1) + 0x21; \
2876 c2 = temp % (0xFF - 0xA1) + 0x21; \
2877 } while (0)
2878
2879#define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2880 do { \
2881 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2882 if (charset == charset_big5_2) \
2883 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2884 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2885 b2 = temp % BIG5_SAME_ROW; \
2886 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2887 } while (0)
2888
2889/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2890 Check if a text is encoded in SJIS. If it is, return
2891 CODING_CATEGORY_MASK_SJIS, else return 0. */
2892
0a28aafb
KH
2893static int
2894detect_coding_sjis (src, src_end, multibytep)
4ed46869 2895 unsigned char *src, *src_end;
0a28aafb 2896 int multibytep;
4ed46869 2897{
b73bfc1c
KH
2898 int c;
2899 /* Dummy for ONE_MORE_BYTE. */
2900 struct coding_system dummy_coding;
2901 struct coding_system *coding = &dummy_coding;
4ed46869 2902
b73bfc1c 2903 while (1)
4ed46869 2904 {
0a28aafb 2905 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
682169fe
KH
2906 if (c < 0x80)
2907 continue;
2908 if (c == 0x80 || c == 0xA0 || c > 0xEF)
2909 return 0;
2910 if (c <= 0x9F || c >= 0xE0)
4ed46869 2911 {
682169fe
KH
2912 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
2913 if (c < 0x40 || c == 0x7F || c > 0xFC)
4ed46869
KH
2914 return 0;
2915 }
2916 }
b73bfc1c 2917 label_end_of_loop:
4ed46869
KH
2918 return CODING_CATEGORY_MASK_SJIS;
2919}
2920
2921/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2922 Check if a text is encoded in BIG5. If it is, return
2923 CODING_CATEGORY_MASK_BIG5, else return 0. */
2924
0a28aafb
KH
2925static int
2926detect_coding_big5 (src, src_end, multibytep)
4ed46869 2927 unsigned char *src, *src_end;
0a28aafb 2928 int multibytep;
4ed46869 2929{
b73bfc1c
KH
2930 int c;
2931 /* Dummy for ONE_MORE_BYTE. */
2932 struct coding_system dummy_coding;
2933 struct coding_system *coding = &dummy_coding;
4ed46869 2934
b73bfc1c 2935 while (1)
4ed46869 2936 {
0a28aafb 2937 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
682169fe
KH
2938 if (c < 0x80)
2939 continue;
2940 if (c < 0xA1 || c > 0xFE)
2941 return 0;
2942 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
2943 if (c < 0x40 || (c > 0x7F && c < 0xA1) || c > 0xFE)
2944 return 0;
4ed46869 2945 }
b73bfc1c 2946 label_end_of_loop:
4ed46869
KH
2947 return CODING_CATEGORY_MASK_BIG5;
2948}
2949
fa42c37f
KH
2950/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2951 Check if a text is encoded in UTF-8. If it is, return
2952 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2953
2954#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2955#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2956#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2957#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2958#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2959#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2960#define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2961
0a28aafb
KH
2962static int
2963detect_coding_utf_8 (src, src_end, multibytep)
fa42c37f 2964 unsigned char *src, *src_end;
0a28aafb 2965 int multibytep;
fa42c37f
KH
2966{
2967 unsigned char c;
2968 int seq_maybe_bytes;
b73bfc1c
KH
2969 /* Dummy for ONE_MORE_BYTE. */
2970 struct coding_system dummy_coding;
2971 struct coding_system *coding = &dummy_coding;
fa42c37f 2972
b73bfc1c 2973 while (1)
fa42c37f 2974 {
0a28aafb 2975 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
fa42c37f
KH
2976 if (UTF_8_1_OCTET_P (c))
2977 continue;
2978 else if (UTF_8_2_OCTET_LEADING_P (c))
2979 seq_maybe_bytes = 1;
2980 else if (UTF_8_3_OCTET_LEADING_P (c))
2981 seq_maybe_bytes = 2;
2982 else if (UTF_8_4_OCTET_LEADING_P (c))
2983 seq_maybe_bytes = 3;
2984 else if (UTF_8_5_OCTET_LEADING_P (c))
2985 seq_maybe_bytes = 4;
2986 else if (UTF_8_6_OCTET_LEADING_P (c))
2987 seq_maybe_bytes = 5;
2988 else
2989 return 0;
2990
2991 do
2992 {
0a28aafb 2993 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
fa42c37f
KH
2994 if (!UTF_8_EXTRA_OCTET_P (c))
2995 return 0;
2996 seq_maybe_bytes--;
2997 }
2998 while (seq_maybe_bytes > 0);
2999 }
3000
b73bfc1c 3001 label_end_of_loop:
fa42c37f
KH
3002 return CODING_CATEGORY_MASK_UTF_8;
3003}
3004
3005/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3006 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
3007 Little Endian (otherwise). If it is, return
3008 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
3009 else return 0. */
3010
3011#define UTF_16_INVALID_P(val) \
3012 (((val) == 0xFFFE) \
3013 || ((val) == 0xFFFF))
3014
3015#define UTF_16_HIGH_SURROGATE_P(val) \
3016 (((val) & 0xD800) == 0xD800)
3017
3018#define UTF_16_LOW_SURROGATE_P(val) \
3019 (((val) & 0xDC00) == 0xDC00)
3020
0a28aafb
KH
3021static int
3022detect_coding_utf_16 (src, src_end, multibytep)
fa42c37f 3023 unsigned char *src, *src_end;
0a28aafb 3024 int multibytep;
fa42c37f 3025{
b73bfc1c 3026 unsigned char c1, c2;
1c7457e2 3027 /* Dummy for ONE_MORE_BYTE_CHECK_MULTIBYTE. */
b73bfc1c
KH
3028 struct coding_system dummy_coding;
3029 struct coding_system *coding = &dummy_coding;
fa42c37f 3030
0a28aafb
KH
3031 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1, multibytep);
3032 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2, multibytep);
b73bfc1c
KH
3033
3034 if ((c1 == 0xFF) && (c2 == 0xFE))
fa42c37f 3035 return CODING_CATEGORY_MASK_UTF_16_LE;
b73bfc1c 3036 else if ((c1 == 0xFE) && (c2 == 0xFF))
fa42c37f
KH
3037 return CODING_CATEGORY_MASK_UTF_16_BE;
3038
b73bfc1c 3039 label_end_of_loop:
fa42c37f
KH
3040 return 0;
3041}
3042
4ed46869
KH
3043/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
3044 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
3045
b73bfc1c 3046static void
4ed46869 3047decode_coding_sjis_big5 (coding, source, destination,
d46c5b12 3048 src_bytes, dst_bytes, sjis_p)
4ed46869
KH
3049 struct coding_system *coding;
3050 unsigned char *source, *destination;
3051 int src_bytes, dst_bytes;
4ed46869
KH
3052 int sjis_p;
3053{
3054 unsigned char *src = source;
3055 unsigned char *src_end = source + src_bytes;
3056 unsigned char *dst = destination;
3057 unsigned char *dst_end = destination + dst_bytes;
b73bfc1c
KH
3058 /* SRC_BASE remembers the start position in source in each loop.
3059 The loop will be exited when there's not enough source code
3060 (within macro ONE_MORE_BYTE), or when there's not enough
3061 destination area to produce a character (within macro
3062 EMIT_CHAR). */
3063 unsigned char *src_base;
3064 Lisp_Object translation_table;
a5d301df 3065
b73bfc1c
KH
3066 if (NILP (Venable_character_translation))
3067 translation_table = Qnil;
3068 else
3069 {
3070 translation_table = coding->translation_table_for_decode;
3071 if (NILP (translation_table))
3072 translation_table = Vstandard_translation_table_for_decode;
3073 }
4ed46869 3074
d46c5b12 3075 coding->produced_char = 0;
b73bfc1c 3076 while (1)
4ed46869 3077 {
85478bc6 3078 int c, charset, c1, c2 = 0;
b73bfc1c
KH
3079
3080 src_base = src;
3081 ONE_MORE_BYTE (c1);
3082
3083 if (c1 < 0x80)
4ed46869 3084 {
b73bfc1c
KH
3085 charset = CHARSET_ASCII;
3086 if (c1 < 0x20)
4ed46869 3087 {
b73bfc1c 3088 if (c1 == '\r')
d46c5b12 3089 {
b73bfc1c 3090 if (coding->eol_type == CODING_EOL_CRLF)
d46c5b12 3091 {
b73bfc1c
KH
3092 ONE_MORE_BYTE (c2);
3093 if (c2 == '\n')
3094 c1 = c2;
b73bfc1c
KH
3095 else
3096 /* To process C2 again, SRC is subtracted by 1. */
3097 src--;
d46c5b12 3098 }
b73bfc1c
KH
3099 else if (coding->eol_type == CODING_EOL_CR)
3100 c1 = '\n';
3101 }
3102 else if (c1 == '\n'
3103 && (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
3104 && (coding->eol_type == CODING_EOL_CR
3105 || coding->eol_type == CODING_EOL_CRLF))
3106 {
3107 coding->result = CODING_FINISH_INCONSISTENT_EOL;
3108 goto label_end_of_loop;
d46c5b12 3109 }
4ed46869 3110 }
4ed46869 3111 }
54f78171 3112 else
b73bfc1c 3113 {
4ed46869
KH
3114 if (sjis_p)
3115 {
682169fe 3116 if (c1 == 0x80 || c1 == 0xA0 || c1 > 0xEF)
b73bfc1c 3117 goto label_invalid_code;
682169fe 3118 if (c1 <= 0x9F || c1 >= 0xE0)
fb88bf2d 3119 {
54f78171
KH
3120 /* SJIS -> JISX0208 */
3121 ONE_MORE_BYTE (c2);
b73bfc1c
KH
3122 if (c2 < 0x40 || c2 == 0x7F || c2 > 0xFC)
3123 goto label_invalid_code;
3124 DECODE_SJIS (c1, c2, c1, c2);
3125 charset = charset_jisx0208;
5e34de15 3126 }
fb88bf2d 3127 else
b73bfc1c
KH
3128 /* SJIS -> JISX0201-Kana */
3129 charset = charset_katakana_jisx0201;
4ed46869 3130 }
fb88bf2d 3131 else
fb88bf2d 3132 {
54f78171 3133 /* BIG5 -> Big5 */
682169fe 3134 if (c1 < 0xA0 || c1 > 0xFE)
b73bfc1c
KH
3135 goto label_invalid_code;
3136 ONE_MORE_BYTE (c2);
3137 if (c2 < 0x40 || (c2 > 0x7E && c2 < 0xA1) || c2 > 0xFE)
3138 goto label_invalid_code;
3139 DECODE_BIG5 (c1, c2, charset, c1, c2);
4ed46869
KH
3140 }
3141 }
4ed46869 3142
b73bfc1c
KH
3143 c = DECODE_ISO_CHARACTER (charset, c1, c2);
3144 EMIT_CHAR (c);
fb88bf2d
KH
3145 continue;
3146
b73bfc1c
KH
3147 label_invalid_code:
3148 coding->errors++;
4ed46869 3149 src = src_base;
b73bfc1c
KH
3150 c = *src++;
3151 EMIT_CHAR (c);
fb88bf2d 3152 }
d46c5b12 3153
b73bfc1c
KH
3154 label_end_of_loop:
3155 coding->consumed = coding->consumed_char = src_base - source;
d46c5b12 3156 coding->produced = dst - destination;
b73bfc1c 3157 return;
4ed46869
KH
3158}
3159
3160/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
b73bfc1c
KH
3161 This function can encode charsets `ascii', `katakana-jisx0201',
3162 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
3163 are sure that all these charsets are registered as official charset
4ed46869
KH
3164 (i.e. do not have extended leading-codes). Characters of other
3165 charsets are produced without any encoding. If SJIS_P is 1, encode
3166 SJIS text, else encode BIG5 text. */
3167
b73bfc1c 3168static void
4ed46869 3169encode_coding_sjis_big5 (coding, source, destination,
d46c5b12 3170 src_bytes, dst_bytes, sjis_p)
4ed46869
KH
3171 struct coding_system *coding;
3172 unsigned char *source, *destination;
3173 int src_bytes, dst_bytes;
4ed46869
KH
3174 int sjis_p;
3175{
3176 unsigned char *src = source;
3177 unsigned char *src_end = source + src_bytes;
3178 unsigned char *dst = destination;
3179 unsigned char *dst_end = destination + dst_bytes;
b73bfc1c
KH
3180 /* SRC_BASE remembers the start position in source in each loop.
3181 The loop will be exited when there's not enough source text to
3182 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3183 there's not enough destination area to produce encoded codes
3184 (within macro EMIT_BYTES). */
3185 unsigned char *src_base;
3186 Lisp_Object translation_table;
4ed46869 3187
b73bfc1c
KH
3188 if (NILP (Venable_character_translation))
3189 translation_table = Qnil;
3190 else
4ed46869 3191 {
39658efc 3192 translation_table = coding->translation_table_for_encode;
b73bfc1c 3193 if (NILP (translation_table))
39658efc 3194 translation_table = Vstandard_translation_table_for_encode;
b73bfc1c 3195 }
a5d301df 3196
b73bfc1c
KH
3197 while (1)
3198 {
3199 int c, charset, c1, c2;
4ed46869 3200
b73bfc1c
KH
3201 src_base = src;
3202 ONE_MORE_CHAR (c);
93dec019 3203
b73bfc1c
KH
3204 /* Now encode the character C. */
3205 if (SINGLE_BYTE_CHAR_P (c))
3206 {
3207 switch (c)
4ed46869 3208 {
b73bfc1c 3209 case '\r':
7371fe0a 3210 if (!(coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
b73bfc1c
KH
3211 {
3212 EMIT_ONE_BYTE (c);
3213 break;
3214 }
3215 c = '\n';
3216 case '\n':
3217 if (coding->eol_type == CODING_EOL_CRLF)
3218 {
3219 EMIT_TWO_BYTES ('\r', c);
3220 break;
3221 }
3222 else if (coding->eol_type == CODING_EOL_CR)
3223 c = '\r';
3224 default:
3225 EMIT_ONE_BYTE (c);
3226 }
3227 }
3228 else
3229 {
3230 SPLIT_CHAR (c, charset, c1, c2);
3231 if (sjis_p)
3232 {
3233 if (charset == charset_jisx0208
3234 || charset == charset_jisx0208_1978)
3235 {
3236 ENCODE_SJIS (c1, c2, c1, c2);
3237 EMIT_TWO_BYTES (c1, c2);
3238 }
39658efc
KH
3239 else if (charset == charset_katakana_jisx0201)
3240 EMIT_ONE_BYTE (c1 | 0x80);
fc53a214
KH
3241 else if (charset == charset_latin_jisx0201)
3242 EMIT_ONE_BYTE (c1);
0eecad43
KH
3243 else if (coding->mode & CODING_MODE_INHIBIT_UNENCODABLE_CHAR)
3244 {
3245 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER);
3246 if (CHARSET_WIDTH (charset) > 1)
3247 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER);
3248 }
b73bfc1c
KH
3249 else
3250 /* There's no way other than producing the internal
3251 codes as is. */
3252 EMIT_BYTES (src_base, src);
4ed46869 3253 }
4ed46869 3254 else
b73bfc1c
KH
3255 {
3256 if (charset == charset_big5_1 || charset == charset_big5_2)
3257 {
3258 ENCODE_BIG5 (charset, c1, c2, c1, c2);
3259 EMIT_TWO_BYTES (c1, c2);
3260 }
0eecad43
KH
3261 else if (coding->mode & CODING_MODE_INHIBIT_UNENCODABLE_CHAR)
3262 {
3263 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER);
3264 if (CHARSET_WIDTH (charset) > 1)
3265 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER);
3266 }
b73bfc1c
KH
3267 else
3268 /* There's no way other than producing the internal
3269 codes as is. */
3270 EMIT_BYTES (src_base, src);
3271 }
4ed46869 3272 }
b73bfc1c 3273 coding->consumed_char++;
4ed46869
KH
3274 }
3275
b73bfc1c
KH
3276 label_end_of_loop:
3277 coding->consumed = src_base - source;
d46c5b12 3278 coding->produced = coding->produced_char = dst - destination;
4ed46869
KH
3279}
3280
3281\f
1397dc18
KH
3282/*** 5. CCL handlers ***/
3283
3284/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3285 Check if a text is encoded in a coding system of which
3286 encoder/decoder are written in CCL program. If it is, return
3287 CODING_CATEGORY_MASK_CCL, else return 0. */
3288
0a28aafb
KH
3289static int
3290detect_coding_ccl (src, src_end, multibytep)
1397dc18 3291 unsigned char *src, *src_end;
0a28aafb 3292 int multibytep;
1397dc18
KH
3293{
3294 unsigned char *valid;
b73bfc1c
KH
3295 int c;
3296 /* Dummy for ONE_MORE_BYTE. */
3297 struct coding_system dummy_coding;
3298 struct coding_system *coding = &dummy_coding;
1397dc18
KH
3299
3300 /* No coding system is assigned to coding-category-ccl. */
3301 if (!coding_system_table[CODING_CATEGORY_IDX_CCL])
3302 return 0;
3303
3304 valid = coding_system_table[CODING_CATEGORY_IDX_CCL]->spec.ccl.valid_codes;
b73bfc1c 3305 while (1)
1397dc18 3306 {
0a28aafb 3307 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
b73bfc1c
KH
3308 if (! valid[c])
3309 return 0;
1397dc18 3310 }
b73bfc1c 3311 label_end_of_loop:
1397dc18
KH
3312 return CODING_CATEGORY_MASK_CCL;
3313}
3314
3315\f
3316/*** 6. End-of-line handlers ***/
4ed46869 3317
b73bfc1c 3318/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
4ed46869 3319
b73bfc1c 3320static void
d46c5b12 3321decode_eol (coding, source, destination, src_bytes, dst_bytes)
4ed46869
KH
3322 struct coding_system *coding;
3323 unsigned char *source, *destination;
3324 int src_bytes, dst_bytes;
4ed46869
KH
3325{
3326 unsigned char *src = source;
4ed46869 3327 unsigned char *dst = destination;
b73bfc1c
KH
3328 unsigned char *src_end = src + src_bytes;
3329 unsigned char *dst_end = dst + dst_bytes;
3330 Lisp_Object translation_table;
3331 /* SRC_BASE remembers the start position in source in each loop.
3332 The loop will be exited when there's not enough source code
3333 (within macro ONE_MORE_BYTE), or when there's not enough
3334 destination area to produce a character (within macro
3335 EMIT_CHAR). */
3336 unsigned char *src_base;
3337 int c;
3338
3339 translation_table = Qnil;
4ed46869
KH
3340 switch (coding->eol_type)
3341 {
3342 case CODING_EOL_CRLF:
b73bfc1c 3343 while (1)
d46c5b12 3344 {
b73bfc1c
KH
3345 src_base = src;
3346 ONE_MORE_BYTE (c);
3347 if (c == '\r')
fb88bf2d 3348 {
b73bfc1c
KH
3349 ONE_MORE_BYTE (c);
3350 if (c != '\n')
3351 {
b73bfc1c
KH
3352 src--;
3353 c = '\r';
3354 }
fb88bf2d 3355 }
b73bfc1c
KH
3356 else if (c == '\n'
3357 && (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL))
d46c5b12 3358 {
b73bfc1c
KH
3359 coding->result = CODING_FINISH_INCONSISTENT_EOL;
3360 goto label_end_of_loop;
d46c5b12 3361 }
b73bfc1c 3362 EMIT_CHAR (c);
d46c5b12 3363 }
b73bfc1c
KH
3364 break;
3365
3366 case CODING_EOL_CR:
3367 while (1)
d46c5b12 3368 {
b73bfc1c
KH
3369 src_base = src;
3370 ONE_MORE_BYTE (c);
3371 if (c == '\n')
3372 {
3373 if (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
3374 {
3375 coding->result = CODING_FINISH_INCONSISTENT_EOL;
3376 goto label_end_of_loop;
3377 }
3378 }
3379 else if (c == '\r')
3380 c = '\n';
3381 EMIT_CHAR (c);
d46c5b12 3382 }
4ed46869
KH
3383 break;
3384
b73bfc1c
KH
3385 default: /* no need for EOL handling */
3386 while (1)
d46c5b12 3387 {
b73bfc1c
KH
3388 src_base = src;
3389 ONE_MORE_BYTE (c);
3390 EMIT_CHAR (c);
d46c5b12 3391 }
4ed46869
KH
3392 }
3393
b73bfc1c
KH
3394 label_end_of_loop:
3395 coding->consumed = coding->consumed_char = src_base - source;
3396 coding->produced = dst - destination;
3397 return;
4ed46869
KH
3398}
3399
3400/* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
b73bfc1c 3401 format of end-of-line according to `coding->eol_type'. It also
8ca3766a 3402 convert multibyte form 8-bit characters to unibyte if
b73bfc1c
KH
3403 CODING->src_multibyte is nonzero. If `coding->mode &
3404 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3405 also means end-of-line. */
4ed46869 3406
b73bfc1c 3407static void
d46c5b12 3408encode_eol (coding, source, destination, src_bytes, dst_bytes)
4ed46869 3409 struct coding_system *coding;
a4244313
KR
3410 const unsigned char *source;
3411 unsigned char *destination;
4ed46869 3412 int src_bytes, dst_bytes;
4ed46869 3413{
a4244313 3414 const unsigned char *src = source;
4ed46869 3415 unsigned char *dst = destination;
a4244313 3416 const unsigned char *src_end = src + src_bytes;
b73bfc1c
KH
3417 unsigned char *dst_end = dst + dst_bytes;
3418 Lisp_Object translation_table;
3419 /* SRC_BASE remembers the start position in source in each loop.
3420 The loop will be exited when there's not enough source text to
3421 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3422 there's not enough destination area to produce encoded codes
3423 (within macro EMIT_BYTES). */
a4244313
KR
3424 const unsigned char *src_base;
3425 unsigned char *tmp;
b73bfc1c
KH
3426 int c;
3427 int selective_display = coding->mode & CODING_MODE_SELECTIVE_DISPLAY;
3428
3429 translation_table = Qnil;
3430 if (coding->src_multibyte
3431 && *(src_end - 1) == LEADING_CODE_8_BIT_CONTROL)
3432 {
3433 src_end--;
3434 src_bytes--;
3435 coding->result = CODING_FINISH_INSUFFICIENT_SRC;
3436 }
fb88bf2d 3437
d46c5b12
KH
3438 if (coding->eol_type == CODING_EOL_CRLF)
3439 {
b73bfc1c 3440 while (src < src_end)
d46c5b12 3441 {
b73bfc1c 3442 src_base = src;
d46c5b12 3443 c = *src++;
b73bfc1c
KH
3444 if (c >= 0x20)
3445 EMIT_ONE_BYTE (c);
3446 else if (c == '\n' || (c == '\r' && selective_display))
3447 EMIT_TWO_BYTES ('\r', '\n');
d46c5b12 3448 else
b73bfc1c 3449 EMIT_ONE_BYTE (c);
d46c5b12 3450 }
ff2b1ea9 3451 src_base = src;
b73bfc1c 3452 label_end_of_loop:
005f0d35 3453 ;
d46c5b12
KH
3454 }
3455 else
4ed46869 3456 {
78a629d2 3457 if (!dst_bytes || src_bytes <= dst_bytes)
4ed46869 3458 {
b73bfc1c
KH
3459 safe_bcopy (src, dst, src_bytes);
3460 src_base = src_end;
3461 dst += src_bytes;
d46c5b12 3462 }
d46c5b12 3463 else
b73bfc1c
KH
3464 {
3465 if (coding->src_multibyte
3466 && *(src + dst_bytes - 1) == LEADING_CODE_8_BIT_CONTROL)
3467 dst_bytes--;
3468 safe_bcopy (src, dst, dst_bytes);
3469 src_base = src + dst_bytes;
3470 dst = destination + dst_bytes;
3471 coding->result = CODING_FINISH_INSUFFICIENT_DST;
3472 }
993824c9 3473 if (coding->eol_type == CODING_EOL_CR)
d46c5b12 3474 {
a4244313
KR
3475 for (tmp = destination; tmp < dst; tmp++)
3476 if (*tmp == '\n') *tmp = '\r';
d46c5b12 3477 }
b73bfc1c 3478 else if (selective_display)
d46c5b12 3479 {
a4244313
KR
3480 for (tmp = destination; tmp < dst; tmp++)
3481 if (*tmp == '\r') *tmp = '\n';
4ed46869 3482 }
4ed46869 3483 }
b73bfc1c
KH
3484 if (coding->src_multibyte)
3485 dst = destination + str_as_unibyte (destination, dst - destination);
4ed46869 3486
b73bfc1c
KH
3487 coding->consumed = src_base - source;
3488 coding->produced = dst - destination;
78a629d2 3489 coding->produced_char = coding->produced;
4ed46869
KH
3490}
3491
3492\f
1397dc18 3493/*** 7. C library functions ***/
4ed46869 3494
cfb43547 3495/* In Emacs Lisp, a coding system is represented by a Lisp symbol which
4ed46869 3496 has a property `coding-system'. The value of this property is a
cfb43547 3497 vector of length 5 (called the coding-vector). Among elements of
4ed46869
KH
3498 this vector, the first (element[0]) and the fifth (element[4])
3499 carry important information for decoding/encoding. Before
3500 decoding/encoding, this information should be set in fields of a
3501 structure of type `coding_system'.
3502
cfb43547 3503 The value of the property `coding-system' can be a symbol of another
4ed46869
KH
3504 subsidiary coding-system. In that case, Emacs gets coding-vector
3505 from that symbol.
3506
3507 `element[0]' contains information to be set in `coding->type'. The
3508 value and its meaning is as follows:
3509
0ef69138
KH
3510 0 -- coding_type_emacs_mule
3511 1 -- coding_type_sjis
3512 2 -- coding_type_iso2022
3513 3 -- coding_type_big5
3514 4 -- coding_type_ccl encoder/decoder written in CCL
3515 nil -- coding_type_no_conversion
3516 t -- coding_type_undecided (automatic conversion on decoding,
3517 no-conversion on encoding)
4ed46869
KH
3518
3519 `element[4]' contains information to be set in `coding->flags' and
3520 `coding->spec'. The meaning varies by `coding->type'.
3521
3522 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3523 of length 32 (of which the first 13 sub-elements are used now).
3524 Meanings of these sub-elements are:
3525
3526 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3527 If the value is an integer of valid charset, the charset is
3528 assumed to be designated to graphic register N initially.
3529
3530 If the value is minus, it is a minus value of charset which
3531 reserves graphic register N, which means that the charset is
3532 not designated initially but should be designated to graphic
3533 register N just before encoding a character in that charset.
3534
3535 If the value is nil, graphic register N is never used on
3536 encoding.
93dec019 3537
4ed46869
KH
3538 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3539 Each value takes t or nil. See the section ISO2022 of
3540 `coding.h' for more information.
3541
3542 If `coding->type' is `coding_type_big5', element[4] is t to denote
3543 BIG5-ETen or nil to denote BIG5-HKU.
3544
3545 If `coding->type' takes the other value, element[4] is ignored.
3546
cfb43547 3547 Emacs Lisp's coding systems also carry information about format of
4ed46869
KH
3548 end-of-line in a value of property `eol-type'. If the value is
3549 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3550 means CODING_EOL_CR. If it is not integer, it should be a vector
3551 of subsidiary coding systems of which property `eol-type' has one
cfb43547 3552 of the above values.
4ed46869
KH
3553
3554*/
3555
3556/* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3557 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3558 is setup so that no conversion is necessary and return -1, else
3559 return 0. */
3560
3561int
e0e989f6
KH
3562setup_coding_system (coding_system, coding)
3563 Lisp_Object coding_system;
4ed46869
KH
3564 struct coding_system *coding;
3565{
d46c5b12 3566 Lisp_Object coding_spec, coding_type, eol_type, plist;
4608c386 3567 Lisp_Object val;
4ed46869 3568
c07c8e12
KH
3569 /* At first, zero clear all members. */
3570 bzero (coding, sizeof (struct coding_system));
3571
d46c5b12 3572 /* Initialize some fields required for all kinds of coding systems. */
774324d6 3573 coding->symbol = coding_system;
d46c5b12
KH
3574 coding->heading_ascii = -1;
3575 coding->post_read_conversion = coding->pre_write_conversion = Qnil;
ec6d2bb8
KH
3576 coding->composing = COMPOSITION_DISABLED;
3577 coding->cmp_data = NULL;
1f5dbf34
KH
3578
3579 if (NILP (coding_system))
3580 goto label_invalid_coding_system;
3581
4608c386 3582 coding_spec = Fget (coding_system, Qcoding_system);
1f5dbf34 3583
4608c386
KH
3584 if (!VECTORP (coding_spec)
3585 || XVECTOR (coding_spec)->size != 5
3586 || !CONSP (XVECTOR (coding_spec)->contents[3]))
4ed46869 3587 goto label_invalid_coding_system;
4608c386 3588
d46c5b12
KH
3589 eol_type = inhibit_eol_conversion ? Qnil : Fget (coding_system, Qeol_type);
3590 if (VECTORP (eol_type))
3591 {
3592 coding->eol_type = CODING_EOL_UNDECIDED;
3593 coding->common_flags = CODING_REQUIRE_DETECTION_MASK;
3594 }
3595 else if (XFASTINT (eol_type) == 1)
3596 {
3597 coding->eol_type = CODING_EOL_CRLF;
3598 coding->common_flags
3599 = CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK;
3600 }
3601 else if (XFASTINT (eol_type) == 2)
3602 {
3603 coding->eol_type = CODING_EOL_CR;
3604 coding->common_flags
3605 = CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK;
3606 }
3607 else
3608 coding->eol_type = CODING_EOL_LF;
3609
3610 coding_type = XVECTOR (coding_spec)->contents[0];
3611 /* Try short cut. */
3612 if (SYMBOLP (coding_type))
3613 {
3614 if (EQ (coding_type, Qt))
3615 {
3616 coding->type = coding_type_undecided;
3617 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
3618 }
3619 else
3620 coding->type = coding_type_no_conversion;
9b96232f
KH
3621 /* Initialize this member. Any thing other than
3622 CODING_CATEGORY_IDX_UTF_16_BE and
3623 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3624 special treatment in detect_eol. */
3625 coding->category_idx = CODING_CATEGORY_IDX_EMACS_MULE;
3626
d46c5b12
KH
3627 return 0;
3628 }
3629
d46c5b12
KH
3630 /* Get values of coding system properties:
3631 `post-read-conversion', `pre-write-conversion',
f967223b 3632 `translation-table-for-decode', `translation-table-for-encode'. */
4608c386 3633 plist = XVECTOR (coding_spec)->contents[3];
b843d1ae 3634 /* Pre & post conversion functions should be disabled if
8ca3766a 3635 inhibit_eol_conversion is nonzero. This is the case that a code
b843d1ae
KH
3636 conversion function is called while those functions are running. */
3637 if (! inhibit_pre_post_conversion)
3638 {
3639 coding->post_read_conversion = Fplist_get (plist, Qpost_read_conversion);
3640 coding->pre_write_conversion = Fplist_get (plist, Qpre_write_conversion);
3641 }
f967223b 3642 val = Fplist_get (plist, Qtranslation_table_for_decode);
4608c386 3643 if (SYMBOLP (val))
f967223b
KH
3644 val = Fget (val, Qtranslation_table_for_decode);
3645 coding->translation_table_for_decode = CHAR_TABLE_P (val) ? val : Qnil;
3646 val = Fplist_get (plist, Qtranslation_table_for_encode);
4608c386 3647 if (SYMBOLP (val))
f967223b
KH
3648 val = Fget (val, Qtranslation_table_for_encode);
3649 coding->translation_table_for_encode = CHAR_TABLE_P (val) ? val : Qnil;
d46c5b12
KH
3650 val = Fplist_get (plist, Qcoding_category);
3651 if (!NILP (val))
3652 {
3653 val = Fget (val, Qcoding_category_index);
3654 if (INTEGERP (val))
3655 coding->category_idx = XINT (val);
3656 else
3657 goto label_invalid_coding_system;
3658 }
3659 else
3660 goto label_invalid_coding_system;
93dec019 3661
ec6d2bb8
KH
3662 /* If the coding system has non-nil `composition' property, enable
3663 composition handling. */
3664 val = Fplist_get (plist, Qcomposition);
3665 if (!NILP (val))
3666 coding->composing = COMPOSITION_NO;
3667
d46c5b12 3668 switch (XFASTINT (coding_type))
4ed46869
KH
3669 {
3670 case 0:
0ef69138 3671 coding->type = coding_type_emacs_mule;
aa72b389
KH
3672 coding->common_flags
3673 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK;
c952af22
KH
3674 if (!NILP (coding->post_read_conversion))
3675 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
3676 if (!NILP (coding->pre_write_conversion))
3677 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
4ed46869
KH
3678 break;
3679
3680 case 1:
3681 coding->type = coding_type_sjis;
c952af22
KH
3682 coding->common_flags
3683 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK;
4ed46869
KH
3684 break;
3685
3686 case 2:
3687 coding->type = coding_type_iso2022;
c952af22
KH
3688 coding->common_flags
3689 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK;
4ed46869 3690 {
70c22245 3691 Lisp_Object val, temp;
4ed46869 3692 Lisp_Object *flags;
d46c5b12 3693 int i, charset, reg_bits = 0;
4ed46869 3694
4608c386 3695 val = XVECTOR (coding_spec)->contents[4];
f44d27ce 3696
4ed46869
KH
3697 if (!VECTORP (val) || XVECTOR (val)->size != 32)
3698 goto label_invalid_coding_system;
3699
3700 flags = XVECTOR (val)->contents;
3701 coding->flags
3702 = ((NILP (flags[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM)
3703 | (NILP (flags[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL)
3704 | (NILP (flags[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL)
3705 | (NILP (flags[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS)
3706 | (NILP (flags[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT)
3707 | (NILP (flags[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT)
3708 | (NILP (flags[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN)
3709 | (NILP (flags[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS)
e0e989f6
KH
3710 | (NILP (flags[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION)
3711 | (NILP (flags[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL)
c4825358
KH
3712 | (NILP (flags[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL)
3713 | (NILP (flags[15]) ? 0 : CODING_FLAG_ISO_SAFE)
3f003981 3714 | (NILP (flags[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA)
c4825358 3715 );
4ed46869
KH
3716
3717 /* Invoke graphic register 0 to plane 0. */
3718 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0;
3719 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3720 CODING_SPEC_ISO_INVOCATION (coding, 1)
3721 = (coding->flags & CODING_FLAG_ISO_SEVEN_BITS ? -1 : 1);
3722 /* Not single shifting at first. */
6e85d753 3723 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0;
e0e989f6 3724 /* Beginning of buffer should also be regarded as bol. */
6e85d753 3725 CODING_SPEC_ISO_BOL (coding) = 1;
4ed46869 3726
70c22245
KH
3727 for (charset = 0; charset <= MAX_CHARSET; charset++)
3728 CODING_SPEC_ISO_REVISION_NUMBER (coding, charset) = 255;
3729 val = Vcharset_revision_alist;
3730 while (CONSP (val))
3731 {
03699b14 3732 charset = get_charset_id (Fcar_safe (XCAR (val)));
70c22245 3733 if (charset >= 0
03699b14 3734 && (temp = Fcdr_safe (XCAR (val)), INTEGERP (temp))
70c22245
KH
3735 && (i = XINT (temp), (i >= 0 && (i + '@') < 128)))
3736 CODING_SPEC_ISO_REVISION_NUMBER (coding, charset) = i;
03699b14 3737 val = XCDR (val);
70c22245
KH
3738 }
3739
4ed46869
KH
3740 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3741 FLAGS[REG] can be one of below:
3742 integer CHARSET: CHARSET occupies register I,
3743 t: designate nothing to REG initially, but can be used
3744 by any charsets,
3745 list of integer, nil, or t: designate the first
3746 element (if integer) to REG initially, the remaining
3747 elements (if integer) is designated to REG on request,
d46c5b12 3748 if an element is t, REG can be used by any charsets,
4ed46869 3749 nil: REG is never used. */
467e7675 3750 for (charset = 0; charset <= MAX_CHARSET; charset++)
1ba9e4ab
KH
3751 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3752 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION;
4ed46869
KH
3753 for (i = 0; i < 4; i++)
3754 {
87323294
PJ
3755 if ((INTEGERP (flags[i])
3756 && (charset = XINT (flags[i]), CHARSET_VALID_P (charset)))
e0e989f6 3757 || (charset = get_charset_id (flags[i])) >= 0)
4ed46869
KH
3758 {
3759 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = charset;
3760 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) = i;
3761 }
3762 else if (EQ (flags[i], Qt))
3763 {
3764 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = -1;
d46c5b12
KH
3765 reg_bits |= 1 << i;
3766 coding->flags |= CODING_FLAG_ISO_DESIGNATION;
4ed46869
KH
3767 }
3768 else if (CONSP (flags[i]))
3769 {
84d60297
RS
3770 Lisp_Object tail;
3771 tail = flags[i];
4ed46869 3772
d46c5b12 3773 coding->flags |= CODING_FLAG_ISO_DESIGNATION;
87323294
PJ
3774 if ((INTEGERP (XCAR (tail))
3775 && (charset = XINT (XCAR (tail)),
3776 CHARSET_VALID_P (charset)))
03699b14 3777 || (charset = get_charset_id (XCAR (tail))) >= 0)
4ed46869
KH
3778 {
3779 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = charset;
3780 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) =i;
3781 }
3782 else
3783 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = -1;
03699b14 3784 tail = XCDR (tail);
4ed46869
KH
3785 while (CONSP (tail))
3786 {
87323294
PJ
3787 if ((INTEGERP (XCAR (tail))
3788 && (charset = XINT (XCAR (tail)),
3789 CHARSET_VALID_P (charset)))
03699b14 3790 || (charset = get_charset_id (XCAR (tail))) >= 0)
70c22245
KH
3791 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3792 = i;
03699b14 3793 else if (EQ (XCAR (tail), Qt))
d46c5b12 3794 reg_bits |= 1 << i;
03699b14 3795 tail = XCDR (tail);
4ed46869
KH
3796 }
3797 }
3798 else
3799 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = -1;
93dec019 3800
4ed46869
KH
3801 CODING_SPEC_ISO_DESIGNATION (coding, i)
3802 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i);
3803 }
3804
d46c5b12 3805 if (reg_bits && ! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT))
4ed46869
KH
3806 {
3807 /* REG 1 can be used only by locking shift in 7-bit env. */
3808 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS)
d46c5b12 3809 reg_bits &= ~2;
4ed46869
KH
3810 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT))
3811 /* Without any shifting, only REG 0 and 1 can be used. */
d46c5b12 3812 reg_bits &= 3;
4ed46869
KH
3813 }
3814
d46c5b12
KH
3815 if (reg_bits)
3816 for (charset = 0; charset <= MAX_CHARSET; charset++)
6e85d753 3817 {
928a85c1 3818 if (CHARSET_DEFINED_P (charset)
96148065
KH
3819 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3820 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
d46c5b12
KH
3821 {
3822 /* There exist some default graphic registers to be
96148065 3823 used by CHARSET. */
d46c5b12
KH
3824
3825 /* We had better avoid designating a charset of
3826 CHARS96 to REG 0 as far as possible. */
3827 if (CHARSET_CHARS (charset) == 96)
3828 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3829 = (reg_bits & 2
3830 ? 1 : (reg_bits & 4 ? 2 : (reg_bits & 8 ? 3 : 0)));
3831 else
3832 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3833 = (reg_bits & 1
3834 ? 0 : (reg_bits & 2 ? 1 : (reg_bits & 4 ? 2 : 3)));
3835 }
6e85d753 3836 }
4ed46869 3837 }
c952af22 3838 coding->common_flags |= CODING_REQUIRE_FLUSHING_MASK;
d46c5b12 3839 coding->spec.iso2022.last_invalid_designation_register = -1;
4ed46869
KH
3840 break;
3841
3842 case 3:
3843 coding->type = coding_type_big5;
c952af22
KH
3844 coding->common_flags
3845 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK;
4ed46869 3846 coding->flags
4608c386 3847 = (NILP (XVECTOR (coding_spec)->contents[4])
4ed46869
KH
3848 ? CODING_FLAG_BIG5_HKU
3849 : CODING_FLAG_BIG5_ETEN);
3850 break;
3851
3852 case 4:
3853 coding->type = coding_type_ccl;
c952af22
KH
3854 coding->common_flags
3855 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK;
4ed46869 3856 {
84d60297 3857 val = XVECTOR (coding_spec)->contents[4];
ef4ced28
KH
3858 if (! CONSP (val)
3859 || setup_ccl_program (&(coding->spec.ccl.decoder),
03699b14 3860 XCAR (val)) < 0
ef4ced28 3861 || setup_ccl_program (&(coding->spec.ccl.encoder),
03699b14 3862 XCDR (val)) < 0)
4ed46869 3863 goto label_invalid_coding_system;
1397dc18
KH
3864
3865 bzero (coding->spec.ccl.valid_codes, 256);
3866 val = Fplist_get (plist, Qvalid_codes);
3867 if (CONSP (val))
3868 {
3869 Lisp_Object this;
3870
03699b14 3871 for (; CONSP (val); val = XCDR (val))
1397dc18 3872 {
03699b14 3873 this = XCAR (val);
1397dc18
KH
3874 if (INTEGERP (this)
3875 && XINT (this) >= 0 && XINT (this) < 256)
3876 coding->spec.ccl.valid_codes[XINT (this)] = 1;
3877 else if (CONSP (this)
03699b14
KR
3878 && INTEGERP (XCAR (this))
3879 && INTEGERP (XCDR (this)))
1397dc18 3880 {
03699b14
KR
3881 int start = XINT (XCAR (this));
3882 int end = XINT (XCDR (this));
1397dc18
KH
3883
3884 if (start >= 0 && start <= end && end < 256)
e133c8fa 3885 while (start <= end)
1397dc18
KH
3886 coding->spec.ccl.valid_codes[start++] = 1;
3887 }
3888 }
3889 }
4ed46869 3890 }
c952af22 3891 coding->common_flags |= CODING_REQUIRE_FLUSHING_MASK;
aaaf0b1e 3892 coding->spec.ccl.cr_carryover = 0;
1c3478b0 3893 coding->spec.ccl.eight_bit_carryover[0] = 0;
4ed46869
KH
3894 break;
3895
27901516
KH
3896 case 5:
3897 coding->type = coding_type_raw_text;
3898 break;
3899
4ed46869 3900 default:
d46c5b12 3901 goto label_invalid_coding_system;
4ed46869
KH
3902 }
3903 return 0;
3904
3905 label_invalid_coding_system:
3906 coding->type = coding_type_no_conversion;
d46c5b12 3907 coding->category_idx = CODING_CATEGORY_IDX_BINARY;
c952af22 3908 coding->common_flags = 0;
dec137e5 3909 coding->eol_type = CODING_EOL_LF;
d46c5b12 3910 coding->pre_write_conversion = coding->post_read_conversion = Qnil;
4ed46869
KH
3911 return -1;
3912}
3913
ec6d2bb8
KH
3914/* Free memory blocks allocated for storing composition information. */
3915
3916void
3917coding_free_composition_data (coding)
3918 struct coding_system *coding;
3919{
3920 struct composition_data *cmp_data = coding->cmp_data, *next;
3921
3922 if (!cmp_data)
3923 return;
3924 /* Memory blocks are chained. At first, rewind to the first, then,
3925 free blocks one by one. */
3926 while (cmp_data->prev)
3927 cmp_data = cmp_data->prev;
3928 while (cmp_data)
3929 {
3930 next = cmp_data->next;
3931 xfree (cmp_data);
3932 cmp_data = next;
3933 }
3934 coding->cmp_data = NULL;
3935}
3936
3937/* Set `char_offset' member of all memory blocks pointed by
3938 coding->cmp_data to POS. */
3939
3940void
3941coding_adjust_composition_offset (coding, pos)
3942 struct coding_system *coding;
3943 int pos;
3944{
3945 struct composition_data *cmp_data;
3946
3947 for (cmp_data = coding->cmp_data; cmp_data; cmp_data = cmp_data->next)
3948 cmp_data->char_offset = pos;
3949}
3950
54f78171
KH
3951/* Setup raw-text or one of its subsidiaries in the structure
3952 coding_system CODING according to the already setup value eol_type
3953 in CODING. CODING should be setup for some coding system in
3954 advance. */
3955
3956void
3957setup_raw_text_coding_system (coding)
3958 struct coding_system *coding;
3959{
3960 if (coding->type != coding_type_raw_text)
3961 {
3962 coding->symbol = Qraw_text;
3963 coding->type = coding_type_raw_text;
3964 if (coding->eol_type != CODING_EOL_UNDECIDED)
3965 {
84d60297
RS
3966 Lisp_Object subsidiaries;
3967 subsidiaries = Fget (Qraw_text, Qeol_type);
54f78171
KH
3968
3969 if (VECTORP (subsidiaries)
3970 && XVECTOR (subsidiaries)->size == 3)
3971 coding->symbol
3972 = XVECTOR (subsidiaries)->contents[coding->eol_type];
3973 }
716e0b0a 3974 setup_coding_system (coding->symbol, coding);
54f78171
KH
3975 }
3976 return;
3977}
3978
4ed46869
KH
3979/* Emacs has a mechanism to automatically detect a coding system if it
3980 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3981 it's impossible to distinguish some coding systems accurately
3982 because they use the same range of codes. So, at first, coding
3983 systems are categorized into 7, those are:
3984
0ef69138 3985 o coding-category-emacs-mule
4ed46869
KH
3986
3987 The category for a coding system which has the same code range
3988 as Emacs' internal format. Assigned the coding-system (Lisp
0ef69138 3989 symbol) `emacs-mule' by default.
4ed46869
KH
3990
3991 o coding-category-sjis
3992
3993 The category for a coding system which has the same code range
3994 as SJIS. Assigned the coding-system (Lisp
7717c392 3995 symbol) `japanese-shift-jis' by default.
4ed46869
KH
3996
3997 o coding-category-iso-7
3998
3999 The category for a coding system which has the same code range
7717c392 4000 as ISO2022 of 7-bit environment. This doesn't use any locking
d46c5b12
KH
4001 shift and single shift functions. This can encode/decode all
4002 charsets. Assigned the coding-system (Lisp symbol)
4003 `iso-2022-7bit' by default.
4004
4005 o coding-category-iso-7-tight
4006
4007 Same as coding-category-iso-7 except that this can
4008 encode/decode only the specified charsets.
4ed46869
KH
4009
4010 o coding-category-iso-8-1
4011
4012 The category for a coding system which has the same code range
4013 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH
4014 for DIMENSION1 charset. This doesn't use any locking shift
4015 and single shift functions. Assigned the coding-system (Lisp
4016 symbol) `iso-latin-1' by default.
4ed46869
KH
4017
4018 o coding-category-iso-8-2
4019
4020 The category for a coding system which has the same code range
4021 as ISO2022 of 8-bit environment and graphic plane 1 used only
7717c392
KH
4022 for DIMENSION2 charset. This doesn't use any locking shift
4023 and single shift functions. Assigned the coding-system (Lisp
4024 symbol) `japanese-iso-8bit' by default.
4ed46869 4025
7717c392 4026 o coding-category-iso-7-else
4ed46869
KH
4027
4028 The category for a coding system which has the same code range
8ca3766a 4029 as ISO2022 of 7-bit environment but uses locking shift or
7717c392
KH
4030 single shift functions. Assigned the coding-system (Lisp
4031 symbol) `iso-2022-7bit-lock' by default.
4032
4033 o coding-category-iso-8-else
4034
4035 The category for a coding system which has the same code range
8ca3766a 4036 as ISO2022 of 8-bit environment but uses locking shift or
7717c392
KH
4037 single shift functions. Assigned the coding-system (Lisp
4038 symbol) `iso-2022-8bit-ss2' by default.
4ed46869
KH
4039
4040 o coding-category-big5
4041
4042 The category for a coding system which has the same code range
4043 as BIG5. Assigned the coding-system (Lisp symbol)
e0e989f6 4044 `cn-big5' by default.
4ed46869 4045
fa42c37f
KH
4046 o coding-category-utf-8
4047
4048 The category for a coding system which has the same code range
38b92c42 4049 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
fa42c37f
KH
4050 symbol) `utf-8' by default.
4051
4052 o coding-category-utf-16-be
4053
4054 The category for a coding system in which a text has an
4055 Unicode signature (cf. Unicode Standard) in the order of BIG
4056 endian at the head. Assigned the coding-system (Lisp symbol)
4057 `utf-16-be' by default.
4058
4059 o coding-category-utf-16-le
4060
4061 The category for a coding system in which a text has an
4062 Unicode signature (cf. Unicode Standard) in the order of
4063 LITTLE endian at the head. Assigned the coding-system (Lisp
4064 symbol) `utf-16-le' by default.
4065
1397dc18
KH
4066 o coding-category-ccl
4067
4068 The category for a coding system of which encoder/decoder is
4069 written in CCL programs. The default value is nil, i.e., no
4070 coding system is assigned.
4071
4ed46869
KH
4072 o coding-category-binary
4073
4074 The category for a coding system not categorized in any of the
4075 above. Assigned the coding-system (Lisp symbol)
e0e989f6 4076 `no-conversion' by default.
4ed46869
KH
4077
4078 Each of them is a Lisp symbol and the value is an actual
cfb43547 4079 `coding-system' (this is also a Lisp symbol) assigned by a user.
4ed46869
KH
4080 What Emacs does actually is to detect a category of coding system.
4081 Then, it uses a `coding-system' assigned to it. If Emacs can't
cfb43547 4082 decide a single possible category, it selects a category of the
4ed46869
KH
4083 highest priority. Priorities of categories are also specified by a
4084 user in a Lisp variable `coding-category-list'.
4085
4086*/
4087
66cfb530
KH
4088static
4089int ascii_skip_code[256];
4090
d46c5b12 4091/* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
4ed46869
KH
4092 If it detects possible coding systems, return an integer in which
4093 appropriate flag bits are set. Flag bits are defined by macros
fa42c37f
KH
4094 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
4095 it should point the table `coding_priorities'. In that case, only
4096 the flag bit for a coding system of the highest priority is set in
0a28aafb
KH
4097 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
4098 range 0x80..0x9F are in multibyte form.
4ed46869 4099
d46c5b12
KH
4100 How many ASCII characters are at the head is returned as *SKIP. */
4101
4102static int
0a28aafb 4103detect_coding_mask (source, src_bytes, priorities, skip, multibytep)
d46c5b12
KH
4104 unsigned char *source;
4105 int src_bytes, *priorities, *skip;
0a28aafb 4106 int multibytep;
4ed46869
KH
4107{
4108 register unsigned char c;
d46c5b12 4109 unsigned char *src = source, *src_end = source + src_bytes;
fa42c37f 4110 unsigned int mask, utf16_examined_p, iso2022_examined_p;
da55a2b7 4111 int i;
4ed46869
KH
4112
4113 /* At first, skip all ASCII characters and control characters except
4114 for three ISO2022 specific control characters. */
66cfb530
KH
4115 ascii_skip_code[ISO_CODE_SO] = 0;
4116 ascii_skip_code[ISO_CODE_SI] = 0;
4117 ascii_skip_code[ISO_CODE_ESC] = 0;
4118
bcf26d6a 4119 label_loop_detect_coding:
66cfb530 4120 while (src < src_end && ascii_skip_code[*src]) src++;
d46c5b12 4121 *skip = src - source;
4ed46869
KH
4122
4123 if (src >= src_end)
4124 /* We found nothing other than ASCII. There's nothing to do. */
d46c5b12 4125 return 0;
4ed46869 4126
8a8147d6 4127 c = *src;
4ed46869
KH
4128 /* The text seems to be encoded in some multilingual coding system.
4129 Now, try to find in which coding system the text is encoded. */
4130 if (c < 0x80)
bcf26d6a
KH
4131 {
4132 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
4133 /* C is an ISO2022 specific control code of C0. */
0a28aafb 4134 mask = detect_coding_iso2022 (src, src_end, multibytep);
1b2af4b0 4135 if (mask == 0)
d46c5b12
KH
4136 {
4137 /* No valid ISO2022 code follows C. Try again. */
4138 src++;
66cfb530
KH
4139 if (c == ISO_CODE_ESC)
4140 ascii_skip_code[ISO_CODE_ESC] = 1;
4141 else
4142 ascii_skip_code[ISO_CODE_SO] = ascii_skip_code[ISO_CODE_SI] = 1;
d46c5b12
KH
4143 goto label_loop_detect_coding;
4144 }
4145 if (priorities)
fa42c37f
KH
4146 {
4147 for (i = 0; i < CODING_CATEGORY_IDX_MAX; i++)
4148 {
4149 if (mask & priorities[i])
4150 return priorities[i];
4151 }
4152 return CODING_CATEGORY_MASK_RAW_TEXT;
4153 }
bcf26d6a 4154 }
d46c5b12 4155 else
c4825358 4156 {
d46c5b12 4157 int try;
4ed46869 4158
0a28aafb 4159 if (multibytep && c == LEADING_CODE_8_BIT_CONTROL)
67091e59 4160 c = src[1] - 0x20;
0a28aafb 4161
d46c5b12
KH
4162 if (c < 0xA0)
4163 {
4164 /* C is the first byte of SJIS character code,
fa42c37f
KH
4165 or a leading-code of Emacs' internal format (emacs-mule),
4166 or the first byte of UTF-16. */
4167 try = (CODING_CATEGORY_MASK_SJIS
4168 | CODING_CATEGORY_MASK_EMACS_MULE
4169 | CODING_CATEGORY_MASK_UTF_16_BE
4170 | CODING_CATEGORY_MASK_UTF_16_LE);
d46c5b12
KH
4171
4172 /* Or, if C is a special latin extra code,
93dec019 4173 or is an ISO2022 specific control code of C1 (SS2 or SS3),
d46c5b12
KH
4174 or is an ISO2022 control-sequence-introducer (CSI),
4175 we should also consider the possibility of ISO2022 codings. */
4176 if ((VECTORP (Vlatin_extra_code_table)
4177 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
4178 || (c == ISO_CODE_SS2 || c == ISO_CODE_SS3)
4179 || (c == ISO_CODE_CSI
4180 && (src < src_end
4181 && (*src == ']'
4182 || ((*src == '0' || *src == '1' || *src == '2')
4183 && src + 1 < src_end
4184 && src[1] == ']')))))
4185 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4186 | CODING_CATEGORY_MASK_ISO_8BIT);
4187 }
c4825358 4188 else
d46c5b12
KH
4189 /* C is a character of ISO2022 in graphic plane right,
4190 or a SJIS's 1-byte character code (i.e. JISX0201),
fa42c37f
KH
4191 or the first byte of BIG5's 2-byte code,
4192 or the first byte of UTF-8/16. */
d46c5b12
KH
4193 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4194 | CODING_CATEGORY_MASK_ISO_8BIT
4195 | CODING_CATEGORY_MASK_SJIS
fa42c37f
KH
4196 | CODING_CATEGORY_MASK_BIG5
4197 | CODING_CATEGORY_MASK_UTF_8
4198 | CODING_CATEGORY_MASK_UTF_16_BE
4199 | CODING_CATEGORY_MASK_UTF_16_LE);
d46c5b12 4200
1397dc18
KH
4201 /* Or, we may have to consider the possibility of CCL. */
4202 if (coding_system_table[CODING_CATEGORY_IDX_CCL]
4203 && (coding_system_table[CODING_CATEGORY_IDX_CCL]
4204 ->spec.ccl.valid_codes)[c])
4205 try |= CODING_CATEGORY_MASK_CCL;
4206
d46c5b12 4207 mask = 0;
fa42c37f 4208 utf16_examined_p = iso2022_examined_p = 0;
d46c5b12
KH
4209 if (priorities)
4210 {
4211 for (i = 0; i < CODING_CATEGORY_IDX_MAX; i++)
4212 {
fa42c37f
KH
4213 if (!iso2022_examined_p
4214 && (priorities[i] & try & CODING_CATEGORY_MASK_ISO))
4215 {
0192762c 4216 mask |= detect_coding_iso2022 (src, src_end, multibytep);
fa42c37f
KH
4217 iso2022_examined_p = 1;
4218 }
5ab13dd0 4219 else if (priorities[i] & try & CODING_CATEGORY_MASK_SJIS)
0a28aafb 4220 mask |= detect_coding_sjis (src, src_end, multibytep);
fa42c37f 4221 else if (priorities[i] & try & CODING_CATEGORY_MASK_UTF_8)
0a28aafb 4222 mask |= detect_coding_utf_8 (src, src_end, multibytep);
fa42c37f
KH
4223 else if (!utf16_examined_p
4224 && (priorities[i] & try &
4225 CODING_CATEGORY_MASK_UTF_16_BE_LE))
4226 {
0a28aafb 4227 mask |= detect_coding_utf_16 (src, src_end, multibytep);
fa42c37f
KH
4228 utf16_examined_p = 1;
4229 }
5ab13dd0 4230 else if (priorities[i] & try & CODING_CATEGORY_MASK_BIG5)
0a28aafb 4231 mask |= detect_coding_big5 (src, src_end, multibytep);
5ab13dd0 4232 else if (priorities[i] & try & CODING_CATEGORY_MASK_EMACS_MULE)
0a28aafb 4233 mask |= detect_coding_emacs_mule (src, src_end, multibytep);
89fa8b36 4234 else if (priorities[i] & try & CODING_CATEGORY_MASK_CCL)
0a28aafb 4235 mask |= detect_coding_ccl (src, src_end, multibytep);
5ab13dd0 4236 else if (priorities[i] & CODING_CATEGORY_MASK_RAW_TEXT)
fa42c37f 4237 mask |= CODING_CATEGORY_MASK_RAW_TEXT;
5ab13dd0 4238 else if (priorities[i] & CODING_CATEGORY_MASK_BINARY)
fa42c37f
KH
4239 mask |= CODING_CATEGORY_MASK_BINARY;
4240 if (mask & priorities[i])
4241 return priorities[i];
d46c5b12
KH
4242 }
4243 return CODING_CATEGORY_MASK_RAW_TEXT;
4244 }
4245 if (try & CODING_CATEGORY_MASK_ISO)
0a28aafb 4246 mask |= detect_coding_iso2022 (src, src_end, multibytep);
d46c5b12 4247 if (try & CODING_CATEGORY_MASK_SJIS)
0a28aafb 4248 mask |= detect_coding_sjis (src, src_end, multibytep);
d46c5b12 4249 if (try & CODING_CATEGORY_MASK_BIG5)
0a28aafb 4250 mask |= detect_coding_big5 (src, src_end, multibytep);
fa42c37f 4251 if (try & CODING_CATEGORY_MASK_UTF_8)
0a28aafb 4252 mask |= detect_coding_utf_8 (src, src_end, multibytep);
fa42c37f 4253 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE)
0a28aafb 4254 mask |= detect_coding_utf_16 (src, src_end, multibytep);
d46c5b12 4255 if (try & CODING_CATEGORY_MASK_EMACS_MULE)
0a28aafb 4256 mask |= detect_coding_emacs_mule (src, src_end, multibytep);
1397dc18 4257 if (try & CODING_CATEGORY_MASK_CCL)
0a28aafb 4258 mask |= detect_coding_ccl (src, src_end, multibytep);
c4825358 4259 }
5ab13dd0 4260 return (mask | CODING_CATEGORY_MASK_RAW_TEXT | CODING_CATEGORY_MASK_BINARY);
4ed46869
KH
4261}
4262
4263/* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4264 The information of the detected coding system is set in CODING. */
4265
4266void
4267detect_coding (coding, src, src_bytes)
4268 struct coding_system *coding;
a4244313 4269 const unsigned char *src;
4ed46869
KH
4270 int src_bytes;
4271{
d46c5b12 4272 unsigned int idx;
da55a2b7 4273 int skip, mask;
84d60297 4274 Lisp_Object val;
4ed46869 4275
84d60297 4276 val = Vcoding_category_list;
64c1e55f
KH
4277 mask = detect_coding_mask (src, src_bytes, coding_priorities, &skip,
4278 coding->src_multibyte);
d46c5b12 4279 coding->heading_ascii = skip;
4ed46869 4280
d46c5b12
KH
4281 if (!mask) return;
4282
4283 /* We found a single coding system of the highest priority in MASK. */
4284 idx = 0;
4285 while (mask && ! (mask & 1)) mask >>= 1, idx++;
4286 if (! mask)
4287 idx = CODING_CATEGORY_IDX_RAW_TEXT;
4ed46869 4288
f5c1dd0d 4289 val = SYMBOL_VALUE (XVECTOR (Vcoding_category_table)->contents[idx]);
d46c5b12
KH
4290
4291 if (coding->eol_type != CODING_EOL_UNDECIDED)
27901516 4292 {
84d60297 4293 Lisp_Object tmp;
d46c5b12 4294
84d60297 4295 tmp = Fget (val, Qeol_type);
d46c5b12
KH
4296 if (VECTORP (tmp))
4297 val = XVECTOR (tmp)->contents[coding->eol_type];
4ed46869 4298 }
b73bfc1c
KH
4299
4300 /* Setup this new coding system while preserving some slots. */
4301 {
4302 int src_multibyte = coding->src_multibyte;
4303 int dst_multibyte = coding->dst_multibyte;
4304
4305 setup_coding_system (val, coding);
4306 coding->src_multibyte = src_multibyte;
4307 coding->dst_multibyte = dst_multibyte;
4308 coding->heading_ascii = skip;
4309 }
4ed46869
KH
4310}
4311
d46c5b12
KH
4312/* Detect how end-of-line of a text of length SRC_BYTES pointed by
4313 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4314 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4315
4316 How many non-eol characters are at the head is returned as *SKIP. */
4ed46869 4317
bc4bc72a
RS
4318#define MAX_EOL_CHECK_COUNT 3
4319
d46c5b12
KH
4320static int
4321detect_eol_type (source, src_bytes, skip)
4322 unsigned char *source;
4323 int src_bytes, *skip;
4ed46869 4324{
d46c5b12 4325 unsigned char *src = source, *src_end = src + src_bytes;
4ed46869 4326 unsigned char c;
bc4bc72a
RS
4327 int total = 0; /* How many end-of-lines are found so far. */
4328 int eol_type = CODING_EOL_UNDECIDED;
4329 int this_eol_type;
4ed46869 4330
d46c5b12
KH
4331 *skip = 0;
4332
bc4bc72a 4333 while (src < src_end && total < MAX_EOL_CHECK_COUNT)
4ed46869
KH
4334 {
4335 c = *src++;
bc4bc72a 4336 if (c == '\n' || c == '\r')
4ed46869 4337 {
d46c5b12
KH
4338 if (*skip == 0)
4339 *skip = src - 1 - source;
bc4bc72a
RS
4340 total++;
4341 if (c == '\n')
4342 this_eol_type = CODING_EOL_LF;
4343 else if (src >= src_end || *src != '\n')
4344 this_eol_type = CODING_EOL_CR;
4ed46869 4345 else
bc4bc72a
RS
4346 this_eol_type = CODING_EOL_CRLF, src++;
4347
4348 if (eol_type == CODING_EOL_UNDECIDED)
4349 /* This is the first end-of-line. */
4350 eol_type = this_eol_type;
4351 else if (eol_type != this_eol_type)
d46c5b12
KH
4352 {
4353 /* The found type is different from what found before. */
4354 eol_type = CODING_EOL_INCONSISTENT;
4355 break;
4356 }
4ed46869
KH
4357 }
4358 }
bc4bc72a 4359
d46c5b12
KH
4360 if (*skip == 0)
4361 *skip = src_end - source;
85a02ca4 4362 return eol_type;
4ed46869
KH
4363}
4364
fa42c37f
KH
4365/* Like detect_eol_type, but detect EOL type in 2-octet
4366 big-endian/little-endian format for coding systems utf-16-be and
4367 utf-16-le. */
4368
4369static int
4370detect_eol_type_in_2_octet_form (source, src_bytes, skip, big_endian_p)
4371 unsigned char *source;
cfb43547 4372 int src_bytes, *skip, big_endian_p;
fa42c37f
KH
4373{
4374 unsigned char *src = source, *src_end = src + src_bytes;
4375 unsigned int c1, c2;
4376 int total = 0; /* How many end-of-lines are found so far. */
4377 int eol_type = CODING_EOL_UNDECIDED;
4378 int this_eol_type;
4379 int msb, lsb;
4380
4381 if (big_endian_p)
4382 msb = 0, lsb = 1;
4383 else
4384 msb = 1, lsb = 0;
4385
4386 *skip = 0;
4387
4388 while ((src + 1) < src_end && total < MAX_EOL_CHECK_COUNT)
4389 {
4390 c1 = (src[msb] << 8) | (src[lsb]);
4391 src += 2;
4392
4393 if (c1 == '\n' || c1 == '\r')
4394 {
4395 if (*skip == 0)
4396 *skip = src - 2 - source;
4397 total++;
4398 if (c1 == '\n')
4399 {
4400 this_eol_type = CODING_EOL_LF;
4401 }
4402 else
4403 {
4404 if ((src + 1) >= src_end)
4405 {
4406 this_eol_type = CODING_EOL_CR;
4407 }
4408 else
4409 {
4410 c2 = (src[msb] << 8) | (src[lsb]);
4411 if (c2 == '\n')
4412 this_eol_type = CODING_EOL_CRLF, src += 2;
4413 else
4414 this_eol_type = CODING_EOL_CR;
4415 }
4416 }
4417
4418 if (eol_type == CODING_EOL_UNDECIDED)
4419 /* This is the first end-of-line. */
4420 eol_type = this_eol_type;
4421 else if (eol_type != this_eol_type)
4422 {
4423 /* The found type is different from what found before. */
4424 eol_type = CODING_EOL_INCONSISTENT;
4425 break;
4426 }
4427 }
4428 }
4429
4430 if (*skip == 0)
4431 *skip = src_end - source;
4432 return eol_type;
4433}
4434
4ed46869
KH
4435/* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4436 is encoded. If it detects an appropriate format of end-of-line, it
4437 sets the information in *CODING. */
4438
4439void
4440detect_eol (coding, src, src_bytes)
4441 struct coding_system *coding;
a4244313 4442 const unsigned char *src;
4ed46869
KH
4443 int src_bytes;
4444{
4608c386 4445 Lisp_Object val;
d46c5b12 4446 int skip;
fa42c37f
KH
4447 int eol_type;
4448
4449 switch (coding->category_idx)
4450 {
4451 case CODING_CATEGORY_IDX_UTF_16_BE:
4452 eol_type = detect_eol_type_in_2_octet_form (src, src_bytes, &skip, 1);
4453 break;
4454 case CODING_CATEGORY_IDX_UTF_16_LE:
4455 eol_type = detect_eol_type_in_2_octet_form (src, src_bytes, &skip, 0);
4456 break;
4457 default:
4458 eol_type = detect_eol_type (src, src_bytes, &skip);
4459 break;
4460 }
d46c5b12
KH
4461
4462 if (coding->heading_ascii > skip)
4463 coding->heading_ascii = skip;
4464 else
4465 skip = coding->heading_ascii;
4ed46869 4466
0ef69138 4467 if (eol_type == CODING_EOL_UNDECIDED)
4ed46869 4468 return;
27901516
KH
4469 if (eol_type == CODING_EOL_INCONSISTENT)
4470 {
4471#if 0
4472 /* This code is suppressed until we find a better way to
992f23f2 4473 distinguish raw text file and binary file. */
27901516
KH
4474
4475 /* If we have already detected that the coding is raw-text, the
4476 coding should actually be no-conversion. */
4477 if (coding->type == coding_type_raw_text)
4478 {
4479 setup_coding_system (Qno_conversion, coding);
4480 return;
4481 }
4482 /* Else, let's decode only text code anyway. */
4483#endif /* 0 */
1b2af4b0 4484 eol_type = CODING_EOL_LF;
27901516
KH
4485 }
4486
4608c386 4487 val = Fget (coding->symbol, Qeol_type);
4ed46869 4488 if (VECTORP (val) && XVECTOR (val)->size == 3)
d46c5b12 4489 {
b73bfc1c
KH
4490 int src_multibyte = coding->src_multibyte;
4491 int dst_multibyte = coding->dst_multibyte;
1cd6b64c 4492 struct composition_data *cmp_data = coding->cmp_data;
b73bfc1c 4493
d46c5b12 4494 setup_coding_system (XVECTOR (val)->contents[eol_type], coding);
b73bfc1c
KH
4495 coding->src_multibyte = src_multibyte;
4496 coding->dst_multibyte = dst_multibyte;
d46c5b12 4497 coding->heading_ascii = skip;
1cd6b64c 4498 coding->cmp_data = cmp_data;
d46c5b12
KH
4499 }
4500}
4501
4502#define CONVERSION_BUFFER_EXTRA_ROOM 256
4503
b73bfc1c
KH
4504#define DECODING_BUFFER_MAG(coding) \
4505 (coding->type == coding_type_iso2022 \
4506 ? 3 \
4507 : (coding->type == coding_type_ccl \
4508 ? coding->spec.ccl.decoder.buf_magnification \
4509 : 2))
d46c5b12
KH
4510
4511/* Return maximum size (bytes) of a buffer enough for decoding
4512 SRC_BYTES of text encoded in CODING. */
4513
4514int
4515decoding_buffer_size (coding, src_bytes)
4516 struct coding_system *coding;
4517 int src_bytes;
4518{
4519 return (src_bytes * DECODING_BUFFER_MAG (coding)
4520 + CONVERSION_BUFFER_EXTRA_ROOM);
4521}
4522
4523/* Return maximum size (bytes) of a buffer enough for encoding
4524 SRC_BYTES of text to CODING. */
4525
4526int
4527encoding_buffer_size (coding, src_bytes)
4528 struct coding_system *coding;
4529 int src_bytes;
4530{
4531 int magnification;
4532
4533 if (coding->type == coding_type_ccl)
a84f1519
KH
4534 {
4535 magnification = coding->spec.ccl.encoder.buf_magnification;
4536 if (coding->eol_type == CODING_EOL_CRLF)
4537 magnification *= 2;
4538 }
b73bfc1c 4539 else if (CODING_REQUIRE_ENCODING (coding))
d46c5b12 4540 magnification = 3;
b73bfc1c
KH
4541 else
4542 magnification = 1;
d46c5b12
KH
4543
4544 return (src_bytes * magnification + CONVERSION_BUFFER_EXTRA_ROOM);
4545}
4546
73be902c
KH
4547/* Working buffer for code conversion. */
4548struct conversion_buffer
4549{
4550 int size; /* size of data. */
4551 int on_stack; /* 1 if allocated by alloca. */
4552 unsigned char *data;
4553};
d46c5b12 4554
73be902c
KH
4555/* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4556#define allocate_conversion_buffer(buf, len) \
4557 do { \
4558 if (len < MAX_ALLOCA) \
4559 { \
4560 buf.data = (unsigned char *) alloca (len); \
4561 buf.on_stack = 1; \
4562 } \
4563 else \
4564 { \
4565 buf.data = (unsigned char *) xmalloc (len); \
4566 buf.on_stack = 0; \
4567 } \
4568 buf.size = len; \
4569 } while (0)
d46c5b12 4570
73be902c
KH
4571/* Double the allocated memory for *BUF. */
4572static void
4573extend_conversion_buffer (buf)
4574 struct conversion_buffer *buf;
d46c5b12 4575{
73be902c 4576 if (buf->on_stack)
d46c5b12 4577 {
73be902c
KH
4578 unsigned char *save = buf->data;
4579 buf->data = (unsigned char *) xmalloc (buf->size * 2);
4580 bcopy (save, buf->data, buf->size);
4581 buf->on_stack = 0;
d46c5b12 4582 }
73be902c
KH
4583 else
4584 {
4585 buf->data = (unsigned char *) xrealloc (buf->data, buf->size * 2);
4586 }
4587 buf->size *= 2;
4588}
4589
4590/* Free the allocated memory for BUF if it is not on stack. */
4591static void
4592free_conversion_buffer (buf)
4593 struct conversion_buffer *buf;
4594{
4595 if (!buf->on_stack)
4596 xfree (buf->data);
d46c5b12
KH
4597}
4598
4599int
4600ccl_coding_driver (coding, source, destination, src_bytes, dst_bytes, encodep)
4601 struct coding_system *coding;
4602 unsigned char *source, *destination;
4603 int src_bytes, dst_bytes, encodep;
4604{
4605 struct ccl_program *ccl
4606 = encodep ? &coding->spec.ccl.encoder : &coding->spec.ccl.decoder;
1c3478b0 4607 unsigned char *dst = destination;
d46c5b12 4608
bd64290d 4609 ccl->suppress_error = coding->suppress_error;
ae9ff118 4610 ccl->last_block = coding->mode & CODING_MODE_LAST_BLOCK;
aaaf0b1e 4611 if (encodep)
80e0ca99
KH
4612 {
4613 /* On encoding, EOL format is converted within ccl_driver. For
4614 that, setup proper information in the structure CCL. */
4615 ccl->eol_type = coding->eol_type;
4616 if (ccl->eol_type ==CODING_EOL_UNDECIDED)
4617 ccl->eol_type = CODING_EOL_LF;
4618 ccl->cr_consumed = coding->spec.ccl.cr_carryover;
b671ed5e 4619 ccl->eight_bit_control = coding->dst_multibyte;
80e0ca99 4620 }
b671ed5e
KH
4621 else
4622 ccl->eight_bit_control = 1;
7272d75c 4623 ccl->multibyte = coding->src_multibyte;
1c3478b0
KH
4624 if (coding->spec.ccl.eight_bit_carryover[0] != 0)
4625 {
4626 /* Move carryover bytes to DESTINATION. */
4627 unsigned char *p = coding->spec.ccl.eight_bit_carryover;
4628 while (*p)
4629 *dst++ = *p++;
4630 coding->spec.ccl.eight_bit_carryover[0] = 0;
4631 if (dst_bytes)
4632 dst_bytes -= dst - destination;
4633 }
4634
4635 coding->produced = (ccl_driver (ccl, source, dst, src_bytes, dst_bytes,
4636 &(coding->consumed))
4637 + dst - destination);
4638
b73bfc1c 4639 if (encodep)
80e0ca99
KH
4640 {
4641 coding->produced_char = coding->produced;
4642 coding->spec.ccl.cr_carryover = ccl->cr_consumed;
4643 }
ade8d05e
KH
4644 else if (!ccl->eight_bit_control)
4645 {
4646 /* The produced bytes forms a valid multibyte sequence. */
4647 coding->produced_char
4648 = multibyte_chars_in_text (destination, coding->produced);
4649 coding->spec.ccl.eight_bit_carryover[0] = 0;
4650 }
b73bfc1c
KH
4651 else
4652 {
1c3478b0
KH
4653 /* On decoding, the destination should always multibyte. But,
4654 CCL program might have been generated an invalid multibyte
4655 sequence. Here we make such a sequence valid as
4656 multibyte. */
b73bfc1c
KH
4657 int bytes
4658 = dst_bytes ? dst_bytes : source + coding->consumed - destination;
1c3478b0
KH
4659
4660 if ((coding->consumed < src_bytes
4661 || !ccl->last_block)
4662 && coding->produced >= 1
4663 && destination[coding->produced - 1] >= 0x80)
4664 {
4665 /* We should not convert the tailing 8-bit codes to
4666 multibyte form even if they doesn't form a valid
4667 multibyte sequence. They may form a valid sequence in
4668 the next call. */
4669 int carryover = 0;
4670
4671 if (destination[coding->produced - 1] < 0xA0)
4672 carryover = 1;
4673 else if (coding->produced >= 2)
4674 {
4675 if (destination[coding->produced - 2] >= 0x80)
4676 {
4677 if (destination[coding->produced - 2] < 0xA0)
4678 carryover = 2;
4679 else if (coding->produced >= 3
4680 && destination[coding->produced - 3] >= 0x80
4681 && destination[coding->produced - 3] < 0xA0)
4682 carryover = 3;
4683 }
4684 }
4685 if (carryover > 0)
4686 {
4687 BCOPY_SHORT (destination + coding->produced - carryover,
4688 coding->spec.ccl.eight_bit_carryover,
4689 carryover);
4690 coding->spec.ccl.eight_bit_carryover[carryover] = 0;
4691 coding->produced -= carryover;
4692 }
4693 }
b73bfc1c
KH
4694 coding->produced = str_as_multibyte (destination, bytes,
4695 coding->produced,
4696 &(coding->produced_char));
4697 }
69f76525 4698
d46c5b12
KH
4699 switch (ccl->status)
4700 {
4701 case CCL_STAT_SUSPEND_BY_SRC:
73be902c 4702 coding->result = CODING_FINISH_INSUFFICIENT_SRC;
d46c5b12
KH
4703 break;
4704 case CCL_STAT_SUSPEND_BY_DST:
73be902c 4705 coding->result = CODING_FINISH_INSUFFICIENT_DST;
d46c5b12 4706 break;
9864ebce
KH
4707 case CCL_STAT_QUIT:
4708 case CCL_STAT_INVALID_CMD:
73be902c 4709 coding->result = CODING_FINISH_INTERRUPT;
9864ebce 4710 break;
d46c5b12 4711 default:
73be902c 4712 coding->result = CODING_FINISH_NORMAL;
d46c5b12
KH
4713 break;
4714 }
73be902c 4715 return coding->result;
4ed46869
KH
4716}
4717
aaaf0b1e
KH
4718/* Decode EOL format of the text at PTR of BYTES length destructively
4719 according to CODING->eol_type. This is called after the CCL
4720 program produced a decoded text at PTR. If we do CRLF->LF
4721 conversion, update CODING->produced and CODING->produced_char. */
4722
4723static void
4724decode_eol_post_ccl (coding, ptr, bytes)
4725 struct coding_system *coding;
4726 unsigned char *ptr;
4727 int bytes;
4728{
4729 Lisp_Object val, saved_coding_symbol;
4730 unsigned char *pend = ptr + bytes;
4731 int dummy;
4732
4733 /* Remember the current coding system symbol. We set it back when
4734 an inconsistent EOL is found so that `last-coding-system-used' is
4735 set to the coding system that doesn't specify EOL conversion. */
4736 saved_coding_symbol = coding->symbol;
4737
4738 coding->spec.ccl.cr_carryover = 0;
4739 if (coding->eol_type == CODING_EOL_UNDECIDED)
4740 {
4741 /* Here, to avoid the call of setup_coding_system, we directly
4742 call detect_eol_type. */
4743 coding->eol_type = detect_eol_type (ptr, bytes, &dummy);
74b01b80
EZ
4744 if (coding->eol_type == CODING_EOL_INCONSISTENT)
4745 coding->eol_type = CODING_EOL_LF;
4746 if (coding->eol_type != CODING_EOL_UNDECIDED)
4747 {
4748 val = Fget (coding->symbol, Qeol_type);
4749 if (VECTORP (val) && XVECTOR (val)->size == 3)
4750 coding->symbol = XVECTOR (val)->contents[coding->eol_type];
4751 }
aaaf0b1e
KH
4752 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
4753 }
4754
74b01b80
EZ
4755 if (coding->eol_type == CODING_EOL_LF
4756 || coding->eol_type == CODING_EOL_UNDECIDED)
aaaf0b1e
KH
4757 {
4758 /* We have nothing to do. */
4759 ptr = pend;
4760 }
4761 else if (coding->eol_type == CODING_EOL_CRLF)
4762 {
4763 unsigned char *pstart = ptr, *p = ptr;
4764
4765 if (! (coding->mode & CODING_MODE_LAST_BLOCK)
4766 && *(pend - 1) == '\r')
4767 {
4768 /* If the last character is CR, we can't handle it here
4769 because LF will be in the not-yet-decoded source text.
9861e777 4770 Record that the CR is not yet processed. */
aaaf0b1e
KH
4771 coding->spec.ccl.cr_carryover = 1;
4772 coding->produced--;
4773 coding->produced_char--;
4774 pend--;
4775 }
4776 while (ptr < pend)
4777 {
4778 if (*ptr == '\r')
4779 {
4780 if (ptr + 1 < pend && *(ptr + 1) == '\n')
4781 {
4782 *p++ = '\n';
4783 ptr += 2;
4784 }
4785 else
4786 {
4787 if (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
4788 goto undo_eol_conversion;
4789 *p++ = *ptr++;
4790 }
4791 }
4792 else if (*ptr == '\n'
4793 && coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
4794 goto undo_eol_conversion;
4795 else
4796 *p++ = *ptr++;
4797 continue;
4798
4799 undo_eol_conversion:
4800 /* We have faced with inconsistent EOL format at PTR.
4801 Convert all LFs before PTR back to CRLFs. */
4802 for (p--, ptr--; p >= pstart; p--)
4803 {
4804 if (*p == '\n')
4805 *ptr-- = '\n', *ptr-- = '\r';
4806 else
4807 *ptr-- = *p;
4808 }
4809 /* If carryover is recorded, cancel it because we don't
4810 convert CRLF anymore. */
4811 if (coding->spec.ccl.cr_carryover)
4812 {
4813 coding->spec.ccl.cr_carryover = 0;
4814 coding->produced++;
4815 coding->produced_char++;
4816 pend++;
4817 }
4818 p = ptr = pend;
4819 coding->eol_type = CODING_EOL_LF;
4820 coding->symbol = saved_coding_symbol;
4821 }
4822 if (p < pend)
4823 {
4824 /* As each two-byte sequence CRLF was converted to LF, (PEND
4825 - P) is the number of deleted characters. */
4826 coding->produced -= pend - p;
4827 coding->produced_char -= pend - p;
4828 }
4829 }
4830 else /* i.e. coding->eol_type == CODING_EOL_CR */
4831 {
4832 unsigned char *p = ptr;
4833
4834 for (; ptr < pend; ptr++)
4835 {
4836 if (*ptr == '\r')
4837 *ptr = '\n';
4838 else if (*ptr == '\n'
4839 && coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
4840 {
4841 for (; p < ptr; p++)
4842 {
4843 if (*p == '\n')
4844 *p = '\r';
4845 }
4846 ptr = pend;
4847 coding->eol_type = CODING_EOL_LF;
4848 coding->symbol = saved_coding_symbol;
4849 }
4850 }
4851 }
4852}
4853
4ed46869
KH
4854/* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4855 decoding, it may detect coding system and format of end-of-line if
b73bfc1c
KH
4856 those are not yet decided. The source should be unibyte, the
4857 result is multibyte if CODING->dst_multibyte is nonzero, else
4858 unibyte. */
4ed46869
KH
4859
4860int
d46c5b12 4861decode_coding (coding, source, destination, src_bytes, dst_bytes)
4ed46869 4862 struct coding_system *coding;
a4244313
KR
4863 const unsigned char *source;
4864 unsigned char *destination;
4ed46869 4865 int src_bytes, dst_bytes;
4ed46869 4866{
9861e777
EZ
4867 int extra = 0;
4868
0ef69138 4869 if (coding->type == coding_type_undecided)
4ed46869
KH
4870 detect_coding (coding, source, src_bytes);
4871
aaaf0b1e
KH
4872 if (coding->eol_type == CODING_EOL_UNDECIDED
4873 && coding->type != coding_type_ccl)
8844fa83
KH
4874 {
4875 detect_eol (coding, source, src_bytes);
4876 /* We had better recover the original eol format if we
8ca3766a 4877 encounter an inconsistent eol format while decoding. */
8844fa83
KH
4878 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
4879 }
4ed46869 4880
b73bfc1c
KH
4881 coding->produced = coding->produced_char = 0;
4882 coding->consumed = coding->consumed_char = 0;
4883 coding->errors = 0;
4884 coding->result = CODING_FINISH_NORMAL;
4885
4ed46869
KH
4886 switch (coding->type)
4887 {
4ed46869 4888 case coding_type_sjis:
b73bfc1c
KH
4889 decode_coding_sjis_big5 (coding, source, destination,
4890 src_bytes, dst_bytes, 1);
4ed46869
KH
4891 break;
4892
4893 case coding_type_iso2022:
b73bfc1c
KH
4894 decode_coding_iso2022 (coding, source, destination,
4895 src_bytes, dst_bytes);
4ed46869
KH
4896 break;
4897
4898 case coding_type_big5:
b73bfc1c
KH
4899 decode_coding_sjis_big5 (coding, source, destination,
4900 src_bytes, dst_bytes, 0);
4901 break;
4902
4903 case coding_type_emacs_mule:
4904 decode_coding_emacs_mule (coding, source, destination,
4905 src_bytes, dst_bytes);
4ed46869
KH
4906 break;
4907
4908 case coding_type_ccl:
aaaf0b1e
KH
4909 if (coding->spec.ccl.cr_carryover)
4910 {
9861e777
EZ
4911 /* Put the CR which was not processed by the previous call
4912 of decode_eol_post_ccl in DESTINATION. It will be
4913 decoded together with the following LF by the call to
4914 decode_eol_post_ccl below. */
aaaf0b1e
KH
4915 *destination = '\r';
4916 coding->produced++;
4917 coding->produced_char++;
4918 dst_bytes--;
9861e777 4919 extra = coding->spec.ccl.cr_carryover;
aaaf0b1e 4920 }
9861e777 4921 ccl_coding_driver (coding, source, destination + extra,
b73bfc1c 4922 src_bytes, dst_bytes, 0);
aaaf0b1e 4923 if (coding->eol_type != CODING_EOL_LF)
9861e777
EZ
4924 {
4925 coding->produced += extra;
4926 coding->produced_char += extra;
4927 decode_eol_post_ccl (coding, destination, coding->produced);
4928 }
d46c5b12
KH
4929 break;
4930
b73bfc1c
KH
4931 default:
4932 decode_eol (coding, source, destination, src_bytes, dst_bytes);
4933 }
4934
4935 if (coding->result == CODING_FINISH_INSUFFICIENT_SRC
e7c9eef9 4936 && coding->mode & CODING_MODE_LAST_BLOCK
b73bfc1c
KH
4937 && coding->consumed == src_bytes)
4938 coding->result = CODING_FINISH_NORMAL;
4939
4940 if (coding->mode & CODING_MODE_LAST_BLOCK
4941 && coding->result == CODING_FINISH_INSUFFICIENT_SRC)
4942 {
a4244313 4943 const unsigned char *src = source + coding->consumed;
b73bfc1c
KH
4944 unsigned char *dst = destination + coding->produced;
4945
4946 src_bytes -= coding->consumed;
bb10be8b 4947 coding->errors++;
b73bfc1c
KH
4948 if (COMPOSING_P (coding))
4949 DECODE_COMPOSITION_END ('1');
4950 while (src_bytes--)
d46c5b12 4951 {
b73bfc1c
KH
4952 int c = *src++;
4953 dst += CHAR_STRING (c, dst);
4954 coding->produced_char++;
d46c5b12 4955 }
b73bfc1c
KH
4956 coding->consumed = coding->consumed_char = src - source;
4957 coding->produced = dst - destination;
73be902c 4958 coding->result = CODING_FINISH_NORMAL;
4ed46869
KH
4959 }
4960
b73bfc1c
KH
4961 if (!coding->dst_multibyte)
4962 {
4963 coding->produced = str_as_unibyte (destination, coding->produced);
4964 coding->produced_char = coding->produced;
4965 }
4ed46869 4966
b73bfc1c
KH
4967 return coding->result;
4968}
52d41803 4969
b73bfc1c
KH
4970/* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4971 multibyteness of the source is CODING->src_multibyte, the
4972 multibyteness of the result is always unibyte. */
4ed46869
KH
4973
4974int
d46c5b12 4975encode_coding (coding, source, destination, src_bytes, dst_bytes)
4ed46869 4976 struct coding_system *coding;
a4244313
KR
4977 const unsigned char *source;
4978 unsigned char *destination;
4ed46869 4979 int src_bytes, dst_bytes;
4ed46869 4980{
b73bfc1c
KH
4981 coding->produced = coding->produced_char = 0;
4982 coding->consumed = coding->consumed_char = 0;
4983 coding->errors = 0;
4984 coding->result = CODING_FINISH_NORMAL;
4ed46869 4985
d46c5b12
KH
4986 switch (coding->type)
4987 {
4ed46869 4988 case coding_type_sjis:
b73bfc1c
KH
4989 encode_coding_sjis_big5 (coding, source, destination,
4990 src_bytes, dst_bytes, 1);
4ed46869
KH
4991 break;
4992
4993 case coding_type_iso2022:
b73bfc1c
KH
4994 encode_coding_iso2022 (coding, source, destination,
4995 src_bytes, dst_bytes);
4ed46869
KH
4996 break;
4997
4998 case coding_type_big5:
b73bfc1c
KH
4999 encode_coding_sjis_big5 (coding, source, destination,
5000 src_bytes, dst_bytes, 0);
5001 break;
5002
5003 case coding_type_emacs_mule:
5004 encode_coding_emacs_mule (coding, source, destination,
5005 src_bytes, dst_bytes);
4ed46869
KH
5006 break;
5007
5008 case coding_type_ccl:
b73bfc1c
KH
5009 ccl_coding_driver (coding, source, destination,
5010 src_bytes, dst_bytes, 1);
d46c5b12
KH
5011 break;
5012
b73bfc1c
KH
5013 default:
5014 encode_eol (coding, source, destination, src_bytes, dst_bytes);
5015 }
5016
73be902c
KH
5017 if (coding->mode & CODING_MODE_LAST_BLOCK
5018 && coding->result == CODING_FINISH_INSUFFICIENT_SRC)
b73bfc1c 5019 {
a4244313 5020 const unsigned char *src = source + coding->consumed;
b73bfc1c
KH
5021 unsigned char *dst = destination + coding->produced;
5022
5023 if (coding->type == coding_type_iso2022)
5024 ENCODE_RESET_PLANE_AND_REGISTER;
5025 if (COMPOSING_P (coding))
5026 *dst++ = ISO_CODE_ESC, *dst++ = '1';
5027 if (coding->consumed < src_bytes)
d46c5b12 5028 {
b73bfc1c
KH
5029 int len = src_bytes - coding->consumed;
5030
fabf4a91 5031 BCOPY_SHORT (src, dst, len);
b73bfc1c
KH
5032 if (coding->src_multibyte)
5033 len = str_as_unibyte (dst, len);
5034 dst += len;
5035 coding->consumed = src_bytes;
d46c5b12 5036 }
b73bfc1c 5037 coding->produced = coding->produced_char = dst - destination;
73be902c 5038 coding->result = CODING_FINISH_NORMAL;
4ed46869
KH
5039 }
5040
bb10be8b
KH
5041 if (coding->result == CODING_FINISH_INSUFFICIENT_SRC
5042 && coding->consumed == src_bytes)
5043 coding->result = CODING_FINISH_NORMAL;
5044
b73bfc1c 5045 return coding->result;
4ed46869
KH
5046}
5047
fb88bf2d
KH
5048/* Scan text in the region between *BEG and *END (byte positions),
5049 skip characters which we don't have to decode by coding system
5050 CODING at the head and tail, then set *BEG and *END to the region
5051 of the text we actually have to convert. The caller should move
b73bfc1c
KH
5052 the gap out of the region in advance if the region is from a
5053 buffer.
4ed46869 5054
d46c5b12
KH
5055 If STR is not NULL, *BEG and *END are indices into STR. */
5056
5057static void
5058shrink_decoding_region (beg, end, coding, str)
5059 int *beg, *end;
5060 struct coding_system *coding;
5061 unsigned char *str;
5062{
fb88bf2d 5063 unsigned char *begp_orig, *begp, *endp_orig, *endp, c;
d46c5b12 5064 int eol_conversion;
88993dfd 5065 Lisp_Object translation_table;
d46c5b12
KH
5066
5067 if (coding->type == coding_type_ccl
5068 || coding->type == coding_type_undecided
b73bfc1c
KH
5069 || coding->eol_type != CODING_EOL_LF
5070 || !NILP (coding->post_read_conversion)
5071 || coding->composing != COMPOSITION_DISABLED)
d46c5b12
KH
5072 {
5073 /* We can't skip any data. */
5074 return;
5075 }
b73bfc1c
KH
5076 if (coding->type == coding_type_no_conversion
5077 || coding->type == coding_type_raw_text
5078 || coding->type == coding_type_emacs_mule)
d46c5b12 5079 {
fb88bf2d
KH
5080 /* We need no conversion, but don't have to skip any data here.
5081 Decoding routine handles them effectively anyway. */
d46c5b12
KH
5082 return;
5083 }
5084
88993dfd
KH
5085 translation_table = coding->translation_table_for_decode;
5086 if (NILP (translation_table) && !NILP (Venable_character_translation))
5087 translation_table = Vstandard_translation_table_for_decode;
5088 if (CHAR_TABLE_P (translation_table))
5089 {
5090 int i;
5091 for (i = 0; i < 128; i++)
5092 if (!NILP (CHAR_TABLE_REF (translation_table, i)))
5093 break;
5094 if (i < 128)
fa46990e 5095 /* Some ASCII character should be translated. We give up
88993dfd
KH
5096 shrinking. */
5097 return;
5098 }
5099
b73bfc1c 5100 if (coding->heading_ascii >= 0)
d46c5b12
KH
5101 /* Detection routine has already found how much we can skip at the
5102 head. */
5103 *beg += coding->heading_ascii;
5104
5105 if (str)
5106 {
5107 begp_orig = begp = str + *beg;
5108 endp_orig = endp = str + *end;
5109 }
5110 else
5111 {
fb88bf2d 5112 begp_orig = begp = BYTE_POS_ADDR (*beg);
d46c5b12
KH
5113 endp_orig = endp = begp + *end - *beg;
5114 }
5115
fa46990e
DL
5116 eol_conversion = (coding->eol_type == CODING_EOL_CR
5117 || coding->eol_type == CODING_EOL_CRLF);
5118
d46c5b12
KH
5119 switch (coding->type)
5120 {
d46c5b12
KH
5121 case coding_type_sjis:
5122 case coding_type_big5:
5123 /* We can skip all ASCII characters at the head. */
5124 if (coding->heading_ascii < 0)
5125 {
5126 if (eol_conversion)
de9d083c 5127 while (begp < endp && *begp < 0x80 && *begp != '\r') begp++;
d46c5b12
KH
5128 else
5129 while (begp < endp && *begp < 0x80) begp++;
5130 }
5131 /* We can skip all ASCII characters at the tail except for the
5132 second byte of SJIS or BIG5 code. */
5133 if (eol_conversion)
de9d083c 5134 while (begp < endp && endp[-1] < 0x80 && endp[-1] != '\r') endp--;
d46c5b12
KH
5135 else
5136 while (begp < endp && endp[-1] < 0x80) endp--;
ee59c65f
RS
5137 /* Do not consider LF as ascii if preceded by CR, since that
5138 confuses eol decoding. */
5139 if (begp < endp && endp < endp_orig && endp[-1] == '\r' && endp[0] == '\n')
5140 endp++;
d46c5b12
KH
5141 if (begp < endp && endp < endp_orig && endp[-1] >= 0x80)
5142 endp++;
5143 break;
5144
b73bfc1c 5145 case coding_type_iso2022:
622fece5
KH
5146 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, 0) != CHARSET_ASCII)
5147 /* We can't skip any data. */
5148 break;
d46c5b12
KH
5149 if (coding->heading_ascii < 0)
5150 {
d46c5b12
KH
5151 /* We can skip all ASCII characters at the head except for a
5152 few control codes. */
5153 while (begp < endp && (c = *begp) < 0x80
5154 && c != ISO_CODE_CR && c != ISO_CODE_SO
5155 && c != ISO_CODE_SI && c != ISO_CODE_ESC
5156 && (!eol_conversion || c != ISO_CODE_LF))
5157 begp++;
5158 }
5159 switch (coding->category_idx)
5160 {
5161 case CODING_CATEGORY_IDX_ISO_8_1:
5162 case CODING_CATEGORY_IDX_ISO_8_2:
5163 /* We can skip all ASCII characters at the tail. */
5164 if (eol_conversion)
de9d083c 5165 while (begp < endp && (c = endp[-1]) < 0x80 && c != '\r') endp--;
d46c5b12
KH
5166 else
5167 while (begp < endp && endp[-1] < 0x80) endp--;
ee59c65f
RS
5168 /* Do not consider LF as ascii if preceded by CR, since that
5169 confuses eol decoding. */
5170 if (begp < endp && endp < endp_orig && endp[-1] == '\r' && endp[0] == '\n')
5171 endp++;
d46c5b12
KH
5172 break;
5173
5174 case CODING_CATEGORY_IDX_ISO_7:
5175 case CODING_CATEGORY_IDX_ISO_7_TIGHT:
de79a6a5 5176 {
8ca3766a 5177 /* We can skip all characters at the tail except for 8-bit
de79a6a5
KH
5178 codes and ESC and the following 2-byte at the tail. */
5179 unsigned char *eight_bit = NULL;
5180
5181 if (eol_conversion)
5182 while (begp < endp
5183 && (c = endp[-1]) != ISO_CODE_ESC && c != '\r')
5184 {
5185 if (!eight_bit && c & 0x80) eight_bit = endp;
5186 endp--;
5187 }
5188 else
5189 while (begp < endp
5190 && (c = endp[-1]) != ISO_CODE_ESC)
5191 {
5192 if (!eight_bit && c & 0x80) eight_bit = endp;
5193 endp--;
5194 }
5195 /* Do not consider LF as ascii if preceded by CR, since that
5196 confuses eol decoding. */
5197 if (begp < endp && endp < endp_orig
5198 && endp[-1] == '\r' && endp[0] == '\n')
5199 endp++;
5200 if (begp < endp && endp[-1] == ISO_CODE_ESC)
5201 {
5202 if (endp + 1 < endp_orig && end[0] == '(' && end[1] == 'B')
5203 /* This is an ASCII designation sequence. We can
5204 surely skip the tail. But, if we have
5205 encountered an 8-bit code, skip only the codes
5206 after that. */
5207 endp = eight_bit ? eight_bit : endp + 2;
5208 else
5209 /* Hmmm, we can't skip the tail. */
5210 endp = endp_orig;
5211 }
5212 else if (eight_bit)
5213 endp = eight_bit;
5214 }
d46c5b12 5215 }
b73bfc1c
KH
5216 break;
5217
5218 default:
5219 abort ();
d46c5b12
KH
5220 }
5221 *beg += begp - begp_orig;
5222 *end += endp - endp_orig;
5223 return;
5224}
5225
5226/* Like shrink_decoding_region but for encoding. */
5227
5228static void
5229shrink_encoding_region (beg, end, coding, str)
5230 int *beg, *end;
5231 struct coding_system *coding;
5232 unsigned char *str;
5233{
5234 unsigned char *begp_orig, *begp, *endp_orig, *endp;
5235 int eol_conversion;
88993dfd 5236 Lisp_Object translation_table;
d46c5b12 5237
b73bfc1c
KH
5238 if (coding->type == coding_type_ccl
5239 || coding->eol_type == CODING_EOL_CRLF
5240 || coding->eol_type == CODING_EOL_CR
87323294 5241 || (coding->cmp_data && coding->cmp_data->used > 0))
d46c5b12 5242 {
b73bfc1c
KH
5243 /* We can't skip any data. */
5244 return;
5245 }
5246 if (coding->type == coding_type_no_conversion
5247 || coding->type == coding_type_raw_text
5248 || coding->type == coding_type_emacs_mule
5249 || coding->type == coding_type_undecided)
5250 {
5251 /* We need no conversion, but don't have to skip any data here.
5252 Encoding routine handles them effectively anyway. */
d46c5b12
KH
5253 return;
5254 }
5255
88993dfd
KH
5256 translation_table = coding->translation_table_for_encode;
5257 if (NILP (translation_table) && !NILP (Venable_character_translation))
5258 translation_table = Vstandard_translation_table_for_encode;
5259 if (CHAR_TABLE_P (translation_table))
5260 {
5261 int i;
5262 for (i = 0; i < 128; i++)
5263 if (!NILP (CHAR_TABLE_REF (translation_table, i)))
5264 break;
5265 if (i < 128)
8ca3766a 5266 /* Some ASCII character should be translated. We give up
88993dfd
KH
5267 shrinking. */
5268 return;
5269 }
5270
d46c5b12
KH
5271 if (str)
5272 {
5273 begp_orig = begp = str + *beg;
5274 endp_orig = endp = str + *end;
5275 }
5276 else
5277 {
fb88bf2d 5278 begp_orig = begp = BYTE_POS_ADDR (*beg);
d46c5b12
KH
5279 endp_orig = endp = begp + *end - *beg;
5280 }
5281
5282 eol_conversion = (coding->eol_type == CODING_EOL_CR
5283 || coding->eol_type == CODING_EOL_CRLF);
5284
5285 /* Here, we don't have to check coding->pre_write_conversion because
5286 the caller is expected to have handled it already. */
5287 switch (coding->type)
5288 {
d46c5b12 5289 case coding_type_iso2022:
622fece5
KH
5290 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, 0) != CHARSET_ASCII)
5291 /* We can't skip any data. */
5292 break;
d46c5b12
KH
5293 if (coding->flags & CODING_FLAG_ISO_DESIGNATE_AT_BOL)
5294 {
93dec019 5295 unsigned char *bol = begp;
d46c5b12
KH
5296 while (begp < endp && *begp < 0x80)
5297 {
5298 begp++;
5299 if (begp[-1] == '\n')
5300 bol = begp;
5301 }
5302 begp = bol;
5303 goto label_skip_tail;
5304 }
5305 /* fall down ... */
5306
b73bfc1c
KH
5307 case coding_type_sjis:
5308 case coding_type_big5:
d46c5b12
KH
5309 /* We can skip all ASCII characters at the head and tail. */
5310 if (eol_conversion)
5311 while (begp < endp && *begp < 0x80 && *begp != '\n') begp++;
5312 else
5313 while (begp < endp && *begp < 0x80) begp++;
5314 label_skip_tail:
5315 if (eol_conversion)
5316 while (begp < endp && endp[-1] < 0x80 && endp[-1] != '\n') endp--;
5317 else
5318 while (begp < endp && *(endp - 1) < 0x80) endp--;
5319 break;
b73bfc1c
KH
5320
5321 default:
5322 abort ();
d46c5b12
KH
5323 }
5324
5325 *beg += begp - begp_orig;
5326 *end += endp - endp_orig;
5327 return;
5328}
5329
88993dfd
KH
5330/* As shrinking conversion region requires some overhead, we don't try
5331 shrinking if the length of conversion region is less than this
5332 value. */
5333static int shrink_conversion_region_threshhold = 1024;
5334
5335#define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5336 do { \
5337 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5338 { \
5339 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5340 else shrink_decoding_region (beg, end, coding, str); \
5341 } \
5342 } while (0)
5343
b843d1ae 5344static Lisp_Object
1c7457e2
KH
5345code_convert_region_unwind (arg)
5346 Lisp_Object arg;
b843d1ae
KH
5347{
5348 inhibit_pre_post_conversion = 0;
1c7457e2 5349 Vlast_coding_system_used = arg;
b843d1ae
KH
5350 return Qnil;
5351}
5352
ec6d2bb8
KH
5353/* Store information about all compositions in the range FROM and TO
5354 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5355 buffer or a string, defaults to the current buffer. */
5356
5357void
5358coding_save_composition (coding, from, to, obj)
5359 struct coding_system *coding;
5360 int from, to;
5361 Lisp_Object obj;
5362{
5363 Lisp_Object prop;
5364 int start, end;
5365
91bee881
KH
5366 if (coding->composing == COMPOSITION_DISABLED)
5367 return;
5368 if (!coding->cmp_data)
5369 coding_allocate_composition_data (coding, from);
ec6d2bb8
KH
5370 if (!find_composition (from, to, &start, &end, &prop, obj)
5371 || end > to)
5372 return;
5373 if (start < from
5374 && (!find_composition (end, to, &start, &end, &prop, obj)
5375 || end > to))
5376 return;
5377 coding->composing = COMPOSITION_NO;
ec6d2bb8
KH
5378 do
5379 {
5380 if (COMPOSITION_VALID_P (start, end, prop))
5381 {
5382 enum composition_method method = COMPOSITION_METHOD (prop);
5383 if (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH
5384 >= COMPOSITION_DATA_SIZE)
5385 coding_allocate_composition_data (coding, from);
5386 /* For relative composition, we remember start and end
5387 positions, for the other compositions, we also remember
5388 components. */
5389 CODING_ADD_COMPOSITION_START (coding, start - from, method);
5390 if (method != COMPOSITION_RELATIVE)
5391 {
5392 /* We must store a*/
5393 Lisp_Object val, ch;
5394
5395 val = COMPOSITION_COMPONENTS (prop);
5396 if (CONSP (val))
5397 while (CONSP (val))
5398 {
5399 ch = XCAR (val), val = XCDR (val);
5400 CODING_ADD_COMPOSITION_COMPONENT (coding, XINT (ch));
5401 }
5402 else if (VECTORP (val) || STRINGP (val))
5403 {
5404 int len = (VECTORP (val)
d5db4077 5405 ? XVECTOR (val)->size : SCHARS (val));
ec6d2bb8
KH
5406 int i;
5407 for (i = 0; i < len; i++)
5408 {
5409 ch = (STRINGP (val)
5410 ? Faref (val, make_number (i))
5411 : XVECTOR (val)->contents[i]);
5412 CODING_ADD_COMPOSITION_COMPONENT (coding, XINT (ch));
5413 }
5414 }
5415 else /* INTEGERP (val) */
5416 CODING_ADD_COMPOSITION_COMPONENT (coding, XINT (val));
5417 }
5418 CODING_ADD_COMPOSITION_END (coding, end - from);
5419 }
5420 start = end;
5421 }
5422 while (start < to
5423 && find_composition (start, to, &start, &end, &prop, obj)
5424 && end <= to);
5425
5426 /* Make coding->cmp_data point to the first memory block. */
5427 while (coding->cmp_data->prev)
5428 coding->cmp_data = coding->cmp_data->prev;
5429 coding->cmp_data_start = 0;
5430}
5431
5432/* Reflect the saved information about compositions to OBJ.
8ca3766a 5433 CODING->cmp_data points to a memory block for the information. OBJ
ec6d2bb8
KH
5434 is a buffer or a string, defaults to the current buffer. */
5435
33fb63eb 5436void
ec6d2bb8
KH
5437coding_restore_composition (coding, obj)
5438 struct coding_system *coding;
5439 Lisp_Object obj;
5440{
5441 struct composition_data *cmp_data = coding->cmp_data;
5442
5443 if (!cmp_data)
5444 return;
5445
5446 while (cmp_data->prev)
5447 cmp_data = cmp_data->prev;
5448
5449 while (cmp_data)
5450 {
5451 int i;
5452
78108bcd
KH
5453 for (i = 0; i < cmp_data->used && cmp_data->data[i] > 0;
5454 i += cmp_data->data[i])
ec6d2bb8
KH
5455 {
5456 int *data = cmp_data->data + i;
5457 enum composition_method method = (enum composition_method) data[3];
5458 Lisp_Object components;
5459
4307d534
KH
5460 if (data[0] < 0 || i + data[0] > cmp_data->used)
5461 /* Invalid composition data. */
5462 break;
5463
ec6d2bb8
KH
5464 if (method == COMPOSITION_RELATIVE)
5465 components = Qnil;
5466 else
5467 {
5468 int len = data[0] - 4, j;
5469 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
5470
b6871cc7
KH
5471 if (method == COMPOSITION_WITH_RULE_ALTCHARS
5472 && len % 2 == 0)
5473 len --;
09721b31
KH
5474 if (len < 1)
5475 /* Invalid composition data. */
5476 break;
ec6d2bb8
KH
5477 for (j = 0; j < len; j++)
5478 args[j] = make_number (data[4 + j]);
5479 components = (method == COMPOSITION_WITH_ALTCHARS
316d4bf9
SM
5480 ? Fstring (len, args)
5481 : Fvector (len, args));
ec6d2bb8
KH
5482 }
5483 compose_text (data[1], data[2], components, Qnil, obj);
5484 }
5485 cmp_data = cmp_data->next;
5486 }
5487}
5488
d46c5b12 5489/* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
fb88bf2d
KH
5490 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5491 coding system CODING, and return the status code of code conversion
5492 (currently, this value has no meaning).
5493
5494 How many characters (and bytes) are converted to how many
5495 characters (and bytes) are recorded in members of the structure
5496 CODING.
d46c5b12 5497
6e44253b 5498 If REPLACE is nonzero, we do various things as if the original text
d46c5b12 5499 is deleted and a new text is inserted. See the comments in
b73bfc1c
KH
5500 replace_range (insdel.c) to know what we are doing.
5501
5502 If REPLACE is zero, it is assumed that the source text is unibyte.
8ca3766a 5503 Otherwise, it is assumed that the source text is multibyte. */
4ed46869
KH
5504
5505int
6e44253b
KH
5506code_convert_region (from, from_byte, to, to_byte, coding, encodep, replace)
5507 int from, from_byte, to, to_byte, encodep, replace;
4ed46869 5508 struct coding_system *coding;
4ed46869 5509{
fb88bf2d 5510 int len = to - from, len_byte = to_byte - from_byte;
72d1a715 5511 int nchars_del = 0, nbytes_del = 0;
fb88bf2d 5512 int require, inserted, inserted_byte;
4b39528c 5513 int head_skip, tail_skip, total_skip = 0;
84d60297 5514 Lisp_Object saved_coding_symbol;
fb88bf2d 5515 int first = 1;
fb88bf2d 5516 unsigned char *src, *dst;
84d60297 5517 Lisp_Object deletion;
e133c8fa 5518 int orig_point = PT, orig_len = len;
6abb9bd9 5519 int prev_Z;
b73bfc1c
KH
5520 int multibyte_p = !NILP (current_buffer->enable_multibyte_characters);
5521
84d60297 5522 deletion = Qnil;
8844fa83 5523 saved_coding_symbol = coding->symbol;
d46c5b12 5524
83fa074f 5525 if (from < PT && PT < to)
e133c8fa
KH
5526 {
5527 TEMP_SET_PT_BOTH (from, from_byte);
5528 orig_point = from;
5529 }
83fa074f 5530
6e44253b 5531 if (replace)
d46c5b12 5532 {
fb88bf2d 5533 int saved_from = from;
e077cc80 5534 int saved_inhibit_modification_hooks;
fb88bf2d 5535
d46c5b12 5536 prepare_to_modify_buffer (from, to, &from);
fb88bf2d
KH
5537 if (saved_from != from)
5538 {
5539 to = from + len;
b73bfc1c 5540 from_byte = CHAR_TO_BYTE (from), to_byte = CHAR_TO_BYTE (to);
fb88bf2d
KH
5541 len_byte = to_byte - from_byte;
5542 }
e077cc80
KH
5543
5544 /* The code conversion routine can not preserve text properties
5545 for now. So, we must remove all text properties in the
5546 region. Here, we must suppress all modification hooks. */
5547 saved_inhibit_modification_hooks = inhibit_modification_hooks;
5548 inhibit_modification_hooks = 1;
5549 Fset_text_properties (make_number (from), make_number (to), Qnil, Qnil);
5550 inhibit_modification_hooks = saved_inhibit_modification_hooks;
d46c5b12 5551 }
d46c5b12
KH
5552
5553 if (! encodep && CODING_REQUIRE_DETECTION (coding))
5554 {
12410ef1 5555 /* We must detect encoding of text and eol format. */
d46c5b12
KH
5556
5557 if (from < GPT && to > GPT)
5558 move_gap_both (from, from_byte);
5559 if (coding->type == coding_type_undecided)
5560 {
fb88bf2d 5561 detect_coding (coding, BYTE_POS_ADDR (from_byte), len_byte);
d46c5b12 5562 if (coding->type == coding_type_undecided)
62b3ef1d
KH
5563 {
5564 /* It seems that the text contains only ASCII, but we
d9aef30f 5565 should not leave it undecided because the deeper
62b3ef1d
KH
5566 decoding routine (decode_coding) tries to detect the
5567 encodings again in vain. */
5568 coding->type = coding_type_emacs_mule;
5569 coding->category_idx = CODING_CATEGORY_IDX_EMACS_MULE;
d280ccb6
KH
5570 /* As emacs-mule decoder will handle composition, we
5571 need this setting to allocate coding->cmp_data
5572 later. */
5573 coding->composing = COMPOSITION_NO;
62b3ef1d 5574 }
d46c5b12 5575 }
aaaf0b1e
KH
5576 if (coding->eol_type == CODING_EOL_UNDECIDED
5577 && coding->type != coding_type_ccl)
d46c5b12 5578 {
d46c5b12
KH
5579 detect_eol (coding, BYTE_POS_ADDR (from_byte), len_byte);
5580 if (coding->eol_type == CODING_EOL_UNDECIDED)
5581 coding->eol_type = CODING_EOL_LF;
5582 /* We had better recover the original eol format if we
8ca3766a 5583 encounter an inconsistent eol format while decoding. */
d46c5b12
KH
5584 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
5585 }
5586 }
5587
d46c5b12
KH
5588 /* Now we convert the text. */
5589
5590 /* For encoding, we must process pre-write-conversion in advance. */
b73bfc1c
KH
5591 if (! inhibit_pre_post_conversion
5592 && encodep
d46c5b12
KH
5593 && SYMBOLP (coding->pre_write_conversion)
5594 && ! NILP (Ffboundp (coding->pre_write_conversion)))
5595 {
2b4f9037
KH
5596 /* The function in pre-write-conversion may put a new text in a
5597 new buffer. */
0007bdd0
KH
5598 struct buffer *prev = current_buffer;
5599 Lisp_Object new;
d46c5b12 5600
1c7457e2 5601 record_unwind_protect (code_convert_region_unwind,
24a948a7 5602 Vlast_coding_system_used);
b843d1ae
KH
5603 /* We should not call any more pre-write/post-read-conversion
5604 functions while this pre-write-conversion is running. */
5605 inhibit_pre_post_conversion = 1;
b39f748c
AS
5606 call2 (coding->pre_write_conversion,
5607 make_number (from), make_number (to));
b843d1ae
KH
5608 inhibit_pre_post_conversion = 0;
5609 /* Discard the unwind protect. */
5610 specpdl_ptr--;
5611
d46c5b12
KH
5612 if (current_buffer != prev)
5613 {
5614 len = ZV - BEGV;
0007bdd0 5615 new = Fcurrent_buffer ();
d46c5b12 5616 set_buffer_internal_1 (prev);
7dae4502 5617 del_range_2 (from, from_byte, to, to_byte, 0);
e133c8fa 5618 TEMP_SET_PT_BOTH (from, from_byte);
0007bdd0
KH
5619 insert_from_buffer (XBUFFER (new), 1, len, 0);
5620 Fkill_buffer (new);
e133c8fa
KH
5621 if (orig_point >= to)
5622 orig_point += len - orig_len;
5623 else if (orig_point > from)
5624 orig_point = from;
5625 orig_len = len;
d46c5b12 5626 to = from + len;
b73bfc1c
KH
5627 from_byte = CHAR_TO_BYTE (from);
5628 to_byte = CHAR_TO_BYTE (to);
d46c5b12 5629 len_byte = to_byte - from_byte;
e133c8fa 5630 TEMP_SET_PT_BOTH (from, from_byte);
d46c5b12
KH
5631 }
5632 }
5633
12410ef1 5634 if (replace)
72d1a715
RS
5635 {
5636 if (! EQ (current_buffer->undo_list, Qt))
5637 deletion = make_buffer_string_both (from, from_byte, to, to_byte, 1);
5638 else
5639 {
5640 nchars_del = to - from;
5641 nbytes_del = to_byte - from_byte;
5642 }
5643 }
12410ef1 5644
ec6d2bb8
KH
5645 if (coding->composing != COMPOSITION_DISABLED)
5646 {
5647 if (encodep)
5648 coding_save_composition (coding, from, to, Fcurrent_buffer ());
5649 else
5650 coding_allocate_composition_data (coding, from);
5651 }
fb88bf2d 5652
ce559e6f
KH
5653 /* Try to skip the heading and tailing ASCIIs. We can't skip them
5654 if we must run CCL program or there are compositions to
5655 encode. */
5656 if (coding->type != coding_type_ccl
5657 && (! coding->cmp_data || coding->cmp_data->used == 0))
4956c225
KH
5658 {
5659 int from_byte_orig = from_byte, to_byte_orig = to_byte;
ec6d2bb8 5660
4956c225
KH
5661 if (from < GPT && GPT < to)
5662 move_gap_both (from, from_byte);
5663 SHRINK_CONVERSION_REGION (&from_byte, &to_byte, coding, NULL, encodep);
5664 if (from_byte == to_byte
5665 && (encodep || NILP (coding->post_read_conversion))
5666 && ! CODING_REQUIRE_FLUSHING (coding))
5667 {
5668 coding->produced = len_byte;
5669 coding->produced_char = len;
5670 if (!replace)
5671 /* We must record and adjust for this new text now. */
5672 adjust_after_insert (from, from_byte_orig, to, to_byte_orig, len);
ce559e6f 5673 coding_free_composition_data (coding);
4956c225
KH
5674 return 0;
5675 }
5676
5677 head_skip = from_byte - from_byte_orig;
5678 tail_skip = to_byte_orig - to_byte;
5679 total_skip = head_skip + tail_skip;
5680 from += head_skip;
5681 to -= tail_skip;
5682 len -= total_skip; len_byte -= total_skip;
5683 }
d46c5b12 5684
8ca3766a 5685 /* For conversion, we must put the gap before the text in addition to
fb88bf2d
KH
5686 making the gap larger for efficient decoding. The required gap
5687 size starts from 2000 which is the magic number used in make_gap.
5688 But, after one batch of conversion, it will be incremented if we
5689 find that it is not enough . */
d46c5b12
KH
5690 require = 2000;
5691
5692 if (GAP_SIZE < require)
5693 make_gap (require - GAP_SIZE);
5694 move_gap_both (from, from_byte);
5695
d46c5b12 5696 inserted = inserted_byte = 0;
fb88bf2d
KH
5697
5698 GAP_SIZE += len_byte;
5699 ZV -= len;
5700 Z -= len;
5701 ZV_BYTE -= len_byte;
5702 Z_BYTE -= len_byte;
5703
d9f9a1bc
GM
5704 if (GPT - BEG < BEG_UNCHANGED)
5705 BEG_UNCHANGED = GPT - BEG;
5706 if (Z - GPT < END_UNCHANGED)
5707 END_UNCHANGED = Z - GPT;
f2558efd 5708
b73bfc1c
KH
5709 if (!encodep && coding->src_multibyte)
5710 {
5711 /* Decoding routines expects that the source text is unibyte.
5712 We must convert 8-bit characters of multibyte form to
5713 unibyte. */
5714 int len_byte_orig = len_byte;
5715 len_byte = str_as_unibyte (GAP_END_ADDR - len_byte, len_byte);
5716 if (len_byte < len_byte_orig)
5717 safe_bcopy (GAP_END_ADDR - len_byte_orig, GAP_END_ADDR - len_byte,
5718 len_byte);
5719 coding->src_multibyte = 0;
5720 }
5721
d46c5b12
KH
5722 for (;;)
5723 {
fb88bf2d 5724 int result;
d46c5b12 5725
ec6d2bb8 5726 /* The buffer memory is now:
b73bfc1c
KH
5727 +--------+converted-text+---------+-------original-text-------+---+
5728 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5729 |<---------------------- GAP ----------------------->| */
ec6d2bb8
KH
5730 src = GAP_END_ADDR - len_byte;
5731 dst = GPT_ADDR + inserted_byte;
5732
d46c5b12 5733 if (encodep)
fb88bf2d 5734 result = encode_coding (coding, src, dst, len_byte, 0);
d46c5b12 5735 else
0e79d667
RS
5736 {
5737 if (coding->composing != COMPOSITION_DISABLED)
5738 coding->cmp_data->char_offset = from + inserted;
5739 result = decode_coding (coding, src, dst, len_byte, 0);
5740 }
ec6d2bb8
KH
5741
5742 /* The buffer memory is now:
b73bfc1c
KH
5743 +--------+-------converted-text----+--+------original-text----+---+
5744 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5745 |<---------------------- GAP ----------------------->| */
ec6d2bb8 5746
d46c5b12
KH
5747 inserted += coding->produced_char;
5748 inserted_byte += coding->produced;
d46c5b12 5749 len_byte -= coding->consumed;
ec6d2bb8
KH
5750
5751 if (result == CODING_FINISH_INSUFFICIENT_CMP)
5752 {
5753 coding_allocate_composition_data (coding, from + inserted);
5754 continue;
5755 }
5756
fb88bf2d 5757 src += coding->consumed;
3636f7a3 5758 dst += coding->produced;
d46c5b12 5759
9864ebce
KH
5760 if (result == CODING_FINISH_NORMAL)
5761 {
5762 src += len_byte;
5763 break;
5764 }
d46c5b12
KH
5765 if (! encodep && result == CODING_FINISH_INCONSISTENT_EOL)
5766 {
fb88bf2d 5767 unsigned char *pend = dst, *p = pend - inserted_byte;
38edf7d4 5768 Lisp_Object eol_type;
d46c5b12
KH
5769
5770 /* Encode LFs back to the original eol format (CR or CRLF). */
5771 if (coding->eol_type == CODING_EOL_CR)
5772 {
5773 while (p < pend) if (*p++ == '\n') p[-1] = '\r';
5774 }
5775 else
5776 {
d46c5b12
KH
5777 int count = 0;
5778
fb88bf2d
KH
5779 while (p < pend) if (*p++ == '\n') count++;
5780 if (src - dst < count)
d46c5b12 5781 {
38edf7d4 5782 /* We don't have sufficient room for encoding LFs
fb88bf2d
KH
5783 back to CRLF. We must record converted and
5784 not-yet-converted text back to the buffer
5785 content, enlarge the gap, then record them out of
5786 the buffer contents again. */
5787 int add = len_byte + inserted_byte;
5788
5789 GAP_SIZE -= add;
5790 ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
5791 GPT += inserted_byte; GPT_BYTE += inserted_byte;
5792 make_gap (count - GAP_SIZE);
5793 GAP_SIZE += add;
5794 ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
5795 GPT -= inserted_byte; GPT_BYTE -= inserted_byte;
5796 /* Don't forget to update SRC, DST, and PEND. */
5797 src = GAP_END_ADDR - len_byte;
5798 dst = GPT_ADDR + inserted_byte;
5799 pend = dst;
d46c5b12 5800 }
d46c5b12
KH
5801 inserted += count;
5802 inserted_byte += count;
fb88bf2d
KH
5803 coding->produced += count;
5804 p = dst = pend + count;
5805 while (count)
5806 {
5807 *--p = *--pend;
5808 if (*p == '\n') count--, *--p = '\r';
5809 }
d46c5b12
KH
5810 }
5811
5812 /* Suppress eol-format conversion in the further conversion. */
5813 coding->eol_type = CODING_EOL_LF;
5814
38edf7d4
KH
5815 /* Set the coding system symbol to that for Unix-like EOL. */
5816 eol_type = Fget (saved_coding_symbol, Qeol_type);
5817 if (VECTORP (eol_type)
5818 && XVECTOR (eol_type)->size == 3
5819 && SYMBOLP (XVECTOR (eol_type)->contents[CODING_EOL_LF]))
5820 coding->symbol = XVECTOR (eol_type)->contents[CODING_EOL_LF];
5821 else
5822 coding->symbol = saved_coding_symbol;
93dec019 5823
fb88bf2d 5824 continue;
d46c5b12
KH
5825 }
5826 if (len_byte <= 0)
944bd420
KH
5827 {
5828 if (coding->type != coding_type_ccl
5829 || coding->mode & CODING_MODE_LAST_BLOCK)
5830 break;
5831 coding->mode |= CODING_MODE_LAST_BLOCK;
5832 continue;
5833 }
d46c5b12
KH
5834 if (result == CODING_FINISH_INSUFFICIENT_SRC)
5835 {
5836 /* The source text ends in invalid codes. Let's just
5837 make them valid buffer contents, and finish conversion. */
70ad9fc4
GM
5838 if (multibyte_p)
5839 {
5840 unsigned char *start = dst;
93dec019 5841
70ad9fc4
GM
5842 inserted += len_byte;
5843 while (len_byte--)
5844 {
5845 int c = *src++;
5846 dst += CHAR_STRING (c, dst);
5847 }
5848
5849 inserted_byte += dst - start;
5850 }
5851 else
5852 {
5853 inserted += len_byte;
5854 inserted_byte += len_byte;
5855 while (len_byte--)
5856 *dst++ = *src++;
5857 }
d46c5b12
KH
5858 break;
5859 }
9864ebce
KH
5860 if (result == CODING_FINISH_INTERRUPT)
5861 {
5862 /* The conversion procedure was interrupted by a user. */
9864ebce
KH
5863 break;
5864 }
5865 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5866 if (coding->consumed < 1)
5867 {
5868 /* It's quite strange to require more memory without
5869 consuming any bytes. Perhaps CCL program bug. */
9864ebce
KH
5870 break;
5871 }
fb88bf2d
KH
5872 if (first)
5873 {
5874 /* We have just done the first batch of conversion which was
8ca3766a 5875 stopped because of insufficient gap. Let's reconsider the
fb88bf2d
KH
5876 required gap size (i.e. SRT - DST) now.
5877
5878 We have converted ORIG bytes (== coding->consumed) into
5879 NEW bytes (coding->produced). To convert the remaining
5880 LEN bytes, we may need REQUIRE bytes of gap, where:
5881 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5882 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5883 Here, we are sure that NEW >= ORIG. */
b3385c28
KH
5884 float ratio;
5885
5886 if (coding->produced <= coding->consumed)
5887 {
5888 /* This happens because of CCL-based coding system with
5889 eol-type CRLF. */
5890 require = 0;
5891 }
5892 else
5893 {
5894 ratio = (coding->produced - coding->consumed) / coding->consumed;
5895 require = len_byte * ratio;
5896 }
fb88bf2d
KH
5897 first = 0;
5898 }
5899 if ((src - dst) < (require + 2000))
5900 {
5901 /* See the comment above the previous call of make_gap. */
5902 int add = len_byte + inserted_byte;
5903
5904 GAP_SIZE -= add;
5905 ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
5906 GPT += inserted_byte; GPT_BYTE += inserted_byte;
5907 make_gap (require + 2000);
5908 GAP_SIZE += add;
5909 ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
5910 GPT -= inserted_byte; GPT_BYTE -= inserted_byte;
fb88bf2d 5911 }
d46c5b12 5912 }
fb88bf2d
KH
5913 if (src - dst > 0) *dst = 0; /* Put an anchor. */
5914
b73bfc1c
KH
5915 if (encodep && coding->dst_multibyte)
5916 {
5917 /* The output is unibyte. We must convert 8-bit characters to
5918 multibyte form. */
5919 if (inserted_byte * 2 > GAP_SIZE)
5920 {
5921 GAP_SIZE -= inserted_byte;
5922 ZV += inserted_byte; Z += inserted_byte;
5923 ZV_BYTE += inserted_byte; Z_BYTE += inserted_byte;
5924 GPT += inserted_byte; GPT_BYTE += inserted_byte;
5925 make_gap (inserted_byte - GAP_SIZE);
5926 GAP_SIZE += inserted_byte;
5927 ZV -= inserted_byte; Z -= inserted_byte;
5928 ZV_BYTE -= inserted_byte; Z_BYTE -= inserted_byte;
5929 GPT -= inserted_byte; GPT_BYTE -= inserted_byte;
5930 }
5931 inserted_byte = str_to_multibyte (GPT_ADDR, GAP_SIZE, inserted_byte);
5932 }
7553d0e1 5933
93dec019 5934 /* If we shrank the conversion area, adjust it now. */
12410ef1
KH
5935 if (total_skip > 0)
5936 {
5937 if (tail_skip > 0)
5938 safe_bcopy (GAP_END_ADDR, GPT_ADDR + inserted_byte, tail_skip);
5939 inserted += total_skip; inserted_byte += total_skip;
5940 GAP_SIZE += total_skip;
5941 GPT -= head_skip; GPT_BYTE -= head_skip;
5942 ZV -= total_skip; ZV_BYTE -= total_skip;
5943 Z -= total_skip; Z_BYTE -= total_skip;
5944 from -= head_skip; from_byte -= head_skip;
5945 to += tail_skip; to_byte += tail_skip;
5946 }
5947
6abb9bd9 5948 prev_Z = Z;
72d1a715
RS
5949 if (! EQ (current_buffer->undo_list, Qt))
5950 adjust_after_replace (from, from_byte, deletion, inserted, inserted_byte);
5951 else
5952 adjust_after_replace_noundo (from, from_byte, nchars_del, nbytes_del,
5953 inserted, inserted_byte);
6abb9bd9 5954 inserted = Z - prev_Z;
4ed46869 5955
ec6d2bb8
KH
5956 if (!encodep && coding->cmp_data && coding->cmp_data->used)
5957 coding_restore_composition (coding, Fcurrent_buffer ());
5958 coding_free_composition_data (coding);
5959
b73bfc1c
KH
5960 if (! inhibit_pre_post_conversion
5961 && ! encodep && ! NILP (coding->post_read_conversion))
d46c5b12 5962 {
2b4f9037 5963 Lisp_Object val;
1c7457e2 5964 Lisp_Object saved_coding_system;
4ed46869 5965
e133c8fa
KH
5966 if (from != PT)
5967 TEMP_SET_PT_BOTH (from, from_byte);
6abb9bd9 5968 prev_Z = Z;
1c7457e2
KH
5969 record_unwind_protect (code_convert_region_unwind,
5970 Vlast_coding_system_used);
5971 saved_coding_system = Vlast_coding_system_used;
5972 Vlast_coding_system_used = coding->symbol;
b843d1ae
KH
5973 /* We should not call any more pre-write/post-read-conversion
5974 functions while this post-read-conversion is running. */
5975 inhibit_pre_post_conversion = 1;
2b4f9037 5976 val = call1 (coding->post_read_conversion, make_number (inserted));
b843d1ae 5977 inhibit_pre_post_conversion = 0;
1c7457e2
KH
5978 coding->symbol = Vlast_coding_system_used;
5979 Vlast_coding_system_used = saved_coding_system;
b843d1ae
KH
5980 /* Discard the unwind protect. */
5981 specpdl_ptr--;
b7826503 5982 CHECK_NUMBER (val);
944bd420 5983 inserted += Z - prev_Z;
e133c8fa
KH
5984 }
5985
5986 if (orig_point >= from)
5987 {
5988 if (orig_point >= from + orig_len)
5989 orig_point += inserted - orig_len;
5990 else
5991 orig_point = from;
5992 TEMP_SET_PT (orig_point);
d46c5b12 5993 }
4ed46869 5994
ec6d2bb8
KH
5995 if (replace)
5996 {
5997 signal_after_change (from, to - from, inserted);
e19539f1 5998 update_compositions (from, from + inserted, CHECK_BORDER);
ec6d2bb8 5999 }
2b4f9037 6000
fb88bf2d 6001 {
12410ef1
KH
6002 coding->consumed = to_byte - from_byte;
6003 coding->consumed_char = to - from;
6004 coding->produced = inserted_byte;
6005 coding->produced_char = inserted;
fb88bf2d 6006 }
7553d0e1 6007
fb88bf2d 6008 return 0;
d46c5b12
KH
6009}
6010
6011Lisp_Object
b73bfc1c
KH
6012run_pre_post_conversion_on_str (str, coding, encodep)
6013 Lisp_Object str;
6014 struct coding_system *coding;
6015 int encodep;
6016{
aed13378 6017 int count = SPECPDL_INDEX ();
cf3b32fc 6018 struct gcpro gcpro1, gcpro2;
b73bfc1c 6019 int multibyte = STRING_MULTIBYTE (str);
3fd9494b
RS
6020 Lisp_Object buffer;
6021 struct buffer *buf;
cf3b32fc 6022 Lisp_Object old_deactivate_mark;
b73bfc1c
KH
6023
6024 record_unwind_protect (Fset_buffer, Fcurrent_buffer ());
1c7457e2
KH
6025 record_unwind_protect (code_convert_region_unwind,
6026 Vlast_coding_system_used);
cf3b32fc
RS
6027 /* It is not crucial to specbind this. */
6028 old_deactivate_mark = Vdeactivate_mark;
6029 GCPRO2 (str, old_deactivate_mark);
3fd9494b
RS
6030
6031 buffer = Fget_buffer_create (build_string (" *code-converting-work*"));
6032 buf = XBUFFER (buffer);
6033
6ed0af67 6034 delete_all_overlays (buf);
3fd9494b
RS
6035 buf->directory = current_buffer->directory;
6036 buf->read_only = Qnil;
6037 buf->filename = Qnil;
6038 buf->undo_list = Qt;
6ed0af67
SM
6039 eassert (buf->overlays_before == NULL);
6040 eassert (buf->overlays_after == NULL);
3fd9494b
RS
6041
6042 set_buffer_internal (buf);
b73bfc1c
KH
6043 /* We must insert the contents of STR as is without
6044 unibyte<->multibyte conversion. For that, we adjust the
6045 multibyteness of the working buffer to that of STR. */
6046 Ferase_buffer ();
3fd9494b
RS
6047 buf->enable_multibyte_characters = multibyte ? Qt : Qnil;
6048
b73bfc1c 6049 insert_from_string (str, 0, 0,
d5db4077 6050 SCHARS (str), SBYTES (str), 0);
b73bfc1c
KH
6051 UNGCPRO;
6052 inhibit_pre_post_conversion = 1;
6053 if (encodep)
6054 call2 (coding->pre_write_conversion, make_number (BEG), make_number (Z));
6055 else
6bac5b12 6056 {
1c7457e2 6057 Vlast_coding_system_used = coding->symbol;
6bac5b12
KH
6058 TEMP_SET_PT_BOTH (BEG, BEG_BYTE);
6059 call1 (coding->post_read_conversion, make_number (Z - BEG));
1c7457e2 6060 coding->symbol = Vlast_coding_system_used;
6bac5b12 6061 }
b73bfc1c 6062 inhibit_pre_post_conversion = 0;
cf3b32fc 6063 Vdeactivate_mark = old_deactivate_mark;
78108bcd 6064 str = make_buffer_string (BEG, Z, 1);
b73bfc1c
KH
6065 return unbind_to (count, str);
6066}
6067
6068Lisp_Object
6069decode_coding_string (str, coding, nocopy)
d46c5b12 6070 Lisp_Object str;
4ed46869 6071 struct coding_system *coding;
b73bfc1c 6072 int nocopy;
4ed46869 6073{
d46c5b12 6074 int len;
73be902c 6075 struct conversion_buffer buf;
da55a2b7 6076 int from, to_byte;
84d60297 6077 Lisp_Object saved_coding_symbol;
d46c5b12 6078 int result;
78108bcd 6079 int require_decoding;
73be902c
KH
6080 int shrinked_bytes = 0;
6081 Lisp_Object newstr;
2391eaa4 6082 int consumed, consumed_char, produced, produced_char;
4ed46869 6083
b73bfc1c 6084 from = 0;
d5db4077 6085 to_byte = SBYTES (str);
4ed46869 6086
8844fa83 6087 saved_coding_symbol = coding->symbol;
764ca8da
KH
6088 coding->src_multibyte = STRING_MULTIBYTE (str);
6089 coding->dst_multibyte = 1;
b73bfc1c 6090 if (CODING_REQUIRE_DETECTION (coding))
d46c5b12
KH
6091 {
6092 /* See the comments in code_convert_region. */
6093 if (coding->type == coding_type_undecided)
6094 {
d5db4077 6095 detect_coding (coding, SDATA (str), to_byte);
d46c5b12 6096 if (coding->type == coding_type_undecided)
d280ccb6
KH
6097 {
6098 coding->type = coding_type_emacs_mule;
6099 coding->category_idx = CODING_CATEGORY_IDX_EMACS_MULE;
6100 /* As emacs-mule decoder will handle composition, we
6101 need this setting to allocate coding->cmp_data
6102 later. */
6103 coding->composing = COMPOSITION_NO;
6104 }
d46c5b12 6105 }
aaaf0b1e
KH
6106 if (coding->eol_type == CODING_EOL_UNDECIDED
6107 && coding->type != coding_type_ccl)
d46c5b12
KH
6108 {
6109 saved_coding_symbol = coding->symbol;
d5db4077 6110 detect_eol (coding, SDATA (str), to_byte);
d46c5b12
KH
6111 if (coding->eol_type == CODING_EOL_UNDECIDED)
6112 coding->eol_type = CODING_EOL_LF;
6113 /* We had better recover the original eol format if we
8ca3766a 6114 encounter an inconsistent eol format while decoding. */
d46c5b12
KH
6115 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
6116 }
6117 }
4ed46869 6118
764ca8da
KH
6119 if (coding->type == coding_type_no_conversion
6120 || coding->type == coding_type_raw_text)
6121 coding->dst_multibyte = 0;
6122
78108bcd 6123 require_decoding = CODING_REQUIRE_DECODING (coding);
ec6d2bb8 6124
b73bfc1c 6125 if (STRING_MULTIBYTE (str))
d46c5b12 6126 {
b73bfc1c
KH
6127 /* Decoding routines expect the source text to be unibyte. */
6128 str = Fstring_as_unibyte (str);
d5db4077 6129 to_byte = SBYTES (str);
b73bfc1c 6130 nocopy = 1;
764ca8da 6131 coding->src_multibyte = 0;
b73bfc1c 6132 }
ec6d2bb8 6133
b73bfc1c 6134 /* Try to skip the heading and tailing ASCIIs. */
78108bcd 6135 if (require_decoding && coding->type != coding_type_ccl)
4956c225 6136 {
d5db4077 6137 SHRINK_CONVERSION_REGION (&from, &to_byte, coding, SDATA (str),
4956c225
KH
6138 0);
6139 if (from == to_byte)
78108bcd 6140 require_decoding = 0;
d5db4077 6141 shrinked_bytes = from + (SBYTES (str) - to_byte);
4956c225 6142 }
b73bfc1c 6143
439ad9ea
KH
6144 if (!require_decoding
6145 && !(SYMBOLP (coding->post_read_conversion)
6146 && !NILP (Ffboundp (coding->post_read_conversion))))
78108bcd 6147 {
d5db4077
KR
6148 coding->consumed = SBYTES (str);
6149 coding->consumed_char = SCHARS (str);
78108bcd
KH
6150 if (coding->dst_multibyte)
6151 {
6152 str = Fstring_as_multibyte (str);
6153 nocopy = 1;
6154 }
d5db4077
KR
6155 coding->produced = SBYTES (str);
6156 coding->produced_char = SCHARS (str);
78108bcd
KH
6157 return (nocopy ? str : Fcopy_sequence (str));
6158 }
6159
6160 if (coding->composing != COMPOSITION_DISABLED)
6161 coding_allocate_composition_data (coding, from);
b73bfc1c 6162 len = decoding_buffer_size (coding, to_byte - from);
73be902c 6163 allocate_conversion_buffer (buf, len);
4ed46869 6164
2391eaa4 6165 consumed = consumed_char = produced = produced_char = 0;
73be902c 6166 while (1)
4ed46869 6167 {
d5db4077 6168 result = decode_coding (coding, SDATA (str) + from + consumed,
73be902c
KH
6169 buf.data + produced, to_byte - from - consumed,
6170 buf.size - produced);
6171 consumed += coding->consumed;
2391eaa4 6172 consumed_char += coding->consumed_char;
73be902c
KH
6173 produced += coding->produced;
6174 produced_char += coding->produced_char;
2391eaa4
KH
6175 if (result == CODING_FINISH_NORMAL
6176 || (result == CODING_FINISH_INSUFFICIENT_SRC
6177 && coding->consumed == 0))
73be902c
KH
6178 break;
6179 if (result == CODING_FINISH_INSUFFICIENT_CMP)
6180 coding_allocate_composition_data (coding, from + produced_char);
6181 else if (result == CODING_FINISH_INSUFFICIENT_DST)
6182 extend_conversion_buffer (&buf);
6183 else if (result == CODING_FINISH_INCONSISTENT_EOL)
6184 {
8844fa83
KH
6185 Lisp_Object eol_type;
6186
73be902c
KH
6187 /* Recover the original EOL format. */
6188 if (coding->eol_type == CODING_EOL_CR)
6189 {
6190 unsigned char *p;
6191 for (p = buf.data; p < buf.data + produced; p++)
6192 if (*p == '\n') *p = '\r';
6193 }
6194 else if (coding->eol_type == CODING_EOL_CRLF)
6195 {
6196 int num_eol = 0;
6197 unsigned char *p0, *p1;
6198 for (p0 = buf.data, p1 = p0 + produced; p0 < p1; p0++)
6199 if (*p0 == '\n') num_eol++;
6200 if (produced + num_eol >= buf.size)
6201 extend_conversion_buffer (&buf);
6202 for (p0 = buf.data + produced, p1 = p0 + num_eol; p0 > buf.data;)
6203 {
6204 *--p1 = *--p0;
6205 if (*p0 == '\n') *--p1 = '\r';
6206 }
6207 produced += num_eol;
6208 produced_char += num_eol;
93dec019 6209 }
8844fa83 6210 /* Suppress eol-format conversion in the further conversion. */
73be902c 6211 coding->eol_type = CODING_EOL_LF;
8844fa83
KH
6212
6213 /* Set the coding system symbol to that for Unix-like EOL. */
6214 eol_type = Fget (saved_coding_symbol, Qeol_type);
6215 if (VECTORP (eol_type)
6216 && XVECTOR (eol_type)->size == 3
6217 && SYMBOLP (XVECTOR (eol_type)->contents[CODING_EOL_LF]))
6218 coding->symbol = XVECTOR (eol_type)->contents[CODING_EOL_LF];
6219 else
6220 coding->symbol = saved_coding_symbol;
6221
6222
73be902c 6223 }
4ed46869 6224 }
d46c5b12 6225
2391eaa4
KH
6226 coding->consumed = consumed;
6227 coding->consumed_char = consumed_char;
6228 coding->produced = produced;
6229 coding->produced_char = produced_char;
6230
78108bcd 6231 if (coding->dst_multibyte)
73be902c
KH
6232 newstr = make_uninit_multibyte_string (produced_char + shrinked_bytes,
6233 produced + shrinked_bytes);
78108bcd 6234 else
73be902c
KH
6235 newstr = make_uninit_string (produced + shrinked_bytes);
6236 if (from > 0)
a4244313
KR
6237 STRING_COPYIN (newstr, 0, SDATA (str), from);
6238 STRING_COPYIN (newstr, from, buf.data, produced);
73be902c 6239 if (shrinked_bytes > from)
a4244313
KR
6240 STRING_COPYIN (newstr, from + produced,
6241 SDATA (str) + to_byte,
6242 shrinked_bytes - from);
73be902c 6243 free_conversion_buffer (&buf);
b73bfc1c 6244
160a708c
KH
6245 coding->consumed += shrinked_bytes;
6246 coding->consumed_char += shrinked_bytes;
6247 coding->produced += shrinked_bytes;
6248 coding->produced_char += shrinked_bytes;
6249
b73bfc1c 6250 if (coding->cmp_data && coding->cmp_data->used)
73be902c 6251 coding_restore_composition (coding, newstr);
b73bfc1c
KH
6252 coding_free_composition_data (coding);
6253
6254 if (SYMBOLP (coding->post_read_conversion)
6255 && !NILP (Ffboundp (coding->post_read_conversion)))
73be902c 6256 newstr = run_pre_post_conversion_on_str (newstr, coding, 0);
b73bfc1c 6257
73be902c 6258 return newstr;
b73bfc1c
KH
6259}
6260
6261Lisp_Object
6262encode_coding_string (str, coding, nocopy)
6263 Lisp_Object str;
6264 struct coding_system *coding;
6265 int nocopy;
6266{
6267 int len;
73be902c 6268 struct conversion_buffer buf;
b73bfc1c 6269 int from, to, to_byte;
b73bfc1c 6270 int result;
73be902c
KH
6271 int shrinked_bytes = 0;
6272 Lisp_Object newstr;
2391eaa4 6273 int consumed, consumed_char, produced, produced_char;
b73bfc1c
KH
6274
6275 if (SYMBOLP (coding->pre_write_conversion)
6276 && !NILP (Ffboundp (coding->pre_write_conversion)))
6bac5b12 6277 str = run_pre_post_conversion_on_str (str, coding, 1);
b73bfc1c
KH
6278
6279 from = 0;
d5db4077
KR
6280 to = SCHARS (str);
6281 to_byte = SBYTES (str);
b73bfc1c 6282
e2c06b17
KH
6283 /* Encoding routines determine the multibyteness of the source text
6284 by coding->src_multibyte. */
6285 coding->src_multibyte = STRING_MULTIBYTE (str);
6286 coding->dst_multibyte = 0;
b73bfc1c 6287 if (! CODING_REQUIRE_ENCODING (coding))
826bfb8b 6288 {
d5db4077
KR
6289 coding->consumed = SBYTES (str);
6290 coding->consumed_char = SCHARS (str);
b73bfc1c
KH
6291 if (STRING_MULTIBYTE (str))
6292 {
6293 str = Fstring_as_unibyte (str);
6294 nocopy = 1;
6295 }
d5db4077
KR
6296 coding->produced = SBYTES (str);
6297 coding->produced_char = SCHARS (str);
b73bfc1c 6298 return (nocopy ? str : Fcopy_sequence (str));
826bfb8b
KH
6299 }
6300
b73bfc1c
KH
6301 if (coding->composing != COMPOSITION_DISABLED)
6302 coding_save_composition (coding, from, to, str);
ec6d2bb8 6303
ce559e6f
KH
6304 /* Try to skip the heading and tailing ASCIIs. We can't skip them
6305 if we must run CCL program or there are compositions to
6306 encode. */
6307 if (coding->type != coding_type_ccl
6308 && (! coding->cmp_data || coding->cmp_data->used == 0))
4956c225 6309 {
d5db4077 6310 SHRINK_CONVERSION_REGION (&from, &to_byte, coding, SDATA (str),
4956c225
KH
6311 1);
6312 if (from == to_byte)
ce559e6f
KH
6313 {
6314 coding_free_composition_data (coding);
6315 return (nocopy ? str : Fcopy_sequence (str));
6316 }
d5db4077 6317 shrinked_bytes = from + (SBYTES (str) - to_byte);
4956c225 6318 }
b73bfc1c
KH
6319
6320 len = encoding_buffer_size (coding, to_byte - from);
73be902c
KH
6321 allocate_conversion_buffer (buf, len);
6322
2391eaa4 6323 consumed = consumed_char = produced = produced_char = 0;
73be902c
KH
6324 while (1)
6325 {
d5db4077 6326 result = encode_coding (coding, SDATA (str) + from + consumed,
73be902c
KH
6327 buf.data + produced, to_byte - from - consumed,
6328 buf.size - produced);
6329 consumed += coding->consumed;
2391eaa4 6330 consumed_char += coding->consumed_char;
13004bef 6331 produced += coding->produced;
2391eaa4
KH
6332 produced_char += coding->produced_char;
6333 if (result == CODING_FINISH_NORMAL
230779b9 6334 || result == CODING_FINISH_INTERRUPT
2391eaa4
KH
6335 || (result == CODING_FINISH_INSUFFICIENT_SRC
6336 && coding->consumed == 0))
73be902c
KH
6337 break;
6338 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6339 extend_conversion_buffer (&buf);
6340 }
6341
2391eaa4
KH
6342 coding->consumed = consumed;
6343 coding->consumed_char = consumed_char;
6344 coding->produced = produced;
6345 coding->produced_char = produced_char;
6346
73be902c 6347 newstr = make_uninit_string (produced + shrinked_bytes);
b73bfc1c 6348 if (from > 0)
a4244313
KR
6349 STRING_COPYIN (newstr, 0, SDATA (str), from);
6350 STRING_COPYIN (newstr, from, buf.data, produced);
73be902c 6351 if (shrinked_bytes > from)
a4244313
KR
6352 STRING_COPYIN (newstr, from + produced,
6353 SDATA (str) + to_byte,
6354 shrinked_bytes - from);
73be902c
KH
6355
6356 free_conversion_buffer (&buf);
ec6d2bb8 6357 coding_free_composition_data (coding);
b73bfc1c 6358
73be902c 6359 return newstr;
4ed46869
KH
6360}
6361
6362\f
6363#ifdef emacs
1397dc18 6364/*** 8. Emacs Lisp library functions ***/
4ed46869 6365
4ed46869 6366DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0,
48b0f3ae
PJ
6367 doc: /* Return t if OBJECT is nil or a coding-system.
6368See the documentation of `make-coding-system' for information
6369about coding-system objects. */)
6370 (obj)
4ed46869
KH
6371 Lisp_Object obj;
6372{
4608c386
KH
6373 if (NILP (obj))
6374 return Qt;
6375 if (!SYMBOLP (obj))
6376 return Qnil;
c2164d91
KH
6377 if (! NILP (Fget (obj, Qcoding_system_define_form)))
6378 return Qt;
4608c386
KH
6379 /* Get coding-spec vector for OBJ. */
6380 obj = Fget (obj, Qcoding_system);
6381 return ((VECTORP (obj) && XVECTOR (obj)->size == 5)
6382 ? Qt : Qnil);
4ed46869
KH
6383}
6384
9d991de8
RS
6385DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system,
6386 Sread_non_nil_coding_system, 1, 1, 0,
48b0f3ae
PJ
6387 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6388 (prompt)
4ed46869
KH
6389 Lisp_Object prompt;
6390{
e0e989f6 6391 Lisp_Object val;
9d991de8
RS
6392 do
6393 {
4608c386
KH
6394 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
6395 Qt, Qnil, Qcoding_system_history, Qnil, Qnil);
9d991de8 6396 }
d5db4077 6397 while (SCHARS (val) == 0);
e0e989f6 6398 return (Fintern (val, Qnil));
4ed46869
KH
6399}
6400
9b787f3e 6401DEFUN ("read-coding-system", Fread_coding_system, Sread_coding_system, 1, 2, 0,
48b0f3ae
PJ
6402 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6403If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
6404 (prompt, default_coding_system)
9b787f3e 6405 Lisp_Object prompt, default_coding_system;
4ed46869 6406{
f44d27ce 6407 Lisp_Object val;
9b787f3e 6408 if (SYMBOLP (default_coding_system))
57d25e6f 6409 default_coding_system = SYMBOL_NAME (default_coding_system);
4608c386 6410 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
9b787f3e
RS
6411 Qt, Qnil, Qcoding_system_history,
6412 default_coding_system, Qnil);
d5db4077 6413 return (SCHARS (val) == 0 ? Qnil : Fintern (val, Qnil));
4ed46869
KH
6414}
6415
6416DEFUN ("check-coding-system", Fcheck_coding_system, Scheck_coding_system,
6417 1, 1, 0,
48b0f3ae
PJ
6418 doc: /* Check validity of CODING-SYSTEM.
6419If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
303cdc2d 6420It is valid if it is nil or a symbol with a non-nil `coding-system' property.
de1d1a40 6421The value of this property should be a vector of length 5. */)
48b0f3ae 6422 (coding_system)
4ed46869
KH
6423 Lisp_Object coding_system;
6424{
a362520d
KH
6425 Lisp_Object define_form;
6426
6427 define_form = Fget (coding_system, Qcoding_system_define_form);
6428 if (! NILP (define_form))
6429 {
6430 Fput (coding_system, Qcoding_system_define_form, Qnil);
6431 safe_eval (define_form);
6432 }
4ed46869
KH
6433 if (!NILP (Fcoding_system_p (coding_system)))
6434 return coding_system;
6435 while (1)
02ba4723 6436 Fsignal (Qcoding_system_error, Fcons (coding_system, Qnil));
4ed46869 6437}
3a73fa5d 6438\f
d46c5b12 6439Lisp_Object
0a28aafb 6440detect_coding_system (src, src_bytes, highest, multibytep)
a4244313 6441 const unsigned char *src;
d46c5b12 6442 int src_bytes, highest;
0a28aafb 6443 int multibytep;
4ed46869
KH
6444{
6445 int coding_mask, eol_type;
d46c5b12
KH
6446 Lisp_Object val, tmp;
6447 int dummy;
4ed46869 6448
0a28aafb 6449 coding_mask = detect_coding_mask (src, src_bytes, NULL, &dummy, multibytep);
d46c5b12
KH
6450 eol_type = detect_eol_type (src, src_bytes, &dummy);
6451 if (eol_type == CODING_EOL_INCONSISTENT)
25b02698 6452 eol_type = CODING_EOL_UNDECIDED;
4ed46869 6453
d46c5b12 6454 if (!coding_mask)
4ed46869 6455 {
27901516 6456 val = Qundecided;
d46c5b12 6457 if (eol_type != CODING_EOL_UNDECIDED)
4ed46869 6458 {
f44d27ce
RS
6459 Lisp_Object val2;
6460 val2 = Fget (Qundecided, Qeol_type);
4ed46869
KH
6461 if (VECTORP (val2))
6462 val = XVECTOR (val2)->contents[eol_type];
6463 }
80e803b4 6464 return (highest ? val : Fcons (val, Qnil));
4ed46869 6465 }
4ed46869 6466
d46c5b12
KH
6467 /* At first, gather possible coding systems in VAL. */
6468 val = Qnil;
fa42c37f 6469 for (tmp = Vcoding_category_list; CONSP (tmp); tmp = XCDR (tmp))
4ed46869 6470 {
fa42c37f
KH
6471 Lisp_Object category_val, category_index;
6472
6473 category_index = Fget (XCAR (tmp), Qcoding_category_index);
6474 category_val = Fsymbol_value (XCAR (tmp));
6475 if (!NILP (category_val)
6476 && NATNUMP (category_index)
6477 && (coding_mask & (1 << XFASTINT (category_index))))
4ed46869 6478 {
fa42c37f 6479 val = Fcons (category_val, val);
d46c5b12
KH
6480 if (highest)
6481 break;
4ed46869
KH
6482 }
6483 }
d46c5b12
KH
6484 if (!highest)
6485 val = Fnreverse (val);
4ed46869 6486
65059037 6487 /* Then, replace the elements with subsidiary coding systems. */
fa42c37f 6488 for (tmp = val; CONSP (tmp); tmp = XCDR (tmp))
4ed46869 6489 {
65059037
RS
6490 if (eol_type != CODING_EOL_UNDECIDED
6491 && eol_type != CODING_EOL_INCONSISTENT)
4ed46869 6492 {
d46c5b12 6493 Lisp_Object eol;
03699b14 6494 eol = Fget (XCAR (tmp), Qeol_type);
d46c5b12 6495 if (VECTORP (eol))
f3fbd155 6496 XSETCAR (tmp, XVECTOR (eol)->contents[eol_type]);
4ed46869
KH
6497 }
6498 }
03699b14 6499 return (highest ? XCAR (val) : val);
93dec019 6500}
4ed46869 6501
d46c5b12
KH
6502DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region,
6503 2, 3, 0,
40fd536c
KH
6504 doc: /* Detect how the byte sequence in the region is encoded.
6505Return a list of possible coding systems used on decoding a byte
6506sequence containing the bytes in the region between START and END when
6507the coding system `undecided' is specified. The list is ordered by
6508priority decided in the current language environment.
48b0f3ae
PJ
6509
6510If only ASCII characters are found, it returns a list of single element
6511`undecided' or its subsidiary coding system according to a detected
6512end-of-line format.
6513
6514If optional argument HIGHEST is non-nil, return the coding system of
6515highest priority. */)
6516 (start, end, highest)
d46c5b12
KH
6517 Lisp_Object start, end, highest;
6518{
6519 int from, to;
6520 int from_byte, to_byte;
682169fe 6521 int include_anchor_byte = 0;
6289dd10 6522
b7826503
PJ
6523 CHECK_NUMBER_COERCE_MARKER (start);
6524 CHECK_NUMBER_COERCE_MARKER (end);
4ed46869 6525
d46c5b12
KH
6526 validate_region (&start, &end);
6527 from = XINT (start), to = XINT (end);
6528 from_byte = CHAR_TO_BYTE (from);
6529 to_byte = CHAR_TO_BYTE (to);
6289dd10 6530
d46c5b12
KH
6531 if (from < GPT && to >= GPT)
6532 move_gap_both (to, to_byte);
c210f766
KH
6533 /* If we an anchor byte `\0' follows the region, we include it in
6534 the detecting source. Then code detectors can handle the tailing
6535 byte sequence more accurately.
6536
7d0393cf 6537 Fix me: This is not a perfect solution. It is better that we
c210f766
KH
6538 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6539 */
682169fe
KH
6540 if (to == Z || (to == GPT && GAP_SIZE > 0))
6541 include_anchor_byte = 1;
d46c5b12 6542 return detect_coding_system (BYTE_POS_ADDR (from_byte),
682169fe 6543 to_byte - from_byte + include_anchor_byte,
0a28aafb
KH
6544 !NILP (highest),
6545 !NILP (current_buffer
6546 ->enable_multibyte_characters));
d46c5b12 6547}
6289dd10 6548
d46c5b12
KH
6549DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string,
6550 1, 2, 0,
eec1f3c7
KH
6551 doc: /* Detect how the byte sequence in STRING is encoded.
6552Return a list of possible coding systems used on decoding a byte
6553sequence containing the bytes in STRING when the coding system
6554`undecided' is specified. The list is ordered by priority decided in
6555the current language environment.
48b0f3ae
PJ
6556
6557If only ASCII characters are found, it returns a list of single element
6558`undecided' or its subsidiary coding system according to a detected
6559end-of-line format.
6560
6561If optional argument HIGHEST is non-nil, return the coding system of
6562highest priority. */)
6563 (string, highest)
d46c5b12
KH
6564 Lisp_Object string, highest;
6565{
b7826503 6566 CHECK_STRING (string);
4ed46869 6567
d5db4077 6568 return detect_coding_system (SDATA (string),
682169fe
KH
6569 /* "+ 1" is to include the anchor byte
6570 `\0'. With this, code detectors can
c210f766
KH
6571 handle the tailing bytes more
6572 accurately. */
d5db4077 6573 SBYTES (string) + 1,
0a28aafb
KH
6574 !NILP (highest),
6575 STRING_MULTIBYTE (string));
4ed46869
KH
6576}
6577
05e6f5dc
KH
6578/* Subroutine for Fsafe_coding_systems_region_internal.
6579
6580 Return a list of coding systems that safely encode the multibyte
b666620c 6581 text between P and PEND. SAFE_CODINGS, if non-nil, is an alist of
05e6f5dc
KH
6582 possible coding systems. If it is nil, it means that we have not
6583 yet found any coding systems.
6584
12d5b185
KH
6585 WORK_TABLE a char-table of which element is set to t once the
6586 element is looked up.
05e6f5dc
KH
6587
6588 If a non-ASCII single byte char is found, set
6589 *single_byte_char_found to 1. */
6590
6591static Lisp_Object
6592find_safe_codings (p, pend, safe_codings, work_table, single_byte_char_found)
6593 unsigned char *p, *pend;
6594 Lisp_Object safe_codings, work_table;
6595 int *single_byte_char_found;
6b89e3aa 6596{
f1ce3dcf 6597 int c, len;
6b89e3aa
KH
6598 Lisp_Object val, ch;
6599 Lisp_Object prev, tail;
177c0ea7 6600
12d5b185
KH
6601 if (NILP (safe_codings))
6602 goto done_safe_codings;
6b89e3aa
KH
6603 while (p < pend)
6604 {
6605 c = STRING_CHAR_AND_LENGTH (p, pend - p, len);
6606 p += len;
6607 if (ASCII_BYTE_P (c))
6608 /* We can ignore ASCII characters here. */
6609 continue;
6610 if (SINGLE_BYTE_CHAR_P (c))
6611 *single_byte_char_found = 1;
6b89e3aa
KH
6612 /* Check the safe coding systems for C. */
6613 ch = make_number (c);
6614 val = Faref (work_table, ch);
6615 if (EQ (val, Qt))
6616 /* This element was already checked. Ignore it. */
6617 continue;
6618 /* Remember that we checked this element. */
6619 Faset (work_table, ch, Qt);
6620
6621 for (prev = tail = safe_codings; CONSP (tail); tail = XCDR (tail))
6622 {
b666620c
KH
6623 Lisp_Object elt, translation_table, hash_table, accept_latin_extra;
6624 int encodable;
6625
6626 elt = XCAR (tail);
6627 if (CONSP (XCDR (elt)))
6628 {
6629 /* This entry has this format now:
6630 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6631 ACCEPT-LATIN-EXTRA ) */
6632 val = XCDR (elt);
6633 encodable = ! NILP (Faref (XCAR (val), ch));
6634 if (! encodable)
6635 {
6636 val = XCDR (val);
6637 translation_table = XCAR (val);
6638 hash_table = XCAR (XCDR (val));
6639 accept_latin_extra = XCAR (XCDR (XCDR (val)));
6640 }
6641 }
6642 else
6643 {
6644 /* This entry has this format now: ( CODING . SAFE-CHARS) */
6645 encodable = ! NILP (Faref (XCDR (elt), ch));
6646 if (! encodable)
6647 {
6648 /* Transform the format to:
6649 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6650 ACCEPT-LATIN-EXTRA ) */
6651 val = Fget (XCAR (elt), Qcoding_system);
6652 translation_table
6653 = Fplist_get (AREF (val, 3),
6654 Qtranslation_table_for_encode);
6655 if (SYMBOLP (translation_table))
6656 translation_table = Fget (translation_table,
6657 Qtranslation_table);
6658 hash_table
6659 = (CHAR_TABLE_P (translation_table)
6660 ? XCHAR_TABLE (translation_table)->extras[1]
6661 : Qnil);
6662 accept_latin_extra
6663 = ((EQ (AREF (val, 0), make_number (2))
6664 && VECTORP (AREF (val, 4)))
58f99379 6665 ? AREF (AREF (val, 4), 16)
b666620c
KH
6666 : Qnil);
6667 XSETCAR (tail, list5 (XCAR (elt), XCDR (elt),
6668 translation_table, hash_table,
6669 accept_latin_extra));
6670 }
6671 }
43e4a82f 6672
b666620c
KH
6673 if (! encodable
6674 && ((CHAR_TABLE_P (translation_table)
6675 && ! NILP (Faref (translation_table, ch)))
6676 || (HASH_TABLE_P (hash_table)
6677 && ! NILP (Fgethash (ch, hash_table, Qnil)))
6678 || (SINGLE_BYTE_CHAR_P (c)
6679 && ! NILP (accept_latin_extra)
6680 && VECTORP (Vlatin_extra_code_table)
6681 && ! NILP (AREF (Vlatin_extra_code_table, c)))))
6682 encodable = 1;
6683 if (encodable)
6684 prev = tail;
6685 else
6b89e3aa 6686 {
7c695ab9 6687 /* Exclude this coding system from SAFE_CODINGS. */
6b89e3aa 6688 if (EQ (tail, safe_codings))
12d5b185
KH
6689 {
6690 safe_codings = XCDR (safe_codings);
6691 if (NILP (safe_codings))
6692 goto done_safe_codings;
6693 }
6b89e3aa
KH
6694 else
6695 XSETCDR (prev, XCDR (tail));
6696 }
6b89e3aa
KH
6697 }
6698 }
12d5b185
KH
6699
6700 done_safe_codings:
6701 /* If the above loop was terminated before P reaches PEND, it means
6702 SAFE_CODINGS was set to nil. If we have not yet found an
6703 non-ASCII single-byte char, check it now. */
6704 if (! *single_byte_char_found)
6705 while (p < pend)
6706 {
6707 c = STRING_CHAR_AND_LENGTH (p, pend - p, len);
6708 p += len;
6709 if (! ASCII_BYTE_P (c)
6710 && SINGLE_BYTE_CHAR_P (c))
6711 {
6712 *single_byte_char_found = 1;
6713 break;
6714 }
6715 }
6b89e3aa
KH
6716 return safe_codings;
6717}
6718
067a6a66
KH
6719DEFUN ("find-coding-systems-region-internal",
6720 Ffind_coding_systems_region_internal,
6721 Sfind_coding_systems_region_internal, 2, 2, 0,
6b89e3aa
KH
6722 doc: /* Internal use only. */)
6723 (start, end)
6724 Lisp_Object start, end;
6725{
6726 Lisp_Object work_table, safe_codings;
6727 int non_ascii_p = 0;
6728 int single_byte_char_found = 0;
6729 const unsigned char *p1, *p1end, *p2, *p2end, *p;
6730
6731 if (STRINGP (start))
6732 {
6733 if (!STRING_MULTIBYTE (start))
6734 return Qt;
6735 p1 = SDATA (start), p1end = p1 + SBYTES (start);
6736 p2 = p2end = p1end;
6737 if (SCHARS (start) != SBYTES (start))
6738 non_ascii_p = 1;
6739 }
6740 else
6741 {
6742 int from, to, stop;
6743
6744 CHECK_NUMBER_COERCE_MARKER (start);
6745 CHECK_NUMBER_COERCE_MARKER (end);
6746 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
6747 args_out_of_range (start, end);
6748 if (NILP (current_buffer->enable_multibyte_characters))
6749 return Qt;
6750 from = CHAR_TO_BYTE (XINT (start));
6751 to = CHAR_TO_BYTE (XINT (end));
6752 stop = from < GPT_BYTE && GPT_BYTE < to ? GPT_BYTE : to;
6753 p1 = BYTE_POS_ADDR (from), p1end = p1 + (stop - from);
6754 if (stop == to)
6755 p2 = p2end = p1end;
6756 else
6757 p2 = BYTE_POS_ADDR (stop), p2end = p2 + (to - stop);
6758 if (XINT (end) - XINT (start) != to - from)
6759 non_ascii_p = 1;
6760 }
6761
6762 if (!non_ascii_p)
6763 {
6764 /* We are sure that the text contains no multibyte character.
6765 Check if it contains eight-bit-graphic. */
6766 p = p1;
6767 for (p = p1; p < p1end && ASCII_BYTE_P (*p); p++);
6768 if (p == p1end)
6769 {
6770 for (p = p2; p < p2end && ASCII_BYTE_P (*p); p++);
6771 if (p == p2end)
6772 return Qt;
6773 }
6774 }
6775
6776 /* The text contains non-ASCII characters. */
6777
6778 work_table = Fmake_char_table (Qchar_coding_system, Qnil);
6779 safe_codings = Fcopy_sequence (XCDR (Vcoding_system_safe_chars));
6780
067a6a66
KH
6781 safe_codings = find_safe_codings (p1, p1end, safe_codings, work_table,
6782 &single_byte_char_found);
6b89e3aa 6783 if (p2 < p2end)
067a6a66
KH
6784 safe_codings = find_safe_codings (p2, p2end, safe_codings, work_table,
6785 &single_byte_char_found);
6b89e3aa
KH
6786 if (EQ (safe_codings, XCDR (Vcoding_system_safe_chars)))
6787 safe_codings = Qt;
6788 else
6789 {
6790 /* Turn safe_codings to a list of coding systems... */
6791 Lisp_Object val;
6792
6793 if (single_byte_char_found)
6794 /* ... and append these for eight-bit chars. */
6795 val = Fcons (Qraw_text,
6796 Fcons (Qemacs_mule, Fcons (Qno_conversion, Qnil)));
6797 else
6798 /* ... and append generic coding systems. */
6799 val = Fcopy_sequence (XCAR (Vcoding_system_safe_chars));
177c0ea7 6800
6b89e3aa
KH
6801 for (; CONSP (safe_codings); safe_codings = XCDR (safe_codings))
6802 val = Fcons (XCAR (XCAR (safe_codings)), val);
6803 safe_codings = val;
6804 }
6805
6806 return safe_codings;
6807}
6808
6809
068a9dbd
KH
6810/* Search from position POS for such characters that are unencodable
6811 accoding to SAFE_CHARS, and return a list of their positions. P
6812 points where in the memory the character at POS exists. Limit the
6813 search at PEND or when Nth unencodable characters are found.
6814
6815 If SAFE_CHARS is a char table, an element for an unencodable
6816 character is nil.
6817
6818 If SAFE_CHARS is nil, all non-ASCII characters are unencodable.
6819
6820 Otherwise, SAFE_CHARS is t, and only eight-bit-contrl and
6821 eight-bit-graphic characters are unencodable. */
6822
6823static Lisp_Object
6824unencodable_char_position (safe_chars, pos, p, pend, n)
6825 Lisp_Object safe_chars;
6826 int pos;
6827 unsigned char *p, *pend;
6828 int n;
6829{
6830 Lisp_Object pos_list;
6831
6832 pos_list = Qnil;
6833 while (p < pend)
6834 {
6835 int len;
6836 int c = STRING_CHAR_AND_LENGTH (p, MAX_MULTIBYTE_LENGTH, len);
7d0393cf 6837
068a9dbd
KH
6838 if (c >= 128
6839 && (CHAR_TABLE_P (safe_chars)
6840 ? NILP (CHAR_TABLE_REF (safe_chars, c))
6841 : (NILP (safe_chars) || c < 256)))
6842 {
6843 pos_list = Fcons (make_number (pos), pos_list);
6844 if (--n <= 0)
6845 break;
6846 }
6847 pos++;
6848 p += len;
6849 }
6850 return Fnreverse (pos_list);
6851}
6852
6853
6854DEFUN ("unencodable-char-position", Funencodable_char_position,
6855 Sunencodable_char_position, 3, 5, 0,
6856 doc: /*
6857Return position of first un-encodable character in a region.
6858START and END specfiy the region and CODING-SYSTEM specifies the
6859encoding to check. Return nil if CODING-SYSTEM does encode the region.
6860
6861If optional 4th argument COUNT is non-nil, it specifies at most how
6862many un-encodable characters to search. In this case, the value is a
6863list of positions.
6864
6865If optional 5th argument STRING is non-nil, it is a string to search
6866for un-encodable characters. In that case, START and END are indexes
6867to the string. */)
6868 (start, end, coding_system, count, string)
6869 Lisp_Object start, end, coding_system, count, string;
6870{
6871 int n;
6872 Lisp_Object safe_chars;
6873 struct coding_system coding;
6874 Lisp_Object positions;
6875 int from, to;
6876 unsigned char *p, *pend;
6877
6878 if (NILP (string))
6879 {
6880 validate_region (&start, &end);
6881 from = XINT (start);
6882 to = XINT (end);
6883 if (NILP (current_buffer->enable_multibyte_characters))
6884 return Qnil;
6885 p = CHAR_POS_ADDR (from);
200c93e2
KH
6886 if (to == GPT)
6887 pend = GPT_ADDR;
6888 else
6889 pend = CHAR_POS_ADDR (to);
068a9dbd
KH
6890 }
6891 else
6892 {
6893 CHECK_STRING (string);
6894 CHECK_NATNUM (start);
6895 CHECK_NATNUM (end);
6896 from = XINT (start);
6897 to = XINT (end);
6898 if (from > to
6899 || to > SCHARS (string))
6900 args_out_of_range_3 (string, start, end);
6901 if (! STRING_MULTIBYTE (string))
6902 return Qnil;
6903 p = SDATA (string) + string_char_to_byte (string, from);
6904 pend = SDATA (string) + string_char_to_byte (string, to);
6905 }
6906
6907 setup_coding_system (Fcheck_coding_system (coding_system), &coding);
6908
6909 if (NILP (count))
6910 n = 1;
6911 else
6912 {
6913 CHECK_NATNUM (count);
6914 n = XINT (count);
6915 }
6916
6917 if (coding.type == coding_type_no_conversion
6918 || coding.type == coding_type_raw_text)
6919 return Qnil;
6920
6921 if (coding.type == coding_type_undecided)
6922 safe_chars = Qnil;
6923 else
6b89e3aa 6924 safe_chars = coding_safe_chars (coding_system);
068a9dbd
KH
6925
6926 if (STRINGP (string)
6927 || from >= GPT || to <= GPT)
6928 positions = unencodable_char_position (safe_chars, from, p, pend, n);
6929 else
6930 {
6931 Lisp_Object args[2];
6932
6933 args[0] = unencodable_char_position (safe_chars, from, p, GPT_ADDR, n);
96d2e64d 6934 n -= XINT (Flength (args[0]));
068a9dbd
KH
6935 if (n <= 0)
6936 positions = args[0];
6937 else
6938 {
6939 args[1] = unencodable_char_position (safe_chars, GPT, GAP_END_ADDR,
6940 pend, n);
6941 positions = Fappend (2, args);
6942 }
6943 }
6944
6945 return (NILP (count) ? Fcar (positions) : positions);
6946}
6947
6948
4031e2bf
KH
6949Lisp_Object
6950code_convert_region1 (start, end, coding_system, encodep)
d46c5b12 6951 Lisp_Object start, end, coding_system;
4031e2bf 6952 int encodep;
3a73fa5d
RS
6953{
6954 struct coding_system coding;
da55a2b7 6955 int from, to;
3a73fa5d 6956
b7826503
PJ
6957 CHECK_NUMBER_COERCE_MARKER (start);
6958 CHECK_NUMBER_COERCE_MARKER (end);
6959 CHECK_SYMBOL (coding_system);
3a73fa5d 6960
d46c5b12
KH
6961 validate_region (&start, &end);
6962 from = XFASTINT (start);
6963 to = XFASTINT (end);
6964
3a73fa5d 6965 if (NILP (coding_system))
d46c5b12
KH
6966 return make_number (to - from);
6967
3a73fa5d 6968 if (setup_coding_system (Fcheck_coding_system (coding_system), &coding) < 0)
d5db4077 6969 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system)));
3a73fa5d 6970
d46c5b12 6971 coding.mode |= CODING_MODE_LAST_BLOCK;
b73bfc1c
KH
6972 coding.src_multibyte = coding.dst_multibyte
6973 = !NILP (current_buffer->enable_multibyte_characters);
fb88bf2d
KH
6974 code_convert_region (from, CHAR_TO_BYTE (from), to, CHAR_TO_BYTE (to),
6975 &coding, encodep, 1);
f072a3e8 6976 Vlast_coding_system_used = coding.symbol;
fb88bf2d 6977 return make_number (coding.produced_char);
4031e2bf
KH
6978}
6979
6980DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region,
6981 3, 3, "r\nzCoding system: ",
48b0f3ae
PJ
6982 doc: /* Decode the current region from the specified coding system.
6983When called from a program, takes three arguments:
6984START, END, and CODING-SYSTEM. START and END are buffer positions.
6985This function sets `last-coding-system-used' to the precise coding system
6986used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6987not fully specified.)
6988It returns the length of the decoded text. */)
6989 (start, end, coding_system)
4031e2bf
KH
6990 Lisp_Object start, end, coding_system;
6991{
6992 return code_convert_region1 (start, end, coding_system, 0);
3a73fa5d
RS
6993}
6994
6995DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region,
6996 3, 3, "r\nzCoding system: ",
48b0f3ae
PJ
6997 doc: /* Encode the current region into the specified coding system.
6998When called from a program, takes three arguments:
6999START, END, and CODING-SYSTEM. START and END are buffer positions.
7000This function sets `last-coding-system-used' to the precise coding system
7001used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7002not fully specified.)
7003It returns the length of the encoded text. */)
7004 (start, end, coding_system)
d46c5b12 7005 Lisp_Object start, end, coding_system;
3a73fa5d 7006{
4031e2bf
KH
7007 return code_convert_region1 (start, end, coding_system, 1);
7008}
3a73fa5d 7009
4031e2bf
KH
7010Lisp_Object
7011code_convert_string1 (string, coding_system, nocopy, encodep)
7012 Lisp_Object string, coding_system, nocopy;
7013 int encodep;
7014{
7015 struct coding_system coding;
3a73fa5d 7016
b7826503
PJ
7017 CHECK_STRING (string);
7018 CHECK_SYMBOL (coding_system);
4ed46869 7019
d46c5b12 7020 if (NILP (coding_system))
4031e2bf 7021 return (NILP (nocopy) ? Fcopy_sequence (string) : string);
4ed46869 7022
d46c5b12 7023 if (setup_coding_system (Fcheck_coding_system (coding_system), &coding) < 0)
d5db4077 7024 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system)));
5f1cd180 7025
d46c5b12 7026 coding.mode |= CODING_MODE_LAST_BLOCK;
b73bfc1c
KH
7027 string = (encodep
7028 ? encode_coding_string (string, &coding, !NILP (nocopy))
7029 : decode_coding_string (string, &coding, !NILP (nocopy)));
f072a3e8 7030 Vlast_coding_system_used = coding.symbol;
ec6d2bb8
KH
7031
7032 return string;
4ed46869
KH
7033}
7034
4ed46869 7035DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string,
e0e989f6 7036 2, 3, 0,
48b0f3ae
PJ
7037 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7038Optional arg NOCOPY non-nil means it is OK to return STRING itself
7039if the decoding operation is trivial.
7040This function sets `last-coding-system-used' to the precise coding system
7041used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7042not fully specified.) */)
7043 (string, coding_system, nocopy)
e0e989f6 7044 Lisp_Object string, coding_system, nocopy;
4ed46869 7045{
f072a3e8 7046 return code_convert_string1 (string, coding_system, nocopy, 0);
4ed46869
KH
7047}
7048
7049DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
e0e989f6 7050 2, 3, 0,
48b0f3ae
PJ
7051 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
7052Optional arg NOCOPY non-nil means it is OK to return STRING itself
7053if the encoding operation is trivial.
7054This function sets `last-coding-system-used' to the precise coding system
7055used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7056not fully specified.) */)
7057 (string, coding_system, nocopy)
e0e989f6 7058 Lisp_Object string, coding_system, nocopy;
4ed46869 7059{
f072a3e8 7060 return code_convert_string1 (string, coding_system, nocopy, 1);
4ed46869 7061}
4031e2bf 7062
ecec61c1 7063/* Encode or decode STRING according to CODING_SYSTEM.
ec6d2bb8
KH
7064 Do not set Vlast_coding_system_used.
7065
7066 This function is called only from macros DECODE_FILE and
7067 ENCODE_FILE, thus we ignore character composition. */
ecec61c1
KH
7068
7069Lisp_Object
7070code_convert_string_norecord (string, coding_system, encodep)
7071 Lisp_Object string, coding_system;
7072 int encodep;
7073{
7074 struct coding_system coding;
7075
b7826503
PJ
7076 CHECK_STRING (string);
7077 CHECK_SYMBOL (coding_system);
ecec61c1
KH
7078
7079 if (NILP (coding_system))
7080 return string;
7081
7082 if (setup_coding_system (Fcheck_coding_system (coding_system), &coding) < 0)
d5db4077 7083 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system)));
ecec61c1 7084
ec6d2bb8 7085 coding.composing = COMPOSITION_DISABLED;
ecec61c1 7086 coding.mode |= CODING_MODE_LAST_BLOCK;
b73bfc1c
KH
7087 return (encodep
7088 ? encode_coding_string (string, &coding, 1)
7089 : decode_coding_string (string, &coding, 1));
ecec61c1 7090}
3a73fa5d 7091\f
4ed46869 7092DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0,
48b0f3ae
PJ
7093 doc: /* Decode a Japanese character which has CODE in shift_jis encoding.
7094Return the corresponding character. */)
7095 (code)
4ed46869
KH
7096 Lisp_Object code;
7097{
7098 unsigned char c1, c2, s1, s2;
7099 Lisp_Object val;
7100
b7826503 7101 CHECK_NUMBER (code);
4ed46869 7102 s1 = (XFASTINT (code)) >> 8, s2 = (XFASTINT (code)) & 0xFF;
55ab7be3
KH
7103 if (s1 == 0)
7104 {
c28a9453
KH
7105 if (s2 < 0x80)
7106 XSETFASTINT (val, s2);
7107 else if (s2 >= 0xA0 || s2 <= 0xDF)
b73bfc1c 7108 XSETFASTINT (val, MAKE_CHAR (charset_katakana_jisx0201, s2, 0));
c28a9453 7109 else
9da8350f 7110 error ("Invalid Shift JIS code: %x", XFASTINT (code));
55ab7be3
KH
7111 }
7112 else
7113 {
87323294 7114 if ((s1 < 0x80 || (s1 > 0x9F && s1 < 0xE0) || s1 > 0xEF)
55ab7be3 7115 || (s2 < 0x40 || s2 == 0x7F || s2 > 0xFC))
9da8350f 7116 error ("Invalid Shift JIS code: %x", XFASTINT (code));
55ab7be3 7117 DECODE_SJIS (s1, s2, c1, c2);
b73bfc1c 7118 XSETFASTINT (val, MAKE_CHAR (charset_jisx0208, c1, c2));
55ab7be3 7119 }
4ed46869
KH
7120 return val;
7121}
7122
7123DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0,
48b0f3ae
PJ
7124 doc: /* Encode a Japanese character CHAR to shift_jis encoding.
7125Return the corresponding code in SJIS. */)
7126 (ch)
4ed46869
KH
7127 Lisp_Object ch;
7128{
bcf26d6a 7129 int charset, c1, c2, s1, s2;
4ed46869
KH
7130 Lisp_Object val;
7131
b7826503 7132 CHECK_NUMBER (ch);
4ed46869 7133 SPLIT_CHAR (XFASTINT (ch), charset, c1, c2);
c28a9453
KH
7134 if (charset == CHARSET_ASCII)
7135 {
7136 val = ch;
7137 }
7138 else if (charset == charset_jisx0208
7139 && c1 > 0x20 && c1 < 0x7F && c2 > 0x20 && c2 < 0x7F)
4ed46869
KH
7140 {
7141 ENCODE_SJIS (c1, c2, s1, s2);
bcf26d6a 7142 XSETFASTINT (val, (s1 << 8) | s2);
4ed46869 7143 }
55ab7be3
KH
7144 else if (charset == charset_katakana_jisx0201
7145 && c1 > 0x20 && c2 < 0xE0)
7146 {
7147 XSETFASTINT (val, c1 | 0x80);
7148 }
4ed46869 7149 else
55ab7be3 7150 error ("Can't encode to shift_jis: %d", XFASTINT (ch));
4ed46869
KH
7151 return val;
7152}
7153
7154DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0,
48b0f3ae
PJ
7155 doc: /* Decode a Big5 character which has CODE in BIG5 coding system.
7156Return the corresponding character. */)
7157 (code)
4ed46869
KH
7158 Lisp_Object code;
7159{
7160 int charset;
7161 unsigned char b1, b2, c1, c2;
7162 Lisp_Object val;
7163
b7826503 7164 CHECK_NUMBER (code);
4ed46869 7165 b1 = (XFASTINT (code)) >> 8, b2 = (XFASTINT (code)) & 0xFF;
c28a9453
KH
7166 if (b1 == 0)
7167 {
7168 if (b2 >= 0x80)
9da8350f 7169 error ("Invalid BIG5 code: %x", XFASTINT (code));
c28a9453
KH
7170 val = code;
7171 }
7172 else
7173 {
7174 if ((b1 < 0xA1 || b1 > 0xFE)
7175 || (b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE))
9da8350f 7176 error ("Invalid BIG5 code: %x", XFASTINT (code));
c28a9453 7177 DECODE_BIG5 (b1, b2, charset, c1, c2);
b73bfc1c 7178 XSETFASTINT (val, MAKE_CHAR (charset, c1, c2));
c28a9453 7179 }
4ed46869
KH
7180 return val;
7181}
7182
7183DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0,
48b0f3ae
PJ
7184 doc: /* Encode the Big5 character CHAR to BIG5 coding system.
7185Return the corresponding character code in Big5. */)
7186 (ch)
4ed46869
KH
7187 Lisp_Object ch;
7188{
bcf26d6a 7189 int charset, c1, c2, b1, b2;
4ed46869
KH
7190 Lisp_Object val;
7191
b7826503 7192 CHECK_NUMBER (ch);
4ed46869 7193 SPLIT_CHAR (XFASTINT (ch), charset, c1, c2);
c28a9453
KH
7194 if (charset == CHARSET_ASCII)
7195 {
7196 val = ch;
7197 }
7198 else if ((charset == charset_big5_1
7199 && (XFASTINT (ch) >= 0x250a1 && XFASTINT (ch) <= 0x271ec))
7200 || (charset == charset_big5_2
7201 && XFASTINT (ch) >= 0x290a1 && XFASTINT (ch) <= 0x2bdb2))
4ed46869
KH
7202 {
7203 ENCODE_BIG5 (charset, c1, c2, b1, b2);
bcf26d6a 7204 XSETFASTINT (val, (b1 << 8) | b2);
4ed46869
KH
7205 }
7206 else
c28a9453 7207 error ("Can't encode to Big5: %d", XFASTINT (ch));
4ed46869
KH
7208 return val;
7209}
3a73fa5d 7210\f
002fdb44 7211DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal,
48b0f3ae
PJ
7212 Sset_terminal_coding_system_internal, 1, 1, 0,
7213 doc: /* Internal use only. */)
7214 (coding_system)
4ed46869
KH
7215 Lisp_Object coding_system;
7216{
b7826503 7217 CHECK_SYMBOL (coding_system);
4ed46869 7218 setup_coding_system (Fcheck_coding_system (coding_system), &terminal_coding);
70c22245 7219 /* We had better not send unsafe characters to terminal. */
0eecad43 7220 terminal_coding.mode |= CODING_MODE_INHIBIT_UNENCODABLE_CHAR;
8ca3766a 7221 /* Character composition should be disabled. */
ec6d2bb8 7222 terminal_coding.composing = COMPOSITION_DISABLED;
bd64290d
KH
7223 /* Error notification should be suppressed. */
7224 terminal_coding.suppress_error = 1;
b73bfc1c
KH
7225 terminal_coding.src_multibyte = 1;
7226 terminal_coding.dst_multibyte = 0;
4ed46869
KH
7227 return Qnil;
7228}
7229
002fdb44 7230DEFUN ("set-safe-terminal-coding-system-internal", Fset_safe_terminal_coding_system_internal,
48b0f3ae 7231 Sset_safe_terminal_coding_system_internal, 1, 1, 0,
ddb67bdc 7232 doc: /* Internal use only. */)
48b0f3ae 7233 (coding_system)
c4825358
KH
7234 Lisp_Object coding_system;
7235{
b7826503 7236 CHECK_SYMBOL (coding_system);
c4825358
KH
7237 setup_coding_system (Fcheck_coding_system (coding_system),
7238 &safe_terminal_coding);
8ca3766a 7239 /* Character composition should be disabled. */
ec6d2bb8 7240 safe_terminal_coding.composing = COMPOSITION_DISABLED;
bd64290d 7241 /* Error notification should be suppressed. */
6239a668 7242 safe_terminal_coding.suppress_error = 1;
b73bfc1c
KH
7243 safe_terminal_coding.src_multibyte = 1;
7244 safe_terminal_coding.dst_multibyte = 0;
c4825358
KH
7245 return Qnil;
7246}
7247
002fdb44
DL
7248DEFUN ("terminal-coding-system", Fterminal_coding_system,
7249 Sterminal_coding_system, 0, 0, 0,
48b0f3ae
PJ
7250 doc: /* Return coding system specified for terminal output. */)
7251 ()
4ed46869
KH
7252{
7253 return terminal_coding.symbol;
7254}
7255
002fdb44 7256DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal,
48b0f3ae
PJ
7257 Sset_keyboard_coding_system_internal, 1, 1, 0,
7258 doc: /* Internal use only. */)
7259 (coding_system)
4ed46869
KH
7260 Lisp_Object coding_system;
7261{
b7826503 7262 CHECK_SYMBOL (coding_system);
4ed46869 7263 setup_coding_system (Fcheck_coding_system (coding_system), &keyboard_coding);
8ca3766a 7264 /* Character composition should be disabled. */
ec6d2bb8 7265 keyboard_coding.composing = COMPOSITION_DISABLED;
4ed46869
KH
7266 return Qnil;
7267}
7268
002fdb44
DL
7269DEFUN ("keyboard-coding-system", Fkeyboard_coding_system,
7270 Skeyboard_coding_system, 0, 0, 0,
48b0f3ae
PJ
7271 doc: /* Return coding system specified for decoding keyboard input. */)
7272 ()
4ed46869
KH
7273{
7274 return keyboard_coding.symbol;
7275}
7276
7277\f
a5d301df
KH
7278DEFUN ("find-operation-coding-system", Ffind_operation_coding_system,
7279 Sfind_operation_coding_system, 1, MANY, 0,
48b0f3ae
PJ
7280 doc: /* Choose a coding system for an operation based on the target name.
7281The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
7282DECODING-SYSTEM is the coding system to use for decoding
7283\(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
7284for encoding (in case OPERATION does encoding).
7285
7286The first argument OPERATION specifies an I/O primitive:
7287 For file I/O, `insert-file-contents' or `write-region'.
7288 For process I/O, `call-process', `call-process-region', or `start-process'.
7289 For network I/O, `open-network-stream'.
7290
7291The remaining arguments should be the same arguments that were passed
7292to the primitive. Depending on which primitive, one of those arguments
7293is selected as the TARGET. For example, if OPERATION does file I/O,
7294whichever argument specifies the file name is TARGET.
7295
7296TARGET has a meaning which depends on OPERATION:
7297 For file I/O, TARGET is a file name.
7298 For process I/O, TARGET is a process name.
7299 For network I/O, TARGET is a service name or a port number
7300
7301This function looks up what specified for TARGET in,
7302`file-coding-system-alist', `process-coding-system-alist',
7303or `network-coding-system-alist' depending on OPERATION.
7304They may specify a coding system, a cons of coding systems,
7305or a function symbol to call.
7306In the last case, we call the function with one argument,
7307which is a list of all the arguments given to this function.
7308
7309usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
7310 (nargs, args)
4ed46869
KH
7311 int nargs;
7312 Lisp_Object *args;
7313{
7314 Lisp_Object operation, target_idx, target, val;
7315 register Lisp_Object chain;
7316
7317 if (nargs < 2)
7318 error ("Too few arguments");
7319 operation = args[0];
7320 if (!SYMBOLP (operation)
7321 || !INTEGERP (target_idx = Fget (operation, Qtarget_idx)))
8ca3766a 7322 error ("Invalid first argument");
4ed46869
KH
7323 if (nargs < 1 + XINT (target_idx))
7324 error ("Too few arguments for operation: %s",
d5db4077 7325 SDATA (SYMBOL_NAME (operation)));
7f787cfd
KH
7326 /* For write-region, if the 6th argument (i.e. VISIT, the 5th
7327 argument to write-region) is string, it must be treated as a
7328 target file name. */
7329 if (EQ (operation, Qwrite_region)
7330 && nargs > 5
7331 && STRINGP (args[5]))
d90ed3b4 7332 target_idx = make_number (4);
4ed46869
KH
7333 target = args[XINT (target_idx) + 1];
7334 if (!(STRINGP (target)
7335 || (EQ (operation, Qopen_network_stream) && INTEGERP (target))))
8ca3766a 7336 error ("Invalid argument %d", XINT (target_idx) + 1);
4ed46869 7337
2e34157c
RS
7338 chain = ((EQ (operation, Qinsert_file_contents)
7339 || EQ (operation, Qwrite_region))
02ba4723 7340 ? Vfile_coding_system_alist
2e34157c 7341 : (EQ (operation, Qopen_network_stream)
02ba4723
KH
7342 ? Vnetwork_coding_system_alist
7343 : Vprocess_coding_system_alist));
4ed46869
KH
7344 if (NILP (chain))
7345 return Qnil;
7346
03699b14 7347 for (; CONSP (chain); chain = XCDR (chain))
4ed46869 7348 {
f44d27ce 7349 Lisp_Object elt;
03699b14 7350 elt = XCAR (chain);
4ed46869
KH
7351
7352 if (CONSP (elt)
7353 && ((STRINGP (target)
03699b14
KR
7354 && STRINGP (XCAR (elt))
7355 && fast_string_match (XCAR (elt), target) >= 0)
7356 || (INTEGERP (target) && EQ (target, XCAR (elt)))))
02ba4723 7357 {
03699b14 7358 val = XCDR (elt);
b19fd4c5
KH
7359 /* Here, if VAL is both a valid coding system and a valid
7360 function symbol, we return VAL as a coding system. */
02ba4723
KH
7361 if (CONSP (val))
7362 return val;
7363 if (! SYMBOLP (val))
7364 return Qnil;
7365 if (! NILP (Fcoding_system_p (val)))
7366 return Fcons (val, val);
b19fd4c5
KH
7367 if (! NILP (Ffboundp (val)))
7368 {
7369 val = call1 (val, Flist (nargs, args));
7370 if (CONSP (val))
7371 return val;
7372 if (SYMBOLP (val) && ! NILP (Fcoding_system_p (val)))
7373 return Fcons (val, val);
7374 }
02ba4723
KH
7375 return Qnil;
7376 }
4ed46869
KH
7377 }
7378 return Qnil;
7379}
7380
1397dc18
KH
7381DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal,
7382 Supdate_coding_systems_internal, 0, 0, 0,
48b0f3ae
PJ
7383 doc: /* Update internal database for ISO2022 and CCL based coding systems.
7384When values of any coding categories are changed, you must
7385call this function. */)
7386 ()
d46c5b12
KH
7387{
7388 int i;
7389
fa42c37f 7390 for (i = CODING_CATEGORY_IDX_EMACS_MULE; i < CODING_CATEGORY_IDX_MAX; i++)
d46c5b12 7391 {
1397dc18
KH
7392 Lisp_Object val;
7393
f5c1dd0d 7394 val = SYMBOL_VALUE (XVECTOR (Vcoding_category_table)->contents[i]);
1397dc18
KH
7395 if (!NILP (val))
7396 {
7397 if (! coding_system_table[i])
7398 coding_system_table[i] = ((struct coding_system *)
7399 xmalloc (sizeof (struct coding_system)));
7400 setup_coding_system (val, coding_system_table[i]);
7401 }
7402 else if (coding_system_table[i])
7403 {
7404 xfree (coding_system_table[i]);
7405 coding_system_table[i] = NULL;
7406 }
d46c5b12 7407 }
1397dc18 7408
d46c5b12
KH
7409 return Qnil;
7410}
7411
66cfb530
KH
7412DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal,
7413 Sset_coding_priority_internal, 0, 0, 0,
48b0f3ae
PJ
7414 doc: /* Update internal database for the current value of `coding-category-list'.
7415This function is internal use only. */)
7416 ()
66cfb530
KH
7417{
7418 int i = 0, idx;
84d60297
RS
7419 Lisp_Object val;
7420
7421 val = Vcoding_category_list;
66cfb530
KH
7422
7423 while (CONSP (val) && i < CODING_CATEGORY_IDX_MAX)
7424 {
03699b14 7425 if (! SYMBOLP (XCAR (val)))
66cfb530 7426 break;
03699b14 7427 idx = XFASTINT (Fget (XCAR (val), Qcoding_category_index));
66cfb530
KH
7428 if (idx >= CODING_CATEGORY_IDX_MAX)
7429 break;
7430 coding_priorities[i++] = (1 << idx);
03699b14 7431 val = XCDR (val);
66cfb530
KH
7432 }
7433 /* If coding-category-list is valid and contains all coding
7434 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
fa42c37f 7435 the following code saves Emacs from crashing. */
66cfb530
KH
7436 while (i < CODING_CATEGORY_IDX_MAX)
7437 coding_priorities[i++] = CODING_CATEGORY_MASK_RAW_TEXT;
7438
7439 return Qnil;
7440}
7441
6b89e3aa
KH
7442DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal,
7443 Sdefine_coding_system_internal, 1, 1, 0,
7444 doc: /* Register CODING-SYSTEM as a base coding system.
7445This function is internal use only. */)
7446 (coding_system)
7447 Lisp_Object coding_system;
7448{
7449 Lisp_Object safe_chars, slot;
7450
7451 if (NILP (Fcheck_coding_system (coding_system)))
7452 Fsignal (Qcoding_system_error, Fcons (coding_system, Qnil));
7453 safe_chars = coding_safe_chars (coding_system);
7454 if (! EQ (safe_chars, Qt) && ! CHAR_TABLE_P (safe_chars))
7455 error ("No valid safe-chars property for %s",
7456 SDATA (SYMBOL_NAME (coding_system)));
7457 if (EQ (safe_chars, Qt))
7458 {
7459 if (NILP (Fmemq (coding_system, XCAR (Vcoding_system_safe_chars))))
7460 XSETCAR (Vcoding_system_safe_chars,
7461 Fcons (coding_system, XCAR (Vcoding_system_safe_chars)));
7462 }
7463 else
7464 {
7465 slot = Fassq (coding_system, XCDR (Vcoding_system_safe_chars));
7466 if (NILP (slot))
7467 XSETCDR (Vcoding_system_safe_chars,
7468 nconc2 (XCDR (Vcoding_system_safe_chars),
7469 Fcons (Fcons (coding_system, safe_chars), Qnil)));
7470 else
7471 XSETCDR (slot, safe_chars);
7472 }
7473 return Qnil;
7474}
7475
4ed46869
KH
7476#endif /* emacs */
7477
7478\f
1397dc18 7479/*** 9. Post-amble ***/
4ed46869 7480
dfcf069d 7481void
4ed46869
KH
7482init_coding_once ()
7483{
7484 int i;
7485
93dec019 7486 /* Emacs' internal format specific initialize routine. */
4ed46869
KH
7487 for (i = 0; i <= 0x20; i++)
7488 emacs_code_class[i] = EMACS_control_code;
7489 emacs_code_class[0x0A] = EMACS_linefeed_code;
7490 emacs_code_class[0x0D] = EMACS_carriage_return_code;
7491 for (i = 0x21 ; i < 0x7F; i++)
7492 emacs_code_class[i] = EMACS_ascii_code;
7493 emacs_code_class[0x7F] = EMACS_control_code;
ec6d2bb8 7494 for (i = 0x80; i < 0xFF; i++)
4ed46869
KH
7495 emacs_code_class[i] = EMACS_invalid_code;
7496 emacs_code_class[LEADING_CODE_PRIVATE_11] = EMACS_leading_code_3;
7497 emacs_code_class[LEADING_CODE_PRIVATE_12] = EMACS_leading_code_3;
7498 emacs_code_class[LEADING_CODE_PRIVATE_21] = EMACS_leading_code_4;
7499 emacs_code_class[LEADING_CODE_PRIVATE_22] = EMACS_leading_code_4;
7500
7501 /* ISO2022 specific initialize routine. */
7502 for (i = 0; i < 0x20; i++)
b73bfc1c 7503 iso_code_class[i] = ISO_control_0;
4ed46869
KH
7504 for (i = 0x21; i < 0x7F; i++)
7505 iso_code_class[i] = ISO_graphic_plane_0;
7506 for (i = 0x80; i < 0xA0; i++)
b73bfc1c 7507 iso_code_class[i] = ISO_control_1;
4ed46869
KH
7508 for (i = 0xA1; i < 0xFF; i++)
7509 iso_code_class[i] = ISO_graphic_plane_1;
7510 iso_code_class[0x20] = iso_code_class[0x7F] = ISO_0x20_or_0x7F;
7511 iso_code_class[0xA0] = iso_code_class[0xFF] = ISO_0xA0_or_0xFF;
7512 iso_code_class[ISO_CODE_CR] = ISO_carriage_return;
7513 iso_code_class[ISO_CODE_SO] = ISO_shift_out;
7514 iso_code_class[ISO_CODE_SI] = ISO_shift_in;
7515 iso_code_class[ISO_CODE_SS2_7] = ISO_single_shift_2_7;
7516 iso_code_class[ISO_CODE_ESC] = ISO_escape;
7517 iso_code_class[ISO_CODE_SS2] = ISO_single_shift_2;
7518 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3;
7519 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer;
7520
e0e989f6
KH
7521 setup_coding_system (Qnil, &keyboard_coding);
7522 setup_coding_system (Qnil, &terminal_coding);
c4825358 7523 setup_coding_system (Qnil, &safe_terminal_coding);
6bc51348 7524 setup_coding_system (Qnil, &default_buffer_file_coding);
9ce27fde 7525
d46c5b12
KH
7526 bzero (coding_system_table, sizeof coding_system_table);
7527
66cfb530
KH
7528 bzero (ascii_skip_code, sizeof ascii_skip_code);
7529 for (i = 0; i < 128; i++)
7530 ascii_skip_code[i] = 1;
7531
9ce27fde
KH
7532#if defined (MSDOS) || defined (WINDOWSNT)
7533 system_eol_type = CODING_EOL_CRLF;
7534#else
7535 system_eol_type = CODING_EOL_LF;
7536#endif
b843d1ae
KH
7537
7538 inhibit_pre_post_conversion = 0;
e0e989f6
KH
7539}
7540
7541#ifdef emacs
7542
dfcf069d 7543void
e0e989f6
KH
7544syms_of_coding ()
7545{
7546 Qtarget_idx = intern ("target-idx");
7547 staticpro (&Qtarget_idx);
7548
bb0115a2
RS
7549 Qcoding_system_history = intern ("coding-system-history");
7550 staticpro (&Qcoding_system_history);
7551 Fset (Qcoding_system_history, Qnil);
7552
9ce27fde 7553 /* Target FILENAME is the first argument. */
e0e989f6 7554 Fput (Qinsert_file_contents, Qtarget_idx, make_number (0));
9ce27fde 7555 /* Target FILENAME is the third argument. */
e0e989f6
KH
7556 Fput (Qwrite_region, Qtarget_idx, make_number (2));
7557
7558 Qcall_process = intern ("call-process");
7559 staticpro (&Qcall_process);
9ce27fde 7560 /* Target PROGRAM is the first argument. */
e0e989f6
KH
7561 Fput (Qcall_process, Qtarget_idx, make_number (0));
7562
7563 Qcall_process_region = intern ("call-process-region");
7564 staticpro (&Qcall_process_region);
9ce27fde 7565 /* Target PROGRAM is the third argument. */
e0e989f6
KH
7566 Fput (Qcall_process_region, Qtarget_idx, make_number (2));
7567
7568 Qstart_process = intern ("start-process");
7569 staticpro (&Qstart_process);
9ce27fde 7570 /* Target PROGRAM is the third argument. */
e0e989f6
KH
7571 Fput (Qstart_process, Qtarget_idx, make_number (2));
7572
7573 Qopen_network_stream = intern ("open-network-stream");
7574 staticpro (&Qopen_network_stream);
9ce27fde 7575 /* Target SERVICE is the fourth argument. */
e0e989f6
KH
7576 Fput (Qopen_network_stream, Qtarget_idx, make_number (3));
7577
4ed46869
KH
7578 Qcoding_system = intern ("coding-system");
7579 staticpro (&Qcoding_system);
7580
7581 Qeol_type = intern ("eol-type");
7582 staticpro (&Qeol_type);
7583
7584 Qbuffer_file_coding_system = intern ("buffer-file-coding-system");
7585 staticpro (&Qbuffer_file_coding_system);
7586
7587 Qpost_read_conversion = intern ("post-read-conversion");
7588 staticpro (&Qpost_read_conversion);
7589
7590 Qpre_write_conversion = intern ("pre-write-conversion");
7591 staticpro (&Qpre_write_conversion);
7592
27901516
KH
7593 Qno_conversion = intern ("no-conversion");
7594 staticpro (&Qno_conversion);
7595
7596 Qundecided = intern ("undecided");
7597 staticpro (&Qundecided);
7598
4ed46869
KH
7599 Qcoding_system_p = intern ("coding-system-p");
7600 staticpro (&Qcoding_system_p);
7601
7602 Qcoding_system_error = intern ("coding-system-error");
7603 staticpro (&Qcoding_system_error);
7604
7605 Fput (Qcoding_system_error, Qerror_conditions,
7606 Fcons (Qcoding_system_error, Fcons (Qerror, Qnil)));
7607 Fput (Qcoding_system_error, Qerror_message,
9ce27fde 7608 build_string ("Invalid coding system"));
4ed46869 7609
d46c5b12
KH
7610 Qcoding_category = intern ("coding-category");
7611 staticpro (&Qcoding_category);
4ed46869
KH
7612 Qcoding_category_index = intern ("coding-category-index");
7613 staticpro (&Qcoding_category_index);
7614
d46c5b12
KH
7615 Vcoding_category_table
7616 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX), Qnil);
7617 staticpro (&Vcoding_category_table);
4ed46869
KH
7618 {
7619 int i;
7620 for (i = 0; i < CODING_CATEGORY_IDX_MAX; i++)
7621 {
d46c5b12
KH
7622 XVECTOR (Vcoding_category_table)->contents[i]
7623 = intern (coding_category_name[i]);
7624 Fput (XVECTOR (Vcoding_category_table)->contents[i],
7625 Qcoding_category_index, make_number (i));
4ed46869
KH
7626 }
7627 }
7628
6b89e3aa
KH
7629 Vcoding_system_safe_chars = Fcons (Qnil, Qnil);
7630 staticpro (&Vcoding_system_safe_chars);
7631
f967223b
KH
7632 Qtranslation_table = intern ("translation-table");
7633 staticpro (&Qtranslation_table);
b666620c 7634 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (2));
bdd9fb48 7635
f967223b
KH
7636 Qtranslation_table_id = intern ("translation-table-id");
7637 staticpro (&Qtranslation_table_id);
84fbb8a0 7638
f967223b
KH
7639 Qtranslation_table_for_decode = intern ("translation-table-for-decode");
7640 staticpro (&Qtranslation_table_for_decode);
a5d301df 7641
f967223b
KH
7642 Qtranslation_table_for_encode = intern ("translation-table-for-encode");
7643 staticpro (&Qtranslation_table_for_encode);
a5d301df 7644
05e6f5dc
KH
7645 Qsafe_chars = intern ("safe-chars");
7646 staticpro (&Qsafe_chars);
7647
7648 Qchar_coding_system = intern ("char-coding-system");
7649 staticpro (&Qchar_coding_system);
7650
7651 /* Intern this now in case it isn't already done.
7652 Setting this variable twice is harmless.
7653 But don't staticpro it here--that is done in alloc.c. */
7654 Qchar_table_extra_slots = intern ("char-table-extra-slots");
7655 Fput (Qsafe_chars, Qchar_table_extra_slots, make_number (0));
067a6a66 7656 Fput (Qchar_coding_system, Qchar_table_extra_slots, make_number (0));
70c22245 7657
1397dc18
KH
7658 Qvalid_codes = intern ("valid-codes");
7659 staticpro (&Qvalid_codes);
7660
9ce27fde
KH
7661 Qemacs_mule = intern ("emacs-mule");
7662 staticpro (&Qemacs_mule);
7663
d46c5b12
KH
7664 Qraw_text = intern ("raw-text");
7665 staticpro (&Qraw_text);
7666
ecf488bc
DL
7667 Qutf_8 = intern ("utf-8");
7668 staticpro (&Qutf_8);
7669
a362520d
KH
7670 Qcoding_system_define_form = intern ("coding-system-define-form");
7671 staticpro (&Qcoding_system_define_form);
7672
4ed46869
KH
7673 defsubr (&Scoding_system_p);
7674 defsubr (&Sread_coding_system);
7675 defsubr (&Sread_non_nil_coding_system);
7676 defsubr (&Scheck_coding_system);
7677 defsubr (&Sdetect_coding_region);
d46c5b12 7678 defsubr (&Sdetect_coding_string);
05e6f5dc 7679 defsubr (&Sfind_coding_systems_region_internal);
068a9dbd 7680 defsubr (&Sunencodable_char_position);
4ed46869
KH
7681 defsubr (&Sdecode_coding_region);
7682 defsubr (&Sencode_coding_region);
7683 defsubr (&Sdecode_coding_string);
7684 defsubr (&Sencode_coding_string);
7685 defsubr (&Sdecode_sjis_char);
7686 defsubr (&Sencode_sjis_char);
7687 defsubr (&Sdecode_big5_char);
7688 defsubr (&Sencode_big5_char);
1ba9e4ab 7689 defsubr (&Sset_terminal_coding_system_internal);
c4825358 7690 defsubr (&Sset_safe_terminal_coding_system_internal);
4ed46869 7691 defsubr (&Sterminal_coding_system);
1ba9e4ab 7692 defsubr (&Sset_keyboard_coding_system_internal);
4ed46869 7693 defsubr (&Skeyboard_coding_system);
a5d301df 7694 defsubr (&Sfind_operation_coding_system);
1397dc18 7695 defsubr (&Supdate_coding_systems_internal);
66cfb530 7696 defsubr (&Sset_coding_priority_internal);
6b89e3aa 7697 defsubr (&Sdefine_coding_system_internal);
4ed46869 7698
4608c386 7699 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list,
48b0f3ae
PJ
7700 doc: /* List of coding systems.
7701
7702Do not alter the value of this variable manually. This variable should be
7703updated by the functions `make-coding-system' and
7704`define-coding-system-alias'. */);
4608c386
KH
7705 Vcoding_system_list = Qnil;
7706
7707 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist,
48b0f3ae
PJ
7708 doc: /* Alist of coding system names.
7709Each element is one element list of coding system name.
7710This variable is given to `completing-read' as TABLE argument.
7711
7712Do not alter the value of this variable manually. This variable should be
7713updated by the functions `make-coding-system' and
7714`define-coding-system-alias'. */);
4608c386
KH
7715 Vcoding_system_alist = Qnil;
7716
4ed46869 7717 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list,
48b0f3ae
PJ
7718 doc: /* List of coding-categories (symbols) ordered by priority.
7719
7720On detecting a coding system, Emacs tries code detection algorithms
7721associated with each coding-category one by one in this order. When
7722one algorithm agrees with a byte sequence of source text, the coding
7723system bound to the corresponding coding-category is selected. */);
4ed46869
KH
7724 {
7725 int i;
7726
7727 Vcoding_category_list = Qnil;
7728 for (i = CODING_CATEGORY_IDX_MAX - 1; i >= 0; i--)
7729 Vcoding_category_list
d46c5b12
KH
7730 = Fcons (XVECTOR (Vcoding_category_table)->contents[i],
7731 Vcoding_category_list);
4ed46869
KH
7732 }
7733
7734 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read,
48b0f3ae
PJ
7735 doc: /* Specify the coding system for read operations.
7736It is useful to bind this variable with `let', but do not set it globally.
7737If the value is a coding system, it is used for decoding on read operation.
7738If not, an appropriate element is used from one of the coding system alists:
7739There are three such tables, `file-coding-system-alist',
7740`process-coding-system-alist', and `network-coding-system-alist'. */);
4ed46869
KH
7741 Vcoding_system_for_read = Qnil;
7742
7743 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write,
48b0f3ae
PJ
7744 doc: /* Specify the coding system for write operations.
7745Programs bind this variable with `let', but you should not set it globally.
7746If the value is a coding system, it is used for encoding of output,
7747when writing it to a file and when sending it to a file or subprocess.
7748
7749If this does not specify a coding system, an appropriate element
7750is used from one of the coding system alists:
7751There are three such tables, `file-coding-system-alist',
7752`process-coding-system-alist', and `network-coding-system-alist'.
7753For output to files, if the above procedure does not specify a coding system,
7754the value of `buffer-file-coding-system' is used. */);
4ed46869
KH
7755 Vcoding_system_for_write = Qnil;
7756
7757 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used,
7c695ab9
DL
7758 doc: /* Coding system used in the latest file or process I/O.
7759Also set by `encode-coding-region', `decode-coding-region',
7760`encode-coding-string' and `decode-coding-string'. */);
4ed46869
KH
7761 Vlast_coding_system_used = Qnil;
7762
9ce27fde 7763 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion,
48b0f3ae
PJ
7764 doc: /* *Non-nil means always inhibit code conversion of end-of-line format.
7765See info node `Coding Systems' and info node `Text and Binary' concerning
7766such conversion. */);
9ce27fde
KH
7767 inhibit_eol_conversion = 0;
7768
ed29121d 7769 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system,
48b0f3ae
PJ
7770 doc: /* Non-nil means process buffer inherits coding system of process output.
7771Bind it to t if the process output is to be treated as if it were a file
7772read from some filesystem. */);
ed29121d
EZ
7773 inherit_process_coding_system = 0;
7774
02ba4723 7775 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist,
48b0f3ae
PJ
7776 doc: /* Alist to decide a coding system to use for a file I/O operation.
7777The format is ((PATTERN . VAL) ...),
7778where PATTERN is a regular expression matching a file name,
7779VAL is a coding system, a cons of coding systems, or a function symbol.
7780If VAL is a coding system, it is used for both decoding and encoding
7781the file contents.
7782If VAL is a cons of coding systems, the car part is used for decoding,
7783and the cdr part is used for encoding.
7784If VAL is a function symbol, the function must return a coding system
0192762c 7785or a cons of coding systems which are used as above. The function gets
ff955d90 7786the arguments with which `find-operation-coding-system' was called.
48b0f3ae
PJ
7787
7788See also the function `find-operation-coding-system'
7789and the variable `auto-coding-alist'. */);
02ba4723
KH
7790 Vfile_coding_system_alist = Qnil;
7791
7792 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist,
48b0f3ae
PJ
7793 doc: /* Alist to decide a coding system to use for a process I/O operation.
7794The format is ((PATTERN . VAL) ...),
7795where PATTERN is a regular expression matching a program name,
7796VAL is a coding system, a cons of coding systems, or a function symbol.
7797If VAL is a coding system, it is used for both decoding what received
7798from the program and encoding what sent to the program.
7799If VAL is a cons of coding systems, the car part is used for decoding,
7800and the cdr part is used for encoding.
7801If VAL is a function symbol, the function must return a coding system
7802or a cons of coding systems which are used as above.
7803
7804See also the function `find-operation-coding-system'. */);
02ba4723
KH
7805 Vprocess_coding_system_alist = Qnil;
7806
7807 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist,
48b0f3ae
PJ
7808 doc: /* Alist to decide a coding system to use for a network I/O operation.
7809The format is ((PATTERN . VAL) ...),
7810where PATTERN is a regular expression matching a network service name
7811or is a port number to connect to,
7812VAL is a coding system, a cons of coding systems, or a function symbol.
7813If VAL is a coding system, it is used for both decoding what received
7814from the network stream and encoding what sent to the network stream.
7815If VAL is a cons of coding systems, the car part is used for decoding,
7816and the cdr part is used for encoding.
7817If VAL is a function symbol, the function must return a coding system
7818or a cons of coding systems which are used as above.
7819
7820See also the function `find-operation-coding-system'. */);
02ba4723 7821 Vnetwork_coding_system_alist = Qnil;
4ed46869 7822
68c45bf0 7823 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system,
75205970
RS
7824 doc: /* Coding system to use with system messages.
7825Also used for decoding keyboard input on X Window system. */);
68c45bf0
PE
7826 Vlocale_coding_system = Qnil;
7827
005f0d35 7828 /* The eol mnemonics are reset in startup.el system-dependently. */
7722baf9 7829 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix,
48b0f3ae 7830 doc: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7722baf9 7831 eol_mnemonic_unix = build_string (":");
4ed46869 7832
7722baf9 7833 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos,
48b0f3ae 7834 doc: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7722baf9 7835 eol_mnemonic_dos = build_string ("\\");
4ed46869 7836
7722baf9 7837 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac,
48b0f3ae 7838 doc: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
7722baf9 7839 eol_mnemonic_mac = build_string ("/");
4ed46869 7840
7722baf9 7841 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided,
48b0f3ae 7842 doc: /* *String displayed in mode line when end-of-line format is not yet determined. */);
7722baf9 7843 eol_mnemonic_undecided = build_string (":");
4ed46869 7844
84fbb8a0 7845 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation,
48b0f3ae 7846 doc: /* *Non-nil enables character translation while encoding and decoding. */);
84fbb8a0 7847 Venable_character_translation = Qt;
bdd9fb48 7848
f967223b 7849 DEFVAR_LISP ("standard-translation-table-for-decode",
48b0f3ae
PJ
7850 &Vstandard_translation_table_for_decode,
7851 doc: /* Table for translating characters while decoding. */);
f967223b 7852 Vstandard_translation_table_for_decode = Qnil;
bdd9fb48 7853
f967223b 7854 DEFVAR_LISP ("standard-translation-table-for-encode",
48b0f3ae
PJ
7855 &Vstandard_translation_table_for_encode,
7856 doc: /* Table for translating characters while encoding. */);
f967223b 7857 Vstandard_translation_table_for_encode = Qnil;
4ed46869
KH
7858
7859 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist,
48b0f3ae
PJ
7860 doc: /* Alist of charsets vs revision numbers.
7861While encoding, if a charset (car part of an element) is found,
7862designate it with the escape sequence identifying revision (cdr part of the element). */);
4ed46869 7863 Vcharset_revision_alist = Qnil;
02ba4723
KH
7864
7865 DEFVAR_LISP ("default-process-coding-system",
7866 &Vdefault_process_coding_system,
48b0f3ae
PJ
7867 doc: /* Cons of coding systems used for process I/O by default.
7868The car part is used for decoding a process output,
7869the cdr part is used for encoding a text to be sent to a process. */);
02ba4723 7870 Vdefault_process_coding_system = Qnil;
c4825358 7871
3f003981 7872 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table,
48b0f3ae
PJ
7873 doc: /* Table of extra Latin codes in the range 128..159 (inclusive).
7874This is a vector of length 256.
7875If Nth element is non-nil, the existence of code N in a file
7876\(or output of subprocess) doesn't prevent it to be detected as
7877a coding system of ISO 2022 variant which has a flag
7878`accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
7879or reading output of a subprocess.
7880Only 128th through 159th elements has a meaning. */);
3f003981 7881 Vlatin_extra_code_table = Fmake_vector (make_number (256), Qnil);
d46c5b12
KH
7882
7883 DEFVAR_LISP ("select-safe-coding-system-function",
7884 &Vselect_safe_coding_system_function,
48b0f3ae
PJ
7885 doc: /* Function to call to select safe coding system for encoding a text.
7886
7887If set, this function is called to force a user to select a proper
7888coding system which can encode the text in the case that a default
7889coding system used in each operation can't encode the text.
7890
7891The default value is `select-safe-coding-system' (which see). */);
d46c5b12
KH
7892 Vselect_safe_coding_system_function = Qnil;
7893
5d5bf4d8
KH
7894 DEFVAR_BOOL ("coding-system-require-warning",
7895 &coding_system_require_warning,
7896 doc: /* Internal use only.
6b89e3aa
KH
7897If non-nil, on writing a file, `select-safe-coding-system-function' is
7898called even if `coding-system-for-write' is non-nil. The command
7899`universal-coding-system-argument' binds this variable to t temporarily. */);
5d5bf4d8
KH
7900 coding_system_require_warning = 0;
7901
7902
22ab2303 7903 DEFVAR_BOOL ("inhibit-iso-escape-detection",
74383408 7904 &inhibit_iso_escape_detection,
48b0f3ae
PJ
7905 doc: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
7906
7907By default, on reading a file, Emacs tries to detect how the text is
7908encoded. This code detection is sensitive to escape sequences. If
7909the sequence is valid as ISO2022, the code is determined as one of
7910the ISO2022 encodings, and the file is decoded by the corresponding
7911coding system (e.g. `iso-2022-7bit').
7912
7913However, there may be a case that you want to read escape sequences in
7914a file as is. In such a case, you can set this variable to non-nil.
7915Then, as the code detection ignores any escape sequences, no file is
7916detected as encoded in some ISO2022 encoding. The result is that all
7917escape sequences become visible in a buffer.
7918
7919The default value is nil, and it is strongly recommended not to change
7920it. That is because many Emacs Lisp source files that contain
7921non-ASCII characters are encoded by the coding system `iso-2022-7bit'
7922in Emacs's distribution, and they won't be decoded correctly on
7923reading if you suppress escape sequence detection.
7924
7925The other way to read escape sequences in a file without decoding is
7926to explicitly specify some coding system that doesn't use ISO2022's
7927escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
74383408 7928 inhibit_iso_escape_detection = 0;
002fdb44
DL
7929
7930 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input,
15c8f9d1
DL
7931 doc: /* Char table for translating self-inserting characters.
7932This is applied to the result of input methods, not their input. See also
7933`keyboard-translate-table'. */);
002fdb44 7934 Vtranslation_table_for_input = Qnil;
4ed46869
KH
7935}
7936
68c45bf0
PE
7937char *
7938emacs_strerror (error_number)
7939 int error_number;
7940{
7941 char *str;
7942
ca9c0567 7943 synchronize_system_messages_locale ();
68c45bf0
PE
7944 str = strerror (error_number);
7945
7946 if (! NILP (Vlocale_coding_system))
7947 {
7948 Lisp_Object dec = code_convert_string_norecord (build_string (str),
7949 Vlocale_coding_system,
7950 0);
d5db4077 7951 str = (char *) SDATA (dec);
68c45bf0
PE
7952 }
7953
7954 return str;
7955}
7956
4ed46869 7957#endif /* emacs */
c2f94ebc 7958
ab5796a9
MB
7959/* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
7960 (do not change this comment) */