HCoop / bpt / emacs.git / blame_incremental
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (non-incremental) | history | HEAD
*** empty log message ***
[bpt/emacs.git] / src / coding.c
... / ...
CommitLineData
1/* Coding system handler (conversion, detection, etc).
2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
3 2006, 2007 Free Software Foundation, Inc.
4 Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
5 2005, 2006, 2007
6 National Institute of Advanced Industrial Science and Technology (AIST)
7 Registration Number H14PRO021
8 Copyright (C) 2003
9 National Institute of Advanced Industrial Science and Technology (AIST)
10 Registration Number H13PRO009
11
12This file is part of GNU Emacs.
13
14GNU Emacs is free software; you can redistribute it and/or modify
15it under the terms of the GNU General Public License as published by
16the Free Software Foundation; either version 3, or (at your option)
17any later version.
18
19GNU Emacs is distributed in the hope that it will be useful,
20but WITHOUT ANY WARRANTY; without even the implied warranty of
21MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
22GNU General Public License for more details.
23
24You should have received a copy of the GNU General Public License
25along with GNU Emacs; see the file COPYING. If not, write to
26the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
27Boston, MA 02110-1301, USA. */
28
29/*** TABLE OF CONTENTS ***
30
31 0. General comments
32 1. Preamble
33 2. Emacs' internal format (emacs-utf-8) handlers
34 3. UTF-8 handlers
35 4. UTF-16 handlers
36 5. Charset-base coding systems handlers
37 6. emacs-mule (old Emacs' internal format) handlers
38 7. ISO2022 handlers
39 8. Shift-JIS and BIG5 handlers
40 9. CCL handlers
41 10. C library functions
42 11. Emacs Lisp library functions
43 12. Postamble
44
45*/
46
47/*** 0. General comments ***
48
49
50CODING SYSTEM
51
52 A coding system is an object for an encoding mechanism that contains
53 information about how to convert byte sequences to character
54 sequences and vice versa. When we say "decode", it means converting
55 a byte sequence of a specific coding system into a character
56 sequence that is represented by Emacs' internal coding system
57 `emacs-utf-8', and when we say "encode", it means converting a
58 character sequence of emacs-utf-8 to a byte sequence of a specific
59 coding system.
60
61 In Emacs Lisp, a coding system is represented by a Lisp symbol. In
62 C level, a coding system is represented by a vector of attributes
63 stored in the hash table Vcharset_hash_table. The conversion from
64 coding system symbol to attributes vector is done by looking up
65 Vcharset_hash_table by the symbol.
66
67 Coding systems are classified into the following types depending on
68 the encoding mechanism. Here's a brief description of the types.
69
70 o UTF-8
71
72 o UTF-16
73
74 o Charset-base coding system
75
76 A coding system defined by one or more (coded) character sets.
77 Decoding and encoding are done by a code converter defined for each
78 character set.
79
80 o Old Emacs internal format (emacs-mule)
81
82 The coding system adopted by old versions of Emacs (20 and 21).
83
84 o ISO2022-base coding system
85
86 The most famous coding system for multiple character sets. X's
87 Compound Text, various EUCs (Extended Unix Code), and coding systems
88 used in the Internet communication such as ISO-2022-JP are all
89 variants of ISO2022.
90
91 o SJIS (or Shift-JIS or MS-Kanji-Code)
92
93 A coding system to encode character sets: ASCII, JISX0201, and
94 JISX0208. Widely used for PC's in Japan. Details are described in
95 section 8.
96
97 o BIG5
98
99 A coding system to encode character sets: ASCII and Big5. Widely
100 used for Chinese (mainly in Taiwan and Hong Kong). Details are
101 described in section 8. In this file, when we write "big5" (all
102 lowercase), we mean the coding system, and when we write "Big5"
103 (capitalized), we mean the character set.
104
105 o CCL
106
107 If a user wants to decode/encode text encoded in a coding system
108 not listed above, he can supply a decoder and an encoder for it in
109 CCL (Code Conversion Language) programs. Emacs executes the CCL
110 program while decoding/encoding.
111
112 o Raw-text
113
114 A coding system for text containing raw eight-bit data. Emacs
115 treats each byte of source text as a character (except for
116 end-of-line conversion).
117
118 o No-conversion
119
120 Like raw text, but don't do end-of-line conversion.
121
122
123END-OF-LINE FORMAT
124
125 How text end-of-line is encoded depends on operating system. For
126 instance, Unix's format is just one byte of LF (line-feed) code,
127 whereas DOS's format is two-byte sequence of `carriage-return' and
128 `line-feed' codes. MacOS's format is usually one byte of
129 `carriage-return'.
130
131 Since text character encoding and end-of-line encoding are
132 independent, any coding system described above can take any format
133 of end-of-line (except for no-conversion).
134
135STRUCT CODING_SYSTEM
136
137 Before using a coding system for code conversion (i.e. decoding and
138 encoding), we setup a structure of type `struct coding_system'.
139 This structure keeps various information about a specific code
140 conversion (e.g. the location of source and destination data).
141
142*/
143
144/* COMMON MACROS */
145
146
147/*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
148
149 These functions check if a byte sequence specified as a source in
150 CODING conforms to the format of XXX, and update the members of
151 DETECT_INFO.
152
153 Return 1 if the byte sequence conforms to XXX, otherwise return 0.
154
155 Below is the template of these functions. */
156
157#if 0
158static int
159detect_coding_XXX (coding, detect_info)
160 struct coding_system *coding;
161 struct coding_detection_info *detect_info;
162{
163 const unsigned char *src = coding->source;
164 const unsigned char *src_end = coding->source + coding->src_bytes;
165 int multibytep = coding->src_multibyte;
166 int consumed_chars = 0;
167 int found = 0;
168 ...;
169
170 while (1)
171 {
172 /* Get one byte from the source. If the souce is exausted, jump
173 to no_more_source:. */
174 ONE_MORE_BYTE (c);
175
176 if (! __C_conforms_to_XXX___ (c))
177 break;
178 if (! __C_strongly_suggests_XXX__ (c))
179 found = CATEGORY_MASK_XXX;
180 }
181 /* The byte sequence is invalid for XXX. */
182 detect_info->rejected |= CATEGORY_MASK_XXX;
183 return 0;
184
185 no_more_source:
186 /* The source exausted successfully. */
187 detect_info->found |= found;
188 return 1;
189}
190#endif
191
192/*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
193
194 These functions decode a byte sequence specified as a source by
195 CODING. The resulting multibyte text goes to a place pointed to by
196 CODING->charbuf, the length of which should not exceed
197 CODING->charbuf_size;
198
199 These functions set the information of original and decoded texts in
200 CODING->consumed, CODING->consumed_char, and CODING->charbuf_used.
201 They also set CODING->result to one of CODING_RESULT_XXX indicating
202 how the decoding is finished.
203
204 Below is the template of these functions. */
205
206#if 0
207static void
208decode_coding_XXXX (coding)
209 struct coding_system *coding;
210{
211 const unsigned char *src = coding->source + coding->consumed;
212 const unsigned char *src_end = coding->source + coding->src_bytes;
213 /* SRC_BASE remembers the start position in source in each loop.
214 The loop will be exited when there's not enough source code, or
215 when there's no room in CHARBUF for a decoded character. */
216 const unsigned char *src_base;
217 /* A buffer to produce decoded characters. */
218 int *charbuf = coding->charbuf + coding->charbuf_used;
219 int *charbuf_end = coding->charbuf + coding->charbuf_size;
220 int multibytep = coding->src_multibyte;
221
222 while (1)
223 {
224 src_base = src;
225 if (charbuf < charbuf_end)
226 /* No more room to produce a decoded character. */
227 break;
228 ONE_MORE_BYTE (c);
229 /* Decode it. */
230 }
231
232 no_more_source:
233 if (src_base < src_end
234 && coding->mode & CODING_MODE_LAST_BLOCK)
235 /* If the source ends by partial bytes to construct a character,
236 treat them as eight-bit raw data. */
237 while (src_base < src_end && charbuf < charbuf_end)
238 *charbuf++ = *src_base++;
239 /* Remember how many bytes and characters we consumed. If the
240 source is multibyte, the bytes and chars are not identical. */
241 coding->consumed = coding->consumed_char = src_base - coding->source;
242 /* Remember how many characters we produced. */
243 coding->charbuf_used = charbuf - coding->charbuf;
244}
245#endif
246
247/*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
248
249 These functions encode SRC_BYTES length text at SOURCE of Emacs'
250 internal multibyte format by CODING. The resulting byte sequence
251 goes to a place pointed to by DESTINATION, the length of which
252 should not exceed DST_BYTES.
253
254 These functions set the information of original and encoded texts in
255 the members produced, produced_char, consumed, and consumed_char of
256 the structure *CODING. They also set the member result to one of
257 CODING_RESULT_XXX indicating how the encoding finished.
258
259 DST_BYTES zero means that source area and destination area are
260 overlapped, which means that we can produce a encoded text until it
261 reaches at the head of not-yet-encoded source text.
262
263 Below is a template of these functions. */
264#if 0
265static void
266encode_coding_XXX (coding)
267 struct coding_system *coding;
268{
269 int multibytep = coding->dst_multibyte;
270 int *charbuf = coding->charbuf;
271 int *charbuf_end = charbuf->charbuf + coding->charbuf_used;
272 unsigned char *dst = coding->destination + coding->produced;
273 unsigned char *dst_end = coding->destination + coding->dst_bytes;
274 unsigned char *adjusted_dst_end = dst_end - _MAX_BYTES_PRODUCED_IN_LOOP_;
275 int produced_chars = 0;
276
277 for (; charbuf < charbuf_end && dst < adjusted_dst_end; charbuf++)
278 {
279 int c = *charbuf;
280 /* Encode C into DST, and increment DST. */
281 }
282 label_no_more_destination:
283 /* How many chars and bytes we produced. */
284 coding->produced_char += produced_chars;
285 coding->produced = dst - coding->destination;
286}
287#endif
288
289\f
290/*** 1. Preamble ***/
291
292#include <config.h>
293#include <stdio.h>
294
295#include "lisp.h"
296#include "buffer.h"
297#include "character.h"
298#include "charset.h"
299#include "ccl.h"
300#include "composite.h"
301#include "coding.h"
302#include "window.h"
303#include "frame.h"
304#include "termhooks.h"
305
306Lisp_Object Vcoding_system_hash_table;
307
308Lisp_Object Qcoding_system, Qcoding_aliases, Qeol_type;
309Lisp_Object Qunix, Qdos;
310extern Lisp_Object Qmac; /* frame.c */
311Lisp_Object Qbuffer_file_coding_system;
312Lisp_Object Qpost_read_conversion, Qpre_write_conversion;
313Lisp_Object Qdefault_char;
314Lisp_Object Qno_conversion, Qundecided;
315Lisp_Object Qcharset, Qiso_2022, Qutf_8, Qutf_16, Qshift_jis, Qbig5;
316Lisp_Object Qbig, Qlittle;
317Lisp_Object Qcoding_system_history;
318Lisp_Object Qvalid_codes;
319Lisp_Object QCcategory, QCmnemonic, QCdefalut_char;
320Lisp_Object QCdecode_translation_table, QCencode_translation_table;
321Lisp_Object QCpost_read_conversion, QCpre_write_conversion;
322Lisp_Object QCascii_compatible_p;
323
324extern Lisp_Object Qinsert_file_contents, Qwrite_region;
325Lisp_Object Qcall_process, Qcall_process_region;
326Lisp_Object Qstart_process, Qopen_network_stream;
327Lisp_Object Qtarget_idx;
328
329Lisp_Object Qinsufficient_source, Qinconsistent_eol, Qinvalid_source;
330Lisp_Object Qinterrupted, Qinsufficient_memory;
331
332extern Lisp_Object Qcompletion_ignore_case;
333
334/* If a symbol has this property, evaluate the value to define the
335 symbol as a coding system. */
336static Lisp_Object Qcoding_system_define_form;
337
338int coding_system_require_warning;
339
340Lisp_Object Vselect_safe_coding_system_function;
341
342/* Mnemonic string for each format of end-of-line. */
343Lisp_Object eol_mnemonic_unix, eol_mnemonic_dos, eol_mnemonic_mac;
344/* Mnemonic string to indicate format of end-of-line is not yet
345 decided. */
346Lisp_Object eol_mnemonic_undecided;
347
348/* Format of end-of-line decided by system. This is Qunix on
349 Unix and Mac, Qdos on DOS/Windows.
350 This has an effect only for external encoding (i.e. for output to
351 file and process), not for in-buffer or Lisp string encoding. */
352static Lisp_Object system_eol_type;
353
354#ifdef emacs
355
356Lisp_Object Vcoding_system_list, Vcoding_system_alist;
357
358Lisp_Object Qcoding_system_p, Qcoding_system_error;
359
360/* Coding system emacs-mule and raw-text are for converting only
361 end-of-line format. */
362Lisp_Object Qemacs_mule, Qraw_text;
363Lisp_Object Qutf_8_emacs;
364
365/* Coding-systems are handed between Emacs Lisp programs and C internal
366 routines by the following three variables. */
367/* Coding-system for reading files and receiving data from process. */
368Lisp_Object Vcoding_system_for_read;
369/* Coding-system for writing files and sending data to process. */
370Lisp_Object Vcoding_system_for_write;
371/* Coding-system actually used in the latest I/O. */
372Lisp_Object Vlast_coding_system_used;
373/* Set to non-nil when an error is detected while code conversion. */
374Lisp_Object Vlast_code_conversion_error;
375/* A vector of length 256 which contains information about special
376 Latin codes (especially for dealing with Microsoft codes). */
377Lisp_Object Vlatin_extra_code_table;
378
379/* Flag to inhibit code conversion of end-of-line format. */
380int inhibit_eol_conversion;
381
382/* Flag to inhibit ISO2022 escape sequence detection. */
383int inhibit_iso_escape_detection;
384
385/* Flag to make buffer-file-coding-system inherit from process-coding. */
386int inherit_process_coding_system;
387
388/* Coding system to be used to encode text for terminal display when
389 terminal coding system is nil. */
390struct coding_system safe_terminal_coding;
391
392Lisp_Object Vfile_coding_system_alist;
393Lisp_Object Vprocess_coding_system_alist;
394Lisp_Object Vnetwork_coding_system_alist;
395
396Lisp_Object Vlocale_coding_system;
397
398#endif /* emacs */
399
400/* Flag to tell if we look up translation table on character code
401 conversion. */
402Lisp_Object Venable_character_translation;
403/* Standard translation table to look up on decoding (reading). */
404Lisp_Object Vstandard_translation_table_for_decode;
405/* Standard translation table to look up on encoding (writing). */
406Lisp_Object Vstandard_translation_table_for_encode;
407
408Lisp_Object Qtranslation_table;
409Lisp_Object Qtranslation_table_id;
410Lisp_Object Qtranslation_table_for_decode;
411Lisp_Object Qtranslation_table_for_encode;
412
413/* Alist of charsets vs revision number. */
414static Lisp_Object Vcharset_revision_table;
415
416/* Default coding systems used for process I/O. */
417Lisp_Object Vdefault_process_coding_system;
418
419/* Char table for translating Quail and self-inserting input. */
420Lisp_Object Vtranslation_table_for_input;
421
422/* Two special coding systems. */
423Lisp_Object Vsjis_coding_system;
424Lisp_Object Vbig5_coding_system;
425
426/* ISO2022 section */
427
428#define CODING_ISO_INITIAL(coding, reg) \
429 (XINT (AREF (AREF (CODING_ID_ATTRS ((coding)->id), \
430 coding_attr_iso_initial), \
431 reg)))
432
433
434#define CODING_ISO_REQUEST(coding, charset_id) \
435 ((charset_id <= (coding)->max_charset_id \
436 ? (coding)->safe_charsets[charset_id] \
437 : -1))
438
439
440#define CODING_ISO_FLAGS(coding) \
441 ((coding)->spec.iso_2022.flags)
442#define CODING_ISO_DESIGNATION(coding, reg) \
443 ((coding)->spec.iso_2022.current_designation[reg])
444#define CODING_ISO_INVOCATION(coding, plane) \
445 ((coding)->spec.iso_2022.current_invocation[plane])
446#define CODING_ISO_SINGLE_SHIFTING(coding) \
447 ((coding)->spec.iso_2022.single_shifting)
448#define CODING_ISO_BOL(coding) \
449 ((coding)->spec.iso_2022.bol)
450#define CODING_ISO_INVOKED_CHARSET(coding, plane) \
451 CODING_ISO_DESIGNATION ((coding), CODING_ISO_INVOCATION ((coding), (plane)))
452
453/* Control characters of ISO2022. */
454 /* code */ /* function */
455#define ISO_CODE_LF 0x0A /* line-feed */
456#define ISO_CODE_CR 0x0D /* carriage-return */
457#define ISO_CODE_SO 0x0E /* shift-out */
458#define ISO_CODE_SI 0x0F /* shift-in */
459#define ISO_CODE_SS2_7 0x19 /* single-shift-2 for 7-bit code */
460#define ISO_CODE_ESC 0x1B /* escape */
461#define ISO_CODE_SS2 0x8E /* single-shift-2 */
462#define ISO_CODE_SS3 0x8F /* single-shift-3 */
463#define ISO_CODE_CSI 0x9B /* control-sequence-introducer */
464
465/* All code (1-byte) of ISO2022 is classified into one of the
466 followings. */
467enum iso_code_class_type
468 {
469 ISO_control_0, /* Control codes in the range
470 0x00..0x1F and 0x7F, except for the
471 following 5 codes. */
472 ISO_shift_out, /* ISO_CODE_SO (0x0E) */
473 ISO_shift_in, /* ISO_CODE_SI (0x0F) */
474 ISO_single_shift_2_7, /* ISO_CODE_SS2_7 (0x19) */
475 ISO_escape, /* ISO_CODE_SO (0x1B) */
476 ISO_control_1, /* Control codes in the range
477 0x80..0x9F, except for the
478 following 3 codes. */
479 ISO_single_shift_2, /* ISO_CODE_SS2 (0x8E) */
480 ISO_single_shift_3, /* ISO_CODE_SS3 (0x8F) */
481 ISO_control_sequence_introducer, /* ISO_CODE_CSI (0x9B) */
482 ISO_0x20_or_0x7F, /* Codes of the values 0x20 or 0x7F. */
483 ISO_graphic_plane_0, /* Graphic codes in the range 0x21..0x7E. */
484 ISO_0xA0_or_0xFF, /* Codes of the values 0xA0 or 0xFF. */
485 ISO_graphic_plane_1 /* Graphic codes in the range 0xA1..0xFE. */
486 };
487
488/** The macros CODING_ISO_FLAG_XXX defines a flag bit of the
489 `iso-flags' attribute of an iso2022 coding system. */
490
491/* If set, produce long-form designation sequence (e.g. ESC $ ( A)
492 instead of the correct short-form sequence (e.g. ESC $ A). */
493#define CODING_ISO_FLAG_LONG_FORM 0x0001
494
495/* If set, reset graphic planes and registers at end-of-line to the
496 initial state. */
497#define CODING_ISO_FLAG_RESET_AT_EOL 0x0002
498
499/* If set, reset graphic planes and registers before any control
500 characters to the initial state. */
501#define CODING_ISO_FLAG_RESET_AT_CNTL 0x0004
502
503/* If set, encode by 7-bit environment. */
504#define CODING_ISO_FLAG_SEVEN_BITS 0x0008
505
506/* If set, use locking-shift function. */
507#define CODING_ISO_FLAG_LOCKING_SHIFT 0x0010
508
509/* If set, use single-shift function. Overwrite
510 CODING_ISO_FLAG_LOCKING_SHIFT. */
511#define CODING_ISO_FLAG_SINGLE_SHIFT 0x0020
512
513/* If set, use designation escape sequence. */
514#define CODING_ISO_FLAG_DESIGNATION 0x0040
515
516/* If set, produce revision number sequence. */
517#define CODING_ISO_FLAG_REVISION 0x0080
518
519/* If set, produce ISO6429's direction specifying sequence. */
520#define CODING_ISO_FLAG_DIRECTION 0x0100
521
522/* If set, assume designation states are reset at beginning of line on
523 output. */
524#define CODING_ISO_FLAG_INIT_AT_BOL 0x0200
525
526/* If set, designation sequence should be placed at beginning of line
527 on output. */
528#define CODING_ISO_FLAG_DESIGNATE_AT_BOL 0x0400
529
530/* If set, do not encode unsafe charactes on output. */
531#define CODING_ISO_FLAG_SAFE 0x0800
532
533/* If set, extra latin codes (128..159) are accepted as a valid code
534 on input. */
535#define CODING_ISO_FLAG_LATIN_EXTRA 0x1000
536
537#define CODING_ISO_FLAG_COMPOSITION 0x2000
538
539#define CODING_ISO_FLAG_EUC_TW_SHIFT 0x4000
540
541#define CODING_ISO_FLAG_USE_ROMAN 0x8000
542
543#define CODING_ISO_FLAG_USE_OLDJIS 0x10000
544
545#define CODING_ISO_FLAG_FULL_SUPPORT 0x100000
546
547/* A character to be produced on output if encoding of the original
548 character is prohibited by CODING_ISO_FLAG_SAFE. */
549#define CODING_INHIBIT_CHARACTER_SUBSTITUTION '?'
550
551
552/* UTF-16 section */
553#define CODING_UTF_16_BOM(coding) \
554 ((coding)->spec.utf_16.bom)
555
556#define CODING_UTF_16_ENDIAN(coding) \
557 ((coding)->spec.utf_16.endian)
558
559#define CODING_UTF_16_SURROGATE(coding) \
560 ((coding)->spec.utf_16.surrogate)
561
562
563/* CCL section */
564#define CODING_CCL_DECODER(coding) \
565 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_decoder)
566#define CODING_CCL_ENCODER(coding) \
567 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_encoder)
568#define CODING_CCL_VALIDS(coding) \
569 (SDATA (AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_valids)))
570
571/* Index for each coding category in `coding_categories' */
572
573enum coding_category
574 {
575 coding_category_iso_7,
576 coding_category_iso_7_tight,
577 coding_category_iso_8_1,
578 coding_category_iso_8_2,
579 coding_category_iso_7_else,
580 coding_category_iso_8_else,
581 coding_category_utf_8,
582 coding_category_utf_16_auto,
583 coding_category_utf_16_be,
584 coding_category_utf_16_le,
585 coding_category_utf_16_be_nosig,
586 coding_category_utf_16_le_nosig,
587 coding_category_charset,
588 coding_category_sjis,
589 coding_category_big5,
590 coding_category_ccl,
591 coding_category_emacs_mule,
592 /* All above are targets of code detection. */
593 coding_category_raw_text,
594 coding_category_undecided,
595 coding_category_max
596 };
597
598/* Definitions of flag bits used in detect_coding_XXXX. */
599#define CATEGORY_MASK_ISO_7 (1 << coding_category_iso_7)
600#define CATEGORY_MASK_ISO_7_TIGHT (1 << coding_category_iso_7_tight)
601#define CATEGORY_MASK_ISO_8_1 (1 << coding_category_iso_8_1)
602#define CATEGORY_MASK_ISO_8_2 (1 << coding_category_iso_8_2)
603#define CATEGORY_MASK_ISO_7_ELSE (1 << coding_category_iso_7_else)
604#define CATEGORY_MASK_ISO_8_ELSE (1 << coding_category_iso_8_else)
605#define CATEGORY_MASK_UTF_8 (1 << coding_category_utf_8)
606#define CATEGORY_MASK_UTF_16_AUTO (1 << coding_category_utf_16_auto)
607#define CATEGORY_MASK_UTF_16_BE (1 << coding_category_utf_16_be)
608#define CATEGORY_MASK_UTF_16_LE (1 << coding_category_utf_16_le)
609#define CATEGORY_MASK_UTF_16_BE_NOSIG (1 << coding_category_utf_16_be_nosig)
610#define CATEGORY_MASK_UTF_16_LE_NOSIG (1 << coding_category_utf_16_le_nosig)
611#define CATEGORY_MASK_CHARSET (1 << coding_category_charset)
612#define CATEGORY_MASK_SJIS (1 << coding_category_sjis)
613#define CATEGORY_MASK_BIG5 (1 << coding_category_big5)
614#define CATEGORY_MASK_CCL (1 << coding_category_ccl)
615#define CATEGORY_MASK_EMACS_MULE (1 << coding_category_emacs_mule)
616#define CATEGORY_MASK_RAW_TEXT (1 << coding_category_raw_text)
617
618/* This value is returned if detect_coding_mask () find nothing other
619 than ASCII characters. */
620#define CATEGORY_MASK_ANY \
621 (CATEGORY_MASK_ISO_7 \
622 | CATEGORY_MASK_ISO_7_TIGHT \
623 | CATEGORY_MASK_ISO_8_1 \
624 | CATEGORY_MASK_ISO_8_2 \
625 | CATEGORY_MASK_ISO_7_ELSE \
626 | CATEGORY_MASK_ISO_8_ELSE \
627 | CATEGORY_MASK_UTF_8 \
628 | CATEGORY_MASK_UTF_16_BE \
629 | CATEGORY_MASK_UTF_16_LE \
630 | CATEGORY_MASK_UTF_16_BE_NOSIG \
631 | CATEGORY_MASK_UTF_16_LE_NOSIG \
632 | CATEGORY_MASK_CHARSET \
633 | CATEGORY_MASK_SJIS \
634 | CATEGORY_MASK_BIG5 \
635 | CATEGORY_MASK_CCL \
636 | CATEGORY_MASK_EMACS_MULE)
637
638
639#define CATEGORY_MASK_ISO_7BIT \
640 (CATEGORY_MASK_ISO_7 | CATEGORY_MASK_ISO_7_TIGHT)
641
642#define CATEGORY_MASK_ISO_8BIT \
643 (CATEGORY_MASK_ISO_8_1 | CATEGORY_MASK_ISO_8_2)
644
645#define CATEGORY_MASK_ISO_ELSE \
646 (CATEGORY_MASK_ISO_7_ELSE | CATEGORY_MASK_ISO_8_ELSE)
647
648#define CATEGORY_MASK_ISO_ESCAPE \
649 (CATEGORY_MASK_ISO_7 \
650 | CATEGORY_MASK_ISO_7_TIGHT \
651 | CATEGORY_MASK_ISO_7_ELSE \
652 | CATEGORY_MASK_ISO_8_ELSE)
653
654#define CATEGORY_MASK_ISO \
655 ( CATEGORY_MASK_ISO_7BIT \
656 | CATEGORY_MASK_ISO_8BIT \
657 | CATEGORY_MASK_ISO_ELSE)
658
659#define CATEGORY_MASK_UTF_16 \
660 (CATEGORY_MASK_UTF_16_BE \
661 | CATEGORY_MASK_UTF_16_LE \
662 | CATEGORY_MASK_UTF_16_BE_NOSIG \
663 | CATEGORY_MASK_UTF_16_LE_NOSIG)
664
665
666/* List of symbols `coding-category-xxx' ordered by priority. This
667 variable is exposed to Emacs Lisp. */
668static Lisp_Object Vcoding_category_list;
669
670/* Table of coding categories (Lisp symbols). This variable is for
671 internal use oly. */
672static Lisp_Object Vcoding_category_table;
673
674/* Table of coding-categories ordered by priority. */
675static enum coding_category coding_priorities[coding_category_max];
676
677/* Nth element is a coding context for the coding system bound to the
678 Nth coding category. */
679static struct coding_system coding_categories[coding_category_max];
680
681/*** Commonly used macros and functions ***/
682
683#ifndef min
684#define min(a, b) ((a) < (b) ? (a) : (b))
685#endif
686#ifndef max
687#define max(a, b) ((a) > (b) ? (a) : (b))
688#endif
689
690#define CODING_GET_INFO(coding, attrs, charset_list) \
691 do { \
692 (attrs) = CODING_ID_ATTRS ((coding)->id); \
693 (charset_list) = CODING_ATTR_CHARSET_LIST (attrs); \
694 } while (0)
695
696
697/* Safely get one byte from the source text pointed by SRC which ends
698 at SRC_END, and set C to that byte. If there are not enough bytes
699 in the source, it jumps to `no_more_source'. If multibytep is
700 nonzero, and a multibyte character is found at SRC, set C to the
701 negative value of the character code. The caller should declare
702 and set these variables appropriately in advance:
703 src, src_end, multibytep */
704
705#define ONE_MORE_BYTE(c) \
706 do { \
707 if (src == src_end) \
708 { \
709 if (src_base < src) \
710 record_conversion_result \
711 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
712 goto no_more_source; \
713 } \
714 c = *src++; \
715 if (multibytep && (c & 0x80)) \
716 { \
717 if ((c & 0xFE) == 0xC0) \
718 c = ((c & 1) << 6) | *src++; \
719 else \
720 { \
721 src--; \
722 c = - string_char (src, &src, NULL); \
723 record_conversion_result \
724 (coding, CODING_RESULT_INVALID_SRC); \
725 } \
726 } \
727 consumed_chars++; \
728 } while (0)
729
730
731#define ONE_MORE_BYTE_NO_CHECK(c) \
732 do { \
733 c = *src++; \
734 if (multibytep && (c & 0x80)) \
735 { \
736 if ((c & 0xFE) == 0xC0) \
737 c = ((c & 1) << 6) | *src++; \
738 else \
739 { \
740 src--; \
741 c = - string_char (src, &src, NULL); \
742 record_conversion_result \
743 (coding, CODING_RESULT_INVALID_SRC); \
744 } \
745 } \
746 consumed_chars++; \
747 } while (0)
748
749
750/* Store a byte C in the place pointed by DST and increment DST to the
751 next free point, and increment PRODUCED_CHARS. The caller should
752 assure that C is 0..127, and declare and set the variable `dst'
753 appropriately in advance.
754*/
755
756
757#define EMIT_ONE_ASCII_BYTE(c) \
758 do { \
759 produced_chars++; \
760 *dst++ = (c); \
761 } while (0)
762
763
764/* Like EMIT_ONE_ASCII_BYTE byt store two bytes; C1 and C2. */
765
766#define EMIT_TWO_ASCII_BYTES(c1, c2) \
767 do { \
768 produced_chars += 2; \
769 *dst++ = (c1), *dst++ = (c2); \
770 } while (0)
771
772
773/* Store a byte C in the place pointed by DST and increment DST to the
774 next free point, and increment PRODUCED_CHARS. If MULTIBYTEP is
775 nonzero, store in an appropriate multibyte from. The caller should
776 declare and set the variables `dst' and `multibytep' appropriately
777 in advance. */
778
779#define EMIT_ONE_BYTE(c) \
780 do { \
781 produced_chars++; \
782 if (multibytep) \
783 { \
784 int ch = (c); \
785 if (ch >= 0x80) \
786 ch = BYTE8_TO_CHAR (ch); \
787 CHAR_STRING_ADVANCE (ch, dst); \
788 } \
789 else \
790 *dst++ = (c); \
791 } while (0)
792
793
794/* Like EMIT_ONE_BYTE, but emit two bytes; C1 and C2. */
795
796#define EMIT_TWO_BYTES(c1, c2) \
797 do { \
798 produced_chars += 2; \
799 if (multibytep) \
800 { \
801 int ch; \
802 \
803 ch = (c1); \
804 if (ch >= 0x80) \
805 ch = BYTE8_TO_CHAR (ch); \
806 CHAR_STRING_ADVANCE (ch, dst); \
807 ch = (c2); \
808 if (ch >= 0x80) \
809 ch = BYTE8_TO_CHAR (ch); \
810 CHAR_STRING_ADVANCE (ch, dst); \
811 } \
812 else \
813 { \
814 *dst++ = (c1); \
815 *dst++ = (c2); \
816 } \
817 } while (0)
818
819
820#define EMIT_THREE_BYTES(c1, c2, c3) \
821 do { \
822 EMIT_ONE_BYTE (c1); \
823 EMIT_TWO_BYTES (c2, c3); \
824 } while (0)
825
826
827#define EMIT_FOUR_BYTES(c1, c2, c3, c4) \
828 do { \
829 EMIT_TWO_BYTES (c1, c2); \
830 EMIT_TWO_BYTES (c3, c4); \
831 } while (0)
832
833
834/* Prototypes for static functions. */
835static void record_conversion_result P_ ((struct coding_system *coding,
836 enum coding_result_code result));
837static int detect_coding_utf_8 P_ ((struct coding_system *,
838 struct coding_detection_info *info));
839static void decode_coding_utf_8 P_ ((struct coding_system *));
840static int encode_coding_utf_8 P_ ((struct coding_system *));
841
842static int detect_coding_utf_16 P_ ((struct coding_system *,
843 struct coding_detection_info *info));
844static void decode_coding_utf_16 P_ ((struct coding_system *));
845static int encode_coding_utf_16 P_ ((struct coding_system *));
846
847static int detect_coding_iso_2022 P_ ((struct coding_system *,
848 struct coding_detection_info *info));
849static void decode_coding_iso_2022 P_ ((struct coding_system *));
850static int encode_coding_iso_2022 P_ ((struct coding_system *));
851
852static int detect_coding_emacs_mule P_ ((struct coding_system *,
853 struct coding_detection_info *info));
854static void decode_coding_emacs_mule P_ ((struct coding_system *));
855static int encode_coding_emacs_mule P_ ((struct coding_system *));
856
857static int detect_coding_sjis P_ ((struct coding_system *,
858 struct coding_detection_info *info));
859static void decode_coding_sjis P_ ((struct coding_system *));
860static int encode_coding_sjis P_ ((struct coding_system *));
861
862static int detect_coding_big5 P_ ((struct coding_system *,
863 struct coding_detection_info *info));
864static void decode_coding_big5 P_ ((struct coding_system *));
865static int encode_coding_big5 P_ ((struct coding_system *));
866
867static int detect_coding_ccl P_ ((struct coding_system *,
868 struct coding_detection_info *info));
869static void decode_coding_ccl P_ ((struct coding_system *));
870static int encode_coding_ccl P_ ((struct coding_system *));
871
872static void decode_coding_raw_text P_ ((struct coding_system *));
873static int encode_coding_raw_text P_ ((struct coding_system *));
874
875static void coding_set_source P_ ((struct coding_system *));
876static void coding_set_destination P_ ((struct coding_system *));
877static void coding_alloc_by_realloc P_ ((struct coding_system *, EMACS_INT));
878static void coding_alloc_by_making_gap P_ ((struct coding_system *,
879 EMACS_INT, EMACS_INT));
880static unsigned char *alloc_destination P_ ((struct coding_system *,
881 EMACS_INT, unsigned char *));
882static void setup_iso_safe_charsets P_ ((Lisp_Object));
883static unsigned char *encode_designation_at_bol P_ ((struct coding_system *,
884 int *, int *,
885 unsigned char *));
886static int detect_eol P_ ((const unsigned char *,
887 EMACS_INT, enum coding_category));
888static Lisp_Object adjust_coding_eol_type P_ ((struct coding_system *, int));
889static void decode_eol P_ ((struct coding_system *));
890static Lisp_Object get_translation_table P_ ((Lisp_Object, int, int *));
891static Lisp_Object get_translation P_ ((Lisp_Object, int *, int *,
892 int, int *, int *));
893static int produce_chars P_ ((struct coding_system *, Lisp_Object, int));
894static INLINE void produce_composition P_ ((struct coding_system *, int *,
895 EMACS_INT));
896static INLINE void produce_charset P_ ((struct coding_system *, int *,
897 EMACS_INT));
898static void produce_annotation P_ ((struct coding_system *, EMACS_INT));
899static int decode_coding P_ ((struct coding_system *));
900static INLINE int *handle_composition_annotation P_ ((EMACS_INT, EMACS_INT,
901 struct coding_system *,
902 int *, EMACS_INT *));
903static INLINE int *handle_charset_annotation P_ ((EMACS_INT, EMACS_INT,
904 struct coding_system *,
905 int *, EMACS_INT *));
906static void consume_chars P_ ((struct coding_system *, Lisp_Object, int));
907static int encode_coding P_ ((struct coding_system *));
908static Lisp_Object make_conversion_work_buffer P_ ((int));
909static Lisp_Object code_conversion_restore P_ ((Lisp_Object));
910static INLINE int char_encodable_p P_ ((int, Lisp_Object));
911static Lisp_Object make_subsidiaries P_ ((Lisp_Object));
912
913static void
914record_conversion_result (struct coding_system *coding,
915 enum coding_result_code result)
916{
917 coding->result = result;
918 switch (result)
919 {
920 case CODING_RESULT_INSUFFICIENT_SRC:
921 Vlast_code_conversion_error = Qinsufficient_source;
922 break;
923 case CODING_RESULT_INCONSISTENT_EOL:
924 Vlast_code_conversion_error = Qinconsistent_eol;
925 break;
926 case CODING_RESULT_INVALID_SRC:
927 Vlast_code_conversion_error = Qinvalid_source;
928 break;
929 case CODING_RESULT_INTERRUPT:
930 Vlast_code_conversion_error = Qinterrupted;
931 break;
932 case CODING_RESULT_INSUFFICIENT_MEM:
933 Vlast_code_conversion_error = Qinsufficient_memory;
934 break;
935 default:
936 Vlast_code_conversion_error = intern ("Unknown error");
937 }
938}
939
940#define CODING_DECODE_CHAR(coding, src, src_base, src_end, charset, code, c) \
941 do { \
942 charset_map_loaded = 0; \
943 c = DECODE_CHAR (charset, code); \
944 if (charset_map_loaded) \
945 { \
946 const unsigned char *orig = coding->source; \
947 EMACS_INT offset; \
948 \
949 coding_set_source (coding); \
950 offset = coding->source - orig; \
951 src += offset; \
952 src_base += offset; \
953 src_end += offset; \
954 } \
955 } while (0)
956
957
958#define ASSURE_DESTINATION(bytes) \
959 do { \
960 if (dst + (bytes) >= dst_end) \
961 { \
962 int more_bytes = charbuf_end - charbuf + (bytes); \
963 \
964 dst = alloc_destination (coding, more_bytes, dst); \
965 dst_end = coding->destination + coding->dst_bytes; \
966 } \
967 } while (0)
968
969
970
971static void
972coding_set_source (coding)
973 struct coding_system *coding;
974{
975 if (BUFFERP (coding->src_object))
976 {
977 struct buffer *buf = XBUFFER (coding->src_object);
978
979 if (coding->src_pos < 0)
980 coding->source = BUF_GAP_END_ADDR (buf) + coding->src_pos_byte;
981 else
982 coding->source = BUF_BYTE_ADDRESS (buf, coding->src_pos_byte);
983 }
984 else if (STRINGP (coding->src_object))
985 {
986 coding->source = SDATA (coding->src_object) + coding->src_pos_byte;
987 }
988 else
989 /* Otherwise, the source is C string and is never relocated
990 automatically. Thus we don't have to update anything. */
991 ;
992}
993
994static void
995coding_set_destination (coding)
996 struct coding_system *coding;
997{
998 if (BUFFERP (coding->dst_object))
999 {
1000 if (coding->src_pos < 0)
1001 {
1002 coding->destination = BEG_ADDR + coding->dst_pos_byte - 1;
1003 coding->dst_bytes = (GAP_END_ADDR
1004 - (coding->src_bytes - coding->consumed)
1005 - coding->destination);
1006 }
1007 else
1008 {
1009 /* We are sure that coding->dst_pos_byte is before the gap
1010 of the buffer. */
1011 coding->destination = (BUF_BEG_ADDR (XBUFFER (coding->dst_object))
1012 + coding->dst_pos_byte - 1);
1013 coding->dst_bytes = (BUF_GAP_END_ADDR (XBUFFER (coding->dst_object))
1014 - coding->destination);
1015 }
1016 }
1017 else
1018 /* Otherwise, the destination is C string and is never relocated
1019 automatically. Thus we don't have to update anything. */
1020 ;
1021}
1022
1023
1024static void
1025coding_alloc_by_realloc (coding, bytes)
1026 struct coding_system *coding;
1027 EMACS_INT bytes;
1028{
1029 coding->destination = (unsigned char *) xrealloc (coding->destination,
1030 coding->dst_bytes + bytes);
1031 coding->dst_bytes += bytes;
1032}
1033
1034static void
1035coding_alloc_by_making_gap (coding, offset, bytes)
1036 struct coding_system *coding;
1037 EMACS_INT offset, bytes;
1038{
1039 if (BUFFERP (coding->dst_object)
1040 && EQ (coding->src_object, coding->dst_object))
1041 {
1042 EMACS_INT add = offset + (coding->src_bytes - coding->consumed);
1043
1044 GPT += offset, GPT_BYTE += offset;
1045 GAP_SIZE -= add; ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
1046 make_gap (bytes);
1047 GAP_SIZE += add; ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
1048 GPT -= offset, GPT_BYTE -= offset;
1049 }
1050 else
1051 {
1052 Lisp_Object this_buffer;
1053
1054 this_buffer = Fcurrent_buffer ();
1055 set_buffer_internal (XBUFFER (coding->dst_object));
1056 make_gap (bytes);
1057 set_buffer_internal (XBUFFER (this_buffer));
1058 }
1059}
1060
1061
1062static unsigned char *
1063alloc_destination (coding, nbytes, dst)
1064 struct coding_system *coding;
1065 EMACS_INT nbytes;
1066 unsigned char *dst;
1067{
1068 EMACS_INT offset = dst - coding->destination;
1069
1070 if (BUFFERP (coding->dst_object))
1071 coding_alloc_by_making_gap (coding, offset, nbytes);
1072 else
1073 coding_alloc_by_realloc (coding, nbytes);
1074 record_conversion_result (coding, CODING_RESULT_SUCCESS);
1075 coding_set_destination (coding);
1076 dst = coding->destination + offset;
1077 return dst;
1078}
1079
1080/** Macros for annotations. */
1081
1082/* Maximum length of annotation data (sum of annotations for
1083 composition and charset). */
1084#define MAX_ANNOTATION_LENGTH (4 + (MAX_COMPOSITION_COMPONENTS * 2) - 1 + 4)
1085
1086/* An annotation data is stored in the array coding->charbuf in this
1087 format:
1088 [ -LENGTH ANNOTATION_MASK NCHARS ... ]
1089 LENGTH is the number of elements in the annotation.
1090 ANNOTATION_MASK is one of CODING_ANNOTATE_XXX_MASK.
1091 NCHARS is the number of characters in the text annotated.
1092
1093 The format of the following elements depend on ANNOTATION_MASK.
1094
1095 In the case of CODING_ANNOTATE_COMPOSITION_MASK, these elements
1096 follows:
1097 ... METHOD [ COMPOSITION-COMPONENTS ... ]
1098 METHOD is one of enum composition_method.
1099 Optionnal COMPOSITION-COMPONENTS are characters and composition
1100 rules.
1101
1102 In the case of CODING_ANNOTATE_CHARSET_MASK, one element CHARSET-ID
1103 follows. */
1104
1105#define ADD_ANNOTATION_DATA(buf, len, mask, nchars) \
1106 do { \
1107 *(buf)++ = -(len); \
1108 *(buf)++ = (mask); \
1109 *(buf)++ = (nchars); \
1110 coding->annotated = 1; \
1111 } while (0);
1112
1113#define ADD_COMPOSITION_DATA(buf, nchars, method) \
1114 do { \
1115 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_COMPOSITION_MASK, nchars); \
1116 *buf++ = method; \
1117 } while (0)
1118
1119
1120#define ADD_CHARSET_DATA(buf, nchars, id) \
1121 do { \
1122 ADD_ANNOTATION_DATA (buf, 4, CODING_ANNOTATE_CHARSET_MASK, nchars); \
1123 *buf++ = id; \
1124 } while (0)
1125
1126\f
1127/*** 2. Emacs' internal format (emacs-utf-8) ***/
1128
1129
1130
1131\f
1132/*** 3. UTF-8 ***/
1133
1134/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1135 Check if a text is encoded in UTF-8. If it is, return 1, else
1136 return 0. */
1137
1138#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
1139#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
1140#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
1141#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
1142#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
1143#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
1144
1145static int
1146detect_coding_utf_8 (coding, detect_info)
1147 struct coding_system *coding;
1148 struct coding_detection_info *detect_info;
1149{
1150 const unsigned char *src = coding->source, *src_base;
1151 const unsigned char *src_end = coding->source + coding->src_bytes;
1152 int multibytep = coding->src_multibyte;
1153 int consumed_chars = 0;
1154 int found = 0;
1155
1156 detect_info->checked |= CATEGORY_MASK_UTF_8;
1157 /* A coding system of this category is always ASCII compatible. */
1158 src += coding->head_ascii;
1159
1160 while (1)
1161 {
1162 int c, c1, c2, c3, c4;
1163
1164 src_base = src;
1165 ONE_MORE_BYTE (c);
1166 if (c < 0 || UTF_8_1_OCTET_P (c))
1167 continue;
1168 ONE_MORE_BYTE (c1);
1169 if (c1 < 0 || ! UTF_8_EXTRA_OCTET_P (c1))
1170 break;
1171 if (UTF_8_2_OCTET_LEADING_P (c))
1172 {
1173 found = CATEGORY_MASK_UTF_8;
1174 continue;
1175 }
1176 ONE_MORE_BYTE (c2);
1177 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
1178 break;
1179 if (UTF_8_3_OCTET_LEADING_P (c))
1180 {
1181 found = CATEGORY_MASK_UTF_8;
1182 continue;
1183 }
1184 ONE_MORE_BYTE (c3);
1185 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
1186 break;
1187 if (UTF_8_4_OCTET_LEADING_P (c))
1188 {
1189 found = CATEGORY_MASK_UTF_8;
1190 continue;
1191 }
1192 ONE_MORE_BYTE (c4);
1193 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
1194 break;
1195 if (UTF_8_5_OCTET_LEADING_P (c))
1196 {
1197 found = CATEGORY_MASK_UTF_8;
1198 continue;
1199 }
1200 break;
1201 }
1202 detect_info->rejected |= CATEGORY_MASK_UTF_8;
1203 return 0;
1204
1205 no_more_source:
1206 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
1207 {
1208 detect_info->rejected |= CATEGORY_MASK_UTF_8;
1209 return 0;
1210 }
1211 detect_info->found |= found;
1212 return 1;
1213}
1214
1215
1216static void
1217decode_coding_utf_8 (coding)
1218 struct coding_system *coding;
1219{
1220 const unsigned char *src = coding->source + coding->consumed;
1221 const unsigned char *src_end = coding->source + coding->src_bytes;
1222 const unsigned char *src_base;
1223 int *charbuf = coding->charbuf + coding->charbuf_used;
1224 int *charbuf_end = coding->charbuf + coding->charbuf_size;
1225 int consumed_chars = 0, consumed_chars_base;
1226 int multibytep = coding->src_multibyte;
1227 Lisp_Object attr, charset_list;
1228
1229 CODING_GET_INFO (coding, attr, charset_list);
1230
1231 while (1)
1232 {
1233 int c, c1, c2, c3, c4, c5;
1234
1235 src_base = src;
1236 consumed_chars_base = consumed_chars;
1237
1238 if (charbuf >= charbuf_end)
1239 break;
1240
1241 ONE_MORE_BYTE (c1);
1242 if (c1 < 0)
1243 {
1244 c = - c1;
1245 }
1246 else if (UTF_8_1_OCTET_P(c1))
1247 {
1248 c = c1;
1249 }
1250 else
1251 {
1252 ONE_MORE_BYTE (c2);
1253 if (c2 < 0 || ! UTF_8_EXTRA_OCTET_P (c2))
1254 goto invalid_code;
1255 if (UTF_8_2_OCTET_LEADING_P (c1))
1256 {
1257 c = ((c1 & 0x1F) << 6) | (c2 & 0x3F);
1258 /* Reject overlong sequences here and below. Encoders
1259 producing them are incorrect, they can be misleading,
1260 and they mess up read/write invariance. */
1261 if (c < 128)
1262 goto invalid_code;
1263 }
1264 else
1265 {
1266 ONE_MORE_BYTE (c3);
1267 if (c3 < 0 || ! UTF_8_EXTRA_OCTET_P (c3))
1268 goto invalid_code;
1269 if (UTF_8_3_OCTET_LEADING_P (c1))
1270 {
1271 c = (((c1 & 0xF) << 12)
1272 | ((c2 & 0x3F) << 6) | (c3 & 0x3F));
1273 if (c < 0x800
1274 || (c >= 0xd800 && c < 0xe000)) /* surrogates (invalid) */
1275 goto invalid_code;
1276 }
1277 else
1278 {
1279 ONE_MORE_BYTE (c4);
1280 if (c4 < 0 || ! UTF_8_EXTRA_OCTET_P (c4))
1281 goto invalid_code;
1282 if (UTF_8_4_OCTET_LEADING_P (c1))
1283 {
1284 c = (((c1 & 0x7) << 18) | ((c2 & 0x3F) << 12)
1285 | ((c3 & 0x3F) << 6) | (c4 & 0x3F));
1286 if (c < 0x10000)
1287 goto invalid_code;
1288 }
1289 else
1290 {
1291 ONE_MORE_BYTE (c5);
1292 if (c5 < 0 || ! UTF_8_EXTRA_OCTET_P (c5))
1293 goto invalid_code;
1294 if (UTF_8_5_OCTET_LEADING_P (c1))
1295 {
1296 c = (((c1 & 0x3) << 24) | ((c2 & 0x3F) << 18)
1297 | ((c3 & 0x3F) << 12) | ((c4 & 0x3F) << 6)
1298 | (c5 & 0x3F));
1299 if ((c > MAX_CHAR) || (c < 0x200000))
1300 goto invalid_code;
1301 }
1302 else
1303 goto invalid_code;
1304 }
1305 }
1306 }
1307 }
1308
1309 *charbuf++ = c;
1310 continue;
1311
1312 invalid_code:
1313 src = src_base;
1314 consumed_chars = consumed_chars_base;
1315 ONE_MORE_BYTE (c);
1316 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
1317 coding->errors++;
1318 }
1319
1320 no_more_source:
1321 coding->consumed_char += consumed_chars_base;
1322 coding->consumed = src_base - coding->source;
1323 coding->charbuf_used = charbuf - coding->charbuf;
1324}
1325
1326
1327static int
1328encode_coding_utf_8 (coding)
1329 struct coding_system *coding;
1330{
1331 int multibytep = coding->dst_multibyte;
1332 int *charbuf = coding->charbuf;
1333 int *charbuf_end = charbuf + coding->charbuf_used;
1334 unsigned char *dst = coding->destination + coding->produced;
1335 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1336 int produced_chars = 0;
1337 int c;
1338
1339 if (multibytep)
1340 {
1341 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
1342
1343 while (charbuf < charbuf_end)
1344 {
1345 unsigned char str[MAX_MULTIBYTE_LENGTH], *p, *pend = str;
1346
1347 ASSURE_DESTINATION (safe_room);
1348 c = *charbuf++;
1349 if (CHAR_BYTE8_P (c))
1350 {
1351 c = CHAR_TO_BYTE8 (c);
1352 EMIT_ONE_BYTE (c);
1353 }
1354 else
1355 {
1356 CHAR_STRING_ADVANCE (c, pend);
1357 for (p = str; p < pend; p++)
1358 EMIT_ONE_BYTE (*p);
1359 }
1360 }
1361 }
1362 else
1363 {
1364 int safe_room = MAX_MULTIBYTE_LENGTH;
1365
1366 while (charbuf < charbuf_end)
1367 {
1368 ASSURE_DESTINATION (safe_room);
1369 c = *charbuf++;
1370 if (CHAR_BYTE8_P (c))
1371 *dst++ = CHAR_TO_BYTE8 (c);
1372 else
1373 dst += CHAR_STRING (c, dst);
1374 produced_chars++;
1375 }
1376 }
1377 record_conversion_result (coding, CODING_RESULT_SUCCESS);
1378 coding->produced_char += produced_chars;
1379 coding->produced = dst - coding->destination;
1380 return 0;
1381}
1382
1383
1384/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1385 Check if a text is encoded in one of UTF-16 based coding systems.
1386 If it is, return 1, else return 0. */
1387
1388#define UTF_16_HIGH_SURROGATE_P(val) \
1389 (((val) & 0xFC00) == 0xD800)
1390
1391#define UTF_16_LOW_SURROGATE_P(val) \
1392 (((val) & 0xFC00) == 0xDC00)
1393
1394#define UTF_16_INVALID_P(val) \
1395 (((val) == 0xFFFE) \
1396 || ((val) == 0xFFFF) \
1397 || UTF_16_LOW_SURROGATE_P (val))
1398
1399
1400static int
1401detect_coding_utf_16 (coding, detect_info)
1402 struct coding_system *coding;
1403 struct coding_detection_info *detect_info;
1404{
1405 const unsigned char *src = coding->source, *src_base = src;
1406 const unsigned char *src_end = coding->source + coding->src_bytes;
1407 int multibytep = coding->src_multibyte;
1408 int consumed_chars = 0;
1409 int c1, c2;
1410
1411 detect_info->checked |= CATEGORY_MASK_UTF_16;
1412 if (coding->mode & CODING_MODE_LAST_BLOCK
1413 && (coding->src_chars & 1))
1414 {
1415 detect_info->rejected |= CATEGORY_MASK_UTF_16;
1416 return 0;
1417 }
1418
1419 ONE_MORE_BYTE (c1);
1420 ONE_MORE_BYTE (c2);
1421 if ((c1 == 0xFF) && (c2 == 0xFE))
1422 {
1423 detect_info->found |= (CATEGORY_MASK_UTF_16_LE
1424 | CATEGORY_MASK_UTF_16_AUTO);
1425 detect_info->rejected |= (CATEGORY_MASK_UTF_16_BE
1426 | CATEGORY_MASK_UTF_16_BE_NOSIG
1427 | CATEGORY_MASK_UTF_16_LE_NOSIG);
1428 }
1429 else if ((c1 == 0xFE) && (c2 == 0xFF))
1430 {
1431 detect_info->found |= (CATEGORY_MASK_UTF_16_BE
1432 | CATEGORY_MASK_UTF_16_AUTO);
1433 detect_info->rejected |= (CATEGORY_MASK_UTF_16_LE
1434 | CATEGORY_MASK_UTF_16_BE_NOSIG
1435 | CATEGORY_MASK_UTF_16_LE_NOSIG);
1436 }
1437 else if (c1 >= 0 && c2 >= 0)
1438 {
1439 detect_info->rejected
1440 |= (CATEGORY_MASK_UTF_16_BE | CATEGORY_MASK_UTF_16_LE);
1441 }
1442 no_more_source:
1443 return 1;
1444}
1445
1446static void
1447decode_coding_utf_16 (coding)
1448 struct coding_system *coding;
1449{
1450 const unsigned char *src = coding->source + coding->consumed;
1451 const unsigned char *src_end = coding->source + coding->src_bytes;
1452 const unsigned char *src_base;
1453 int *charbuf = coding->charbuf + coding->charbuf_used;
1454 int *charbuf_end = coding->charbuf + coding->charbuf_size;
1455 int consumed_chars = 0, consumed_chars_base;
1456 int multibytep = coding->src_multibyte;
1457 enum utf_16_bom_type bom = CODING_UTF_16_BOM (coding);
1458 enum utf_16_endian_type endian = CODING_UTF_16_ENDIAN (coding);
1459 int surrogate = CODING_UTF_16_SURROGATE (coding);
1460 Lisp_Object attr, charset_list;
1461
1462 CODING_GET_INFO (coding, attr, charset_list);
1463
1464 if (bom == utf_16_with_bom)
1465 {
1466 int c, c1, c2;
1467
1468 src_base = src;
1469 ONE_MORE_BYTE (c1);
1470 ONE_MORE_BYTE (c2);
1471 c = (c1 << 8) | c2;
1472
1473 if (endian == utf_16_big_endian
1474 ? c != 0xFEFF : c != 0xFFFE)
1475 {
1476 /* The first two bytes are not BOM. Treat them as bytes
1477 for a normal character. */
1478 src = src_base;
1479 coding->errors++;
1480 }
1481 CODING_UTF_16_BOM (coding) = utf_16_without_bom;
1482 }
1483 else if (bom == utf_16_detect_bom)
1484 {
1485 /* We have already tried to detect BOM and failed in
1486 detect_coding. */
1487 CODING_UTF_16_BOM (coding) = utf_16_without_bom;
1488 }
1489
1490 while (1)
1491 {
1492 int c, c1, c2;
1493
1494 src_base = src;
1495 consumed_chars_base = consumed_chars;
1496
1497 if (charbuf + 2 >= charbuf_end)
1498 break;
1499
1500 ONE_MORE_BYTE (c1);
1501 if (c1 < 0)
1502 {
1503 *charbuf++ = -c1;
1504 continue;
1505 }
1506 ONE_MORE_BYTE (c2);
1507 if (c2 < 0)
1508 {
1509 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
1510 *charbuf++ = -c2;
1511 continue;
1512 }
1513 c = (endian == utf_16_big_endian
1514 ? ((c1 << 8) | c2) : ((c2 << 8) | c1));
1515 if (surrogate)
1516 {
1517 if (! UTF_16_LOW_SURROGATE_P (c))
1518 {
1519 if (endian == utf_16_big_endian)
1520 c1 = surrogate >> 8, c2 = surrogate & 0xFF;
1521 else
1522 c1 = surrogate & 0xFF, c2 = surrogate >> 8;
1523 *charbuf++ = c1;
1524 *charbuf++ = c2;
1525 coding->errors++;
1526 if (UTF_16_HIGH_SURROGATE_P (c))
1527 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1528 else
1529 *charbuf++ = c;
1530 }
1531 else
1532 {
1533 c = ((surrogate - 0xD800) << 10) | (c - 0xDC00);
1534 CODING_UTF_16_SURROGATE (coding) = surrogate = 0;
1535 *charbuf++ = 0x10000 + c;
1536 }
1537 }
1538 else
1539 {
1540 if (UTF_16_HIGH_SURROGATE_P (c))
1541 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1542 else
1543 *charbuf++ = c;
1544 }
1545 }
1546
1547 no_more_source:
1548 coding->consumed_char += consumed_chars_base;
1549 coding->consumed = src_base - coding->source;
1550 coding->charbuf_used = charbuf - coding->charbuf;
1551}
1552
1553static int
1554encode_coding_utf_16 (coding)
1555 struct coding_system *coding;
1556{
1557 int multibytep = coding->dst_multibyte;
1558 int *charbuf = coding->charbuf;
1559 int *charbuf_end = charbuf + coding->charbuf_used;
1560 unsigned char *dst = coding->destination + coding->produced;
1561 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1562 int safe_room = 8;
1563 enum utf_16_bom_type bom = CODING_UTF_16_BOM (coding);
1564 int big_endian = CODING_UTF_16_ENDIAN (coding) == utf_16_big_endian;
1565 int produced_chars = 0;
1566 Lisp_Object attrs, charset_list;
1567 int c;
1568
1569 CODING_GET_INFO (coding, attrs, charset_list);
1570
1571 if (bom != utf_16_without_bom)
1572 {
1573 ASSURE_DESTINATION (safe_room);
1574 if (big_endian)
1575 EMIT_TWO_BYTES (0xFE, 0xFF);
1576 else
1577 EMIT_TWO_BYTES (0xFF, 0xFE);
1578 CODING_UTF_16_BOM (coding) = utf_16_without_bom;
1579 }
1580
1581 while (charbuf < charbuf_end)
1582 {
1583 ASSURE_DESTINATION (safe_room);
1584 c = *charbuf++;
1585 if (c >= MAX_UNICODE_CHAR)
1586 c = coding->default_char;
1587
1588 if (c < 0x10000)
1589 {
1590 if (big_endian)
1591 EMIT_TWO_BYTES (c >> 8, c & 0xFF);
1592 else
1593 EMIT_TWO_BYTES (c & 0xFF, c >> 8);
1594 }
1595 else
1596 {
1597 int c1, c2;
1598
1599 c -= 0x10000;
1600 c1 = (c >> 10) + 0xD800;
1601 c2 = (c & 0x3FF) + 0xDC00;
1602 if (big_endian)
1603 EMIT_FOUR_BYTES (c1 >> 8, c1 & 0xFF, c2 >> 8, c2 & 0xFF);
1604 else
1605 EMIT_FOUR_BYTES (c1 & 0xFF, c1 >> 8, c2 & 0xFF, c2 >> 8);
1606 }
1607 }
1608 record_conversion_result (coding, CODING_RESULT_SUCCESS);
1609 coding->produced = dst - coding->destination;
1610 coding->produced_char += produced_chars;
1611 return 0;
1612}
1613
1614\f
1615/*** 6. Old Emacs' internal format (emacs-mule) ***/
1616
1617/* Emacs' internal format for representation of multiple character
1618 sets is a kind of multi-byte encoding, i.e. characters are
1619 represented by variable-length sequences of one-byte codes.
1620
1621 ASCII characters and control characters (e.g. `tab', `newline') are
1622 represented by one-byte sequences which are their ASCII codes, in
1623 the range 0x00 through 0x7F.
1624
1625 8-bit characters of the range 0x80..0x9F are represented by
1626 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
1627 code + 0x20).
1628
1629 8-bit characters of the range 0xA0..0xFF are represented by
1630 one-byte sequences which are their 8-bit code.
1631
1632 The other characters are represented by a sequence of `base
1633 leading-code', optional `extended leading-code', and one or two
1634 `position-code's. The length of the sequence is determined by the
1635 base leading-code. Leading-code takes the range 0x81 through 0x9D,
1636 whereas extended leading-code and position-code take the range 0xA0
1637 through 0xFF. See `charset.h' for more details about leading-code
1638 and position-code.
1639
1640 --- CODE RANGE of Emacs' internal format ---
1641 character set range
1642 ------------- -----
1643 ascii 0x00..0x7F
1644 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
1645 eight-bit-graphic 0xA0..0xBF
1646 ELSE 0x81..0x9D + [0xA0..0xFF]+
1647 ---------------------------------------------
1648
1649 As this is the internal character representation, the format is
1650 usually not used externally (i.e. in a file or in a data sent to a
1651 process). But, it is possible to have a text externally in this
1652 format (i.e. by encoding by the coding system `emacs-mule').
1653
1654 In that case, a sequence of one-byte codes has a slightly different
1655 form.
1656
1657 At first, all characters in eight-bit-control are represented by
1658 one-byte sequences which are their 8-bit code.
1659
1660 Next, character composition data are represented by the byte
1661 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
1662 where,
1663 METHOD is 0xF0 plus one of composition method (enum
1664 composition_method),
1665
1666 BYTES is 0xA0 plus a byte length of this composition data,
1667
1668 CHARS is 0x20 plus a number of characters composed by this
1669 data,
1670
1671 COMPONENTs are characters of multibye form or composition
1672 rules encoded by two-byte of ASCII codes.
1673
1674 In addition, for backward compatibility, the following formats are
1675 also recognized as composition data on decoding.
1676
1677 0x80 MSEQ ...
1678 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
1679
1680 Here,
1681 MSEQ is a multibyte form but in these special format:
1682 ASCII: 0xA0 ASCII_CODE+0x80,
1683 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
1684 RULE is a one byte code of the range 0xA0..0xF0 that
1685 represents a composition rule.
1686 */
1687
1688char emacs_mule_bytes[256];
1689
1690int
1691emacs_mule_char (coding, src, nbytes, nchars, id)
1692 struct coding_system *coding;
1693 const unsigned char *src;
1694 int *nbytes, *nchars, *id;
1695{
1696 const unsigned char *src_end = coding->source + coding->src_bytes;
1697 const unsigned char *src_base = src;
1698 int multibytep = coding->src_multibyte;
1699 struct charset *charset;
1700 unsigned code;
1701 int c;
1702 int consumed_chars = 0;
1703
1704 ONE_MORE_BYTE (c);
1705 if (c < 0)
1706 {
1707 c = -c;
1708 charset = emacs_mule_charset[0];
1709 }
1710 else
1711 {
1712 if (c >= 0xA0)
1713 {
1714 /* Old style component character of a compostion. */
1715 if (c == 0xA0)
1716 {
1717 ONE_MORE_BYTE (c);
1718 c -= 0x80;
1719 }
1720 else
1721 c -= 0x20;
1722 }
1723
1724 switch (emacs_mule_bytes[c])
1725 {
1726 case 2:
1727 if (! (charset = emacs_mule_charset[c]))
1728 goto invalid_code;
1729 ONE_MORE_BYTE (c);
1730 if (c < 0xA0)
1731 goto invalid_code;
1732 code = c & 0x7F;
1733 break;
1734
1735 case 3:
1736 if (c == EMACS_MULE_LEADING_CODE_PRIVATE_11
1737 || c == EMACS_MULE_LEADING_CODE_PRIVATE_12)
1738 {
1739 ONE_MORE_BYTE (c);
1740 if (c < 0xA0 || ! (charset = emacs_mule_charset[c]))
1741 goto invalid_code;
1742 ONE_MORE_BYTE (c);
1743 if (c < 0xA0)
1744 goto invalid_code;
1745 code = c & 0x7F;
1746 }
1747 else
1748 {
1749 if (! (charset = emacs_mule_charset[c]))
1750 goto invalid_code;
1751 ONE_MORE_BYTE (c);
1752 if (c < 0xA0)
1753 goto invalid_code;
1754 code = (c & 0x7F) << 8;
1755 ONE_MORE_BYTE (c);
1756 if (c < 0xA0)
1757 goto invalid_code;
1758 code |= c & 0x7F;
1759 }
1760 break;
1761
1762 case 4:
1763 ONE_MORE_BYTE (c);
1764 if (c < 0 || ! (charset = emacs_mule_charset[c]))
1765 goto invalid_code;
1766 ONE_MORE_BYTE (c);
1767 if (c < 0xA0)
1768 goto invalid_code;
1769 code = (c & 0x7F) << 8;
1770 ONE_MORE_BYTE (c);
1771 if (c < 0xA0)
1772 goto invalid_code;
1773 code |= c & 0x7F;
1774 break;
1775
1776 case 1:
1777 code = c;
1778 charset = CHARSET_FROM_ID (ASCII_BYTE_P (code)
1779 ? charset_ascii : charset_eight_bit);
1780 break;
1781
1782 default:
1783 abort ();
1784 }
1785 c = DECODE_CHAR (charset, code);
1786 if (c < 0)
1787 goto invalid_code;
1788 }
1789 *nbytes = src - src_base;
1790 *nchars = consumed_chars;
1791 if (id)
1792 *id = charset->id;
1793 return c;
1794
1795 no_more_source:
1796 return -2;
1797
1798 invalid_code:
1799 return -1;
1800}
1801
1802
1803/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1804 Check if a text is encoded in `emacs-mule'. If it is, return 1,
1805 else return 0. */
1806
1807static int
1808detect_coding_emacs_mule (coding, detect_info)
1809 struct coding_system *coding;
1810 struct coding_detection_info *detect_info;
1811{
1812 const unsigned char *src = coding->source, *src_base;
1813 const unsigned char *src_end = coding->source + coding->src_bytes;
1814 int multibytep = coding->src_multibyte;
1815 int consumed_chars = 0;
1816 int c;
1817 int found = 0;
1818
1819 detect_info->checked |= CATEGORY_MASK_EMACS_MULE;
1820 /* A coding system of this category is always ASCII compatible. */
1821 src += coding->head_ascii;
1822
1823 while (1)
1824 {
1825 src_base = src;
1826 ONE_MORE_BYTE (c);
1827 if (c < 0)
1828 continue;
1829 if (c == 0x80)
1830 {
1831 /* Perhaps the start of composite character. We simple skip
1832 it because analyzing it is too heavy for detecting. But,
1833 at least, we check that the composite character
1834 constitues of more than 4 bytes. */
1835 const unsigned char *src_base;
1836
1837 repeat:
1838 src_base = src;
1839 do
1840 {
1841 ONE_MORE_BYTE (c);
1842 }
1843 while (c >= 0xA0);
1844
1845 if (src - src_base <= 4)
1846 break;
1847 found = CATEGORY_MASK_EMACS_MULE;
1848 if (c == 0x80)
1849 goto repeat;
1850 }
1851
1852 if (c < 0x80)
1853 {
1854 if (c < 0x20
1855 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO))
1856 break;
1857 }
1858 else
1859 {
1860 int more_bytes = emacs_mule_bytes[*src_base] - 1;
1861
1862 while (more_bytes > 0)
1863 {
1864 ONE_MORE_BYTE (c);
1865 if (c < 0xA0)
1866 {
1867 src--; /* Unread the last byte. */
1868 break;
1869 }
1870 more_bytes--;
1871 }
1872 if (more_bytes != 0)
1873 break;
1874 found = CATEGORY_MASK_EMACS_MULE;
1875 }
1876 }
1877 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
1878 return 0;
1879
1880 no_more_source:
1881 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
1882 {
1883 detect_info->rejected |= CATEGORY_MASK_EMACS_MULE;
1884 return 0;
1885 }
1886 detect_info->found |= found;
1887 return 1;
1888}
1889
1890
1891/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1892
1893/* Decode a character represented as a component of composition
1894 sequence of Emacs 20/21 style at SRC. Set C to that character and
1895 update SRC to the head of next character (or an encoded composition
1896 rule). If SRC doesn't points a composition component, set C to -1.
1897 If SRC points an invalid byte sequence, global exit by a return
1898 value 0. */
1899
1900#define DECODE_EMACS_MULE_COMPOSITION_CHAR(buf) \
1901 if (1) \
1902 { \
1903 int c; \
1904 int nbytes, nchars; \
1905 \
1906 if (src == src_end) \
1907 break; \
1908 c = emacs_mule_char (coding, src, &nbytes, &nchars, NULL);\
1909 if (c < 0) \
1910 { \
1911 if (c == -2) \
1912 break; \
1913 goto invalid_code; \
1914 } \
1915 *buf++ = c; \
1916 src += nbytes; \
1917 consumed_chars += nchars; \
1918 } \
1919 else
1920
1921
1922/* Decode a composition rule represented as a component of composition
1923 sequence of Emacs 20 style at SRC. Store the decoded rule in *BUF,
1924 and increment BUF. If SRC points an invalid byte sequence, set C
1925 to -1. */
1926
1927#define DECODE_EMACS_MULE_COMPOSITION_RULE_20(buf) \
1928 do { \
1929 int c, gref, nref; \
1930 \
1931 if (src >= src_end) \
1932 goto invalid_code; \
1933 ONE_MORE_BYTE_NO_CHECK (c); \
1934 c -= 0xA0; \
1935 if (c < 0 || c >= 81) \
1936 goto invalid_code; \
1937 \
1938 gref = c / 9, nref = c % 9; \
1939 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
1940 } while (0)
1941
1942
1943/* Decode a composition rule represented as a component of composition
1944 sequence of Emacs 21 style at SRC. Store the decoded rule in *BUF,
1945 and increment BUF. If SRC points an invalid byte sequence, set C
1946 to -1. */
1947
1948#define DECODE_EMACS_MULE_COMPOSITION_RULE_21(buf) \
1949 do { \
1950 int gref, nref; \
1951 \
1952 if (src + 1>= src_end) \
1953 goto invalid_code; \
1954 ONE_MORE_BYTE_NO_CHECK (gref); \
1955 gref -= 0x20; \
1956 ONE_MORE_BYTE_NO_CHECK (nref); \
1957 nref -= 0x20; \
1958 if (gref < 0 || gref >= 81 \
1959 || nref < 0 || nref >= 81) \
1960 goto invalid_code; \
1961 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
1962 } while (0)
1963
1964
1965#define DECODE_EMACS_MULE_21_COMPOSITION(c) \
1966 do { \
1967 /* Emacs 21 style format. The first three bytes at SRC are \
1968 (METHOD - 0xF2), (BYTES - 0xA0), (CHARS - 0xA0), where BYTES is \
1969 the byte length of this composition information, CHARS is the \
1970 number of characters composed by this composition. */ \
1971 enum composition_method method = c - 0xF2; \
1972 int *charbuf_base = charbuf; \
1973 int consumed_chars_limit; \
1974 int nbytes, nchars; \
1975 \
1976 ONE_MORE_BYTE (c); \
1977 if (c < 0) \
1978 goto invalid_code; \
1979 nbytes = c - 0xA0; \
1980 if (nbytes < 3) \
1981 goto invalid_code; \
1982 ONE_MORE_BYTE (c); \
1983 if (c < 0) \
1984 goto invalid_code; \
1985 nchars = c - 0xA0; \
1986 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
1987 consumed_chars_limit = consumed_chars_base + nbytes; \
1988 if (method != COMPOSITION_RELATIVE) \
1989 { \
1990 int i = 0; \
1991 while (consumed_chars < consumed_chars_limit) \
1992 { \
1993 if (i % 2 && method != COMPOSITION_WITH_ALTCHARS) \
1994 DECODE_EMACS_MULE_COMPOSITION_RULE_21 (charbuf); \
1995 else \
1996 DECODE_EMACS_MULE_COMPOSITION_CHAR (charbuf); \
1997 i++; \
1998 } \
1999 if (consumed_chars < consumed_chars_limit) \
2000 goto invalid_code; \
2001 charbuf_base[0] -= i; \
2002 } \
2003 } while (0)
2004
2005
2006#define DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION(c) \
2007 do { \
2008 /* Emacs 20 style format for relative composition. */ \
2009 /* Store multibyte form of characters to be composed. */ \
2010 enum composition_method method = COMPOSITION_RELATIVE; \
2011 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
2012 int *buf = components; \
2013 int i, j; \
2014 \
2015 src = src_base; \
2016 ONE_MORE_BYTE (c); /* skip 0x80 */ \
2017 for (i = 0; *src >= 0xA0 && i < MAX_COMPOSITION_COMPONENTS; i++) \
2018 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2019 if (i < 2) \
2020 goto invalid_code; \
2021 ADD_COMPOSITION_DATA (charbuf, i, method); \
2022 for (j = 0; j < i; j++) \
2023 *charbuf++ = components[j]; \
2024 } while (0)
2025
2026
2027#define DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION(c) \
2028 do { \
2029 /* Emacs 20 style format for rule-base composition. */ \
2030 /* Store multibyte form of characters to be composed. */ \
2031 enum composition_method method = COMPOSITION_WITH_RULE; \
2032 int *charbuf_base = charbuf; \
2033 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
2034 int *buf = components; \
2035 int i, j; \
2036 \
2037 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2038 for (i = 1; i < MAX_COMPOSITION_COMPONENTS; i++) \
2039 { \
2040 if (*src < 0xA0) \
2041 break; \
2042 DECODE_EMACS_MULE_COMPOSITION_RULE_20 (buf); \
2043 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
2044 } \
2045 if (i <= 1 || (buf - components) % 2 == 0) \
2046 goto invalid_code; \
2047 if (charbuf + i + (i / 2) + 1 >= charbuf_end) \
2048 goto no_more_source; \
2049 ADD_COMPOSITION_DATA (charbuf, i, method); \
2050 i = i * 2 - 1; \
2051 for (j = 0; j < i; j++) \
2052 *charbuf++ = components[j]; \
2053 charbuf_base[0] -= i; \
2054 for (j = 0; j < i; j += 2) \
2055 *charbuf++ = components[j]; \
2056 } while (0)
2057
2058
2059static void
2060decode_coding_emacs_mule (coding)
2061 struct coding_system *coding;
2062{
2063 const unsigned char *src = coding->source + coding->consumed;
2064 const unsigned char *src_end = coding->source + coding->src_bytes;
2065 const unsigned char *src_base;
2066 int *charbuf = coding->charbuf + coding->charbuf_used;
2067 int *charbuf_end
2068 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
2069 int consumed_chars = 0, consumed_chars_base;
2070 int multibytep = coding->src_multibyte;
2071 Lisp_Object attrs, charset_list;
2072 int char_offset = coding->produced_char;
2073 int last_offset = char_offset;
2074 int last_id = charset_ascii;
2075
2076 CODING_GET_INFO (coding, attrs, charset_list);
2077
2078 while (1)
2079 {
2080 int c;
2081
2082 src_base = src;
2083 consumed_chars_base = consumed_chars;
2084
2085 if (charbuf >= charbuf_end)
2086 break;
2087
2088 ONE_MORE_BYTE (c);
2089 if (c < 0)
2090 {
2091 *charbuf++ = -c;
2092 char_offset++;
2093 }
2094 else if (c < 0x80)
2095 {
2096 *charbuf++ = c;
2097 char_offset++;
2098 }
2099 else if (c == 0x80)
2100 {
2101 ONE_MORE_BYTE (c);
2102 if (c < 0)
2103 goto invalid_code;
2104 if (c - 0xF2 >= COMPOSITION_RELATIVE
2105 && c - 0xF2 <= COMPOSITION_WITH_RULE_ALTCHARS)
2106 DECODE_EMACS_MULE_21_COMPOSITION (c);
2107 else if (c < 0xC0)
2108 DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION (c);
2109 else if (c == 0xFF)
2110 DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION (c);
2111 else
2112 goto invalid_code;
2113 }
2114 else if (c < 0xA0 && emacs_mule_bytes[c] > 1)
2115 {
2116 int nbytes, nchars;
2117 int id;
2118
2119 src = src_base;
2120 consumed_chars = consumed_chars_base;
2121 c = emacs_mule_char (coding, src, &nbytes, &nchars, &id);
2122 if (c < 0)
2123 {
2124 if (c == -2)
2125 break;
2126 goto invalid_code;
2127 }
2128 if (last_id != id)
2129 {
2130 if (last_id != charset_ascii)
2131 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
2132 last_id = id;
2133 last_offset = char_offset;
2134 }
2135 *charbuf++ = c;
2136 src += nbytes;
2137 consumed_chars += nchars;
2138 char_offset++;
2139 }
2140 else
2141 goto invalid_code;
2142 continue;
2143
2144 invalid_code:
2145 src = src_base;
2146 consumed_chars = consumed_chars_base;
2147 ONE_MORE_BYTE (c);
2148 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
2149 char_offset++;
2150 coding->errors++;
2151 }
2152
2153 no_more_source:
2154 if (last_id != charset_ascii)
2155 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
2156 coding->consumed_char += consumed_chars_base;
2157 coding->consumed = src_base - coding->source;
2158 coding->charbuf_used = charbuf - coding->charbuf;
2159}
2160
2161
2162#define EMACS_MULE_LEADING_CODES(id, codes) \
2163 do { \
2164 if (id < 0xA0) \
2165 codes[0] = id, codes[1] = 0; \
2166 else if (id < 0xE0) \
2167 codes[0] = 0x9A, codes[1] = id; \
2168 else if (id < 0xF0) \
2169 codes[0] = 0x9B, codes[1] = id; \
2170 else if (id < 0xF5) \
2171 codes[0] = 0x9C, codes[1] = id; \
2172 else \
2173 codes[0] = 0x9D, codes[1] = id; \
2174 } while (0);
2175
2176
2177static int
2178encode_coding_emacs_mule (coding)
2179 struct coding_system *coding;
2180{
2181 int multibytep = coding->dst_multibyte;
2182 int *charbuf = coding->charbuf;
2183 int *charbuf_end = charbuf + coding->charbuf_used;
2184 unsigned char *dst = coding->destination + coding->produced;
2185 unsigned char *dst_end = coding->destination + coding->dst_bytes;
2186 int safe_room = 8;
2187 int produced_chars = 0;
2188 Lisp_Object attrs, charset_list;
2189 int c;
2190 int preferred_charset_id = -1;
2191
2192 CODING_GET_INFO (coding, attrs, charset_list);
2193 if (! EQ (charset_list, Vemacs_mule_charset_list))
2194 {
2195 CODING_ATTR_CHARSET_LIST (attrs)
2196 = charset_list = Vemacs_mule_charset_list;
2197 }
2198
2199 while (charbuf < charbuf_end)
2200 {
2201 ASSURE_DESTINATION (safe_room);
2202 c = *charbuf++;
2203
2204 if (c < 0)
2205 {
2206 /* Handle an annotation. */
2207 switch (*charbuf)
2208 {
2209 case CODING_ANNOTATE_COMPOSITION_MASK:
2210 /* Not yet implemented. */
2211 break;
2212 case CODING_ANNOTATE_CHARSET_MASK:
2213 preferred_charset_id = charbuf[3];
2214 if (preferred_charset_id >= 0
2215 && NILP (Fmemq (make_number (preferred_charset_id),
2216 charset_list)))
2217 preferred_charset_id = -1;
2218 break;
2219 default:
2220 abort ();
2221 }
2222 charbuf += -c - 1;
2223 continue;
2224 }
2225
2226 if (ASCII_CHAR_P (c))
2227 EMIT_ONE_ASCII_BYTE (c);
2228 else if (CHAR_BYTE8_P (c))
2229 {
2230 c = CHAR_TO_BYTE8 (c);
2231 EMIT_ONE_BYTE (c);
2232 }
2233 else
2234 {
2235 struct charset *charset;
2236 unsigned code;
2237 int dimension;
2238 int emacs_mule_id;
2239 unsigned char leading_codes[2];
2240
2241 if (preferred_charset_id >= 0)
2242 {
2243 charset = CHARSET_FROM_ID (preferred_charset_id);
2244 if (! CHAR_CHARSET_P (c, charset))
2245 charset = char_charset (c, charset_list, NULL);
2246 }
2247 else
2248 charset = char_charset (c, charset_list, &code);
2249 if (! charset)
2250 {
2251 c = coding->default_char;
2252 if (ASCII_CHAR_P (c))
2253 {
2254 EMIT_ONE_ASCII_BYTE (c);
2255 continue;
2256 }
2257 charset = char_charset (c, charset_list, &code);
2258 }
2259 dimension = CHARSET_DIMENSION (charset);
2260 emacs_mule_id = CHARSET_EMACS_MULE_ID (charset);
2261 EMACS_MULE_LEADING_CODES (emacs_mule_id, leading_codes);
2262 EMIT_ONE_BYTE (leading_codes[0]);
2263 if (leading_codes[1])
2264 EMIT_ONE_BYTE (leading_codes[1]);
2265 if (dimension == 1)
2266 EMIT_ONE_BYTE (code | 0x80);
2267 else
2268 {
2269 code |= 0x8080;
2270 EMIT_ONE_BYTE (code >> 8);
2271 EMIT_ONE_BYTE (code & 0xFF);
2272 }
2273 }
2274 }
2275 record_conversion_result (coding, CODING_RESULT_SUCCESS);
2276 coding->produced_char += produced_chars;
2277 coding->produced = dst - coding->destination;
2278 return 0;
2279}
2280
2281\f
2282/*** 7. ISO2022 handlers ***/
2283
2284/* The following note describes the coding system ISO2022 briefly.
2285 Since the intention of this note is to help understand the
2286 functions in this file, some parts are NOT ACCURATE or are OVERLY
2287 SIMPLIFIED. For thorough understanding, please refer to the
2288 original document of ISO2022. This is equivalent to the standard
2289 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
2290
2291 ISO2022 provides many mechanisms to encode several character sets
2292 in 7-bit and 8-bit environments. For 7-bit environments, all text
2293 is encoded using bytes less than 128. This may make the encoded
2294 text a little bit longer, but the text passes more easily through
2295 several types of gateway, some of which strip off the MSB (Most
2296 Significant Bit).
2297
2298 There are two kinds of character sets: control character sets and
2299 graphic character sets. The former contain control characters such
2300 as `newline' and `escape' to provide control functions (control
2301 functions are also provided by escape sequences). The latter
2302 contain graphic characters such as 'A' and '-'. Emacs recognizes
2303 two control character sets and many graphic character sets.
2304
2305 Graphic character sets are classified into one of the following
2306 four classes, according to the number of bytes (DIMENSION) and
2307 number of characters in one dimension (CHARS) of the set:
2308 - DIMENSION1_CHARS94
2309 - DIMENSION1_CHARS96
2310 - DIMENSION2_CHARS94
2311 - DIMENSION2_CHARS96
2312
2313 In addition, each character set is assigned an identification tag,
2314 unique for each set, called the "final character" (denoted as <F>
2315 hereafter). The <F> of each character set is decided by ECMA(*)
2316 when it is registered in ISO. The code range of <F> is 0x30..0x7F
2317 (0x30..0x3F are for private use only).
2318
2319 Note (*): ECMA = European Computer Manufacturers Association
2320
2321 Here are examples of graphic character sets [NAME(<F>)]:
2322 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
2323 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
2324 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
2325 o DIMENSION2_CHARS96 -- none for the moment
2326
2327 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
2328 C0 [0x00..0x1F] -- control character plane 0
2329 GL [0x20..0x7F] -- graphic character plane 0
2330 C1 [0x80..0x9F] -- control character plane 1
2331 GR [0xA0..0xFF] -- graphic character plane 1
2332
2333 A control character set is directly designated and invoked to C0 or
2334 C1 by an escape sequence. The most common case is that:
2335 - ISO646's control character set is designated/invoked to C0, and
2336 - ISO6429's control character set is designated/invoked to C1,
2337 and usually these designations/invocations are omitted in encoded
2338 text. In a 7-bit environment, only C0 can be used, and a control
2339 character for C1 is encoded by an appropriate escape sequence to
2340 fit into the environment. All control characters for C1 are
2341 defined to have corresponding escape sequences.
2342
2343 A graphic character set is at first designated to one of four
2344 graphic registers (G0 through G3), then these graphic registers are
2345 invoked to GL or GR. These designations and invocations can be
2346 done independently. The most common case is that G0 is invoked to
2347 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
2348 these invocations and designations are omitted in encoded text.
2349 In a 7-bit environment, only GL can be used.
2350
2351 When a graphic character set of CHARS94 is invoked to GL, codes
2352 0x20 and 0x7F of the GL area work as control characters SPACE and
2353 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
2354 be used.
2355
2356 There are two ways of invocation: locking-shift and single-shift.
2357 With locking-shift, the invocation lasts until the next different
2358 invocation, whereas with single-shift, the invocation affects the
2359 following character only and doesn't affect the locking-shift
2360 state. Invocations are done by the following control characters or
2361 escape sequences:
2362
2363 ----------------------------------------------------------------------
2364 abbrev function cntrl escape seq description
2365 ----------------------------------------------------------------------
2366 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
2367 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
2368 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
2369 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
2370 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
2371 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
2372 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
2373 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
2374 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
2375 ----------------------------------------------------------------------
2376 (*) These are not used by any known coding system.
2377
2378 Control characters for these functions are defined by macros
2379 ISO_CODE_XXX in `coding.h'.
2380
2381 Designations are done by the following escape sequences:
2382 ----------------------------------------------------------------------
2383 escape sequence description
2384 ----------------------------------------------------------------------
2385 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
2386 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
2387 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
2388 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
2389 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
2390 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
2391 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
2392 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
2393 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
2394 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
2395 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
2396 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
2397 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
2398 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
2399 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
2400 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
2401 ----------------------------------------------------------------------
2402
2403 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
2404 of dimension 1, chars 94, and final character <F>, etc...
2405
2406 Note (*): Although these designations are not allowed in ISO2022,
2407 Emacs accepts them on decoding, and produces them on encoding
2408 CHARS96 character sets in a coding system which is characterized as
2409 7-bit environment, non-locking-shift, and non-single-shift.
2410
2411 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
2412 '(' must be omitted. We refer to this as "short-form" hereafter.
2413
2414 Now you may notice that there are a lot of ways of encoding the
2415 same multilingual text in ISO2022. Actually, there exist many
2416 coding systems such as Compound Text (used in X11's inter client
2417 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
2418 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
2419 localized platforms), and all of these are variants of ISO2022.
2420
2421 In addition to the above, Emacs handles two more kinds of escape
2422 sequences: ISO6429's direction specification and Emacs' private
2423 sequence for specifying character composition.
2424
2425 ISO6429's direction specification takes the following form:
2426 o CSI ']' -- end of the current direction
2427 o CSI '0' ']' -- end of the current direction
2428 o CSI '1' ']' -- start of left-to-right text
2429 o CSI '2' ']' -- start of right-to-left text
2430 The control character CSI (0x9B: control sequence introducer) is
2431 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
2432
2433 Character composition specification takes the following form:
2434 o ESC '0' -- start relative composition
2435 o ESC '1' -- end composition
2436 o ESC '2' -- start rule-base composition (*)
2437 o ESC '3' -- start relative composition with alternate chars (**)
2438 o ESC '4' -- start rule-base composition with alternate chars (**)
2439 Since these are not standard escape sequences of any ISO standard,
2440 the use of them with these meanings is restricted to Emacs only.
2441
2442 (*) This form is used only in Emacs 20.7 and older versions,
2443 but newer versions can safely decode it.
2444 (**) This form is used only in Emacs 21.1 and newer versions,
2445 and older versions can't decode it.
2446
2447 Here's a list of example usages of these composition escape
2448 sequences (categorized by `enum composition_method').
2449
2450 COMPOSITION_RELATIVE:
2451 ESC 0 CHAR [ CHAR ] ESC 1
2452 COMPOSITION_WITH_RULE:
2453 ESC 2 CHAR [ RULE CHAR ] ESC 1
2454 COMPOSITION_WITH_ALTCHARS:
2455 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
2456 COMPOSITION_WITH_RULE_ALTCHARS:
2457 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
2458
2459enum iso_code_class_type iso_code_class[256];
2460
2461#define SAFE_CHARSET_P(coding, id) \
2462 ((id) <= (coding)->max_charset_id \
2463 && (coding)->safe_charsets[id] >= 0)
2464
2465
2466#define SHIFT_OUT_OK(category) \
2467 (CODING_ISO_INITIAL (&coding_categories[category], 1) >= 0)
2468
2469static void
2470setup_iso_safe_charsets (attrs)
2471 Lisp_Object attrs;
2472{
2473 Lisp_Object charset_list, safe_charsets;
2474 Lisp_Object request;
2475 Lisp_Object reg_usage;
2476 Lisp_Object tail;
2477 int reg94, reg96;
2478 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
2479 int max_charset_id;
2480
2481 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2482 if ((flags & CODING_ISO_FLAG_FULL_SUPPORT)
2483 && ! EQ (charset_list, Viso_2022_charset_list))
2484 {
2485 CODING_ATTR_CHARSET_LIST (attrs)
2486 = charset_list = Viso_2022_charset_list;
2487 ASET (attrs, coding_attr_safe_charsets, Qnil);
2488 }
2489
2490 if (STRINGP (AREF (attrs, coding_attr_safe_charsets)))
2491 return;
2492
2493 max_charset_id = 0;
2494 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2495 {
2496 int id = XINT (XCAR (tail));
2497 if (max_charset_id < id)
2498 max_charset_id = id;
2499 }
2500
2501 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
2502 make_number (255));
2503 request = AREF (attrs, coding_attr_iso_request);
2504 reg_usage = AREF (attrs, coding_attr_iso_usage);
2505 reg94 = XINT (XCAR (reg_usage));
2506 reg96 = XINT (XCDR (reg_usage));
2507
2508 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2509 {
2510 Lisp_Object id;
2511 Lisp_Object reg;
2512 struct charset *charset;
2513
2514 id = XCAR (tail);
2515 charset = CHARSET_FROM_ID (XINT (id));
2516 reg = Fcdr (Fassq (id, request));
2517 if (! NILP (reg))
2518 SSET (safe_charsets, XINT (id), XINT (reg));
2519 else if (charset->iso_chars_96)
2520 {
2521 if (reg96 < 4)
2522 SSET (safe_charsets, XINT (id), reg96);
2523 }
2524 else
2525 {
2526 if (reg94 < 4)
2527 SSET (safe_charsets, XINT (id), reg94);
2528 }
2529 }
2530 ASET (attrs, coding_attr_safe_charsets, safe_charsets);
2531}
2532
2533
2534/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2535 Check if a text is encoded in one of ISO-2022 based codig systems.
2536 If it is, return 1, else return 0. */
2537
2538static int
2539detect_coding_iso_2022 (coding, detect_info)
2540 struct coding_system *coding;
2541 struct coding_detection_info *detect_info;
2542{
2543 const unsigned char *src = coding->source, *src_base = src;
2544 const unsigned char *src_end = coding->source + coding->src_bytes;
2545 int multibytep = coding->src_multibyte;
2546 int single_shifting = 0;
2547 int id;
2548 int c, c1;
2549 int consumed_chars = 0;
2550 int i;
2551 int rejected = 0;
2552 int found = 0;
2553
2554 detect_info->checked |= CATEGORY_MASK_ISO;
2555
2556 for (i = coding_category_iso_7; i <= coding_category_iso_8_else; i++)
2557 {
2558 struct coding_system *this = &(coding_categories[i]);
2559 Lisp_Object attrs, val;
2560
2561 attrs = CODING_ID_ATTRS (this->id);
2562 if (CODING_ISO_FLAGS (this) & CODING_ISO_FLAG_FULL_SUPPORT
2563 && ! EQ (CODING_ATTR_SAFE_CHARSETS (attrs), Viso_2022_charset_list))
2564 setup_iso_safe_charsets (attrs);
2565 val = CODING_ATTR_SAFE_CHARSETS (attrs);
2566 this->max_charset_id = SCHARS (val) - 1;
2567 this->safe_charsets = (char *) SDATA (val);
2568 }
2569
2570 /* A coding system of this category is always ASCII compatible. */
2571 src += coding->head_ascii;
2572
2573 while (rejected != CATEGORY_MASK_ISO)
2574 {
2575 src_base = src;
2576 ONE_MORE_BYTE (c);
2577 switch (c)
2578 {
2579 case ISO_CODE_ESC:
2580 if (inhibit_iso_escape_detection)
2581 break;
2582 single_shifting = 0;
2583 ONE_MORE_BYTE (c);
2584 if (c >= '(' && c <= '/')
2585 {
2586 /* Designation sequence for a charset of dimension 1. */
2587 ONE_MORE_BYTE (c1);
2588 if (c1 < ' ' || c1 >= 0x80
2589 || (id = iso_charset_table[0][c >= ','][c1]) < 0)
2590 /* Invalid designation sequence. Just ignore. */
2591 break;
2592 }
2593 else if (c == '$')
2594 {
2595 /* Designation sequence for a charset of dimension 2. */
2596 ONE_MORE_BYTE (c);
2597 if (c >= '@' && c <= 'B')
2598 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
2599 id = iso_charset_table[1][0][c];
2600 else if (c >= '(' && c <= '/')
2601 {
2602 ONE_MORE_BYTE (c1);
2603 if (c1 < ' ' || c1 >= 0x80
2604 || (id = iso_charset_table[1][c >= ','][c1]) < 0)
2605 /* Invalid designation sequence. Just ignore. */
2606 break;
2607 }
2608 else
2609 /* Invalid designation sequence. Just ignore it. */
2610 break;
2611 }
2612 else if (c == 'N' || c == 'O')
2613 {
2614 /* ESC <Fe> for SS2 or SS3. */
2615 single_shifting = 1;
2616 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
2617 break;
2618 }
2619 else if (c >= '0' && c <= '4')
2620 {
2621 /* ESC <Fp> for start/end composition. */
2622 found |= CATEGORY_MASK_ISO;
2623 break;
2624 }
2625 else
2626 {
2627 /* Invalid escape sequence. Just ignore it. */
2628 break;
2629 }
2630
2631 /* We found a valid designation sequence for CHARSET. */
2632 rejected |= CATEGORY_MASK_ISO_8BIT;
2633 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7],
2634 id))
2635 found |= CATEGORY_MASK_ISO_7;
2636 else
2637 rejected |= CATEGORY_MASK_ISO_7;
2638 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_tight],
2639 id))
2640 found |= CATEGORY_MASK_ISO_7_TIGHT;
2641 else
2642 rejected |= CATEGORY_MASK_ISO_7_TIGHT;
2643 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_else],
2644 id))
2645 found |= CATEGORY_MASK_ISO_7_ELSE;
2646 else
2647 rejected |= CATEGORY_MASK_ISO_7_ELSE;
2648 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_8_else],
2649 id))
2650 found |= CATEGORY_MASK_ISO_8_ELSE;
2651 else
2652 rejected |= CATEGORY_MASK_ISO_8_ELSE;
2653 break;
2654
2655 case ISO_CODE_SO:
2656 case ISO_CODE_SI:
2657 /* Locking shift out/in. */
2658 if (inhibit_iso_escape_detection)
2659 break;
2660 single_shifting = 0;
2661 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_8BIT;
2662 break;
2663
2664 case ISO_CODE_CSI:
2665 /* Control sequence introducer. */
2666 single_shifting = 0;
2667 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2668 found |= CATEGORY_MASK_ISO_8_ELSE;
2669 goto check_extra_latin;
2670
2671 case ISO_CODE_SS2:
2672 case ISO_CODE_SS3:
2673 /* Single shift. */
2674 if (inhibit_iso_escape_detection)
2675 break;
2676 single_shifting = 0;
2677 rejected |= CATEGORY_MASK_ISO_7BIT;
2678 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
2679 & CODING_ISO_FLAG_SINGLE_SHIFT)
2680 found |= CATEGORY_MASK_ISO_8_1, single_shifting = 1;
2681 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
2682 & CODING_ISO_FLAG_SINGLE_SHIFT)
2683 found |= CATEGORY_MASK_ISO_8_2, single_shifting = 1;
2684 if (single_shifting)
2685 break;
2686 goto check_extra_latin;
2687
2688 default:
2689 if (c < 0)
2690 continue;
2691 if (c < 0x80)
2692 {
2693 single_shifting = 0;
2694 break;
2695 }
2696 if (c >= 0xA0)
2697 {
2698 rejected |= CATEGORY_MASK_ISO_7BIT | CATEGORY_MASK_ISO_7_ELSE;
2699 found |= CATEGORY_MASK_ISO_8_1;
2700 /* Check the length of succeeding codes of the range
2701 0xA0..0FF. If the byte length is even, we include
2702 CATEGORY_MASK_ISO_8_2 in `found'. We can check this
2703 only when we are not single shifting. */
2704 if (! single_shifting
2705 && ! (rejected & CATEGORY_MASK_ISO_8_2))
2706 {
2707 int i = 1;
2708 while (src < src_end)
2709 {
2710 ONE_MORE_BYTE (c);
2711 if (c < 0xA0)
2712 break;
2713 i++;
2714 }
2715
2716 if (i & 1 && src < src_end)
2717 rejected |= CATEGORY_MASK_ISO_8_2;
2718 else
2719 found |= CATEGORY_MASK_ISO_8_2;
2720 }
2721 break;
2722 }
2723 check_extra_latin:
2724 single_shifting = 0;
2725 if (! VECTORP (Vlatin_extra_code_table)
2726 || NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
2727 {
2728 rejected = CATEGORY_MASK_ISO;
2729 break;
2730 }
2731 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
2732 & CODING_ISO_FLAG_LATIN_EXTRA)
2733 found |= CATEGORY_MASK_ISO_8_1;
2734 else
2735 rejected |= CATEGORY_MASK_ISO_8_1;
2736 rejected |= CATEGORY_MASK_ISO_8_2;
2737 }
2738 }
2739 detect_info->rejected |= CATEGORY_MASK_ISO;
2740 return 0;
2741
2742 no_more_source:
2743 detect_info->rejected |= rejected;
2744 detect_info->found |= (found & ~rejected);
2745 return 1;
2746}
2747
2748
2749/* Set designation state into CODING. Set CHARS_96 to -1 if the
2750 escape sequence should be kept. */
2751#define DECODE_DESIGNATION(reg, dim, chars_96, final) \
2752 do { \
2753 int id, prev; \
2754 \
2755 if (final < '0' || final >= 128 \
2756 || ((id = ISO_CHARSET_TABLE (dim, chars_96, final)) < 0) \
2757 || !SAFE_CHARSET_P (coding, id)) \
2758 { \
2759 CODING_ISO_DESIGNATION (coding, reg) = -2; \
2760 chars_96 = -1; \
2761 break; \
2762 } \
2763 prev = CODING_ISO_DESIGNATION (coding, reg); \
2764 if (id == charset_jisx0201_roman) \
2765 { \
2766 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
2767 id = charset_ascii; \
2768 } \
2769 else if (id == charset_jisx0208_1978) \
2770 { \
2771 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
2772 id = charset_jisx0208; \
2773 } \
2774 CODING_ISO_DESIGNATION (coding, reg) = id; \
2775 /* If there was an invalid designation to REG previously, and this \
2776 designation is ASCII to REG, we should keep this designation \
2777 sequence. */ \
2778 if (prev == -2 && id == charset_ascii) \
2779 chars_96 = -1; \
2780 } while (0)
2781
2782
2783#define MAYBE_FINISH_COMPOSITION() \
2784 do { \
2785 int i; \
2786 if (composition_state == COMPOSING_NO) \
2787 break; \
2788 /* It is assured that we have enough room for producing \
2789 characters stored in the table `components'. */ \
2790 if (charbuf + component_idx > charbuf_end) \
2791 goto no_more_source; \
2792 composition_state = COMPOSING_NO; \
2793 if (method == COMPOSITION_RELATIVE \
2794 || method == COMPOSITION_WITH_ALTCHARS) \
2795 { \
2796 for (i = 0; i < component_idx; i++) \
2797 *charbuf++ = components[i]; \
2798 char_offset += component_idx; \
2799 } \
2800 else \
2801 { \
2802 for (i = 0; i < component_idx; i += 2) \
2803 *charbuf++ = components[i]; \
2804 char_offset += (component_idx / 2) + 1; \
2805 } \
2806 } while (0)
2807
2808
2809/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
2810 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
2811 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
2812 ESC 3 : altchar composition : ESC 3 CHAR ... ESC 0 CHAR ... ESC 1
2813 ESC 4 : alt&rule composition : ESC 4 CHAR RULE ... CHAR ESC 0 CHAR ... ESC 1
2814 */
2815
2816#define DECODE_COMPOSITION_START(c1) \
2817 do { \
2818 if (c1 == '0' \
2819 && composition_state == COMPOSING_COMPONENT_RULE) \
2820 { \
2821 component_len = component_idx; \
2822 composition_state = COMPOSING_CHAR; \
2823 } \
2824 else \
2825 { \
2826 const unsigned char *p; \
2827 \
2828 MAYBE_FINISH_COMPOSITION (); \
2829 if (charbuf + MAX_COMPOSITION_COMPONENTS > charbuf_end) \
2830 goto no_more_source; \
2831 for (p = src; p < src_end - 1; p++) \
2832 if (*p == ISO_CODE_ESC && p[1] == '1') \
2833 break; \
2834 if (p == src_end - 1) \
2835 { \
2836 /* The current composition doesn't end in the current \
2837 source. */ \
2838 record_conversion_result \
2839 (coding, CODING_RESULT_INSUFFICIENT_SRC); \
2840 goto no_more_source; \
2841 } \
2842 \
2843 /* This is surely the start of a composition. */ \
2844 method = (c1 == '0' ? COMPOSITION_RELATIVE \
2845 : c1 == '2' ? COMPOSITION_WITH_RULE \
2846 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
2847 : COMPOSITION_WITH_RULE_ALTCHARS); \
2848 composition_state = (c1 <= '2' ? COMPOSING_CHAR \
2849 : COMPOSING_COMPONENT_CHAR); \
2850 component_idx = component_len = 0; \
2851 } \
2852 } while (0)
2853
2854
2855/* Handle compositoin end sequence ESC 1. */
2856
2857#define DECODE_COMPOSITION_END() \
2858 do { \
2859 int nchars = (component_len > 0 ? component_idx - component_len \
2860 : method == COMPOSITION_RELATIVE ? component_idx \
2861 : (component_idx + 1) / 2); \
2862 int i; \
2863 int *saved_charbuf = charbuf; \
2864 \
2865 ADD_COMPOSITION_DATA (charbuf, nchars, method); \
2866 if (method != COMPOSITION_RELATIVE) \
2867 { \
2868 if (component_len == 0) \
2869 for (i = 0; i < component_idx; i++) \
2870 *charbuf++ = components[i]; \
2871 else \
2872 for (i = 0; i < component_len; i++) \
2873 *charbuf++ = components[i]; \
2874 *saved_charbuf = saved_charbuf - charbuf; \
2875 } \
2876 if (method == COMPOSITION_WITH_RULE) \
2877 for (i = 0; i < component_idx; i += 2, char_offset++) \
2878 *charbuf++ = components[i]; \
2879 else \
2880 for (i = component_len; i < component_idx; i++, char_offset++) \
2881 *charbuf++ = components[i]; \
2882 coding->annotated = 1; \
2883 composition_state = COMPOSING_NO; \
2884 } while (0)
2885
2886
2887/* Decode a composition rule from the byte C1 (and maybe one more byte
2888 from SRC) and store one encoded composition rule in
2889 coding->cmp_data. */
2890
2891#define DECODE_COMPOSITION_RULE(c1) \
2892 do { \
2893 (c1) -= 32; \
2894 if (c1 < 81) /* old format (before ver.21) */ \
2895 { \
2896 int gref = (c1) / 9; \
2897 int nref = (c1) % 9; \
2898 if (gref == 4) gref = 10; \
2899 if (nref == 4) nref = 10; \
2900 c1 = COMPOSITION_ENCODE_RULE (gref, nref); \
2901 } \
2902 else if (c1 < 93) /* new format (after ver.21) */ \
2903 { \
2904 ONE_MORE_BYTE (c2); \
2905 c1 = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
2906 } \
2907 else \
2908 c1 = 0; \
2909 } while (0)
2910
2911
2912/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
2913
2914static void
2915decode_coding_iso_2022 (coding)
2916 struct coding_system *coding;
2917{
2918 const unsigned char *src = coding->source + coding->consumed;
2919 const unsigned char *src_end = coding->source + coding->src_bytes;
2920 const unsigned char *src_base;
2921 int *charbuf = coding->charbuf + coding->charbuf_used;
2922 int *charbuf_end
2923 = coding->charbuf + coding->charbuf_size - 4 - MAX_ANNOTATION_LENGTH;
2924 int consumed_chars = 0, consumed_chars_base;
2925 int multibytep = coding->src_multibyte;
2926 /* Charsets invoked to graphic plane 0 and 1 respectively. */
2927 int charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
2928 int charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
2929 int charset_id_2, charset_id_3;
2930 struct charset *charset;
2931 int c;
2932 /* For handling composition sequence. */
2933#define COMPOSING_NO 0
2934#define COMPOSING_CHAR 1
2935#define COMPOSING_RULE 2
2936#define COMPOSING_COMPONENT_CHAR 3
2937#define COMPOSING_COMPONENT_RULE 4
2938
2939 int composition_state = COMPOSING_NO;
2940 enum composition_method method;
2941 int components[MAX_COMPOSITION_COMPONENTS * 2 + 1];
2942 int component_idx;
2943 int component_len;
2944 Lisp_Object attrs, charset_list;
2945 int char_offset = coding->produced_char;
2946 int last_offset = char_offset;
2947 int last_id = charset_ascii;
2948
2949 CODING_GET_INFO (coding, attrs, charset_list);
2950 setup_iso_safe_charsets (attrs);
2951 /* Charset list may have been changed. */
2952 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2953 coding->safe_charsets = (char *) SDATA (CODING_ATTR_SAFE_CHARSETS(attrs));
2954
2955 while (1)
2956 {
2957 int c1, c2;
2958
2959 src_base = src;
2960 consumed_chars_base = consumed_chars;
2961
2962 if (charbuf >= charbuf_end)
2963 break;
2964
2965 ONE_MORE_BYTE (c1);
2966 if (c1 < 0)
2967 goto invalid_code;
2968
2969 /* We produce at most one character. */
2970 switch (iso_code_class [c1])
2971 {
2972 case ISO_0x20_or_0x7F:
2973 if (composition_state != COMPOSING_NO)
2974 {
2975 if (composition_state == COMPOSING_RULE
2976 || composition_state == COMPOSING_COMPONENT_RULE)
2977 {
2978 DECODE_COMPOSITION_RULE (c1);
2979 components[component_idx++] = c1;
2980 composition_state--;
2981 continue;
2982 }
2983 }
2984 if (charset_id_0 < 0
2985 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_0)))
2986 /* This is SPACE or DEL. */
2987 charset = CHARSET_FROM_ID (charset_ascii);
2988 else
2989 charset = CHARSET_FROM_ID (charset_id_0);
2990 break;
2991
2992 case ISO_graphic_plane_0:
2993 if (composition_state != COMPOSING_NO)
2994 {
2995 if (composition_state == COMPOSING_RULE
2996 || composition_state == COMPOSING_COMPONENT_RULE)
2997 {
2998 DECODE_COMPOSITION_RULE (c1);
2999 components[component_idx++] = c1;
3000 composition_state--;
3001 continue;
3002 }
3003 }
3004 if (charset_id_0 < 0)
3005 charset = CHARSET_FROM_ID (charset_ascii);
3006 else
3007 charset = CHARSET_FROM_ID (charset_id_0);
3008 break;
3009
3010 case ISO_0xA0_or_0xFF:
3011 if (charset_id_1 < 0
3012 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_1))
3013 || CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS)
3014 goto invalid_code;
3015 /* This is a graphic character, we fall down ... */
3016
3017 case ISO_graphic_plane_1:
3018 if (charset_id_1 < 0)
3019 goto invalid_code;
3020 charset = CHARSET_FROM_ID (charset_id_1);
3021 break;
3022
3023 case ISO_control_0:
3024 MAYBE_FINISH_COMPOSITION ();
3025 charset = CHARSET_FROM_ID (charset_ascii);
3026 break;
3027
3028 case ISO_control_1:
3029 MAYBE_FINISH_COMPOSITION ();
3030 goto invalid_code;
3031
3032 case ISO_shift_out:
3033 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3034 || CODING_ISO_DESIGNATION (coding, 1) < 0)
3035 goto invalid_code;
3036 CODING_ISO_INVOCATION (coding, 0) = 1;
3037 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3038 continue;
3039
3040 case ISO_shift_in:
3041 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT))
3042 goto invalid_code;
3043 CODING_ISO_INVOCATION (coding, 0) = 0;
3044 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3045 continue;
3046
3047 case ISO_single_shift_2_7:
3048 case ISO_single_shift_2:
3049 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3050 goto invalid_code;
3051 /* SS2 is handled as an escape sequence of ESC 'N' */
3052 c1 = 'N';
3053 goto label_escape_sequence;
3054
3055 case ISO_single_shift_3:
3056 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
3057 goto invalid_code;
3058 /* SS2 is handled as an escape sequence of ESC 'O' */
3059 c1 = 'O';
3060 goto label_escape_sequence;
3061
3062 case ISO_control_sequence_introducer:
3063 /* CSI is handled as an escape sequence of ESC '[' ... */
3064 c1 = '[';
3065 goto label_escape_sequence;
3066
3067 case ISO_escape:
3068 ONE_MORE_BYTE (c1);
3069 label_escape_sequence:
3070 /* Escape sequences handled here are invocation,
3071 designation, direction specification, and character
3072 composition specification. */
3073 switch (c1)
3074 {
3075 case '&': /* revision of following character set */
3076 ONE_MORE_BYTE (c1);
3077 if (!(c1 >= '@' && c1 <= '~'))
3078 goto invalid_code;
3079 ONE_MORE_BYTE (c1);
3080 if (c1 != ISO_CODE_ESC)
3081 goto invalid_code;
3082 ONE_MORE_BYTE (c1);
3083 goto label_escape_sequence;
3084
3085 case '$': /* designation of 2-byte character set */
3086 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3087 goto invalid_code;
3088 {
3089 int reg, chars96;
3090
3091 ONE_MORE_BYTE (c1);
3092 if (c1 >= '@' && c1 <= 'B')
3093 { /* designation of JISX0208.1978, GB2312.1980,
3094 or JISX0208.1980 */
3095 reg = 0, chars96 = 0;
3096 }
3097 else if (c1 >= 0x28 && c1 <= 0x2B)
3098 { /* designation of DIMENSION2_CHARS94 character set */
3099 reg = c1 - 0x28, chars96 = 0;
3100 ONE_MORE_BYTE (c1);
3101 }
3102 else if (c1 >= 0x2C && c1 <= 0x2F)
3103 { /* designation of DIMENSION2_CHARS96 character set */
3104 reg = c1 - 0x2C, chars96 = 1;
3105 ONE_MORE_BYTE (c1);
3106 }
3107 else
3108 goto invalid_code;
3109 DECODE_DESIGNATION (reg, 2, chars96, c1);
3110 /* We must update these variables now. */
3111 if (reg == 0)
3112 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3113 else if (reg == 1)
3114 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3115 if (chars96 < 0)
3116 goto invalid_code;
3117 }
3118 continue;
3119
3120 case 'n': /* invocation of locking-shift-2 */
3121 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3122 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3123 goto invalid_code;
3124 CODING_ISO_INVOCATION (coding, 0) = 2;
3125 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3126 continue;
3127
3128 case 'o': /* invocation of locking-shift-3 */
3129 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
3130 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3131 goto invalid_code;
3132 CODING_ISO_INVOCATION (coding, 0) = 3;
3133 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3134 continue;
3135
3136 case 'N': /* invocation of single-shift-2 */
3137 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3138 || CODING_ISO_DESIGNATION (coding, 2) < 0)
3139 goto invalid_code;
3140 charset_id_2 = CODING_ISO_DESIGNATION (coding, 2);
3141 if (charset_id_2 < 0)
3142 charset = CHARSET_FROM_ID (charset_ascii);
3143 else
3144 charset = CHARSET_FROM_ID (charset_id_2);
3145 ONE_MORE_BYTE (c1);
3146 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
3147 goto invalid_code;
3148 break;
3149
3150 case 'O': /* invocation of single-shift-3 */
3151 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3152 || CODING_ISO_DESIGNATION (coding, 3) < 0)
3153 goto invalid_code;
3154 charset_id_3 = CODING_ISO_DESIGNATION (coding, 3);
3155 if (charset_id_3 < 0)
3156 charset = CHARSET_FROM_ID (charset_ascii);
3157 else
3158 charset = CHARSET_FROM_ID (charset_id_3);
3159 ONE_MORE_BYTE (c1);
3160 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
3161 goto invalid_code;
3162 break;
3163
3164 case '0': case '2': case '3': case '4': /* start composition */
3165 if (! (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK))
3166 goto invalid_code;
3167 DECODE_COMPOSITION_START (c1);
3168 continue;
3169
3170 case '1': /* end composition */
3171 if (composition_state == COMPOSING_NO)
3172 goto invalid_code;
3173 DECODE_COMPOSITION_END ();
3174 continue;
3175
3176 case '[': /* specification of direction */
3177 if (! CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DIRECTION)
3178 goto invalid_code;
3179 /* For the moment, nested direction is not supported.
3180 So, `coding->mode & CODING_MODE_DIRECTION' zero means
3181 left-to-right, and nozero means right-to-left. */
3182 ONE_MORE_BYTE (c1);
3183 switch (c1)
3184 {
3185 case ']': /* end of the current direction */
3186 coding->mode &= ~CODING_MODE_DIRECTION;
3187
3188 case '0': /* end of the current direction */
3189 case '1': /* start of left-to-right direction */
3190 ONE_MORE_BYTE (c1);
3191 if (c1 == ']')
3192 coding->mode &= ~CODING_MODE_DIRECTION;
3193 else
3194 goto invalid_code;
3195 break;
3196
3197 case '2': /* start of right-to-left direction */
3198 ONE_MORE_BYTE (c1);
3199 if (c1 == ']')
3200 coding->mode |= CODING_MODE_DIRECTION;
3201 else
3202 goto invalid_code;
3203 break;
3204
3205 default:
3206 goto invalid_code;
3207 }
3208 continue;
3209
3210 case '%':
3211 ONE_MORE_BYTE (c1);
3212 if (c1 == '/')
3213 {
3214 /* CTEXT extended segment:
3215 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
3216 We keep these bytes as is for the moment.
3217 They may be decoded by post-read-conversion. */
3218 int dim, M, L;
3219 int size;
3220
3221 ONE_MORE_BYTE (dim);
3222 ONE_MORE_BYTE (M);
3223 ONE_MORE_BYTE (L);
3224 size = ((M - 128) * 128) + (L - 128);
3225 if (charbuf + 8 + size > charbuf_end)
3226 goto break_loop;
3227 *charbuf++ = ISO_CODE_ESC;
3228 *charbuf++ = '%';
3229 *charbuf++ = '/';
3230 *charbuf++ = dim;
3231 *charbuf++ = BYTE8_TO_CHAR (M);
3232 *charbuf++ = BYTE8_TO_CHAR (L);
3233 while (size-- > 0)
3234 {
3235 ONE_MORE_BYTE (c1);
3236 *charbuf++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
3237 }
3238 }
3239 else if (c1 == 'G')
3240 {
3241 /* XFree86 extension for embedding UTF-8 in CTEXT:
3242 ESC % G --UTF-8-BYTES-- ESC % @
3243 We keep these bytes as is for the moment.
3244 They may be decoded by post-read-conversion. */
3245 int *p = charbuf;
3246
3247 if (p + 6 > charbuf_end)
3248 goto break_loop;
3249 *p++ = ISO_CODE_ESC;
3250 *p++ = '%';
3251 *p++ = 'G';
3252 while (p < charbuf_end)
3253 {
3254 ONE_MORE_BYTE (c1);
3255 if (c1 == ISO_CODE_ESC
3256 && src + 1 < src_end
3257 && src[0] == '%'
3258 && src[1] == '@')
3259 {
3260 src += 2;
3261 break;
3262 }
3263 *p++ = ASCII_BYTE_P (c1) ? c1 : BYTE8_TO_CHAR (c1);
3264 }
3265 if (p + 3 > charbuf_end)
3266 goto break_loop;
3267 *p++ = ISO_CODE_ESC;
3268 *p++ = '%';
3269 *p++ = '@';
3270 charbuf = p;
3271 }
3272 else
3273 goto invalid_code;
3274 continue;
3275 break;
3276
3277 default:
3278 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
3279 goto invalid_code;
3280 {
3281 int reg, chars96;
3282
3283 if (c1 >= 0x28 && c1 <= 0x2B)
3284 { /* designation of DIMENSION1_CHARS94 character set */
3285 reg = c1 - 0x28, chars96 = 0;
3286 ONE_MORE_BYTE (c1);
3287 }
3288 else if (c1 >= 0x2C && c1 <= 0x2F)
3289 { /* designation of DIMENSION1_CHARS96 character set */
3290 reg = c1 - 0x2C, chars96 = 1;
3291 ONE_MORE_BYTE (c1);
3292 }
3293 else
3294 goto invalid_code;
3295 DECODE_DESIGNATION (reg, 1, chars96, c1);
3296 /* We must update these variables now. */
3297 if (reg == 0)
3298 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
3299 else if (reg == 1)
3300 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
3301 if (chars96 < 0)
3302 goto invalid_code;
3303 }
3304 continue;
3305 }
3306 }
3307
3308 if (charset->id != charset_ascii
3309 && last_id != charset->id)
3310 {
3311 if (last_id != charset_ascii)
3312 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
3313 last_id = charset->id;
3314 last_offset = char_offset;
3315 }
3316
3317 /* Now we know CHARSET and 1st position code C1 of a character.
3318 Produce a decoded character while getting 2nd position code
3319 C2 if necessary. */
3320 c1 &= 0x7F;
3321 if (CHARSET_DIMENSION (charset) > 1)
3322 {
3323 ONE_MORE_BYTE (c2);
3324 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
3325 /* C2 is not in a valid range. */
3326 goto invalid_code;
3327 c1 = (c1 << 8) | (c2 & 0x7F);
3328 if (CHARSET_DIMENSION (charset) > 2)
3329 {
3330 ONE_MORE_BYTE (c2);
3331 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
3332 /* C2 is not in a valid range. */
3333 goto invalid_code;
3334 c1 = (c1 << 8) | (c2 & 0x7F);
3335 }
3336 }
3337
3338 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c1, c);
3339 if (c < 0)
3340 {
3341 MAYBE_FINISH_COMPOSITION ();
3342 for (; src_base < src; src_base++, char_offset++)
3343 {
3344 if (ASCII_BYTE_P (*src_base))
3345 *charbuf++ = *src_base;
3346 else
3347 *charbuf++ = BYTE8_TO_CHAR (*src_base);
3348 }
3349 }
3350 else if (composition_state == COMPOSING_NO)
3351 {
3352 *charbuf++ = c;
3353 char_offset++;
3354 }
3355 else
3356 {
3357 components[component_idx++] = c;
3358 if (method == COMPOSITION_WITH_RULE
3359 || (method == COMPOSITION_WITH_RULE_ALTCHARS
3360 && composition_state == COMPOSING_COMPONENT_CHAR))
3361 composition_state++;
3362 }
3363 continue;
3364
3365 invalid_code:
3366 MAYBE_FINISH_COMPOSITION ();
3367 src = src_base;
3368 consumed_chars = consumed_chars_base;
3369 ONE_MORE_BYTE (c);
3370 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
3371 char_offset++;
3372 coding->errors++;
3373 continue;
3374
3375 break_loop:
3376 break;
3377 }
3378
3379 no_more_source:
3380 if (last_id != charset_ascii)
3381 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
3382 coding->consumed_char += consumed_chars_base;
3383 coding->consumed = src_base - coding->source;
3384 coding->charbuf_used = charbuf - coding->charbuf;
3385}
3386
3387
3388/* ISO2022 encoding stuff. */
3389
3390/*
3391 It is not enough to say just "ISO2022" on encoding, we have to
3392 specify more details. In Emacs, each coding system of ISO2022
3393 variant has the following specifications:
3394 1. Initial designation to G0 thru G3.
3395 2. Allows short-form designation?
3396 3. ASCII should be designated to G0 before control characters?
3397 4. ASCII should be designated to G0 at end of line?
3398 5. 7-bit environment or 8-bit environment?
3399 6. Use locking-shift?
3400 7. Use Single-shift?
3401 And the following two are only for Japanese:
3402 8. Use ASCII in place of JIS0201-1976-Roman?
3403 9. Use JISX0208-1983 in place of JISX0208-1978?
3404 These specifications are encoded in CODING_ISO_FLAGS (coding) as flag bits
3405 defined by macros CODING_ISO_FLAG_XXX. See `coding.h' for more
3406 details.
3407*/
3408
3409/* Produce codes (escape sequence) for designating CHARSET to graphic
3410 register REG at DST, and increment DST. If <final-char> of CHARSET is
3411 '@', 'A', or 'B' and the coding system CODING allows, produce
3412 designation sequence of short-form. */
3413
3414#define ENCODE_DESIGNATION(charset, reg, coding) \
3415 do { \
3416 unsigned char final_char = CHARSET_ISO_FINAL (charset); \
3417 char *intermediate_char_94 = "()*+"; \
3418 char *intermediate_char_96 = ",-./"; \
3419 int revision = -1; \
3420 int c; \
3421 \
3422 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_REVISION) \
3423 revision = CHARSET_ISO_REVISION (charset); \
3424 \
3425 if (revision >= 0) \
3426 { \
3427 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '&'); \
3428 EMIT_ONE_BYTE ('@' + revision); \
3429 } \
3430 EMIT_ONE_ASCII_BYTE (ISO_CODE_ESC); \
3431 if (CHARSET_DIMENSION (charset) == 1) \
3432 { \
3433 if (! CHARSET_ISO_CHARS_96 (charset)) \
3434 c = intermediate_char_94[reg]; \
3435 else \
3436 c = intermediate_char_96[reg]; \
3437 EMIT_ONE_ASCII_BYTE (c); \
3438 } \
3439 else \
3440 { \
3441 EMIT_ONE_ASCII_BYTE ('$'); \
3442 if (! CHARSET_ISO_CHARS_96 (charset)) \
3443 { \
3444 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LONG_FORM \
3445 || reg != 0 \
3446 || final_char < '@' || final_char > 'B') \
3447 EMIT_ONE_ASCII_BYTE (intermediate_char_94[reg]); \
3448 } \
3449 else \
3450 EMIT_ONE_ASCII_BYTE (intermediate_char_96[reg]); \
3451 } \
3452 EMIT_ONE_ASCII_BYTE (final_char); \
3453 \
3454 CODING_ISO_DESIGNATION (coding, reg) = CHARSET_ID (charset); \
3455 } while (0)
3456
3457
3458/* The following two macros produce codes (control character or escape
3459 sequence) for ISO2022 single-shift functions (single-shift-2 and
3460 single-shift-3). */
3461
3462#define ENCODE_SINGLE_SHIFT_2 \
3463 do { \
3464 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3465 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'N'); \
3466 else \
3467 EMIT_ONE_BYTE (ISO_CODE_SS2); \
3468 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
3469 } while (0)
3470
3471
3472#define ENCODE_SINGLE_SHIFT_3 \
3473 do { \
3474 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3475 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'O'); \
3476 else \
3477 EMIT_ONE_BYTE (ISO_CODE_SS3); \
3478 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
3479 } while (0)
3480
3481
3482/* The following four macros produce codes (control character or
3483 escape sequence) for ISO2022 locking-shift functions (shift-in,
3484 shift-out, locking-shift-2, and locking-shift-3). */
3485
3486#define ENCODE_SHIFT_IN \
3487 do { \
3488 EMIT_ONE_ASCII_BYTE (ISO_CODE_SI); \
3489 CODING_ISO_INVOCATION (coding, 0) = 0; \
3490 } while (0)
3491
3492
3493#define ENCODE_SHIFT_OUT \
3494 do { \
3495 EMIT_ONE_ASCII_BYTE (ISO_CODE_SO); \
3496 CODING_ISO_INVOCATION (coding, 0) = 1; \
3497 } while (0)
3498
3499
3500#define ENCODE_LOCKING_SHIFT_2 \
3501 do { \
3502 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3503 CODING_ISO_INVOCATION (coding, 0) = 2; \
3504 } while (0)
3505
3506
3507#define ENCODE_LOCKING_SHIFT_3 \
3508 do { \
3509 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3510 CODING_ISO_INVOCATION (coding, 0) = 3; \
3511 } while (0)
3512
3513
3514/* Produce codes for a DIMENSION1 character whose character set is
3515 CHARSET and whose position-code is C1. Designation and invocation
3516 sequences are also produced in advance if necessary. */
3517
3518#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
3519 do { \
3520 int id = CHARSET_ID (charset); \
3521 \
3522 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_ROMAN) \
3523 && id == charset_ascii) \
3524 { \
3525 id = charset_jisx0201_roman; \
3526 charset = CHARSET_FROM_ID (id); \
3527 } \
3528 \
3529 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
3530 { \
3531 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3532 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
3533 else \
3534 EMIT_ONE_BYTE (c1 | 0x80); \
3535 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
3536 break; \
3537 } \
3538 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
3539 { \
3540 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
3541 break; \
3542 } \
3543 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
3544 { \
3545 EMIT_ONE_BYTE (c1 | 0x80); \
3546 break; \
3547 } \
3548 else \
3549 /* Since CHARSET is not yet invoked to any graphic planes, we \
3550 must invoke it, or, at first, designate it to some graphic \
3551 register. Then repeat the loop to actually produce the \
3552 character. */ \
3553 dst = encode_invocation_designation (charset, coding, dst, \
3554 &produced_chars); \
3555 } while (1)
3556
3557
3558/* Produce codes for a DIMENSION2 character whose character set is
3559 CHARSET and whose position-codes are C1 and C2. Designation and
3560 invocation codes are also produced in advance if necessary. */
3561
3562#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
3563 do { \
3564 int id = CHARSET_ID (charset); \
3565 \
3566 if ((CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_USE_OLDJIS) \
3567 && id == charset_jisx0208) \
3568 { \
3569 id = charset_jisx0208_1978; \
3570 charset = CHARSET_FROM_ID (id); \
3571 } \
3572 \
3573 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
3574 { \
3575 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
3576 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
3577 else \
3578 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
3579 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
3580 break; \
3581 } \
3582 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
3583 { \
3584 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
3585 break; \
3586 } \
3587 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
3588 { \
3589 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
3590 break; \
3591 } \
3592 else \
3593 /* Since CHARSET is not yet invoked to any graphic planes, we \
3594 must invoke it, or, at first, designate it to some graphic \
3595 register. Then repeat the loop to actually produce the \
3596 character. */ \
3597 dst = encode_invocation_designation (charset, coding, dst, \
3598 &produced_chars); \
3599 } while (1)
3600
3601
3602#define ENCODE_ISO_CHARACTER(charset, c) \
3603 do { \
3604 int code = ENCODE_CHAR ((charset),(c)); \
3605 \
3606 if (CHARSET_DIMENSION (charset) == 1) \
3607 ENCODE_ISO_CHARACTER_DIMENSION1 ((charset), code); \
3608 else \
3609 ENCODE_ISO_CHARACTER_DIMENSION2 ((charset), code >> 8, code & 0xFF); \
3610 } while (0)
3611
3612
3613/* Produce designation and invocation codes at a place pointed by DST
3614 to use CHARSET. The element `spec.iso_2022' of *CODING is updated.
3615 Return new DST. */
3616
3617unsigned char *
3618encode_invocation_designation (charset, coding, dst, p_nchars)
3619 struct charset *charset;
3620 struct coding_system *coding;
3621 unsigned char *dst;
3622 int *p_nchars;
3623{
3624 int multibytep = coding->dst_multibyte;
3625 int produced_chars = *p_nchars;
3626 int reg; /* graphic register number */
3627 int id = CHARSET_ID (charset);
3628
3629 /* At first, check designations. */
3630 for (reg = 0; reg < 4; reg++)
3631 if (id == CODING_ISO_DESIGNATION (coding, reg))
3632 break;
3633
3634 if (reg >= 4)
3635 {
3636 /* CHARSET is not yet designated to any graphic registers. */
3637 /* At first check the requested designation. */
3638 reg = CODING_ISO_REQUEST (coding, id);
3639 if (reg < 0)
3640 /* Since CHARSET requests no special designation, designate it
3641 to graphic register 0. */
3642 reg = 0;
3643
3644 ENCODE_DESIGNATION (charset, reg, coding);
3645 }
3646
3647 if (CODING_ISO_INVOCATION (coding, 0) != reg
3648 && CODING_ISO_INVOCATION (coding, 1) != reg)
3649 {
3650 /* Since the graphic register REG is not invoked to any graphic
3651 planes, invoke it to graphic plane 0. */
3652 switch (reg)
3653 {
3654 case 0: /* graphic register 0 */
3655 ENCODE_SHIFT_IN;
3656 break;
3657
3658 case 1: /* graphic register 1 */
3659 ENCODE_SHIFT_OUT;
3660 break;
3661
3662 case 2: /* graphic register 2 */
3663 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3664 ENCODE_SINGLE_SHIFT_2;
3665 else
3666 ENCODE_LOCKING_SHIFT_2;
3667 break;
3668
3669 case 3: /* graphic register 3 */
3670 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
3671 ENCODE_SINGLE_SHIFT_3;
3672 else
3673 ENCODE_LOCKING_SHIFT_3;
3674 break;
3675 }
3676 }
3677
3678 *p_nchars = produced_chars;
3679 return dst;
3680}
3681
3682/* The following three macros produce codes for indicating direction
3683 of text. */
3684#define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
3685 do { \
3686 if (CODING_ISO_FLAGS (coding) == CODING_ISO_FLAG_SEVEN_BITS) \
3687 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '['); \
3688 else \
3689 EMIT_ONE_BYTE (ISO_CODE_CSI); \
3690 } while (0)
3691
3692
3693#define ENCODE_DIRECTION_R2L() \
3694 do { \
3695 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
3696 EMIT_TWO_ASCII_BYTES ('2', ']'); \
3697 } while (0)
3698
3699
3700#define ENCODE_DIRECTION_L2R() \
3701 do { \
3702 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
3703 EMIT_TWO_ASCII_BYTES ('0', ']'); \
3704 } while (0)
3705
3706
3707/* Produce codes for designation and invocation to reset the graphic
3708 planes and registers to initial state. */
3709#define ENCODE_RESET_PLANE_AND_REGISTER() \
3710 do { \
3711 int reg; \
3712 struct charset *charset; \
3713 \
3714 if (CODING_ISO_INVOCATION (coding, 0) != 0) \
3715 ENCODE_SHIFT_IN; \
3716 for (reg = 0; reg < 4; reg++) \
3717 if (CODING_ISO_INITIAL (coding, reg) >= 0 \
3718 && (CODING_ISO_DESIGNATION (coding, reg) \
3719 != CODING_ISO_INITIAL (coding, reg))) \
3720 { \
3721 charset = CHARSET_FROM_ID (CODING_ISO_INITIAL (coding, reg)); \
3722 ENCODE_DESIGNATION (charset, reg, coding); \
3723 } \
3724 } while (0)
3725
3726
3727/* Produce designation sequences of charsets in the line started from
3728 SRC to a place pointed by DST, and return updated DST.
3729
3730 If the current block ends before any end-of-line, we may fail to
3731 find all the necessary designations. */
3732
3733static unsigned char *
3734encode_designation_at_bol (coding, charbuf, charbuf_end, dst)
3735 struct coding_system *coding;
3736 int *charbuf, *charbuf_end;
3737 unsigned char *dst;
3738{
3739 struct charset *charset;
3740 /* Table of charsets to be designated to each graphic register. */
3741 int r[4];
3742 int c, found = 0, reg;
3743 int produced_chars = 0;
3744 int multibytep = coding->dst_multibyte;
3745 Lisp_Object attrs;
3746 Lisp_Object charset_list;
3747
3748 attrs = CODING_ID_ATTRS (coding->id);
3749 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
3750 if (EQ (charset_list, Qiso_2022))
3751 charset_list = Viso_2022_charset_list;
3752
3753 for (reg = 0; reg < 4; reg++)
3754 r[reg] = -1;
3755
3756 while (found < 4)
3757 {
3758 int id;
3759
3760 c = *charbuf++;
3761 if (c == '\n')
3762 break;
3763 charset = char_charset (c, charset_list, NULL);
3764 id = CHARSET_ID (charset);
3765 reg = CODING_ISO_REQUEST (coding, id);
3766 if (reg >= 0 && r[reg] < 0)
3767 {
3768 found++;
3769 r[reg] = id;
3770 }
3771 }
3772
3773 if (found)
3774 {
3775 for (reg = 0; reg < 4; reg++)
3776 if (r[reg] >= 0
3777 && CODING_ISO_DESIGNATION (coding, reg) != r[reg])
3778 ENCODE_DESIGNATION (CHARSET_FROM_ID (r[reg]), reg, coding);
3779 }
3780
3781 return dst;
3782}
3783
3784/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
3785
3786static int
3787encode_coding_iso_2022 (coding)
3788 struct coding_system *coding;
3789{
3790 int multibytep = coding->dst_multibyte;
3791 int *charbuf = coding->charbuf;
3792 int *charbuf_end = charbuf + coding->charbuf_used;
3793 unsigned char *dst = coding->destination + coding->produced;
3794 unsigned char *dst_end = coding->destination + coding->dst_bytes;
3795 int safe_room = 16;
3796 int bol_designation
3797 = (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL
3798 && CODING_ISO_BOL (coding));
3799 int produced_chars = 0;
3800 Lisp_Object attrs, eol_type, charset_list;
3801 int ascii_compatible;
3802 int c;
3803 int preferred_charset_id = -1;
3804
3805 CODING_GET_INFO (coding, attrs, charset_list);
3806 eol_type = CODING_ID_EOL_TYPE (coding->id);
3807 if (VECTORP (eol_type))
3808 eol_type = Qunix;
3809
3810 setup_iso_safe_charsets (attrs);
3811 /* Charset list may have been changed. */
3812 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
3813 coding->safe_charsets = (char *) SDATA (CODING_ATTR_SAFE_CHARSETS(attrs));
3814
3815 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
3816
3817 while (charbuf < charbuf_end)
3818 {
3819 ASSURE_DESTINATION (safe_room);
3820
3821 if (bol_designation)
3822 {
3823 unsigned char *dst_prev = dst;
3824
3825 /* We have to produce designation sequences if any now. */
3826 dst = encode_designation_at_bol (coding, charbuf, charbuf_end, dst);
3827 bol_designation = 0;
3828 /* We are sure that designation sequences are all ASCII bytes. */
3829 produced_chars += dst - dst_prev;
3830 }
3831
3832 c = *charbuf++;
3833
3834 if (c < 0)
3835 {
3836 /* Handle an annotation. */
3837 switch (*charbuf)
3838 {
3839 case CODING_ANNOTATE_COMPOSITION_MASK:
3840 /* Not yet implemented. */
3841 break;
3842 case CODING_ANNOTATE_CHARSET_MASK:
3843 preferred_charset_id = charbuf[2];
3844 if (preferred_charset_id >= 0
3845 && NILP (Fmemq (make_number (preferred_charset_id),
3846 charset_list)))
3847 preferred_charset_id = -1;
3848 break;
3849 default:
3850 abort ();
3851 }
3852 charbuf += -c - 1;
3853 continue;
3854 }
3855
3856 /* Now encode the character C. */
3857 if (c < 0x20 || c == 0x7F)
3858 {
3859 if (c == '\n'
3860 || (c == '\r' && EQ (eol_type, Qmac)))
3861 {
3862 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
3863 ENCODE_RESET_PLANE_AND_REGISTER ();
3864 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_INIT_AT_BOL)
3865 {
3866 int i;
3867
3868 for (i = 0; i < 4; i++)
3869 CODING_ISO_DESIGNATION (coding, i)
3870 = CODING_ISO_INITIAL (coding, i);
3871 }
3872 bol_designation
3873 = CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL;
3874 }
3875 else if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_CNTL)
3876 ENCODE_RESET_PLANE_AND_REGISTER ();
3877 EMIT_ONE_ASCII_BYTE (c);
3878 }
3879 else if (ASCII_CHAR_P (c))
3880 {
3881 if (ascii_compatible)
3882 EMIT_ONE_ASCII_BYTE (c);
3883 else
3884 {
3885 struct charset *charset = CHARSET_FROM_ID (charset_ascii);
3886 ENCODE_ISO_CHARACTER (charset, c);
3887 }
3888 }
3889 else if (CHAR_BYTE8_P (c))
3890 {
3891 c = CHAR_TO_BYTE8 (c);
3892 EMIT_ONE_BYTE (c);
3893 }
3894 else
3895 {
3896 struct charset *charset;
3897
3898 if (preferred_charset_id >= 0)
3899 {
3900 charset = CHARSET_FROM_ID (preferred_charset_id);
3901 if (! CHAR_CHARSET_P (c, charset))
3902 charset = char_charset (c, charset_list, NULL);
3903 }
3904 else
3905 charset = char_charset (c, charset_list, NULL);
3906 if (!charset)
3907 {
3908 if (coding->mode & CODING_MODE_SAFE_ENCODING)
3909 {
3910 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
3911 charset = CHARSET_FROM_ID (charset_ascii);
3912 }
3913 else
3914 {
3915 c = coding->default_char;
3916 charset = char_charset (c, charset_list, NULL);
3917 }
3918 }
3919 ENCODE_ISO_CHARACTER (charset, c);
3920 }
3921 }
3922
3923 if (coding->mode & CODING_MODE_LAST_BLOCK
3924 && CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
3925 {
3926 ASSURE_DESTINATION (safe_room);
3927 ENCODE_RESET_PLANE_AND_REGISTER ();
3928 }
3929 record_conversion_result (coding, CODING_RESULT_SUCCESS);
3930 CODING_ISO_BOL (coding) = bol_designation;
3931 coding->produced_char += produced_chars;
3932 coding->produced = dst - coding->destination;
3933 return 0;
3934}
3935
3936\f
3937/*** 8,9. SJIS and BIG5 handlers ***/
3938
3939/* Although SJIS and BIG5 are not ISO's coding system, they are used
3940 quite widely. So, for the moment, Emacs supports them in the bare
3941 C code. But, in the future, they may be supported only by CCL. */
3942
3943/* SJIS is a coding system encoding three character sets: ASCII, right
3944 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
3945 as is. A character of charset katakana-jisx0201 is encoded by
3946 "position-code + 0x80". A character of charset japanese-jisx0208
3947 is encoded in 2-byte but two position-codes are divided and shifted
3948 so that it fit in the range below.
3949
3950 --- CODE RANGE of SJIS ---
3951 (character set) (range)
3952 ASCII 0x00 .. 0x7F
3953 KATAKANA-JISX0201 0xA0 .. 0xDF
3954 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
3955 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
3956 -------------------------------
3957
3958*/
3959
3960/* BIG5 is a coding system encoding two character sets: ASCII and
3961 Big5. An ASCII character is encoded as is. Big5 is a two-byte
3962 character set and is encoded in two-byte.
3963
3964 --- CODE RANGE of BIG5 ---
3965 (character set) (range)
3966 ASCII 0x00 .. 0x7F
3967 Big5 (1st byte) 0xA1 .. 0xFE
3968 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
3969 --------------------------
3970
3971 */
3972
3973/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3974 Check if a text is encoded in SJIS. If it is, return
3975 CATEGORY_MASK_SJIS, else return 0. */
3976
3977static int
3978detect_coding_sjis (coding, detect_info)
3979 struct coding_system *coding;
3980 struct coding_detection_info *detect_info;
3981{
3982 const unsigned char *src = coding->source, *src_base;
3983 const unsigned char *src_end = coding->source + coding->src_bytes;
3984 int multibytep = coding->src_multibyte;
3985 int consumed_chars = 0;
3986 int found = 0;
3987 int c;
3988
3989 detect_info->checked |= CATEGORY_MASK_SJIS;
3990 /* A coding system of this category is always ASCII compatible. */
3991 src += coding->head_ascii;
3992
3993 while (1)
3994 {
3995 src_base = src;
3996 ONE_MORE_BYTE (c);
3997 if (c < 0x80)
3998 continue;
3999 if ((c >= 0x81 && c <= 0x9F) || (c >= 0xE0 && c <= 0xEF))
4000 {
4001 ONE_MORE_BYTE (c);
4002 if (c < 0x40 || c == 0x7F || c > 0xFC)
4003 break;
4004 found = CATEGORY_MASK_SJIS;
4005 }
4006 else if (c >= 0xA0 && c < 0xE0)
4007 found = CATEGORY_MASK_SJIS;
4008 else
4009 break;
4010 }
4011 detect_info->rejected |= CATEGORY_MASK_SJIS;
4012 return 0;
4013
4014 no_more_source:
4015 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
4016 {
4017 detect_info->rejected |= CATEGORY_MASK_SJIS;
4018 return 0;
4019 }
4020 detect_info->found |= found;
4021 return 1;
4022}
4023
4024/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4025 Check if a text is encoded in BIG5. If it is, return
4026 CATEGORY_MASK_BIG5, else return 0. */
4027
4028static int
4029detect_coding_big5 (coding, detect_info)
4030 struct coding_system *coding;
4031 struct coding_detection_info *detect_info;
4032{
4033 const unsigned char *src = coding->source, *src_base;
4034 const unsigned char *src_end = coding->source + coding->src_bytes;
4035 int multibytep = coding->src_multibyte;
4036 int consumed_chars = 0;
4037 int found = 0;
4038 int c;
4039
4040 detect_info->checked |= CATEGORY_MASK_BIG5;
4041 /* A coding system of this category is always ASCII compatible. */
4042 src += coding->head_ascii;
4043
4044 while (1)
4045 {
4046 src_base = src;
4047 ONE_MORE_BYTE (c);
4048 if (c < 0x80)
4049 continue;
4050 if (c >= 0xA1)
4051 {
4052 ONE_MORE_BYTE (c);
4053 if (c < 0x40 || (c >= 0x7F && c <= 0xA0))
4054 return 0;
4055 found = CATEGORY_MASK_BIG5;
4056 }
4057 else
4058 break;
4059 }
4060 detect_info->rejected |= CATEGORY_MASK_BIG5;
4061 return 0;
4062
4063 no_more_source:
4064 if (src_base < src && coding->mode & CODING_MODE_LAST_BLOCK)
4065 {
4066 detect_info->rejected |= CATEGORY_MASK_BIG5;
4067 return 0;
4068 }
4069 detect_info->found |= found;
4070 return 1;
4071}
4072
4073/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
4074 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
4075
4076static void
4077decode_coding_sjis (coding)
4078 struct coding_system *coding;
4079{
4080 const unsigned char *src = coding->source + coding->consumed;
4081 const unsigned char *src_end = coding->source + coding->src_bytes;
4082 const unsigned char *src_base;
4083 int *charbuf = coding->charbuf + coding->charbuf_used;
4084 int *charbuf_end
4085 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
4086 int consumed_chars = 0, consumed_chars_base;
4087 int multibytep = coding->src_multibyte;
4088 struct charset *charset_roman, *charset_kanji, *charset_kana;
4089 struct charset *charset_kanji2;
4090 Lisp_Object attrs, charset_list, val;
4091 int char_offset = coding->produced_char;
4092 int last_offset = char_offset;
4093 int last_id = charset_ascii;
4094
4095 CODING_GET_INFO (coding, attrs, charset_list);
4096
4097 val = charset_list;
4098 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4099 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4100 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4101 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
4102
4103 while (1)
4104 {
4105 int c, c1;
4106 struct charset *charset;
4107
4108 src_base = src;
4109 consumed_chars_base = consumed_chars;
4110
4111 if (charbuf >= charbuf_end)
4112 break;
4113
4114 ONE_MORE_BYTE (c);
4115 if (c < 0)
4116 goto invalid_code;
4117 if (c < 0x80)
4118 charset = charset_roman;
4119 else if (c == 0x80 || c == 0xA0)
4120 goto invalid_code;
4121 else if (c >= 0xA1 && c <= 0xDF)
4122 {
4123 /* SJIS -> JISX0201-Kana */
4124 c &= 0x7F;
4125 charset = charset_kana;
4126 }
4127 else if (c <= 0xEF)
4128 {
4129 /* SJIS -> JISX0208 */
4130 ONE_MORE_BYTE (c1);
4131 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
4132 goto invalid_code;
4133 c = (c << 8) | c1;
4134 SJIS_TO_JIS (c);
4135 charset = charset_kanji;
4136 }
4137 else if (c <= 0xFC && charset_kanji2)
4138 {
4139 /* SJIS -> JISX0213-2 */
4140 ONE_MORE_BYTE (c1);
4141 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
4142 goto invalid_code;
4143 c = (c << 8) | c1;
4144 SJIS_TO_JIS2 (c);
4145 charset = charset_kanji2;
4146 }
4147 else
4148 goto invalid_code;
4149 if (charset->id != charset_ascii
4150 && last_id != charset->id)
4151 {
4152 if (last_id != charset_ascii)
4153 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
4154 last_id = charset->id;
4155 last_offset = char_offset;
4156 }
4157 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
4158 *charbuf++ = c;
4159 char_offset++;
4160 continue;
4161
4162 invalid_code:
4163 src = src_base;
4164 consumed_chars = consumed_chars_base;
4165 ONE_MORE_BYTE (c);
4166 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
4167 char_offset++;
4168 coding->errors++;
4169 }
4170
4171 no_more_source:
4172 if (last_id != charset_ascii)
4173 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
4174 coding->consumed_char += consumed_chars_base;
4175 coding->consumed = src_base - coding->source;
4176 coding->charbuf_used = charbuf - coding->charbuf;
4177}
4178
4179static void
4180decode_coding_big5 (coding)
4181 struct coding_system *coding;
4182{
4183 const unsigned char *src = coding->source + coding->consumed;
4184 const unsigned char *src_end = coding->source + coding->src_bytes;
4185 const unsigned char *src_base;
4186 int *charbuf = coding->charbuf + coding->charbuf_used;
4187 int *charbuf_end
4188 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
4189 int consumed_chars = 0, consumed_chars_base;
4190 int multibytep = coding->src_multibyte;
4191 struct charset *charset_roman, *charset_big5;
4192 Lisp_Object attrs, charset_list, val;
4193 int char_offset = coding->produced_char;
4194 int last_offset = char_offset;
4195 int last_id = charset_ascii;
4196
4197 CODING_GET_INFO (coding, attrs, charset_list);
4198 val = charset_list;
4199 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4200 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4201
4202 while (1)
4203 {
4204 int c, c1;
4205 struct charset *charset;
4206
4207 src_base = src;
4208 consumed_chars_base = consumed_chars;
4209
4210 if (charbuf >= charbuf_end)
4211 break;
4212
4213 ONE_MORE_BYTE (c);
4214
4215 if (c < 0)
4216 goto invalid_code;
4217 if (c < 0x80)
4218 charset = charset_roman;
4219 else
4220 {
4221 /* BIG5 -> Big5 */
4222 if (c < 0xA1 || c > 0xFE)
4223 goto invalid_code;
4224 ONE_MORE_BYTE (c1);
4225 if (c1 < 0x40 || (c1 > 0x7E && c1 < 0xA1) || c1 > 0xFE)
4226 goto invalid_code;
4227 c = c << 8 | c1;
4228 charset = charset_big5;
4229 }
4230 if (charset->id != charset_ascii
4231 && last_id != charset->id)
4232 {
4233 if (last_id != charset_ascii)
4234 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
4235 last_id = charset->id;
4236 last_offset = char_offset;
4237 }
4238 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
4239 *charbuf++ = c;
4240 char_offset++;
4241 continue;
4242
4243 invalid_code:
4244 src = src_base;
4245 consumed_chars = consumed_chars_base;
4246 ONE_MORE_BYTE (c);
4247 *charbuf++ = c < 0 ? -c : BYTE8_TO_CHAR (c);
4248 char_offset++;
4249 coding->errors++;
4250 }
4251
4252 no_more_source:
4253 if (last_id != charset_ascii)
4254 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
4255 coding->consumed_char += consumed_chars_base;
4256 coding->consumed = src_base - coding->source;
4257 coding->charbuf_used = charbuf - coding->charbuf;
4258}
4259
4260/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
4261 This function can encode charsets `ascii', `katakana-jisx0201',
4262 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
4263 are sure that all these charsets are registered as official charset
4264 (i.e. do not have extended leading-codes). Characters of other
4265 charsets are produced without any encoding. If SJIS_P is 1, encode
4266 SJIS text, else encode BIG5 text. */
4267
4268static int
4269encode_coding_sjis (coding)
4270 struct coding_system *coding;
4271{
4272 int multibytep = coding->dst_multibyte;
4273 int *charbuf = coding->charbuf;
4274 int *charbuf_end = charbuf + coding->charbuf_used;
4275 unsigned char *dst = coding->destination + coding->produced;
4276 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4277 int safe_room = 4;
4278 int produced_chars = 0;
4279 Lisp_Object attrs, charset_list, val;
4280 int ascii_compatible;
4281 struct charset *charset_roman, *charset_kanji, *charset_kana;
4282 struct charset *charset_kanji2;
4283 int c;
4284
4285 CODING_GET_INFO (coding, attrs, charset_list);
4286 val = charset_list;
4287 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4288 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4289 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4290 charset_kanji2 = NILP (val) ? NULL : CHARSET_FROM_ID (XINT (XCAR (val)));
4291
4292 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4293
4294 while (charbuf < charbuf_end)
4295 {
4296 ASSURE_DESTINATION (safe_room);
4297 c = *charbuf++;
4298 /* Now encode the character C. */
4299 if (ASCII_CHAR_P (c) && ascii_compatible)
4300 EMIT_ONE_ASCII_BYTE (c);
4301 else if (CHAR_BYTE8_P (c))
4302 {
4303 c = CHAR_TO_BYTE8 (c);
4304 EMIT_ONE_BYTE (c);
4305 }
4306 else
4307 {
4308 unsigned code;
4309 struct charset *charset = char_charset (c, charset_list, &code);
4310
4311 if (!charset)
4312 {
4313 if (coding->mode & CODING_MODE_SAFE_ENCODING)
4314 {
4315 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4316 charset = CHARSET_FROM_ID (charset_ascii);
4317 }
4318 else
4319 {
4320 c = coding->default_char;
4321 charset = char_charset (c, charset_list, &code);
4322 }
4323 }
4324 if (code == CHARSET_INVALID_CODE (charset))
4325 abort ();
4326 if (charset == charset_kanji)
4327 {
4328 int c1, c2;
4329 JIS_TO_SJIS (code);
4330 c1 = code >> 8, c2 = code & 0xFF;
4331 EMIT_TWO_BYTES (c1, c2);
4332 }
4333 else if (charset == charset_kana)
4334 EMIT_ONE_BYTE (code | 0x80);
4335 else if (charset_kanji2 && charset == charset_kanji2)
4336 {
4337 int c1, c2;
4338
4339 c1 = code >> 8;
4340 if (c1 == 0x21 || (c1 >= 0x23 && c1 < 0x25)
4341 || (c1 >= 0x2C && c1 <= 0x2F) || c1 >= 0x6E)
4342 {
4343 JIS_TO_SJIS2 (code);
4344 c1 = code >> 8, c2 = code & 0xFF;
4345 EMIT_TWO_BYTES (c1, c2);
4346 }
4347 else
4348 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4349 }
4350 else
4351 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4352 }
4353 }
4354 record_conversion_result (coding, CODING_RESULT_SUCCESS);
4355 coding->produced_char += produced_chars;
4356 coding->produced = dst - coding->destination;
4357 return 0;
4358}
4359
4360static int
4361encode_coding_big5 (coding)
4362 struct coding_system *coding;
4363{
4364 int multibytep = coding->dst_multibyte;
4365 int *charbuf = coding->charbuf;
4366 int *charbuf_end = charbuf + coding->charbuf_used;
4367 unsigned char *dst = coding->destination + coding->produced;
4368 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4369 int safe_room = 4;
4370 int produced_chars = 0;
4371 Lisp_Object attrs, charset_list, val;
4372 int ascii_compatible;
4373 struct charset *charset_roman, *charset_big5;
4374 int c;
4375
4376 CODING_GET_INFO (coding, attrs, charset_list);
4377 val = charset_list;
4378 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
4379 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
4380 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4381
4382 while (charbuf < charbuf_end)
4383 {
4384 ASSURE_DESTINATION (safe_room);
4385 c = *charbuf++;
4386 /* Now encode the character C. */
4387 if (ASCII_CHAR_P (c) && ascii_compatible)
4388 EMIT_ONE_ASCII_BYTE (c);
4389 else if (CHAR_BYTE8_P (c))
4390 {
4391 c = CHAR_TO_BYTE8 (c);
4392 EMIT_ONE_BYTE (c);
4393 }
4394 else
4395 {
4396 unsigned code;
4397 struct charset *charset = char_charset (c, charset_list, &code);
4398
4399 if (! charset)
4400 {
4401 if (coding->mode & CODING_MODE_SAFE_ENCODING)
4402 {
4403 code = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4404 charset = CHARSET_FROM_ID (charset_ascii);
4405 }
4406 else
4407 {
4408 c = coding->default_char;
4409 charset = char_charset (c, charset_list, &code);
4410 }
4411 }
4412 if (code == CHARSET_INVALID_CODE (charset))
4413 abort ();
4414 if (charset == charset_big5)
4415 {
4416 int c1, c2;
4417
4418 c1 = code >> 8, c2 = code & 0xFF;
4419 EMIT_TWO_BYTES (c1, c2);
4420 }
4421 else
4422 EMIT_ONE_ASCII_BYTE (code & 0x7F);
4423 }
4424 }
4425 record_conversion_result (coding, CODING_RESULT_SUCCESS);
4426 coding->produced_char += produced_chars;
4427 coding->produced = dst - coding->destination;
4428 return 0;
4429}
4430
4431\f
4432/*** 10. CCL handlers ***/
4433
4434/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4435 Check if a text is encoded in a coding system of which
4436 encoder/decoder are written in CCL program. If it is, return
4437 CATEGORY_MASK_CCL, else return 0. */
4438
4439static int
4440detect_coding_ccl (coding, detect_info)
4441 struct coding_system *coding;
4442 struct coding_detection_info *detect_info;
4443{
4444 const unsigned char *src = coding->source, *src_base;
4445 const unsigned char *src_end = coding->source + coding->src_bytes;
4446 int multibytep = coding->src_multibyte;
4447 int consumed_chars = 0;
4448 int found = 0;
4449 unsigned char *valids;
4450 int head_ascii = coding->head_ascii;
4451 Lisp_Object attrs;
4452
4453 detect_info->checked |= CATEGORY_MASK_CCL;
4454
4455 coding = &coding_categories[coding_category_ccl];
4456 valids = CODING_CCL_VALIDS (coding);
4457 attrs = CODING_ID_ATTRS (coding->id);
4458 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
4459 src += head_ascii;
4460
4461 while (1)
4462 {
4463 int c;
4464
4465 src_base = src;
4466 ONE_MORE_BYTE (c);
4467 if (c < 0 || ! valids[c])
4468 break;
4469 if ((valids[c] > 1))
4470 found = CATEGORY_MASK_CCL;
4471 }
4472 detect_info->rejected |= CATEGORY_MASK_CCL;
4473 return 0;
4474
4475 no_more_source:
4476 detect_info->found |= found;
4477 return 1;
4478}
4479
4480static void
4481decode_coding_ccl (coding)
4482 struct coding_system *coding;
4483{
4484 const unsigned char *src = coding->source + coding->consumed;
4485 const unsigned char *src_end = coding->source + coding->src_bytes;
4486 int *charbuf = coding->charbuf + coding->charbuf_used;
4487 int *charbuf_end = coding->charbuf + coding->charbuf_size;
4488 int consumed_chars = 0;
4489 int multibytep = coding->src_multibyte;
4490 struct ccl_program ccl;
4491 int source_charbuf[1024];
4492 int source_byteidx[1024];
4493 Lisp_Object attrs, charset_list;
4494
4495 CODING_GET_INFO (coding, attrs, charset_list);
4496 setup_ccl_program (&ccl, CODING_CCL_DECODER (coding));
4497
4498 while (src < src_end)
4499 {
4500 const unsigned char *p = src;
4501 int *source, *source_end;
4502 int i = 0;
4503
4504 if (multibytep)
4505 while (i < 1024 && p < src_end)
4506 {
4507 source_byteidx[i] = p - src;
4508 source_charbuf[i++] = STRING_CHAR_ADVANCE (p);
4509 }
4510 else
4511 while (i < 1024 && p < src_end)
4512 source_charbuf[i++] = *p++;
4513
4514 if (p == src_end && coding->mode & CODING_MODE_LAST_BLOCK)
4515 ccl.last_block = 1;
4516
4517 source = source_charbuf;
4518 source_end = source + i;
4519 while (source < source_end)
4520 {
4521 ccl_driver (&ccl, source, charbuf,
4522 source_end - source, charbuf_end - charbuf,
4523 charset_list);
4524 source += ccl.consumed;
4525 charbuf += ccl.produced;
4526 if (ccl.status != CCL_STAT_SUSPEND_BY_DST)
4527 break;
4528 }
4529 if (source < source_end)
4530 src += source_byteidx[source - source_charbuf];
4531 else
4532 src = p;
4533 consumed_chars += source - source_charbuf;
4534
4535 if (ccl.status != CCL_STAT_SUSPEND_BY_SRC
4536 && ccl.status != CODING_RESULT_INSUFFICIENT_SRC)
4537 break;
4538 }
4539
4540 switch (ccl.status)
4541 {
4542 case CCL_STAT_SUSPEND_BY_SRC:
4543 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
4544 break;
4545 case CCL_STAT_SUSPEND_BY_DST:
4546 break;
4547 case CCL_STAT_QUIT:
4548 case CCL_STAT_INVALID_CMD:
4549 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
4550 break;
4551 default:
4552 record_conversion_result (coding, CODING_RESULT_SUCCESS);
4553 break;
4554 }
4555 coding->consumed_char += consumed_chars;
4556 coding->consumed = src - coding->source;
4557 coding->charbuf_used = charbuf - coding->charbuf;
4558}
4559
4560static int
4561encode_coding_ccl (coding)
4562 struct coding_system *coding;
4563{
4564 struct ccl_program ccl;
4565 int multibytep = coding->dst_multibyte;
4566 int *charbuf = coding->charbuf;
4567 int *charbuf_end = charbuf + coding->charbuf_used;
4568 unsigned char *dst = coding->destination + coding->produced;
4569 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4570 int destination_charbuf[1024];
4571 int i, produced_chars = 0;
4572 Lisp_Object attrs, charset_list;
4573
4574 CODING_GET_INFO (coding, attrs, charset_list);
4575 setup_ccl_program (&ccl, CODING_CCL_ENCODER (coding));
4576
4577 ccl.last_block = coding->mode & CODING_MODE_LAST_BLOCK;
4578 ccl.dst_multibyte = coding->dst_multibyte;
4579
4580 while (charbuf < charbuf_end)
4581 {
4582 ccl_driver (&ccl, charbuf, destination_charbuf,
4583 charbuf_end - charbuf, 1024, charset_list);
4584 if (multibytep)
4585 {
4586 ASSURE_DESTINATION (ccl.produced * 2);
4587 for (i = 0; i < ccl.produced; i++)
4588 EMIT_ONE_BYTE (destination_charbuf[i] & 0xFF);
4589 }
4590 else
4591 {
4592 ASSURE_DESTINATION (ccl.produced);
4593 for (i = 0; i < ccl.produced; i++)
4594 *dst++ = destination_charbuf[i] & 0xFF;
4595 produced_chars += ccl.produced;
4596 }
4597 charbuf += ccl.consumed;
4598 if (ccl.status == CCL_STAT_QUIT
4599 || ccl.status == CCL_STAT_INVALID_CMD)
4600 break;
4601 }
4602
4603 switch (ccl.status)
4604 {
4605 case CCL_STAT_SUSPEND_BY_SRC:
4606 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_SRC);
4607 break;
4608 case CCL_STAT_SUSPEND_BY_DST:
4609 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_DST);
4610 break;
4611 case CCL_STAT_QUIT:
4612 case CCL_STAT_INVALID_CMD:
4613 record_conversion_result (coding, CODING_RESULT_INTERRUPT);
4614 break;
4615 default:
4616 record_conversion_result (coding, CODING_RESULT_SUCCESS);
4617 break;
4618 }
4619
4620 coding->produced_char += produced_chars;
4621 coding->produced = dst - coding->destination;
4622 return 0;
4623}
4624
4625
4626\f
4627/*** 10, 11. no-conversion handlers ***/
4628
4629/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
4630
4631static void
4632decode_coding_raw_text (coding)
4633 struct coding_system *coding;
4634{
4635 coding->chars_at_source = 1;
4636 coding->consumed_char = 0;
4637 coding->consumed = 0;
4638 record_conversion_result (coding, CODING_RESULT_SUCCESS);
4639}
4640
4641static int
4642encode_coding_raw_text (coding)
4643 struct coding_system *coding;
4644{
4645 int multibytep = coding->dst_multibyte;
4646 int *charbuf = coding->charbuf;
4647 int *charbuf_end = coding->charbuf + coding->charbuf_used;
4648 unsigned char *dst = coding->destination + coding->produced;
4649 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4650 int produced_chars = 0;
4651 int c;
4652
4653 if (multibytep)
4654 {
4655 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
4656
4657 if (coding->src_multibyte)
4658 while (charbuf < charbuf_end)
4659 {
4660 ASSURE_DESTINATION (safe_room);
4661 c = *charbuf++;
4662 if (ASCII_CHAR_P (c))
4663 EMIT_ONE_ASCII_BYTE (c);
4664 else if (CHAR_BYTE8_P (c))
4665 {
4666 c = CHAR_TO_BYTE8 (c);
4667 EMIT_ONE_BYTE (c);
4668 }
4669 else
4670 {
4671 unsigned char str[MAX_MULTIBYTE_LENGTH], *p0 = str, *p1 = str;
4672
4673 CHAR_STRING_ADVANCE (c, p1);
4674 while (p0 < p1)
4675 {
4676 EMIT_ONE_BYTE (*p0);
4677 p0++;
4678 }
4679 }
4680 }
4681 else
4682 while (charbuf < charbuf_end)
4683 {
4684 ASSURE_DESTINATION (safe_room);
4685 c = *charbuf++;
4686 EMIT_ONE_BYTE (c);
4687 }
4688 }
4689 else
4690 {
4691 if (coding->src_multibyte)
4692 {
4693 int safe_room = MAX_MULTIBYTE_LENGTH;
4694
4695 while (charbuf < charbuf_end)
4696 {
4697 ASSURE_DESTINATION (safe_room);
4698 c = *charbuf++;
4699 if (ASCII_CHAR_P (c))
4700 *dst++ = c;
4701 else if (CHAR_BYTE8_P (c))
4702 *dst++ = CHAR_TO_BYTE8 (c);
4703 else
4704 CHAR_STRING_ADVANCE (c, dst);
4705 produced_chars++;
4706 }
4707 }
4708 else
4709 {
4710 ASSURE_DESTINATION (charbuf_end - charbuf);
4711 while (charbuf < charbuf_end && dst < dst_end)
4712 *dst++ = *charbuf++;
4713 produced_chars = dst - (coding->destination + coding->dst_bytes);
4714 }
4715 }
4716 record_conversion_result (coding, CODING_RESULT_SUCCESS);
4717 coding->produced_char += produced_chars;
4718 coding->produced = dst - coding->destination;
4719 return 0;
4720}
4721
4722/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
4723 Check if a text is encoded in a charset-based coding system. If it
4724 is, return 1, else return 0. */
4725
4726static int
4727detect_coding_charset (coding, detect_info)
4728 struct coding_system *coding;
4729 struct coding_detection_info *detect_info;
4730{
4731 const unsigned char *src = coding->source, *src_base;
4732 const unsigned char *src_end = coding->source + coding->src_bytes;
4733 int multibytep = coding->src_multibyte;
4734 int consumed_chars = 0;
4735 Lisp_Object attrs, valids;
4736 int found = 0;
4737 int head_ascii = coding->head_ascii;
4738
4739 detect_info->checked |= CATEGORY_MASK_CHARSET;
4740
4741 coding = &coding_categories[coding_category_charset];
4742 attrs = CODING_ID_ATTRS (coding->id);
4743 valids = AREF (attrs, coding_attr_charset_valids);
4744
4745 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
4746 src += head_ascii;
4747
4748 while (1)
4749 {
4750 int c;
4751 Lisp_Object val;
4752 struct charset *charset;
4753 int dim, idx;
4754
4755 src_base = src;
4756 ONE_MORE_BYTE (c);
4757 if (c < 0)
4758 continue;
4759 val = AREF (valids, c);
4760 if (NILP (val))
4761 break;
4762 if (c >= 0x80)
4763 found = CATEGORY_MASK_CHARSET;
4764 if (INTEGERP (val))
4765 {
4766 charset = CHARSET_FROM_ID (XFASTINT (val));
4767 dim = CHARSET_DIMENSION (charset);
4768 for (idx = 1; idx < dim; idx++)
4769 {
4770 if (src == src_end)
4771 goto too_short;
4772 ONE_MORE_BYTE (c);
4773 if (c < charset->code_space[(dim - 1 - idx) * 2]
4774 || c > charset->code_space[(dim - 1 - idx) * 2 + 1])
4775 break;
4776 }
4777 if (idx < dim)
4778 break;
4779 }
4780 else
4781 {
4782 idx = 1;
4783 for (; CONSP (val); val = XCDR (val))
4784 {
4785 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
4786 dim = CHARSET_DIMENSION (charset);
4787 while (idx < dim)
4788 {
4789 if (src == src_end)
4790 goto too_short;
4791 ONE_MORE_BYTE (c);
4792 if (c < charset->code_space[(dim - 1 - idx) * 4]
4793 || c > charset->code_space[(dim - 1 - idx) * 4 + 1])
4794 break;
4795 idx++;
4796 }
4797 if (idx == dim)
4798 {
4799 val = Qnil;
4800 break;
4801 }
4802 }
4803 if (CONSP (val))
4804 break;
4805 }
4806 }
4807 too_short:
4808 detect_info->rejected |= CATEGORY_MASK_CHARSET;
4809 return 0;
4810
4811 no_more_source:
4812 detect_info->found |= found;
4813 return 1;
4814}
4815
4816static void
4817decode_coding_charset (coding)
4818 struct coding_system *coding;
4819{
4820 const unsigned char *src = coding->source + coding->consumed;
4821 const unsigned char *src_end = coding->source + coding->src_bytes;
4822 const unsigned char *src_base;
4823 int *charbuf = coding->charbuf + coding->charbuf_used;
4824 int *charbuf_end
4825 = coding->charbuf + coding->charbuf_size - MAX_ANNOTATION_LENGTH;
4826 int consumed_chars = 0, consumed_chars_base;
4827 int multibytep = coding->src_multibyte;
4828 Lisp_Object attrs, charset_list, valids;
4829 int char_offset = coding->produced_char;
4830 int last_offset = char_offset;
4831 int last_id = charset_ascii;
4832
4833 CODING_GET_INFO (coding, attrs, charset_list);
4834 valids = AREF (attrs, coding_attr_charset_valids);
4835
4836 while (1)
4837 {
4838 int c;
4839 Lisp_Object val;
4840 struct charset *charset;
4841 int dim;
4842 int len = 1;
4843 unsigned code;
4844
4845 src_base = src;
4846 consumed_chars_base = consumed_chars;
4847
4848 if (charbuf >= charbuf_end)
4849 break;
4850
4851 ONE_MORE_BYTE (c);
4852 if (c < 0)
4853 goto invalid_code;
4854 code = c;
4855
4856 val = AREF (valids, c);
4857 if (NILP (val))
4858 goto invalid_code;
4859 if (INTEGERP (val))
4860 {
4861 charset = CHARSET_FROM_ID (XFASTINT (val));
4862 dim = CHARSET_DIMENSION (charset);
4863 while (len < dim)
4864 {
4865 ONE_MORE_BYTE (c);
4866 code = (code << 8) | c;
4867 len++;
4868 }
4869 CODING_DECODE_CHAR (coding, src, src_base, src_end,
4870 charset, code, c);
4871 }
4872 else
4873 {
4874 /* VAL is a list of charset IDs. It is assured that the
4875 list is sorted by charset dimensions (smaller one
4876 comes first). */
4877 while (CONSP (val))
4878 {
4879 charset = CHARSET_FROM_ID (XFASTINT (XCAR (val)));
4880 dim = CHARSET_DIMENSION (charset);
4881 while (len < dim)
4882 {
4883 ONE_MORE_BYTE (c);
4884 code = (code << 8) | c;
4885 len++;
4886 }
4887 CODING_DECODE_CHAR (coding, src, src_base,
4888 src_end, charset, code, c);
4889 if (c >= 0)
4890 break;
4891 val = XCDR (val);
4892 }
4893 }
4894 if (c < 0)
4895 goto invalid_code;
4896 if (charset->id != charset_ascii
4897 && last_id != charset->id)
4898 {
4899 if (last_id != charset_ascii)
4900 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
4901 last_id = charset->id;
4902 last_offset = char_offset;
4903 }
4904
4905 *charbuf++ = c;
4906 char_offset++;
4907 continue;
4908
4909 invalid_code:
4910 src = src_base;
4911 consumed_chars = consumed_chars_base;
4912 ONE_MORE_BYTE (c);
4913 *charbuf++ = c < 0 ? -c : ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
4914 char_offset++;
4915 coding->errors++;
4916 }
4917
4918 no_more_source:
4919 if (last_id != charset_ascii)
4920 ADD_CHARSET_DATA (charbuf, char_offset - last_offset, last_id);
4921 coding->consumed_char += consumed_chars_base;
4922 coding->consumed = src_base - coding->source;
4923 coding->charbuf_used = charbuf - coding->charbuf;
4924}
4925
4926static int
4927encode_coding_charset (coding)
4928 struct coding_system *coding;
4929{
4930 int multibytep = coding->dst_multibyte;
4931 int *charbuf = coding->charbuf;
4932 int *charbuf_end = charbuf + coding->charbuf_used;
4933 unsigned char *dst = coding->destination + coding->produced;
4934 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4935 int safe_room = MAX_MULTIBYTE_LENGTH;
4936 int produced_chars = 0;
4937 Lisp_Object attrs, charset_list;
4938 int ascii_compatible;
4939 int c;
4940
4941 CODING_GET_INFO (coding, attrs, charset_list);
4942 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4943
4944 while (charbuf < charbuf_end)
4945 {
4946 struct charset *charset;
4947 unsigned code;
4948
4949 ASSURE_DESTINATION (safe_room);
4950 c = *charbuf++;
4951 if (ascii_compatible && ASCII_CHAR_P (c))
4952 EMIT_ONE_ASCII_BYTE (c);
4953 else if (CHAR_BYTE8_P (c))
4954 {
4955 c = CHAR_TO_BYTE8 (c);
4956 EMIT_ONE_BYTE (c);
4957 }
4958 else
4959 {
4960 charset = char_charset (c, charset_list, &code);
4961 if (charset)
4962 {
4963 if (CHARSET_DIMENSION (charset) == 1)
4964 EMIT_ONE_BYTE (code);
4965 else if (CHARSET_DIMENSION (charset) == 2)
4966 EMIT_TWO_BYTES (code >> 8, code & 0xFF);
4967 else if (CHARSET_DIMENSION (charset) == 3)
4968 EMIT_THREE_BYTES (code >> 16, (code >> 8) & 0xFF, code & 0xFF);
4969 else
4970 EMIT_FOUR_BYTES (code >> 24, (code >> 16) & 0xFF,
4971 (code >> 8) & 0xFF, code & 0xFF);
4972 }
4973 else
4974 {
4975 if (coding->mode & CODING_MODE_SAFE_ENCODING)
4976 c = CODING_INHIBIT_CHARACTER_SUBSTITUTION;
4977 else
4978 c = coding->default_char;
4979 EMIT_ONE_BYTE (c);
4980 }
4981 }
4982 }
4983
4984 record_conversion_result (coding, CODING_RESULT_SUCCESS);
4985 coding->produced_char += produced_chars;
4986 coding->produced = dst - coding->destination;
4987 return 0;
4988}
4989
4990\f
4991/*** 7. C library functions ***/
4992
4993/* Setup coding context CODING from information about CODING_SYSTEM.
4994 If CODING_SYSTEM is nil, `no-conversion' is assumed. If
4995 CODING_SYSTEM is invalid, signal an error. */
4996
4997void
4998setup_coding_system (coding_system, coding)
4999 Lisp_Object coding_system;
5000 struct coding_system *coding;
5001{
5002 Lisp_Object attrs;
5003 Lisp_Object eol_type;
5004 Lisp_Object coding_type;
5005 Lisp_Object val;
5006
5007 if (NILP (coding_system))
5008 coding_system = Qundecided;
5009
5010 CHECK_CODING_SYSTEM_GET_ID (coding_system, coding->id);
5011
5012 attrs = CODING_ID_ATTRS (coding->id);
5013 eol_type = CODING_ID_EOL_TYPE (coding->id);
5014
5015 coding->mode = 0;
5016 coding->head_ascii = -1;
5017 if (VECTORP (eol_type))
5018 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5019 | CODING_REQUIRE_DETECTION_MASK);
5020 else if (! EQ (eol_type, Qunix))
5021 coding->common_flags = (CODING_REQUIRE_DECODING_MASK
5022 | CODING_REQUIRE_ENCODING_MASK);
5023 else
5024 coding->common_flags = 0;
5025 if (! NILP (CODING_ATTR_POST_READ (attrs)))
5026 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5027 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
5028 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
5029 if (! NILP (CODING_ATTR_FOR_UNIBYTE (attrs)))
5030 coding->common_flags |= CODING_FOR_UNIBYTE_MASK;
5031
5032 val = CODING_ATTR_SAFE_CHARSETS (attrs);
5033 coding->max_charset_id = SCHARS (val) - 1;
5034 coding->safe_charsets = (char *) SDATA (val);
5035 coding->default_char = XINT (CODING_ATTR_DEFAULT_CHAR (attrs));
5036
5037 coding_type = CODING_ATTR_TYPE (attrs);
5038 if (EQ (coding_type, Qundecided))
5039 {
5040 coding->detector = NULL;
5041 coding->decoder = decode_coding_raw_text;
5042 coding->encoder = encode_coding_raw_text;
5043 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
5044 }
5045 else if (EQ (coding_type, Qiso_2022))
5046 {
5047 int i;
5048 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5049
5050 /* Invoke graphic register 0 to plane 0. */
5051 CODING_ISO_INVOCATION (coding, 0) = 0;
5052 /* Invoke graphic register 1 to plane 1 if we can use 8-bit. */
5053 CODING_ISO_INVOCATION (coding, 1)
5054 = (flags & CODING_ISO_FLAG_SEVEN_BITS ? -1 : 1);
5055 /* Setup the initial status of designation. */
5056 for (i = 0; i < 4; i++)
5057 CODING_ISO_DESIGNATION (coding, i) = CODING_ISO_INITIAL (coding, i);
5058 /* Not single shifting initially. */
5059 CODING_ISO_SINGLE_SHIFTING (coding) = 0;
5060 /* Beginning of buffer should also be regarded as bol. */
5061 CODING_ISO_BOL (coding) = 1;
5062 coding->detector = detect_coding_iso_2022;
5063 coding->decoder = decode_coding_iso_2022;
5064 coding->encoder = encode_coding_iso_2022;
5065 if (flags & CODING_ISO_FLAG_SAFE)
5066 coding->mode |= CODING_MODE_SAFE_ENCODING;
5067 coding->common_flags
5068 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5069 | CODING_REQUIRE_FLUSHING_MASK);
5070 if (flags & CODING_ISO_FLAG_COMPOSITION)
5071 coding->common_flags |= CODING_ANNOTATE_COMPOSITION_MASK;
5072 if (flags & CODING_ISO_FLAG_DESIGNATION)
5073 coding->common_flags |= CODING_ANNOTATE_CHARSET_MASK;
5074 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5075 {
5076 setup_iso_safe_charsets (attrs);
5077 val = CODING_ATTR_SAFE_CHARSETS (attrs);
5078 coding->max_charset_id = SCHARS (val) - 1;
5079 coding->safe_charsets = (char *) SDATA (val);
5080 }
5081 CODING_ISO_FLAGS (coding) = flags;
5082 }
5083 else if (EQ (coding_type, Qcharset))
5084 {
5085 coding->detector = detect_coding_charset;
5086 coding->decoder = decode_coding_charset;
5087 coding->encoder = encode_coding_charset;
5088 coding->common_flags
5089 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5090 }
5091 else if (EQ (coding_type, Qutf_8))
5092 {
5093 coding->detector = detect_coding_utf_8;
5094 coding->decoder = decode_coding_utf_8;
5095 coding->encoder = encode_coding_utf_8;
5096 coding->common_flags
5097 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5098 }
5099 else if (EQ (coding_type, Qutf_16))
5100 {
5101 val = AREF (attrs, coding_attr_utf_16_bom);
5102 CODING_UTF_16_BOM (coding) = (CONSP (val) ? utf_16_detect_bom
5103 : EQ (val, Qt) ? utf_16_with_bom
5104 : utf_16_without_bom);
5105 val = AREF (attrs, coding_attr_utf_16_endian);
5106 CODING_UTF_16_ENDIAN (coding) = (EQ (val, Qbig) ? utf_16_big_endian
5107 : utf_16_little_endian);
5108 CODING_UTF_16_SURROGATE (coding) = 0;
5109 coding->detector = detect_coding_utf_16;
5110 coding->decoder = decode_coding_utf_16;
5111 coding->encoder = encode_coding_utf_16;
5112 coding->common_flags
5113 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5114 if (CODING_UTF_16_BOM (coding) == utf_16_detect_bom)
5115 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
5116 }
5117 else if (EQ (coding_type, Qccl))
5118 {
5119 coding->detector = detect_coding_ccl;
5120 coding->decoder = decode_coding_ccl;
5121 coding->encoder = encode_coding_ccl;
5122 coding->common_flags
5123 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
5124 | CODING_REQUIRE_FLUSHING_MASK);
5125 }
5126 else if (EQ (coding_type, Qemacs_mule))
5127 {
5128 coding->detector = detect_coding_emacs_mule;
5129 coding->decoder = decode_coding_emacs_mule;
5130 coding->encoder = encode_coding_emacs_mule;
5131 coding->common_flags
5132 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5133 if (! NILP (AREF (attrs, coding_attr_emacs_mule_full))
5134 && ! EQ (CODING_ATTR_CHARSET_LIST (attrs), Vemacs_mule_charset_list))
5135 {
5136 Lisp_Object tail, safe_charsets;
5137 int max_charset_id = 0;
5138
5139 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5140 tail = XCDR (tail))
5141 if (max_charset_id < XFASTINT (XCAR (tail)))
5142 max_charset_id = XFASTINT (XCAR (tail));
5143 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
5144 make_number (255));
5145 for (tail = Vemacs_mule_charset_list; CONSP (tail);
5146 tail = XCDR (tail))
5147 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
5148 coding->max_charset_id = max_charset_id;
5149 coding->safe_charsets = (char *) SDATA (safe_charsets);
5150 }
5151 }
5152 else if (EQ (coding_type, Qshift_jis))
5153 {
5154 coding->detector = detect_coding_sjis;
5155 coding->decoder = decode_coding_sjis;
5156 coding->encoder = encode_coding_sjis;
5157 coding->common_flags
5158 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5159 }
5160 else if (EQ (coding_type, Qbig5))
5161 {
5162 coding->detector = detect_coding_big5;
5163 coding->decoder = decode_coding_big5;
5164 coding->encoder = encode_coding_big5;
5165 coding->common_flags
5166 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
5167 }
5168 else /* EQ (coding_type, Qraw_text) */
5169 {
5170 coding->detector = NULL;
5171 coding->decoder = decode_coding_raw_text;
5172 coding->encoder = encode_coding_raw_text;
5173 if (! EQ (eol_type, Qunix))
5174 {
5175 coding->common_flags |= CODING_REQUIRE_DECODING_MASK;
5176 if (! VECTORP (eol_type))
5177 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK;
5178 }
5179
5180 }
5181
5182 return;
5183}
5184
5185/* Return a list of charsets supported by CODING. */
5186
5187Lisp_Object
5188coding_charset_list (coding)
5189 struct coding_system *coding;
5190{
5191 Lisp_Object attrs, charset_list;
5192
5193 CODING_GET_INFO (coding, attrs, charset_list);
5194 if (EQ (CODING_ATTR_TYPE (attrs), Qiso_2022))
5195 {
5196 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
5197
5198 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
5199 charset_list = Viso_2022_charset_list;
5200 }
5201 else if (EQ (CODING_ATTR_TYPE (attrs), Qemacs_mule))
5202 {
5203 charset_list = Vemacs_mule_charset_list;
5204 }
5205 return charset_list;
5206}
5207
5208
5209/* Return raw-text or one of its subsidiaries that has the same
5210 eol_type as CODING-SYSTEM. */
5211
5212Lisp_Object
5213raw_text_coding_system (coding_system)
5214 Lisp_Object coding_system;
5215{
5216 Lisp_Object spec, attrs;
5217 Lisp_Object eol_type, raw_text_eol_type;
5218
5219 if (NILP (coding_system))
5220 return Qraw_text;
5221 spec = CODING_SYSTEM_SPEC (coding_system);
5222 attrs = AREF (spec, 0);
5223
5224 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
5225 return coding_system;
5226
5227 eol_type = AREF (spec, 2);
5228 if (VECTORP (eol_type))
5229 return Qraw_text;
5230 spec = CODING_SYSTEM_SPEC (Qraw_text);
5231 raw_text_eol_type = AREF (spec, 2);
5232 return (EQ (eol_type, Qunix) ? AREF (raw_text_eol_type, 0)
5233 : EQ (eol_type, Qdos) ? AREF (raw_text_eol_type, 1)
5234 : AREF (raw_text_eol_type, 2));
5235}
5236
5237
5238/* If CODING_SYSTEM doesn't specify end-of-line format but PARENT
5239 does, return one of the subsidiary that has the same eol-spec as
5240 PARENT. Otherwise, return CODING_SYSTEM. If PARENT is nil,
5241 inherit end-of-line format from the system's setting
5242 (system_eol_type). */
5243
5244Lisp_Object
5245coding_inherit_eol_type (coding_system, parent)
5246 Lisp_Object coding_system, parent;
5247{
5248 Lisp_Object spec, eol_type;
5249
5250 if (NILP (coding_system))
5251 coding_system = Qraw_text;
5252 spec = CODING_SYSTEM_SPEC (coding_system);
5253 eol_type = AREF (spec, 2);
5254 if (VECTORP (eol_type))
5255 {
5256 Lisp_Object parent_eol_type;
5257
5258 if (! NILP (parent))
5259 {
5260 Lisp_Object parent_spec;
5261
5262 parent_spec = CODING_SYSTEM_SPEC (parent);
5263 parent_eol_type = AREF (parent_spec, 2);
5264 }
5265 else
5266 parent_eol_type = system_eol_type;
5267 if (EQ (parent_eol_type, Qunix))
5268 coding_system = AREF (eol_type, 0);
5269 else if (EQ (parent_eol_type, Qdos))
5270 coding_system = AREF (eol_type, 1);
5271 else if (EQ (parent_eol_type, Qmac))
5272 coding_system = AREF (eol_type, 2);
5273 }
5274 return coding_system;
5275}
5276
5277/* Emacs has a mechanism to automatically detect a coding system if it
5278 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
5279 it's impossible to distinguish some coding systems accurately
5280 because they use the same range of codes. So, at first, coding
5281 systems are categorized into 7, those are:
5282
5283 o coding-category-emacs-mule
5284
5285 The category for a coding system which has the same code range
5286 as Emacs' internal format. Assigned the coding-system (Lisp
5287 symbol) `emacs-mule' by default.
5288
5289 o coding-category-sjis
5290
5291 The category for a coding system which has the same code range
5292 as SJIS. Assigned the coding-system (Lisp
5293 symbol) `japanese-shift-jis' by default.
5294
5295 o coding-category-iso-7
5296
5297 The category for a coding system which has the same code range
5298 as ISO2022 of 7-bit environment. This doesn't use any locking
5299 shift and single shift functions. This can encode/decode all
5300 charsets. Assigned the coding-system (Lisp symbol)
5301 `iso-2022-7bit' by default.
5302
5303 o coding-category-iso-7-tight
5304
5305 Same as coding-category-iso-7 except that this can
5306 encode/decode only the specified charsets.
5307
5308 o coding-category-iso-8-1
5309
5310 The category for a coding system which has the same code range
5311 as ISO2022 of 8-bit environment and graphic plane 1 used only
5312 for DIMENSION1 charset. This doesn't use any locking shift
5313 and single shift functions. Assigned the coding-system (Lisp
5314 symbol) `iso-latin-1' by default.
5315
5316 o coding-category-iso-8-2
5317
5318 The category for a coding system which has the same code range
5319 as ISO2022 of 8-bit environment and graphic plane 1 used only
5320 for DIMENSION2 charset. This doesn't use any locking shift
5321 and single shift functions. Assigned the coding-system (Lisp
5322 symbol) `japanese-iso-8bit' by default.
5323
5324 o coding-category-iso-7-else
5325
5326 The category for a coding system which has the same code range
5327 as ISO2022 of 7-bit environemnt but uses locking shift or
5328 single shift functions. Assigned the coding-system (Lisp
5329 symbol) `iso-2022-7bit-lock' by default.
5330
5331 o coding-category-iso-8-else
5332
5333 The category for a coding system which has the same code range
5334 as ISO2022 of 8-bit environemnt but uses locking shift or
5335 single shift functions. Assigned the coding-system (Lisp
5336 symbol) `iso-2022-8bit-ss2' by default.
5337
5338 o coding-category-big5
5339
5340 The category for a coding system which has the same code range
5341 as BIG5. Assigned the coding-system (Lisp symbol)
5342 `cn-big5' by default.
5343
5344 o coding-category-utf-8
5345
5346 The category for a coding system which has the same code range
5347 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
5348 symbol) `utf-8' by default.
5349
5350 o coding-category-utf-16-be
5351
5352 The category for a coding system in which a text has an
5353 Unicode signature (cf. Unicode Standard) in the order of BIG
5354 endian at the head. Assigned the coding-system (Lisp symbol)
5355 `utf-16-be' by default.
5356
5357 o coding-category-utf-16-le
5358
5359 The category for a coding system in which a text has an
5360 Unicode signature (cf. Unicode Standard) in the order of
5361 LITTLE endian at the head. Assigned the coding-system (Lisp
5362 symbol) `utf-16-le' by default.
5363
5364 o coding-category-ccl
5365
5366 The category for a coding system of which encoder/decoder is
5367 written in CCL programs. The default value is nil, i.e., no
5368 coding system is assigned.
5369
5370 o coding-category-binary
5371
5372 The category for a coding system not categorized in any of the
5373 above. Assigned the coding-system (Lisp symbol)
5374 `no-conversion' by default.
5375
5376 Each of them is a Lisp symbol and the value is an actual
5377 `coding-system's (this is also a Lisp symbol) assigned by a user.
5378 What Emacs does actually is to detect a category of coding system.
5379 Then, it uses a `coding-system' assigned to it. If Emacs can't
5380 decide only one possible category, it selects a category of the
5381 highest priority. Priorities of categories are also specified by a
5382 user in a Lisp variable `coding-category-list'.
5383
5384*/
5385
5386#define EOL_SEEN_NONE 0
5387#define EOL_SEEN_LF 1
5388#define EOL_SEEN_CR 2
5389#define EOL_SEEN_CRLF 4
5390
5391/* Detect how end-of-line of a text of length SRC_BYTES pointed by
5392 SOURCE is encoded. If CATEGORY is one of
5393 coding_category_utf_16_XXXX, assume that CR and LF are encoded by
5394 two-byte, else they are encoded by one-byte.
5395
5396 Return one of EOL_SEEN_XXX. */
5397
5398#define MAX_EOL_CHECK_COUNT 3
5399
5400static int
5401detect_eol (source, src_bytes, category)
5402 const unsigned char *source;
5403 EMACS_INT src_bytes;
5404 enum coding_category category;
5405{
5406 const unsigned char *src = source, *src_end = src + src_bytes;
5407 unsigned char c;
5408 int total = 0;
5409 int eol_seen = EOL_SEEN_NONE;
5410
5411 if ((1 << category) & CATEGORY_MASK_UTF_16)
5412 {
5413 int msb, lsb;
5414
5415 msb = category == (coding_category_utf_16_le
5416 | coding_category_utf_16_le_nosig);
5417 lsb = 1 - msb;
5418
5419 while (src + 1 < src_end)
5420 {
5421 c = src[lsb];
5422 if (src[msb] == 0 && (c == '\n' || c == '\r'))
5423 {
5424 int this_eol;
5425
5426 if (c == '\n')
5427 this_eol = EOL_SEEN_LF;
5428 else if (src + 3 >= src_end
5429 || src[msb + 2] != 0
5430 || src[lsb + 2] != '\n')
5431 this_eol = EOL_SEEN_CR;
5432 else
5433 this_eol = EOL_SEEN_CRLF;
5434
5435 if (eol_seen == EOL_SEEN_NONE)
5436 /* This is the first end-of-line. */
5437 eol_seen = this_eol;
5438 else if (eol_seen != this_eol)
5439 {
5440 /* The found type is different from what found before. */
5441 eol_seen = EOL_SEEN_LF;
5442 break;
5443 }
5444 if (++total == MAX_EOL_CHECK_COUNT)
5445 break;
5446 }
5447 src += 2;
5448 }
5449 }
5450 else
5451 {
5452 while (src < src_end)
5453 {
5454 c = *src++;
5455 if (c == '\n' || c == '\r')
5456 {
5457 int this_eol;
5458
5459 if (c == '\n')
5460 this_eol = EOL_SEEN_LF;
5461 else if (src >= src_end || *src != '\n')
5462 this_eol = EOL_SEEN_CR;
5463 else
5464 this_eol = EOL_SEEN_CRLF, src++;
5465
5466 if (eol_seen == EOL_SEEN_NONE)
5467 /* This is the first end-of-line. */
5468 eol_seen = this_eol;
5469 else if (eol_seen != this_eol)
5470 {
5471 /* The found type is different from what found before. */
5472 eol_seen = EOL_SEEN_LF;
5473 break;
5474 }
5475 if (++total == MAX_EOL_CHECK_COUNT)
5476 break;
5477 }
5478 }
5479 }
5480 return eol_seen;
5481}
5482
5483
5484static Lisp_Object
5485adjust_coding_eol_type (coding, eol_seen)
5486 struct coding_system *coding;
5487 int eol_seen;
5488{
5489 Lisp_Object eol_type;
5490
5491 eol_type = CODING_ID_EOL_TYPE (coding->id);
5492 if (eol_seen & EOL_SEEN_LF)
5493 {
5494 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 0));
5495 eol_type = Qunix;
5496 }
5497 else if (eol_seen & EOL_SEEN_CRLF)
5498 {
5499 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 1));
5500 eol_type = Qdos;
5501 }
5502 else if (eol_seen & EOL_SEEN_CR)
5503 {
5504 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 2));
5505 eol_type = Qmac;
5506 }
5507 return eol_type;
5508}
5509
5510/* Detect how a text specified in CODING is encoded. If a coding
5511 system is detected, update fields of CODING by the detected coding
5512 system. */
5513
5514void
5515detect_coding (coding)
5516 struct coding_system *coding;
5517{
5518 const unsigned char *src, *src_end;
5519
5520 coding->consumed = coding->consumed_char = 0;
5521 coding->produced = coding->produced_char = 0;
5522 coding_set_source (coding);
5523
5524 src_end = coding->source + coding->src_bytes;
5525
5526 /* If we have not yet decided the text encoding type, detect it
5527 now. */
5528 if (EQ (CODING_ATTR_TYPE (CODING_ID_ATTRS (coding->id)), Qundecided))
5529 {
5530 int c, i;
5531 struct coding_detection_info detect_info;
5532
5533 detect_info.checked = detect_info.found = detect_info.rejected = 0;
5534 for (i = 0, src = coding->source; src < src_end; i++, src++)
5535 {
5536 c = *src;
5537 if (c & 0x80)
5538 break;
5539 if (c < 0x20
5540 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
5541 && ! inhibit_iso_escape_detection
5542 && ! detect_info.checked)
5543 {
5544 coding->head_ascii = src - (coding->source + coding->consumed);
5545 if (detect_coding_iso_2022 (coding, &detect_info))
5546 {
5547 /* We have scanned the whole data. */
5548 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
5549 /* We didn't find an 8-bit code. */
5550 src = src_end;
5551 break;
5552 }
5553 }
5554 }
5555 coding->head_ascii = src - (coding->source + coding->consumed);
5556
5557 if (coding->head_ascii < coding->src_bytes
5558 || detect_info.found)
5559 {
5560 enum coding_category category;
5561 struct coding_system *this;
5562
5563 if (coding->head_ascii == coding->src_bytes)
5564 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
5565 for (i = 0; i < coding_category_raw_text; i++)
5566 {
5567 category = coding_priorities[i];
5568 this = coding_categories + category;
5569 if (detect_info.found & (1 << category))
5570 break;
5571 }
5572 else
5573 for (i = 0; i < coding_category_raw_text; i++)
5574 {
5575 category = coding_priorities[i];
5576 this = coding_categories + category;
5577 if (this->id < 0)
5578 {
5579 /* No coding system of this category is defined. */
5580 detect_info.rejected |= (1 << category);
5581 }
5582 else if (category >= coding_category_raw_text)
5583 continue;
5584 else if (detect_info.checked & (1 << category))
5585 {
5586 if (detect_info.found & (1 << category))
5587 break;
5588 }
5589 else if ((*(this->detector)) (coding, &detect_info)
5590 && detect_info.found & (1 << category))
5591 {
5592 if (category == coding_category_utf_16_auto)
5593 {
5594 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
5595 category = coding_category_utf_16_le;
5596 else
5597 category = coding_category_utf_16_be;
5598 }
5599 break;
5600 }
5601 }
5602
5603 if (i < coding_category_raw_text)
5604 setup_coding_system (CODING_ID_NAME (this->id), coding);
5605 else if (detect_info.rejected == CATEGORY_MASK_ANY)
5606 setup_coding_system (Qraw_text, coding);
5607 else if (detect_info.rejected)
5608 for (i = 0; i < coding_category_raw_text; i++)
5609 if (! (detect_info.rejected & (1 << coding_priorities[i])))
5610 {
5611 this = coding_categories + coding_priorities[i];
5612 setup_coding_system (CODING_ID_NAME (this->id), coding);
5613 break;
5614 }
5615 }
5616 }
5617 else if (XINT (CODING_ATTR_CATEGORY (CODING_ID_ATTRS (coding->id)))
5618 == coding_category_utf_16_auto)
5619 {
5620 Lisp_Object coding_systems;
5621 struct coding_detection_info detect_info;
5622
5623 coding_systems
5624 = AREF (CODING_ID_ATTRS (coding->id), coding_attr_utf_16_bom);
5625 detect_info.found = detect_info.rejected = 0;
5626 if (CONSP (coding_systems)
5627 && detect_coding_utf_16 (coding, &detect_info))
5628 {
5629 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
5630 setup_coding_system (XCAR (coding_systems), coding);
5631 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
5632 setup_coding_system (XCDR (coding_systems), coding);
5633 }
5634 }
5635}
5636
5637
5638static void
5639decode_eol (coding)
5640 struct coding_system *coding;
5641{
5642 Lisp_Object eol_type;
5643 unsigned char *p, *pbeg, *pend;
5644
5645 eol_type = CODING_ID_EOL_TYPE (coding->id);
5646 if (EQ (eol_type, Qunix))
5647 return;
5648
5649 if (NILP (coding->dst_object))
5650 pbeg = coding->destination;
5651 else
5652 pbeg = BYTE_POS_ADDR (coding->dst_pos_byte);
5653 pend = pbeg + coding->produced;
5654
5655 if (VECTORP (eol_type))
5656 {
5657 int eol_seen = EOL_SEEN_NONE;
5658
5659 for (p = pbeg; p < pend; p++)
5660 {
5661 if (*p == '\n')
5662 eol_seen |= EOL_SEEN_LF;
5663 else if (*p == '\r')
5664 {
5665 if (p + 1 < pend && *(p + 1) == '\n')
5666 {
5667 eol_seen |= EOL_SEEN_CRLF;
5668 p++;
5669 }
5670 else
5671 eol_seen |= EOL_SEEN_CR;
5672 }
5673 }
5674 if (eol_seen != EOL_SEEN_NONE
5675 && eol_seen != EOL_SEEN_LF
5676 && eol_seen != EOL_SEEN_CRLF
5677 && eol_seen != EOL_SEEN_CR)
5678 eol_seen = EOL_SEEN_LF;
5679 if (eol_seen != EOL_SEEN_NONE)
5680 eol_type = adjust_coding_eol_type (coding, eol_seen);
5681 }
5682
5683 if (EQ (eol_type, Qmac))
5684 {
5685 for (p = pbeg; p < pend; p++)
5686 if (*p == '\r')
5687 *p = '\n';
5688 }
5689 else if (EQ (eol_type, Qdos))
5690 {
5691 int n = 0;
5692
5693 if (NILP (coding->dst_object))
5694 {
5695 /* Start deleting '\r' from the tail to minimize the memory
5696 movement. */
5697 for (p = pend - 2; p >= pbeg; p--)
5698 if (*p == '\r')
5699 {
5700 safe_bcopy ((char *) (p + 1), (char *) p, pend-- - p - 1);
5701 n++;
5702 }
5703 }
5704 else
5705 {
5706 int pos_byte = coding->dst_pos_byte;
5707 int pos = coding->dst_pos;
5708 int pos_end = pos + coding->produced_char - 1;
5709
5710 while (pos < pos_end)
5711 {
5712 p = BYTE_POS_ADDR (pos_byte);
5713 if (*p == '\r' && p[1] == '\n')
5714 {
5715 del_range_2 (pos, pos_byte, pos + 1, pos_byte + 1, 0);
5716 n++;
5717 pos_end--;
5718 }
5719 pos++;
5720 pos_byte += BYTES_BY_CHAR_HEAD (*p);
5721 }
5722 }
5723 coding->produced -= n;
5724 coding->produced_char -= n;
5725 }
5726}
5727
5728
5729/* Return a translation table (or list of them) from coding system
5730 attribute vector ATTRS for encoding (ENCODEP is nonzero) or
5731 decoding (ENCODEP is zero). */
5732
5733static Lisp_Object
5734get_translation_table (attrs, encodep, max_lookup)
5735 Lisp_Object attrs;
5736 int encodep, *max_lookup;
5737{
5738 Lisp_Object standard, translation_table;
5739 Lisp_Object val;
5740
5741 if (encodep)
5742 translation_table = CODING_ATTR_ENCODE_TBL (attrs),
5743 standard = Vstandard_translation_table_for_encode;
5744 else
5745 translation_table = CODING_ATTR_DECODE_TBL (attrs),
5746 standard = Vstandard_translation_table_for_decode;
5747 if (NILP (translation_table))
5748 translation_table = standard;
5749 else
5750 {
5751 if (SYMBOLP (translation_table))
5752 translation_table = Fget (translation_table, Qtranslation_table);
5753 else if (CONSP (translation_table))
5754 {
5755 translation_table = Fcopy_sequence (translation_table);
5756 for (val = translation_table; CONSP (val); val = XCDR (val))
5757 if (SYMBOLP (XCAR (val)))
5758 XSETCAR (val, Fget (XCAR (val), Qtranslation_table));
5759 }
5760 if (CHAR_TABLE_P (standard))
5761 {
5762 if (CONSP (translation_table))
5763 translation_table = nconc2 (translation_table,
5764 Fcons (standard, Qnil));
5765 else
5766 translation_table = Fcons (translation_table,
5767 Fcons (standard, Qnil));
5768 }
5769 }
5770
5771 if (max_lookup)
5772 {
5773 *max_lookup = 1;
5774 if (CHAR_TABLE_P (translation_table)
5775 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (translation_table)) > 1)
5776 {
5777 val = XCHAR_TABLE (translation_table)->extras[1];
5778 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
5779 *max_lookup = XFASTINT (val);
5780 }
5781 else if (CONSP (translation_table))
5782 {
5783 Lisp_Object tail, val;
5784
5785 for (tail = translation_table; CONSP (tail); tail = XCDR (tail))
5786 if (CHAR_TABLE_P (XCAR (tail))
5787 && CHAR_TABLE_EXTRA_SLOTS (XCHAR_TABLE (XCAR (tail))) > 1)
5788 {
5789 val = XCHAR_TABLE (XCAR (tail))->extras[1];
5790 if (NATNUMP (val) && *max_lookup < XFASTINT (val))
5791 *max_lookup = XFASTINT (val);
5792 }
5793 }
5794 }
5795 return translation_table;
5796}
5797
5798#define LOOKUP_TRANSLATION_TABLE(table, c, trans) \
5799 do { \
5800 trans = Qnil; \
5801 if (CHAR_TABLE_P (table)) \
5802 { \
5803 trans = CHAR_TABLE_REF (table, c); \
5804 if (CHARACTERP (trans)) \
5805 c = XFASTINT (trans), trans = Qnil; \
5806 } \
5807 else if (CONSP (table)) \
5808 { \
5809 Lisp_Object tail; \
5810 \
5811 for (tail = table; CONSP (tail); tail = XCDR (tail)) \
5812 if (CHAR_TABLE_P (XCAR (tail))) \
5813 { \
5814 trans = CHAR_TABLE_REF (XCAR (tail), c); \
5815 if (CHARACTERP (trans)) \
5816 c = XFASTINT (trans), trans = Qnil; \
5817 else if (! NILP (trans)) \
5818 break; \
5819 } \
5820 } \
5821 } while (0)
5822
5823
5824static Lisp_Object
5825get_translation (val, buf, buf_end, last_block, from_nchars, to_nchars)
5826 Lisp_Object val;
5827 int *buf, *buf_end;
5828 int last_block;
5829 int *from_nchars, *to_nchars;
5830{
5831 /* VAL is TO or (([FROM-CHAR ...] . TO) ...) where TO is TO-CHAR or
5832 [TO-CHAR ...]. */
5833 if (CONSP (val))
5834 {
5835 Lisp_Object from, tail;
5836 int i, len;
5837
5838 for (tail = val; CONSP (tail); tail = XCDR (tail))
5839 {
5840 val = XCAR (tail);
5841 from = XCAR (val);
5842 len = ASIZE (from);
5843 for (i = 0; i < len; i++)
5844 {
5845 if (buf + i == buf_end)
5846 {
5847 if (! last_block)
5848 return Qt;
5849 break;
5850 }
5851 if (XINT (AREF (from, i)) != buf[i])
5852 break;
5853 }
5854 if (i == len)
5855 {
5856 val = XCDR (val);
5857 *from_nchars = len;
5858 break;
5859 }
5860 }
5861 if (! CONSP (tail))
5862 return Qnil;
5863 }
5864 if (VECTORP (val))
5865 *buf = XINT (AREF (val, 0)), *to_nchars = ASIZE (val);
5866 else
5867 *buf = XINT (val);
5868 return val;
5869}
5870
5871
5872static int
5873produce_chars (coding, translation_table, last_block)
5874 struct coding_system *coding;
5875 Lisp_Object translation_table;
5876 int last_block;
5877{
5878 unsigned char *dst = coding->destination + coding->produced;
5879 unsigned char *dst_end = coding->destination + coding->dst_bytes;
5880 int produced;
5881 int produced_chars = 0;
5882 int carryover = 0;
5883
5884 if (! coding->chars_at_source)
5885 {
5886 /* Characters are in coding->charbuf. */
5887 int *buf = coding->charbuf;
5888 int *buf_end = buf + coding->charbuf_used;
5889
5890 if (BUFFERP (coding->src_object)
5891 && EQ (coding->src_object, coding->dst_object))
5892 dst_end = ((unsigned char *) coding->source) + coding->consumed;
5893
5894 while (buf < buf_end)
5895 {
5896 int c = *buf, i;
5897
5898 if (c >= 0)
5899 {
5900 int from_nchars = 1, to_nchars = 1;
5901 Lisp_Object trans = Qnil;
5902
5903 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
5904 if (! NILP (trans))
5905 {
5906 trans = get_translation (trans, buf, buf_end, last_block,
5907 &from_nchars, &to_nchars);
5908 if (EQ (trans, Qt))
5909 break;
5910 c = *buf;
5911 }
5912
5913 if (dst + MAX_MULTIBYTE_LENGTH * to_nchars > dst_end)
5914 {
5915 dst = alloc_destination (coding,
5916 buf_end - buf
5917 + MAX_MULTIBYTE_LENGTH * to_nchars,
5918 dst);
5919 dst_end = coding->destination + coding->dst_bytes;
5920 }
5921
5922 for (i = 0; i < to_nchars; i++)
5923 {
5924 if (i > 0)
5925 c = XINT (AREF (trans, i));
5926 if (coding->dst_multibyte
5927 || ! CHAR_BYTE8_P (c))
5928 CHAR_STRING_ADVANCE (c, dst);
5929 else
5930 *dst++ = CHAR_TO_BYTE8 (c);
5931 }
5932 produced_chars += to_nchars;
5933 *buf++ = to_nchars;
5934 while (--from_nchars > 0)
5935 *buf++ = 0;
5936 }
5937 else
5938 /* This is an annotation datum. (-C) is the length. */
5939 buf += -c;
5940 }
5941 carryover = buf_end - buf;
5942 }
5943 else
5944 {
5945 const unsigned char *src = coding->source;
5946 const unsigned char *src_end = src + coding->src_bytes;
5947 Lisp_Object eol_type;
5948
5949 eol_type = CODING_ID_EOL_TYPE (coding->id);
5950
5951 if (coding->src_multibyte != coding->dst_multibyte)
5952 {
5953 if (coding->src_multibyte)
5954 {
5955 int multibytep = 1;
5956 int consumed_chars;
5957
5958 while (1)
5959 {
5960 const unsigned char *src_base = src;
5961 int c;
5962
5963 ONE_MORE_BYTE (c);
5964 if (c == '\r')
5965 {
5966 if (EQ (eol_type, Qdos))
5967 {
5968 if (src == src_end)
5969 {
5970 record_conversion_result
5971 (coding, CODING_RESULT_INSUFFICIENT_SRC);
5972 goto no_more_source;
5973 }
5974 if (*src == '\n')
5975 c = *src++;
5976 }
5977 else if (EQ (eol_type, Qmac))
5978 c = '\n';
5979 }
5980 if (dst == dst_end)
5981 {
5982 coding->consumed = src - coding->source;
5983
5984 if (EQ (coding->src_object, coding->dst_object))
5985 dst_end = (unsigned char *) src;
5986 if (dst == dst_end)
5987 {
5988 dst = alloc_destination (coding, src_end - src + 1,
5989 dst);
5990 dst_end = coding->destination + coding->dst_bytes;
5991 coding_set_source (coding);
5992 src = coding->source + coding->consumed;
5993 src_end = coding->source + coding->src_bytes;
5994 }
5995 }
5996 *dst++ = c;
5997 produced_chars++;
5998 }
5999 no_more_source:
6000 ;
6001 }
6002 else
6003 while (src < src_end)
6004 {
6005 int multibytep = 1;
6006 int c = *src++;
6007
6008 if (c == '\r')
6009 {
6010 if (EQ (eol_type, Qdos))
6011 {
6012 if (src < src_end
6013 && *src == '\n')
6014 c = *src++;
6015 }
6016 else if (EQ (eol_type, Qmac))
6017 c = '\n';
6018 }
6019 if (dst >= dst_end - 1)
6020 {
6021 coding->consumed = src - coding->source;
6022
6023 if (EQ (coding->src_object, coding->dst_object))
6024 dst_end = (unsigned char *) src;
6025 if (dst >= dst_end - 1)
6026 {
6027 dst = alloc_destination (coding, src_end - src + 2,
6028 dst);
6029 dst_end = coding->destination + coding->dst_bytes;
6030 coding_set_source (coding);
6031 src = coding->source + coding->consumed;
6032 src_end = coding->source + coding->src_bytes;
6033 }
6034 }
6035 EMIT_ONE_BYTE (c);
6036 }
6037 }
6038 else
6039 {
6040 if (!EQ (coding->src_object, coding->dst_object))
6041 {
6042 int require = coding->src_bytes - coding->dst_bytes;
6043
6044 if (require > 0)
6045 {
6046 EMACS_INT offset = src - coding->source;
6047
6048 dst = alloc_destination (coding, require, dst);
6049 coding_set_source (coding);
6050 src = coding->source + offset;
6051 src_end = coding->source + coding->src_bytes;
6052 }
6053 }
6054 produced_chars = coding->src_chars;
6055 while (src < src_end)
6056 {
6057 int c = *src++;
6058
6059 if (c == '\r')
6060 {
6061 if (EQ (eol_type, Qdos))
6062 {
6063 if (src < src_end
6064 && *src == '\n')
6065 c = *src++;
6066 produced_chars--;
6067 }
6068 else if (EQ (eol_type, Qmac))
6069 c = '\n';
6070 }
6071 *dst++ = c;
6072 }
6073 }
6074 coding->consumed = coding->src_bytes;
6075 coding->consumed_char = coding->src_chars;
6076 }
6077
6078 produced = dst - (coding->destination + coding->produced);
6079 if (BUFFERP (coding->dst_object) && produced_chars > 0)
6080 insert_from_gap (produced_chars, produced);
6081 coding->produced += produced;
6082 coding->produced_char += produced_chars;
6083 return carryover;
6084}
6085
6086/* Compose text in CODING->object according to the annotation data at
6087 CHARBUF. CHARBUF is an array:
6088 [ -LENGTH ANNOTATION_MASK FROM TO METHOD COMP_LEN [ COMPONENTS... ] ]
6089 */
6090
6091static INLINE void
6092produce_composition (coding, charbuf, pos)
6093 struct coding_system *coding;
6094 int *charbuf;
6095 EMACS_INT pos;
6096{
6097 int len;
6098 EMACS_INT to;
6099 enum composition_method method;
6100 Lisp_Object components;
6101
6102 len = -charbuf[0];
6103 to = pos + charbuf[2];
6104 if (to <= pos)
6105 return;
6106 method = (enum composition_method) (charbuf[3]);
6107
6108 if (method == COMPOSITION_RELATIVE)
6109 components = Qnil;
6110 else if (method >= COMPOSITION_WITH_RULE
6111 && method <= COMPOSITION_WITH_RULE_ALTCHARS)
6112 {
6113 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
6114 int i;
6115
6116 len -= 4;
6117 charbuf += 4;
6118 for (i = 0; i < len; i++)
6119 {
6120 args[i] = make_number (charbuf[i]);
6121 if (charbuf[i] < 0)
6122 return;
6123 }
6124 components = (method == COMPOSITION_WITH_ALTCHARS
6125 ? Fstring (len, args) : Fvector (len, args));
6126 }
6127 else
6128 return;
6129 compose_text (pos, to, components, Qnil, coding->dst_object);
6130}
6131
6132
6133/* Put `charset' property on text in CODING->object according to
6134 the annotation data at CHARBUF. CHARBUF is an array:
6135 [ -LENGTH ANNOTATION_MASK NCHARS CHARSET-ID ]
6136 */
6137
6138static INLINE void
6139produce_charset (coding, charbuf, pos)
6140 struct coding_system *coding;
6141 int *charbuf;
6142 EMACS_INT pos;
6143{
6144 EMACS_INT from = pos - charbuf[2];
6145 struct charset *charset = CHARSET_FROM_ID (charbuf[3]);
6146
6147 Fput_text_property (make_number (from), make_number (pos),
6148 Qcharset, CHARSET_NAME (charset),
6149 coding->dst_object);
6150}
6151
6152
6153#define CHARBUF_SIZE 0x4000
6154
6155#define ALLOC_CONVERSION_WORK_AREA(coding) \
6156 do { \
6157 int size = CHARBUF_SIZE;; \
6158 \
6159 coding->charbuf = NULL; \
6160 while (size > 1024) \
6161 { \
6162 coding->charbuf = (int *) alloca (sizeof (int) * size); \
6163 if (coding->charbuf) \
6164 break; \
6165 size >>= 1; \
6166 } \
6167 if (! coding->charbuf) \
6168 { \
6169 record_conversion_result (coding, CODING_RESULT_INSUFFICIENT_MEM); \
6170 return coding->result; \
6171 } \
6172 coding->charbuf_size = size; \
6173 } while (0)
6174
6175
6176static void
6177produce_annotation (coding, pos)
6178 struct coding_system *coding;
6179 EMACS_INT pos;
6180{
6181 int *charbuf = coding->charbuf;
6182 int *charbuf_end = charbuf + coding->charbuf_used;
6183
6184 if (NILP (coding->dst_object))
6185 return;
6186
6187 while (charbuf < charbuf_end)
6188 {
6189 if (*charbuf >= 0)
6190 pos += *charbuf++;
6191 else
6192 {
6193 int len = -*charbuf;
6194 switch (charbuf[1])
6195 {
6196 case CODING_ANNOTATE_COMPOSITION_MASK:
6197 produce_composition (coding, charbuf, pos);
6198 break;
6199 case CODING_ANNOTATE_CHARSET_MASK:
6200 produce_charset (coding, charbuf, pos);
6201 break;
6202 default:
6203 abort ();
6204 }
6205 charbuf += len;
6206 }
6207 }
6208}
6209
6210/* Decode the data at CODING->src_object into CODING->dst_object.
6211 CODING->src_object is a buffer, a string, or nil.
6212 CODING->dst_object is a buffer.
6213
6214 If CODING->src_object is a buffer, it must be the current buffer.
6215 In this case, if CODING->src_pos is positive, it is a position of
6216 the source text in the buffer, otherwise, the source text is in the
6217 gap area of the buffer, and CODING->src_pos specifies the offset of
6218 the text from GPT (which must be the same as PT). If this is the
6219 same buffer as CODING->dst_object, CODING->src_pos must be
6220 negative.
6221
6222 If CODING->src_object is a string, CODING->src_pos is an index to
6223 that string.
6224
6225 If CODING->src_object is nil, CODING->source must already point to
6226 the non-relocatable memory area. In this case, CODING->src_pos is
6227 an offset from CODING->source.
6228
6229 The decoded data is inserted at the current point of the buffer
6230 CODING->dst_object.
6231*/
6232
6233static int
6234decode_coding (coding)
6235 struct coding_system *coding;
6236{
6237 Lisp_Object attrs;
6238 Lisp_Object undo_list;
6239 Lisp_Object translation_table;
6240 int carryover;
6241 int i;
6242
6243 if (BUFFERP (coding->src_object)
6244 && coding->src_pos > 0
6245 && coding->src_pos < GPT
6246 && coding->src_pos + coding->src_chars > GPT)
6247 move_gap_both (coding->src_pos, coding->src_pos_byte);
6248
6249 undo_list = Qt;
6250 if (BUFFERP (coding->dst_object))
6251 {
6252 if (current_buffer != XBUFFER (coding->dst_object))
6253 set_buffer_internal (XBUFFER (coding->dst_object));
6254 if (GPT != PT)
6255 move_gap_both (PT, PT_BYTE);
6256 undo_list = current_buffer->undo_list;
6257 current_buffer->undo_list = Qt;
6258 }
6259
6260 coding->consumed = coding->consumed_char = 0;
6261 coding->produced = coding->produced_char = 0;
6262 coding->chars_at_source = 0;
6263 record_conversion_result (coding, CODING_RESULT_SUCCESS);
6264 coding->errors = 0;
6265
6266 ALLOC_CONVERSION_WORK_AREA (coding);
6267
6268 attrs = CODING_ID_ATTRS (coding->id);
6269 translation_table = get_translation_table (attrs, 0, NULL);
6270
6271 carryover = 0;
6272 do
6273 {
6274 EMACS_INT pos = coding->dst_pos + coding->produced_char;
6275
6276 coding_set_source (coding);
6277 coding->annotated = 0;
6278 coding->charbuf_used = carryover;
6279 (*(coding->decoder)) (coding);
6280 coding_set_destination (coding);
6281 carryover = produce_chars (coding, translation_table, 0);
6282 if (coding->annotated)
6283 produce_annotation (coding, pos);
6284 for (i = 0; i < carryover; i++)
6285 coding->charbuf[i]
6286 = coding->charbuf[coding->charbuf_used - carryover + i];
6287 }
6288 while (coding->consumed < coding->src_bytes
6289 && (coding->result == CODING_RESULT_SUCCESS
6290 || coding->result == CODING_RESULT_INVALID_SRC));
6291
6292 if (carryover > 0)
6293 {
6294 coding_set_destination (coding);
6295 coding->charbuf_used = carryover;
6296 produce_chars (coding, translation_table, 1);
6297 }
6298
6299 coding->carryover_bytes = 0;
6300 if (coding->consumed < coding->src_bytes)
6301 {
6302 int nbytes = coding->src_bytes - coding->consumed;
6303 const unsigned char *src;
6304
6305 coding_set_source (coding);
6306 coding_set_destination (coding);
6307 src = coding->source + coding->consumed;
6308
6309 if (coding->mode & CODING_MODE_LAST_BLOCK)
6310 {
6311 /* Flush out unprocessed data as binary chars. We are sure
6312 that the number of data is less than the size of
6313 coding->charbuf. */
6314 coding->charbuf_used = 0;
6315 while (nbytes-- > 0)
6316 {
6317 int c = *src++;
6318
6319 if (c & 0x80)
6320 c = BYTE8_TO_CHAR (c);
6321 coding->charbuf[coding->charbuf_used++] = c;
6322 }
6323 produce_chars (coding, Qnil, 1);
6324 }
6325 else
6326 {
6327 /* Record unprocessed bytes in coding->carryover. We are
6328 sure that the number of data is less than the size of
6329 coding->carryover. */
6330 unsigned char *p = coding->carryover;
6331
6332 coding->carryover_bytes = nbytes;
6333 while (nbytes-- > 0)
6334 *p++ = *src++;
6335 }
6336 coding->consumed = coding->src_bytes;
6337 }
6338
6339 if (! EQ (CODING_ID_EOL_TYPE (coding->id), Qunix))
6340 decode_eol (coding);
6341 if (BUFFERP (coding->dst_object))
6342 {
6343 current_buffer->undo_list = undo_list;
6344 record_insert (coding->dst_pos, coding->produced_char);
6345 }
6346 return coding->result;
6347}
6348
6349
6350/* Extract an annotation datum from a composition starting at POS and
6351 ending before LIMIT of CODING->src_object (buffer or string), store
6352 the data in BUF, set *STOP to a starting position of the next
6353 composition (if any) or to LIMIT, and return the address of the
6354 next element of BUF.
6355
6356 If such an annotation is not found, set *STOP to a starting
6357 position of a composition after POS (if any) or to LIMIT, and
6358 return BUF. */
6359
6360static INLINE int *
6361handle_composition_annotation (pos, limit, coding, buf, stop)
6362 EMACS_INT pos, limit;
6363 struct coding_system *coding;
6364 int *buf;
6365 EMACS_INT *stop;
6366{
6367 EMACS_INT start, end;
6368 Lisp_Object prop;
6369
6370 if (! find_composition (pos, limit, &start, &end, &prop, coding->src_object)
6371 || end > limit)
6372 *stop = limit;
6373 else if (start > pos)
6374 *stop = start;
6375 else
6376 {
6377 if (start == pos)
6378 {
6379 /* We found a composition. Store the corresponding
6380 annotation data in BUF. */
6381 int *head = buf;
6382 enum composition_method method = COMPOSITION_METHOD (prop);
6383 int nchars = COMPOSITION_LENGTH (prop);
6384
6385 ADD_COMPOSITION_DATA (buf, nchars, method);
6386 if (method != COMPOSITION_RELATIVE)
6387 {
6388 Lisp_Object components;
6389 int len, i, i_byte;
6390
6391 components = COMPOSITION_COMPONENTS (prop);
6392 if (VECTORP (components))
6393 {
6394 len = XVECTOR (components)->size;
6395 for (i = 0; i < len; i++)
6396 *buf++ = XINT (AREF (components, i));
6397 }
6398 else if (STRINGP (components))
6399 {
6400 len = SCHARS (components);
6401 i = i_byte = 0;
6402 while (i < len)
6403 {
6404 FETCH_STRING_CHAR_ADVANCE (*buf, components, i, i_byte);
6405 buf++;
6406 }
6407 }
6408 else if (INTEGERP (components))
6409 {
6410 len = 1;
6411 *buf++ = XINT (components);
6412 }
6413 else if (CONSP (components))
6414 {
6415 for (len = 0; CONSP (components);
6416 len++, components = XCDR (components))
6417 *buf++ = XINT (XCAR (components));
6418 }
6419 else
6420 abort ();
6421 *head -= len;
6422 }
6423 }
6424
6425 if (find_composition (end, limit, &start, &end, &prop,
6426 coding->src_object)
6427 && end <= limit)
6428 *stop = start;
6429 else
6430 *stop = limit;
6431 }
6432 return buf;
6433}
6434
6435
6436/* Extract an annotation datum from a text property `charset' at POS of
6437 CODING->src_object (buffer of string), store the data in BUF, set
6438 *STOP to the position where the value of `charset' property changes
6439 (limiting by LIMIT), and return the address of the next element of
6440 BUF.
6441
6442 If the property value is nil, set *STOP to the position where the
6443 property value is non-nil (limiting by LIMIT), and return BUF. */
6444
6445static INLINE int *
6446handle_charset_annotation (pos, limit, coding, buf, stop)
6447 EMACS_INT pos, limit;
6448 struct coding_system *coding;
6449 int *buf;
6450 EMACS_INT *stop;
6451{
6452 Lisp_Object val, next;
6453 int id;
6454
6455 val = Fget_text_property (make_number (pos), Qcharset, coding->src_object);
6456 if (! NILP (val) && CHARSETP (val))
6457 id = XINT (CHARSET_SYMBOL_ID (val));
6458 else
6459 id = -1;
6460 ADD_CHARSET_DATA (buf, 0, id);
6461 next = Fnext_single_property_change (make_number (pos), Qcharset,
6462 coding->src_object,
6463 make_number (limit));
6464 *stop = XINT (next);
6465 return buf;
6466}
6467
6468
6469static void
6470consume_chars (coding, translation_table, max_lookup)
6471 struct coding_system *coding;
6472 Lisp_Object translation_table;
6473 int max_lookup;
6474{
6475 int *buf = coding->charbuf;
6476 int *buf_end = coding->charbuf + coding->charbuf_size;
6477 const unsigned char *src = coding->source + coding->consumed;
6478 const unsigned char *src_end = coding->source + coding->src_bytes;
6479 EMACS_INT pos = coding->src_pos + coding->consumed_char;
6480 EMACS_INT end_pos = coding->src_pos + coding->src_chars;
6481 int multibytep = coding->src_multibyte;
6482 Lisp_Object eol_type;
6483 int c;
6484 EMACS_INT stop, stop_composition, stop_charset;
6485 int *lookup_buf = NULL;
6486
6487 if (! NILP (translation_table))
6488 lookup_buf = alloca (sizeof (int) * max_lookup);
6489
6490 eol_type = CODING_ID_EOL_TYPE (coding->id);
6491 if (VECTORP (eol_type))
6492 eol_type = Qunix;
6493
6494 /* Note: composition handling is not yet implemented. */
6495 coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
6496
6497 if (NILP (coding->src_object))
6498 stop = stop_composition = stop_charset = end_pos;
6499 else
6500 {
6501 if (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK)
6502 stop = stop_composition = pos;
6503 else
6504 stop = stop_composition = end_pos;
6505 if (coding->common_flags & CODING_ANNOTATE_CHARSET_MASK)
6506 stop = stop_charset = pos;
6507 else
6508 stop_charset = end_pos;
6509 }
6510
6511 /* Compensate for CRLF and conversion. */
6512 buf_end -= 1 + MAX_ANNOTATION_LENGTH;
6513 while (buf < buf_end)
6514 {
6515 Lisp_Object trans;
6516
6517 if (pos == stop)
6518 {
6519 if (pos == end_pos)
6520 break;
6521 if (pos == stop_composition)
6522 buf = handle_composition_annotation (pos, end_pos, coding,
6523 buf, &stop_composition);
6524 if (pos == stop_charset)
6525 buf = handle_charset_annotation (pos, end_pos, coding,
6526 buf, &stop_charset);
6527 stop = (stop_composition < stop_charset
6528 ? stop_composition : stop_charset);
6529 }
6530
6531 if (! multibytep)
6532 {
6533 EMACS_INT bytes;
6534
6535 if (coding->encoder == encode_coding_raw_text)
6536 c = *src++, pos++;
6537 else if ((bytes = MULTIBYTE_LENGTH (src, src_end)) > 0)
6538 c = STRING_CHAR_ADVANCE (src), pos += bytes;
6539 else
6540 c = BYTE8_TO_CHAR (*src), src++, pos++;
6541 }
6542 else
6543 c = STRING_CHAR_ADVANCE (src), pos++;
6544 if ((c == '\r') && (coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
6545 c = '\n';
6546 if (! EQ (eol_type, Qunix))
6547 {
6548 if (c == '\n')
6549 {
6550 if (EQ (eol_type, Qdos))
6551 *buf++ = '\r';
6552 else
6553 c = '\r';
6554 }
6555 }
6556
6557 trans = Qnil;
6558 LOOKUP_TRANSLATION_TABLE (translation_table, c, trans);
6559 if (NILP (trans))
6560 *buf++ = c;
6561 else
6562 {
6563 int from_nchars = 1, to_nchars = 1;
6564 int *lookup_buf_end;
6565 const unsigned char *p = src;
6566 int i;
6567
6568 lookup_buf[0] = c;
6569 for (i = 1; i < max_lookup && p < src_end; i++)
6570 lookup_buf[i] = STRING_CHAR_ADVANCE (p);
6571 lookup_buf_end = lookup_buf + i;
6572 trans = get_translation (trans, lookup_buf, lookup_buf_end, 1,
6573 &from_nchars, &to_nchars);
6574 if (EQ (trans, Qt)
6575 || buf + to_nchars > buf_end)
6576 break;
6577 *buf++ = *lookup_buf;
6578 for (i = 1; i < to_nchars; i++)
6579 *buf++ = XINT (AREF (trans, i));
6580 for (i = 1; i < from_nchars; i++, pos++)
6581 src += MULTIBYTE_LENGTH_NO_CHECK (src);
6582 }
6583 }
6584
6585 coding->consumed = src - coding->source;
6586 coding->consumed_char = pos - coding->src_pos;
6587 coding->charbuf_used = buf - coding->charbuf;
6588 coding->chars_at_source = 0;
6589}
6590
6591
6592/* Encode the text at CODING->src_object into CODING->dst_object.
6593 CODING->src_object is a buffer or a string.
6594 CODING->dst_object is a buffer or nil.
6595
6596 If CODING->src_object is a buffer, it must be the current buffer.
6597 In this case, if CODING->src_pos is positive, it is a position of
6598 the source text in the buffer, otherwise. the source text is in the
6599 gap area of the buffer, and coding->src_pos specifies the offset of
6600 the text from GPT (which must be the same as PT). If this is the
6601 same buffer as CODING->dst_object, CODING->src_pos must be
6602 negative and CODING should not have `pre-write-conversion'.
6603
6604 If CODING->src_object is a string, CODING should not have
6605 `pre-write-conversion'.
6606
6607 If CODING->dst_object is a buffer, the encoded data is inserted at
6608 the current point of that buffer.
6609
6610 If CODING->dst_object is nil, the encoded data is placed at the
6611 memory area specified by CODING->destination. */
6612
6613static int
6614encode_coding (coding)
6615 struct coding_system *coding;
6616{
6617 Lisp_Object attrs;
6618 Lisp_Object translation_table;
6619 int max_lookup;
6620
6621 attrs = CODING_ID_ATTRS (coding->id);
6622 if (coding->encoder == encode_coding_raw_text)
6623 translation_table = Qnil, max_lookup = 0;
6624 else
6625 translation_table = get_translation_table (attrs, 1, &max_lookup);
6626
6627 if (BUFFERP (coding->dst_object))
6628 {
6629 set_buffer_internal (XBUFFER (coding->dst_object));
6630 coding->dst_multibyte
6631 = ! NILP (current_buffer->enable_multibyte_characters);
6632 }
6633
6634 coding->consumed = coding->consumed_char = 0;
6635 coding->produced = coding->produced_char = 0;
6636 record_conversion_result (coding, CODING_RESULT_SUCCESS);
6637 coding->errors = 0;
6638
6639 ALLOC_CONVERSION_WORK_AREA (coding);
6640
6641 do {
6642 coding_set_source (coding);
6643 consume_chars (coding, translation_table, max_lookup);
6644 coding_set_destination (coding);
6645 (*(coding->encoder)) (coding);
6646 } while (coding->consumed_char < coding->src_chars);
6647
6648 if (BUFFERP (coding->dst_object) && coding->produced_char > 0)
6649 insert_from_gap (coding->produced_char, coding->produced);
6650
6651 return (coding->result);
6652}
6653
6654
6655/* Name (or base name) of work buffer for code conversion. */
6656static Lisp_Object Vcode_conversion_workbuf_name;
6657
6658/* A working buffer used by the top level conversion. Once it is
6659 created, it is never destroyed. It has the name
6660 Vcode_conversion_workbuf_name. The other working buffers are
6661 destroyed after the use is finished, and their names are modified
6662 versions of Vcode_conversion_workbuf_name. */
6663static Lisp_Object Vcode_conversion_reused_workbuf;
6664
6665/* 1 iff Vcode_conversion_reused_workbuf is already in use. */
6666static int reused_workbuf_in_use;
6667
6668
6669/* Return a working buffer of code convesion. MULTIBYTE specifies the
6670 multibyteness of returning buffer. */
6671
6672static Lisp_Object
6673make_conversion_work_buffer (multibyte)
6674 int multibyte;
6675{
6676 Lisp_Object name, workbuf;
6677 struct buffer *current;
6678
6679 if (reused_workbuf_in_use++)
6680 {
6681 name = Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name, Qnil);
6682 workbuf = Fget_buffer_create (name);
6683 }
6684 else
6685 {
6686 name = Vcode_conversion_workbuf_name;
6687 workbuf = Fget_buffer_create (name);
6688 if (NILP (Vcode_conversion_reused_workbuf))
6689 Vcode_conversion_reused_workbuf = workbuf;
6690 }
6691 current = current_buffer;
6692 set_buffer_internal (XBUFFER (workbuf));
6693 Ferase_buffer ();
6694 current_buffer->undo_list = Qt;
6695 current_buffer->enable_multibyte_characters = multibyte ? Qt : Qnil;
6696 set_buffer_internal (current);
6697 return workbuf;
6698}
6699
6700
6701static Lisp_Object
6702code_conversion_restore (arg)
6703 Lisp_Object arg;
6704{
6705 Lisp_Object current, workbuf;
6706 struct gcpro gcpro1;
6707
6708 GCPRO1 (arg);
6709 current = XCAR (arg);
6710 workbuf = XCDR (arg);
6711 if (! NILP (workbuf))
6712 {
6713 if (EQ (workbuf, Vcode_conversion_reused_workbuf))
6714 reused_workbuf_in_use = 0;
6715 else if (! NILP (Fbuffer_live_p (workbuf)))
6716 Fkill_buffer (workbuf);
6717 }
6718 set_buffer_internal (XBUFFER (current));
6719 UNGCPRO;
6720 return Qnil;
6721}
6722
6723Lisp_Object
6724code_conversion_save (with_work_buf, multibyte)
6725 int with_work_buf, multibyte;
6726{
6727 Lisp_Object workbuf = Qnil;
6728
6729 if (with_work_buf)
6730 workbuf = make_conversion_work_buffer (multibyte);
6731 record_unwind_protect (code_conversion_restore,
6732 Fcons (Fcurrent_buffer (), workbuf));
6733 return workbuf;
6734}
6735
6736int
6737decode_coding_gap (coding, chars, bytes)
6738 struct coding_system *coding;
6739 EMACS_INT chars, bytes;
6740{
6741 int count = specpdl_ptr - specpdl;
6742 Lisp_Object attrs;
6743
6744 code_conversion_save (0, 0);
6745
6746 coding->src_object = Fcurrent_buffer ();
6747 coding->src_chars = chars;
6748 coding->src_bytes = bytes;
6749 coding->src_pos = -chars;
6750 coding->src_pos_byte = -bytes;
6751 coding->src_multibyte = chars < bytes;
6752 coding->dst_object = coding->src_object;
6753 coding->dst_pos = PT;
6754 coding->dst_pos_byte = PT_BYTE;
6755 coding->dst_multibyte = ! NILP (current_buffer->enable_multibyte_characters);
6756
6757 if (CODING_REQUIRE_DETECTION (coding))
6758 detect_coding (coding);
6759
6760 coding->mode |= CODING_MODE_LAST_BLOCK;
6761 current_buffer->text->inhibit_shrinking = 1;
6762 decode_coding (coding);
6763 current_buffer->text->inhibit_shrinking = 0;
6764
6765 attrs = CODING_ID_ATTRS (coding->id);
6766 if (! NILP (CODING_ATTR_POST_READ (attrs)))
6767 {
6768 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
6769 Lisp_Object val;
6770
6771 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
6772 val = call1 (CODING_ATTR_POST_READ (attrs),
6773 make_number (coding->produced_char));
6774 CHECK_NATNUM (val);
6775 coding->produced_char += Z - prev_Z;
6776 coding->produced += Z_BYTE - prev_Z_BYTE;
6777 }
6778
6779 unbind_to (count, Qnil);
6780 return coding->result;
6781}
6782
6783int
6784encode_coding_gap (coding, chars, bytes)
6785 struct coding_system *coding;
6786 EMACS_INT chars, bytes;
6787{
6788 int count = specpdl_ptr - specpdl;
6789
6790 code_conversion_save (0, 0);
6791
6792 coding->src_object = Fcurrent_buffer ();
6793 coding->src_chars = chars;
6794 coding->src_bytes = bytes;
6795 coding->src_pos = -chars;
6796 coding->src_pos_byte = -bytes;
6797 coding->src_multibyte = chars < bytes;
6798 coding->dst_object = coding->src_object;
6799 coding->dst_pos = PT;
6800 coding->dst_pos_byte = PT_BYTE;
6801
6802 encode_coding (coding);
6803
6804 unbind_to (count, Qnil);
6805 return coding->result;
6806}
6807
6808
6809/* Decode the text in the range FROM/FROM_BYTE and TO/TO_BYTE in
6810 SRC_OBJECT into DST_OBJECT by coding context CODING.
6811
6812 SRC_OBJECT is a buffer, a string, or Qnil.
6813
6814 If it is a buffer, the text is at point of the buffer. FROM and TO
6815 are positions in the buffer.
6816
6817 If it is a string, the text is at the beginning of the string.
6818 FROM and TO are indices to the string.
6819
6820 If it is nil, the text is at coding->source. FROM and TO are
6821 indices to coding->source.
6822
6823 DST_OBJECT is a buffer, Qt, or Qnil.
6824
6825 If it is a buffer, the decoded text is inserted at point of the
6826 buffer. If the buffer is the same as SRC_OBJECT, the source text
6827 is deleted.
6828
6829 If it is Qt, a string is made from the decoded text, and
6830 set in CODING->dst_object.
6831
6832 If it is Qnil, the decoded text is stored at CODING->destination.
6833 The caller must allocate CODING->dst_bytes bytes at
6834 CODING->destination by xmalloc. If the decoded text is longer than
6835 CODING->dst_bytes, CODING->destination is relocated by xrealloc.
6836 */
6837
6838void
6839decode_coding_object (coding, src_object, from, from_byte, to, to_byte,
6840 dst_object)
6841 struct coding_system *coding;
6842 Lisp_Object src_object;
6843 EMACS_INT from, from_byte, to, to_byte;
6844 Lisp_Object dst_object;
6845{
6846 int count = specpdl_ptr - specpdl;
6847 unsigned char *destination;
6848 EMACS_INT dst_bytes;
6849 EMACS_INT chars = to - from;
6850 EMACS_INT bytes = to_byte - from_byte;
6851 Lisp_Object attrs;
6852 Lisp_Object buffer;
6853 int saved_pt = -1, saved_pt_byte;
6854
6855 buffer = Fcurrent_buffer ();
6856
6857 if (NILP (dst_object))
6858 {
6859 destination = coding->destination;
6860 dst_bytes = coding->dst_bytes;
6861 }
6862
6863 coding->src_object = src_object;
6864 coding->src_chars = chars;
6865 coding->src_bytes = bytes;
6866 coding->src_multibyte = chars < bytes;
6867
6868 if (STRINGP (src_object))
6869 {
6870 coding->src_pos = from;
6871 coding->src_pos_byte = from_byte;
6872 }
6873 else if (BUFFERP (src_object))
6874 {
6875 set_buffer_internal (XBUFFER (src_object));
6876 if (from != GPT)
6877 move_gap_both (from, from_byte);
6878 if (EQ (src_object, dst_object))
6879 {
6880 saved_pt = PT, saved_pt_byte = PT_BYTE;
6881 TEMP_SET_PT_BOTH (from, from_byte);
6882 del_range_both (from, from_byte, to, to_byte, 1);
6883 coding->src_pos = -chars;
6884 coding->src_pos_byte = -bytes;
6885 }
6886 else
6887 {
6888 coding->src_pos = from;
6889 coding->src_pos_byte = from_byte;
6890 }
6891 }
6892
6893 if (CODING_REQUIRE_DETECTION (coding))
6894 detect_coding (coding);
6895 attrs = CODING_ID_ATTRS (coding->id);
6896
6897 if (EQ (dst_object, Qt)
6898 || (! NILP (CODING_ATTR_POST_READ (attrs))
6899 && NILP (dst_object)))
6900 {
6901 coding->dst_object = code_conversion_save (1, 1);
6902 coding->dst_pos = BEG;
6903 coding->dst_pos_byte = BEG_BYTE;
6904 coding->dst_multibyte = 1;
6905 }
6906 else if (BUFFERP (dst_object))
6907 {
6908 code_conversion_save (0, 0);
6909 coding->dst_object = dst_object;
6910 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
6911 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
6912 coding->dst_multibyte
6913 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
6914 }
6915 else
6916 {
6917 code_conversion_save (0, 0);
6918 coding->dst_object = Qnil;
6919 coding->dst_multibyte = 1;
6920 }
6921
6922 decode_coding (coding);
6923
6924 if (BUFFERP (coding->dst_object))
6925 set_buffer_internal (XBUFFER (coding->dst_object));
6926
6927 if (! NILP (CODING_ATTR_POST_READ (attrs)))
6928 {
6929 struct gcpro gcpro1, gcpro2;
6930 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
6931 Lisp_Object val;
6932
6933 TEMP_SET_PT_BOTH (coding->dst_pos, coding->dst_pos_byte);
6934 GCPRO2 (coding->src_object, coding->dst_object);
6935 val = safe_call1 (CODING_ATTR_POST_READ (attrs),
6936 make_number (coding->produced_char));
6937 UNGCPRO;
6938 CHECK_NATNUM (val);
6939 coding->produced_char += Z - prev_Z;
6940 coding->produced += Z_BYTE - prev_Z_BYTE;
6941 }
6942
6943 if (EQ (dst_object, Qt))
6944 {
6945 coding->dst_object = Fbuffer_string ();
6946 }
6947 else if (NILP (dst_object) && BUFFERP (coding->dst_object))
6948 {
6949 set_buffer_internal (XBUFFER (coding->dst_object));
6950 if (dst_bytes < coding->produced)
6951 {
6952 destination
6953 = (unsigned char *) xrealloc (destination, coding->produced);
6954 if (! destination)
6955 {
6956 record_conversion_result (coding,
6957 CODING_RESULT_INSUFFICIENT_DST);
6958 unbind_to (count, Qnil);
6959 return;
6960 }
6961 if (BEGV < GPT && GPT < BEGV + coding->produced_char)
6962 move_gap_both (BEGV, BEGV_BYTE);
6963 bcopy (BEGV_ADDR, destination, coding->produced);
6964 coding->destination = destination;
6965 }
6966 }
6967
6968 if (saved_pt >= 0)
6969 {
6970 /* This is the case of:
6971 (BUFFERP (src_object) && EQ (src_object, dst_object))
6972 As we have moved PT while replacing the original buffer
6973 contents, we must recover it now. */
6974 set_buffer_internal (XBUFFER (src_object));
6975 if (saved_pt < from)
6976 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
6977 else if (saved_pt < from + chars)
6978 TEMP_SET_PT_BOTH (from, from_byte);
6979 else if (! NILP (current_buffer->enable_multibyte_characters))
6980 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
6981 saved_pt_byte + (coding->produced - bytes));
6982 else
6983 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
6984 saved_pt_byte + (coding->produced - bytes));
6985 }
6986
6987 unbind_to (count, coding->dst_object);
6988}
6989
6990
6991void
6992encode_coding_object (coding, src_object, from, from_byte, to, to_byte,
6993 dst_object)
6994 struct coding_system *coding;
6995 Lisp_Object src_object;
6996 EMACS_INT from, from_byte, to, to_byte;
6997 Lisp_Object dst_object;
6998{
6999 int count = specpdl_ptr - specpdl;
7000 EMACS_INT chars = to - from;
7001 EMACS_INT bytes = to_byte - from_byte;
7002 Lisp_Object attrs;
7003 Lisp_Object buffer;
7004 int saved_pt = -1, saved_pt_byte;
7005 int kill_src_buffer = 0;
7006
7007 buffer = Fcurrent_buffer ();
7008
7009 coding->src_object = src_object;
7010 coding->src_chars = chars;
7011 coding->src_bytes = bytes;
7012 coding->src_multibyte = chars < bytes;
7013
7014 attrs = CODING_ID_ATTRS (coding->id);
7015
7016 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
7017 {
7018 coding->src_object = code_conversion_save (1, coding->src_multibyte);
7019 set_buffer_internal (XBUFFER (coding->src_object));
7020 if (STRINGP (src_object))
7021 insert_from_string (src_object, from, from_byte, chars, bytes, 0);
7022 else if (BUFFERP (src_object))
7023 insert_from_buffer (XBUFFER (src_object), from, chars, 0);
7024 else
7025 insert_1_both (coding->source + from, chars, bytes, 0, 0, 0);
7026
7027 if (EQ (src_object, dst_object))
7028 {
7029 set_buffer_internal (XBUFFER (src_object));
7030 saved_pt = PT, saved_pt_byte = PT_BYTE;
7031 del_range_both (from, from_byte, to, to_byte, 1);
7032 set_buffer_internal (XBUFFER (coding->src_object));
7033 }
7034
7035 {
7036 Lisp_Object args[3];
7037
7038 args[0] = CODING_ATTR_PRE_WRITE (attrs);
7039 args[1] = make_number (BEG);
7040 args[2] = make_number (Z);
7041 safe_call (3, args);
7042 }
7043 if (XBUFFER (coding->src_object) != current_buffer)
7044 kill_src_buffer = 1;
7045 coding->src_object = Fcurrent_buffer ();
7046 if (BEG != GPT)
7047 move_gap_both (BEG, BEG_BYTE);
7048 coding->src_chars = Z - BEG;
7049 coding->src_bytes = Z_BYTE - BEG_BYTE;
7050 coding->src_pos = BEG;
7051 coding->src_pos_byte = BEG_BYTE;
7052 coding->src_multibyte = Z < Z_BYTE;
7053 }
7054 else if (STRINGP (src_object))
7055 {
7056 code_conversion_save (0, 0);
7057 coding->src_pos = from;
7058 coding->src_pos_byte = from_byte;
7059 }
7060 else if (BUFFERP (src_object))
7061 {
7062 code_conversion_save (0, 0);
7063 set_buffer_internal (XBUFFER (src_object));
7064 if (EQ (src_object, dst_object))
7065 {
7066 saved_pt = PT, saved_pt_byte = PT_BYTE;
7067 coding->src_object = del_range_1 (from, to, 1, 1);
7068 coding->src_pos = 0;
7069 coding->src_pos_byte = 0;
7070 }
7071 else
7072 {
7073 if (from < GPT && to >= GPT)
7074 move_gap_both (from, from_byte);
7075 coding->src_pos = from;
7076 coding->src_pos_byte = from_byte;
7077 }
7078 }
7079 else
7080 code_conversion_save (0, 0);
7081
7082 if (BUFFERP (dst_object))
7083 {
7084 coding->dst_object = dst_object;
7085 if (EQ (src_object, dst_object))
7086 {
7087 coding->dst_pos = from;
7088 coding->dst_pos_byte = from_byte;
7089 }
7090 else
7091 {
7092 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
7093 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
7094 }
7095 coding->dst_multibyte
7096 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
7097 }
7098 else if (EQ (dst_object, Qt))
7099 {
7100 coding->dst_object = Qnil;
7101 coding->dst_bytes = coding->src_chars;
7102 if (coding->dst_bytes == 0)
7103 coding->dst_bytes = 1;
7104 coding->destination = (unsigned char *) xmalloc (coding->dst_bytes);
7105 coding->dst_multibyte = 0;
7106 }
7107 else
7108 {
7109 coding->dst_object = Qnil;
7110 coding->dst_multibyte = 0;
7111 }
7112
7113 encode_coding (coding);
7114
7115 if (EQ (dst_object, Qt))
7116 {
7117 if (BUFFERP (coding->dst_object))
7118 coding->dst_object = Fbuffer_string ();
7119 else
7120 {
7121 coding->dst_object
7122 = make_unibyte_string ((char *) coding->destination,
7123 coding->produced);
7124 xfree (coding->destination);
7125 }
7126 }
7127
7128 if (saved_pt >= 0)
7129 {
7130 /* This is the case of:
7131 (BUFFERP (src_object) && EQ (src_object, dst_object))
7132 As we have moved PT while replacing the original buffer
7133 contents, we must recover it now. */
7134 set_buffer_internal (XBUFFER (src_object));
7135 if (saved_pt < from)
7136 TEMP_SET_PT_BOTH (saved_pt, saved_pt_byte);
7137 else if (saved_pt < from + chars)
7138 TEMP_SET_PT_BOTH (from, from_byte);
7139 else if (! NILP (current_buffer->enable_multibyte_characters))
7140 TEMP_SET_PT_BOTH (saved_pt + (coding->produced_char - chars),
7141 saved_pt_byte + (coding->produced - bytes));
7142 else
7143 TEMP_SET_PT_BOTH (saved_pt + (coding->produced - bytes),
7144 saved_pt_byte + (coding->produced - bytes));
7145 }
7146
7147 if (kill_src_buffer)
7148 Fkill_buffer (coding->src_object);
7149 unbind_to (count, Qnil);
7150}
7151
7152
7153Lisp_Object
7154preferred_coding_system ()
7155{
7156 int id = coding_categories[coding_priorities[0]].id;
7157
7158 return CODING_ID_NAME (id);
7159}
7160
7161\f
7162#ifdef emacs
7163/*** 8. Emacs Lisp library functions ***/
7164
7165DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0,
7166 doc: /* Return t if OBJECT is nil or a coding-system.
7167See the documentation of `define-coding-system' for information
7168about coding-system objects. */)
7169 (obj)
7170 Lisp_Object obj;
7171{
7172 if (NILP (obj)
7173 || CODING_SYSTEM_ID (obj) >= 0)
7174 return Qt;
7175 if (! SYMBOLP (obj)
7176 || NILP (Fget (obj, Qcoding_system_define_form)))
7177 return Qnil;
7178 return Qt;
7179}
7180
7181DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system,
7182 Sread_non_nil_coding_system, 1, 1, 0,
7183 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
7184 (prompt)
7185 Lisp_Object prompt;
7186{
7187 Lisp_Object val;
7188 do
7189 {
7190 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
7191 Qt, Qnil, Qcoding_system_history, Qnil, Qnil);
7192 }
7193 while (SCHARS (val) == 0);
7194 return (Fintern (val, Qnil));
7195}
7196
7197DEFUN ("read-coding-system", Fread_coding_system, Sread_coding_system, 1, 2, 0,
7198 doc: /* Read a coding system from the minibuffer, prompting with string PROMPT.
7199If the user enters null input, return second argument DEFAULT-CODING-SYSTEM.
7200Ignores case when completing coding systems (all Emacs coding systems
7201are lower-case). */)
7202 (prompt, default_coding_system)
7203 Lisp_Object prompt, default_coding_system;
7204{
7205 Lisp_Object val;
7206 int count = SPECPDL_INDEX ();
7207
7208 if (SYMBOLP (default_coding_system))
7209 default_coding_system = SYMBOL_NAME (default_coding_system);
7210 specbind (Qcompletion_ignore_case, Qt);
7211 val = Fcompleting_read (prompt, Vcoding_system_alist, Qnil,
7212 Qt, Qnil, Qcoding_system_history,
7213 default_coding_system, Qnil);
7214 unbind_to (count, Qnil);
7215 return (SCHARS (val) == 0 ? Qnil : Fintern (val, Qnil));
7216}
7217
7218DEFUN ("check-coding-system", Fcheck_coding_system, Scheck_coding_system,
7219 1, 1, 0,
7220 doc: /* Check validity of CODING-SYSTEM.
7221If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
7222It is valid if it is nil or a symbol defined as a coding system by the
7223function `define-coding-system'. */)
7224 (coding_system)
7225 Lisp_Object coding_system;
7226{
7227 Lisp_Object define_form;
7228
7229 define_form = Fget (coding_system, Qcoding_system_define_form);
7230 if (! NILP (define_form))
7231 {
7232 Fput (coding_system, Qcoding_system_define_form, Qnil);
7233 safe_eval (define_form);
7234 }
7235 if (!NILP (Fcoding_system_p (coding_system)))
7236 return coding_system;
7237 xsignal1 (Qcoding_system_error, coding_system);
7238}
7239
7240\f
7241/* Detect how the bytes at SRC of length SRC_BYTES are encoded. If
7242 HIGHEST is nonzero, return the coding system of the highest
7243 priority among the detected coding systems. Otherwize return a
7244 list of detected coding systems sorted by their priorities. If
7245 MULTIBYTEP is nonzero, it is assumed that the bytes are in correct
7246 multibyte form but contains only ASCII and eight-bit chars.
7247 Otherwise, the bytes are raw bytes.
7248
7249 CODING-SYSTEM controls the detection as below:
7250
7251 If it is nil, detect both text-format and eol-format. If the
7252 text-format part of CODING-SYSTEM is already specified
7253 (e.g. `iso-latin-1'), detect only eol-format. If the eol-format
7254 part of CODING-SYSTEM is already specified (e.g. `undecided-unix'),
7255 detect only text-format. */
7256
7257Lisp_Object
7258detect_coding_system (src, src_chars, src_bytes, highest, multibytep,
7259 coding_system)
7260 const unsigned char *src;
7261 int src_chars, src_bytes, highest;
7262 int multibytep;
7263 Lisp_Object coding_system;
7264{
7265 const unsigned char *src_end = src + src_bytes;
7266 Lisp_Object attrs, eol_type;
7267 Lisp_Object val;
7268 struct coding_system coding;
7269 int id;
7270 struct coding_detection_info detect_info;
7271 enum coding_category base_category;
7272
7273 if (NILP (coding_system))
7274 coding_system = Qundecided;
7275 setup_coding_system (coding_system, &coding);
7276 attrs = CODING_ID_ATTRS (coding.id);
7277 eol_type = CODING_ID_EOL_TYPE (coding.id);
7278 coding_system = CODING_ATTR_BASE_NAME (attrs);
7279
7280 coding.source = src;
7281 coding.src_chars = src_chars;
7282 coding.src_bytes = src_bytes;
7283 coding.src_multibyte = multibytep;
7284 coding.consumed = 0;
7285 coding.mode |= CODING_MODE_LAST_BLOCK;
7286
7287 detect_info.checked = detect_info.found = detect_info.rejected = 0;
7288
7289 /* At first, detect text-format if necessary. */
7290 base_category = XINT (CODING_ATTR_CATEGORY (attrs));
7291 if (base_category == coding_category_undecided)
7292 {
7293 enum coding_category category;
7294 struct coding_system *this;
7295 int c, i;
7296
7297 /* Skip all ASCII bytes except for a few ISO2022 controls. */
7298 for (i = 0; src < src_end; i++, src++)
7299 {
7300 c = *src;
7301 if (c & 0x80)
7302 break;
7303 if (c < 0x20
7304 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO)
7305 && ! inhibit_iso_escape_detection)
7306 {
7307 coding.head_ascii = src - coding.source;
7308 if (detect_coding_iso_2022 (&coding, &detect_info))
7309 {
7310 /* We have scanned the whole data. */
7311 if (! (detect_info.rejected & CATEGORY_MASK_ISO_7_ELSE))
7312 /* We didn't find an 8-bit code. */
7313 src = src_end;
7314 break;
7315 }
7316 }
7317 }
7318 coding.head_ascii = src - coding.source;
7319
7320 if (src < src_end
7321 || detect_info.found)
7322 {
7323 if (src == src_end)
7324 /* As all bytes are 7-bit, we can ignore non-ISO-2022 codings. */
7325 for (i = 0; i < coding_category_raw_text; i++)
7326 {
7327 category = coding_priorities[i];
7328 this = coding_categories + category;
7329 if (detect_info.found & (1 << category))
7330 break;
7331 }
7332 else
7333 for (i = 0; i < coding_category_raw_text; i++)
7334 {
7335 category = coding_priorities[i];
7336 this = coding_categories + category;
7337
7338 if (this->id < 0)
7339 {
7340 /* No coding system of this category is defined. */
7341 detect_info.rejected |= (1 << category);
7342 }
7343 else if (category >= coding_category_raw_text)
7344 continue;
7345 else if (detect_info.checked & (1 << category))
7346 {
7347 if (highest
7348 && (detect_info.found & (1 << category)))
7349 break;
7350 }
7351 else
7352 {
7353 if ((*(this->detector)) (&coding, &detect_info)
7354 && highest
7355 && (detect_info.found & (1 << category)))
7356 {
7357 if (category == coding_category_utf_16_auto)
7358 {
7359 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
7360 category = coding_category_utf_16_le;
7361 else
7362 category = coding_category_utf_16_be;
7363 }
7364 break;
7365 }
7366 }
7367 }
7368 }
7369
7370 if (detect_info.rejected == CATEGORY_MASK_ANY)
7371 {
7372 detect_info.found = CATEGORY_MASK_RAW_TEXT;
7373 id = coding_categories[coding_category_raw_text].id;
7374 val = Fcons (make_number (id), Qnil);
7375 }
7376 else if (! detect_info.rejected && ! detect_info.found)
7377 {
7378 detect_info.found = CATEGORY_MASK_ANY;
7379 id = coding_categories[coding_category_undecided].id;
7380 val = Fcons (make_number (id), Qnil);
7381 }
7382 else if (highest)
7383 {
7384 if (detect_info.found)
7385 {
7386 detect_info.found = 1 << category;
7387 val = Fcons (make_number (this->id), Qnil);
7388 }
7389 else
7390 for (i = 0; i < coding_category_raw_text; i++)
7391 if (! (detect_info.rejected & (1 << coding_priorities[i])))
7392 {
7393 detect_info.found = 1 << coding_priorities[i];
7394 id = coding_categories[coding_priorities[i]].id;
7395 val = Fcons (make_number (id), Qnil);
7396 break;
7397 }
7398 }
7399 else
7400 {
7401 int mask = detect_info.rejected | detect_info.found;
7402 int found = 0;
7403 val = Qnil;
7404
7405 for (i = coding_category_raw_text - 1; i >= 0; i--)
7406 {
7407 category = coding_priorities[i];
7408 if (! (mask & (1 << category)))
7409 {
7410 found |= 1 << category;
7411 id = coding_categories[category].id;
7412 if (id >= 0)
7413 val = Fcons (make_number (id), val);
7414 }
7415 }
7416 for (i = coding_category_raw_text - 1; i >= 0; i--)
7417 {
7418 category = coding_priorities[i];
7419 if (detect_info.found & (1 << category))
7420 {
7421 id = coding_categories[category].id;
7422 val = Fcons (make_number (id), val);
7423 }
7424 }
7425 detect_info.found |= found;
7426 }
7427 }
7428 else if (base_category == coding_category_utf_16_auto)
7429 {
7430 if (detect_coding_utf_16 (&coding, &detect_info))
7431 {
7432 struct coding_system *this;
7433
7434 if (detect_info.found & CATEGORY_MASK_UTF_16_LE)
7435 this = coding_categories + coding_category_utf_16_le;
7436 else if (detect_info.found & CATEGORY_MASK_UTF_16_BE)
7437 this = coding_categories + coding_category_utf_16_be;
7438 else if (detect_info.rejected & CATEGORY_MASK_UTF_16_LE_NOSIG)
7439 this = coding_categories + coding_category_utf_16_be_nosig;
7440 else
7441 this = coding_categories + coding_category_utf_16_le_nosig;
7442 val = Fcons (make_number (this->id), Qnil);
7443 }
7444 }
7445 else
7446 {
7447 detect_info.found = 1 << XINT (CODING_ATTR_CATEGORY (attrs));
7448 val = Fcons (make_number (coding.id), Qnil);
7449 }
7450
7451 /* Then, detect eol-format if necessary. */
7452 {
7453 int normal_eol = -1, utf_16_be_eol = -1, utf_16_le_eol;
7454 Lisp_Object tail;
7455
7456 if (VECTORP (eol_type))
7457 {
7458 if (detect_info.found & ~CATEGORY_MASK_UTF_16)
7459 normal_eol = detect_eol (coding.source, src_bytes,
7460 coding_category_raw_text);
7461 if (detect_info.found & (CATEGORY_MASK_UTF_16_BE
7462 | CATEGORY_MASK_UTF_16_BE_NOSIG))
7463 utf_16_be_eol = detect_eol (coding.source, src_bytes,
7464 coding_category_utf_16_be);
7465 if (detect_info.found & (CATEGORY_MASK_UTF_16_LE
7466 | CATEGORY_MASK_UTF_16_LE_NOSIG))
7467 utf_16_le_eol = detect_eol (coding.source, src_bytes,
7468 coding_category_utf_16_le);
7469 }
7470 else
7471 {
7472 if (EQ (eol_type, Qunix))
7473 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_LF;
7474 else if (EQ (eol_type, Qdos))
7475 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CRLF;
7476 else
7477 normal_eol = utf_16_be_eol = utf_16_le_eol = EOL_SEEN_CR;
7478 }
7479
7480 for (tail = val; CONSP (tail); tail = XCDR (tail))
7481 {
7482 enum coding_category category;
7483 int this_eol;
7484
7485 id = XINT (XCAR (tail));
7486 attrs = CODING_ID_ATTRS (id);
7487 category = XINT (CODING_ATTR_CATEGORY (attrs));
7488 eol_type = CODING_ID_EOL_TYPE (id);
7489 if (VECTORP (eol_type))
7490 {
7491 if (category == coding_category_utf_16_be
7492 || category == coding_category_utf_16_be_nosig)
7493 this_eol = utf_16_be_eol;
7494 else if (category == coding_category_utf_16_le
7495 || category == coding_category_utf_16_le_nosig)
7496 this_eol = utf_16_le_eol;
7497 else
7498 this_eol = normal_eol;
7499
7500 if (this_eol == EOL_SEEN_LF)
7501 XSETCAR (tail, AREF (eol_type, 0));
7502 else if (this_eol == EOL_SEEN_CRLF)
7503 XSETCAR (tail, AREF (eol_type, 1));
7504 else if (this_eol == EOL_SEEN_CR)
7505 XSETCAR (tail, AREF (eol_type, 2));
7506 else
7507 XSETCAR (tail, CODING_ID_NAME (id));
7508 }
7509 else
7510 XSETCAR (tail, CODING_ID_NAME (id));
7511 }
7512 }
7513
7514 return (highest ? XCAR (val) : val);
7515}
7516
7517
7518DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region,
7519 2, 3, 0,
7520 doc: /* Detect coding system of the text in the region between START and END.
7521Return a list of possible coding systems ordered by priority.
7522
7523If only ASCII characters are found (except for such ISO-2022 control
7524characters ISO-2022 as ESC), it returns a list of single element
7525`undecided' or its subsidiary coding system according to a detected
7526end-of-line format.
7527
7528If optional argument HIGHEST is non-nil, return the coding system of
7529highest priority. */)
7530 (start, end, highest)
7531 Lisp_Object start, end, highest;
7532{
7533 int from, to;
7534 int from_byte, to_byte;
7535
7536 CHECK_NUMBER_COERCE_MARKER (start);
7537 CHECK_NUMBER_COERCE_MARKER (end);
7538
7539 validate_region (&start, &end);
7540 from = XINT (start), to = XINT (end);
7541 from_byte = CHAR_TO_BYTE (from);
7542 to_byte = CHAR_TO_BYTE (to);
7543
7544 if (from < GPT && to >= GPT)
7545 move_gap_both (to, to_byte);
7546
7547 return detect_coding_system (BYTE_POS_ADDR (from_byte),
7548 to - from, to_byte - from_byte,
7549 !NILP (highest),
7550 !NILP (current_buffer
7551 ->enable_multibyte_characters),
7552 Qnil);
7553}
7554
7555DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string,
7556 1, 2, 0,
7557 doc: /* Detect coding system of the text in STRING.
7558Return a list of possible coding systems ordered by priority.
7559
7560If only ASCII characters are found (except for such ISO-2022 control
7561characters ISO-2022 as ESC), it returns a list of single element
7562`undecided' or its subsidiary coding system according to a detected
7563end-of-line format.
7564
7565If optional argument HIGHEST is non-nil, return the coding system of
7566highest priority. */)
7567 (string, highest)
7568 Lisp_Object string, highest;
7569{
7570 CHECK_STRING (string);
7571
7572 return detect_coding_system (SDATA (string),
7573 SCHARS (string), SBYTES (string),
7574 !NILP (highest), STRING_MULTIBYTE (string),
7575 Qnil);
7576}
7577
7578
7579static INLINE int
7580char_encodable_p (c, attrs)
7581 int c;
7582 Lisp_Object attrs;
7583{
7584 Lisp_Object tail;
7585 struct charset *charset;
7586 Lisp_Object translation_table;
7587
7588 translation_table = CODING_ATTR_TRANS_TBL (attrs);
7589 if (! NILP (translation_table))
7590 c = translate_char (translation_table, c);
7591 for (tail = CODING_ATTR_CHARSET_LIST (attrs);
7592 CONSP (tail); tail = XCDR (tail))
7593 {
7594 charset = CHARSET_FROM_ID (XINT (XCAR (tail)));
7595 if (CHAR_CHARSET_P (c, charset))
7596 break;
7597 }
7598 return (! NILP (tail));
7599}
7600
7601
7602/* Return a list of coding systems that safely encode the text between
7603 START and END. If EXCLUDE is non-nil, it is a list of coding
7604 systems not to check. The returned list doesn't contain any such
7605 coding systems. In any case, if the text contains only ASCII or is
7606 unibyte, return t. */
7607
7608DEFUN ("find-coding-systems-region-internal",
7609 Ffind_coding_systems_region_internal,
7610 Sfind_coding_systems_region_internal, 2, 3, 0,
7611 doc: /* Internal use only. */)
7612 (start, end, exclude)
7613 Lisp_Object start, end, exclude;
7614{
7615 Lisp_Object coding_attrs_list, safe_codings;
7616 EMACS_INT start_byte, end_byte;
7617 const unsigned char *p, *pbeg, *pend;
7618 int c;
7619 Lisp_Object tail, elt;
7620
7621 if (STRINGP (start))
7622 {
7623 if (!STRING_MULTIBYTE (start)
7624 || SCHARS (start) == SBYTES (start))
7625 return Qt;
7626 start_byte = 0;
7627 end_byte = SBYTES (start);
7628 }
7629 else
7630 {
7631 CHECK_NUMBER_COERCE_MARKER (start);
7632 CHECK_NUMBER_COERCE_MARKER (end);
7633 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
7634 args_out_of_range (start, end);
7635 if (NILP (current_buffer->enable_multibyte_characters))
7636 return Qt;
7637 start_byte = CHAR_TO_BYTE (XINT (start));
7638 end_byte = CHAR_TO_BYTE (XINT (end));
7639 if (XINT (end) - XINT (start) == end_byte - start_byte)
7640 return Qt;
7641
7642 if (XINT (start) < GPT && XINT (end) > GPT)
7643 {
7644 if ((GPT - XINT (start)) < (XINT (end) - GPT))
7645 move_gap_both (XINT (start), start_byte);
7646 else
7647 move_gap_both (XINT (end), end_byte);
7648 }
7649 }
7650
7651 coding_attrs_list = Qnil;
7652 for (tail = Vcoding_system_list; CONSP (tail); tail = XCDR (tail))
7653 if (NILP (exclude)
7654 || NILP (Fmemq (XCAR (tail), exclude)))
7655 {
7656 Lisp_Object attrs;
7657
7658 attrs = AREF (CODING_SYSTEM_SPEC (XCAR (tail)), 0);
7659 if (EQ (XCAR (tail), CODING_ATTR_BASE_NAME (attrs))
7660 && ! EQ (CODING_ATTR_TYPE (attrs), Qundecided))
7661 {
7662 ASET (attrs, coding_attr_trans_tbl,
7663 get_translation_table (attrs, 1, NULL));
7664 coding_attrs_list = Fcons (attrs, coding_attrs_list);
7665 }
7666 }
7667
7668 if (STRINGP (start))
7669 p = pbeg = SDATA (start);
7670 else
7671 p = pbeg = BYTE_POS_ADDR (start_byte);
7672 pend = p + (end_byte - start_byte);
7673
7674 while (p < pend && ASCII_BYTE_P (*p)) p++;
7675 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
7676
7677 while (p < pend)
7678 {
7679 if (ASCII_BYTE_P (*p))
7680 p++;
7681 else
7682 {
7683 c = STRING_CHAR_ADVANCE (p);
7684
7685 charset_map_loaded = 0;
7686 for (tail = coding_attrs_list; CONSP (tail);)
7687 {
7688 elt = XCAR (tail);
7689 if (NILP (elt))
7690 tail = XCDR (tail);
7691 else if (char_encodable_p (c, elt))
7692 tail = XCDR (tail);
7693 else if (CONSP (XCDR (tail)))
7694 {
7695 XSETCAR (tail, XCAR (XCDR (tail)));
7696 XSETCDR (tail, XCDR (XCDR (tail)));
7697 }
7698 else
7699 {
7700 XSETCAR (tail, Qnil);
7701 tail = XCDR (tail);
7702 }
7703 }
7704 if (charset_map_loaded)
7705 {
7706 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
7707
7708 if (STRINGP (start))
7709 pbeg = SDATA (start);
7710 else
7711 pbeg = BYTE_POS_ADDR (start_byte);
7712 p = pbeg + p_offset;
7713 pend = pbeg + pend_offset;
7714 }
7715 }
7716 }
7717
7718 safe_codings = list2 (Qraw_text, Qno_conversion);
7719 for (tail = coding_attrs_list; CONSP (tail); tail = XCDR (tail))
7720 if (! NILP (XCAR (tail)))
7721 safe_codings = Fcons (CODING_ATTR_BASE_NAME (XCAR (tail)), safe_codings);
7722
7723 return safe_codings;
7724}
7725
7726
7727DEFUN ("unencodable-char-position", Funencodable_char_position,
7728 Sunencodable_char_position, 3, 5, 0,
7729 doc: /*
7730Return position of first un-encodable character in a region.
7731START and END specfiy the region and CODING-SYSTEM specifies the
7732encoding to check. Return nil if CODING-SYSTEM does encode the region.
7733
7734If optional 4th argument COUNT is non-nil, it specifies at most how
7735many un-encodable characters to search. In this case, the value is a
7736list of positions.
7737
7738If optional 5th argument STRING is non-nil, it is a string to search
7739for un-encodable characters. In that case, START and END are indexes
7740to the string. */)
7741 (start, end, coding_system, count, string)
7742 Lisp_Object start, end, coding_system, count, string;
7743{
7744 int n;
7745 struct coding_system coding;
7746 Lisp_Object attrs, charset_list, translation_table;
7747 Lisp_Object positions;
7748 int from, to;
7749 const unsigned char *p, *stop, *pend;
7750 int ascii_compatible;
7751
7752 setup_coding_system (Fcheck_coding_system (coding_system), &coding);
7753 attrs = CODING_ID_ATTRS (coding.id);
7754 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
7755 return Qnil;
7756 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
7757 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
7758 translation_table = get_translation_table (attrs, 1, NULL);
7759
7760 if (NILP (string))
7761 {
7762 validate_region (&start, &end);
7763 from = XINT (start);
7764 to = XINT (end);
7765 if (NILP (current_buffer->enable_multibyte_characters)
7766 || (ascii_compatible
7767 && (to - from) == (CHAR_TO_BYTE (to) - (CHAR_TO_BYTE (from)))))
7768 return Qnil;
7769 p = CHAR_POS_ADDR (from);
7770 pend = CHAR_POS_ADDR (to);
7771 if (from < GPT && to >= GPT)
7772 stop = GPT_ADDR;
7773 else
7774 stop = pend;
7775 }
7776 else
7777 {
7778 CHECK_STRING (string);
7779 CHECK_NATNUM (start);
7780 CHECK_NATNUM (end);
7781 from = XINT (start);
7782 to = XINT (end);
7783 if (from > to
7784 || to > SCHARS (string))
7785 args_out_of_range_3 (string, start, end);
7786 if (! STRING_MULTIBYTE (string))
7787 return Qnil;
7788 p = SDATA (string) + string_char_to_byte (string, from);
7789 stop = pend = SDATA (string) + string_char_to_byte (string, to);
7790 if (ascii_compatible && (to - from) == (pend - p))
7791 return Qnil;
7792 }
7793
7794 if (NILP (count))
7795 n = 1;
7796 else
7797 {
7798 CHECK_NATNUM (count);
7799 n = XINT (count);
7800 }
7801
7802 positions = Qnil;
7803 while (1)
7804 {
7805 int c;
7806
7807 if (ascii_compatible)
7808 while (p < stop && ASCII_BYTE_P (*p))
7809 p++, from++;
7810 if (p >= stop)
7811 {
7812 if (p >= pend)
7813 break;
7814 stop = pend;
7815 p = GAP_END_ADDR;
7816 }
7817
7818 c = STRING_CHAR_ADVANCE (p);
7819 if (! (ASCII_CHAR_P (c) && ascii_compatible)
7820 && ! char_charset (translate_char (translation_table, c),
7821 charset_list, NULL))
7822 {
7823 positions = Fcons (make_number (from), positions);
7824 n--;
7825 if (n == 0)
7826 break;
7827 }
7828
7829 from++;
7830 }
7831
7832 return (NILP (count) ? Fcar (positions) : Fnreverse (positions));
7833}
7834
7835
7836DEFUN ("check-coding-systems-region", Fcheck_coding_systems_region,
7837 Scheck_coding_systems_region, 3, 3, 0,
7838 doc: /* Check if the region is encodable by coding systems.
7839
7840START and END are buffer positions specifying the region.
7841CODING-SYSTEM-LIST is a list of coding systems to check.
7842
7843The value is an alist ((CODING-SYSTEM POS0 POS1 ...) ...), where
7844CODING-SYSTEM is a member of CODING-SYSTEM-LIst and can't encode the
7845whole region, POS0, POS1, ... are buffer positions where non-encodable
7846characters are found.
7847
7848If all coding systems in CODING-SYSTEM-LIST can encode the region, the
7849value is nil.
7850
7851START may be a string. In that case, check if the string is
7852encodable, and the value contains indices to the string instead of
7853buffer positions. END is ignored. */)
7854 (start, end, coding_system_list)
7855 Lisp_Object start, end, coding_system_list;
7856{
7857 Lisp_Object list;
7858 EMACS_INT start_byte, end_byte;
7859 int pos;
7860 const unsigned char *p, *pbeg, *pend;
7861 int c;
7862 Lisp_Object tail, elt, attrs;
7863
7864 if (STRINGP (start))
7865 {
7866 if (!STRING_MULTIBYTE (start)
7867 && SCHARS (start) != SBYTES (start))
7868 return Qnil;
7869 start_byte = 0;
7870 end_byte = SBYTES (start);
7871 pos = 0;
7872 }
7873 else
7874 {
7875 CHECK_NUMBER_COERCE_MARKER (start);
7876 CHECK_NUMBER_COERCE_MARKER (end);
7877 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
7878 args_out_of_range (start, end);
7879 if (NILP (current_buffer->enable_multibyte_characters))
7880 return Qnil;
7881 start_byte = CHAR_TO_BYTE (XINT (start));
7882 end_byte = CHAR_TO_BYTE (XINT (end));
7883 if (XINT (end) - XINT (start) == end_byte - start_byte)
7884 return Qt;
7885
7886 if (XINT (start) < GPT && XINT (end) > GPT)
7887 {
7888 if ((GPT - XINT (start)) < (XINT (end) - GPT))
7889 move_gap_both (XINT (start), start_byte);
7890 else
7891 move_gap_both (XINT (end), end_byte);
7892 }
7893 pos = XINT (start);
7894 }
7895
7896 list = Qnil;
7897 for (tail = coding_system_list; CONSP (tail); tail = XCDR (tail))
7898 {
7899 elt = XCAR (tail);
7900 attrs = AREF (CODING_SYSTEM_SPEC (elt), 0);
7901 ASET (attrs, coding_attr_trans_tbl,
7902 get_translation_table (attrs, 1, NULL));
7903 list = Fcons (Fcons (elt, Fcons (attrs, Qnil)), list);
7904 }
7905
7906 if (STRINGP (start))
7907 p = pbeg = SDATA (start);
7908 else
7909 p = pbeg = BYTE_POS_ADDR (start_byte);
7910 pend = p + (end_byte - start_byte);
7911
7912 while (p < pend && ASCII_BYTE_P (*p)) p++, pos++;
7913 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
7914
7915 while (p < pend)
7916 {
7917 if (ASCII_BYTE_P (*p))
7918 p++;
7919 else
7920 {
7921 c = STRING_CHAR_ADVANCE (p);
7922
7923 charset_map_loaded = 0;
7924 for (tail = list; CONSP (tail); tail = XCDR (tail))
7925 {
7926 elt = XCDR (XCAR (tail));
7927 if (! char_encodable_p (c, XCAR (elt)))
7928 XSETCDR (elt, Fcons (make_number (pos), XCDR (elt)));
7929 }
7930 if (charset_map_loaded)
7931 {
7932 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
7933
7934 if (STRINGP (start))
7935 pbeg = SDATA (start);
7936 else
7937 pbeg = BYTE_POS_ADDR (start_byte);
7938 p = pbeg + p_offset;
7939 pend = pbeg + pend_offset;
7940 }
7941 }
7942 pos++;
7943 }
7944
7945 tail = list;
7946 list = Qnil;
7947 for (; CONSP (tail); tail = XCDR (tail))
7948 {
7949 elt = XCAR (tail);
7950 if (CONSP (XCDR (XCDR (elt))))
7951 list = Fcons (Fcons (XCAR (elt), Fnreverse (XCDR (XCDR (elt)))),
7952 list);
7953 }
7954
7955 return list;
7956}
7957
7958
7959Lisp_Object
7960code_convert_region (start, end, coding_system, dst_object, encodep, norecord)
7961 Lisp_Object start, end, coding_system, dst_object;
7962 int encodep, norecord;
7963{
7964 struct coding_system coding;
7965 EMACS_INT from, from_byte, to, to_byte;
7966 Lisp_Object src_object;
7967
7968 CHECK_NUMBER_COERCE_MARKER (start);
7969 CHECK_NUMBER_COERCE_MARKER (end);
7970 if (NILP (coding_system))
7971 coding_system = Qno_conversion;
7972 else
7973 CHECK_CODING_SYSTEM (coding_system);
7974 src_object = Fcurrent_buffer ();
7975 if (NILP (dst_object))
7976 dst_object = src_object;
7977 else if (! EQ (dst_object, Qt))
7978 CHECK_BUFFER (dst_object);
7979
7980 validate_region (&start, &end);
7981 from = XFASTINT (start);
7982 from_byte = CHAR_TO_BYTE (from);
7983 to = XFASTINT (end);
7984 to_byte = CHAR_TO_BYTE (to);
7985
7986 setup_coding_system (coding_system, &coding);
7987 coding.mode |= CODING_MODE_LAST_BLOCK;
7988
7989 if (encodep)
7990 encode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
7991 dst_object);
7992 else
7993 decode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
7994 dst_object);
7995 if (! norecord)
7996 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
7997
7998 return (BUFFERP (dst_object)
7999 ? make_number (coding.produced_char)
8000 : coding.dst_object);
8001}
8002
8003
8004DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region,
8005 3, 4, "r\nzCoding system: ",
8006 doc: /* Decode the current region from the specified coding system.
8007When called from a program, takes four arguments:
8008 START, END, CODING-SYSTEM, and DESTINATION.
8009START and END are buffer positions.
8010
8011Optional 4th arguments DESTINATION specifies where the decoded text goes.
8012If nil, the region between START and END is replace by the decoded text.
8013If buffer, the decoded text is inserted in the buffer.
8014If t, the decoded text is returned.
8015
8016This function sets `last-coding-system-used' to the precise coding system
8017used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
8018not fully specified.)
8019It returns the length of the decoded text. */)
8020 (start, end, coding_system, destination)
8021 Lisp_Object start, end, coding_system, destination;
8022{
8023 return code_convert_region (start, end, coding_system, destination, 0, 0);
8024}
8025
8026DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region,
8027 3, 4, "r\nzCoding system: ",
8028 doc: /* Encode the current region by specified coding system.
8029When called from a program, takes three arguments:
8030START, END, and CODING-SYSTEM. START and END are buffer positions.
8031
8032Optional 4th arguments DESTINATION specifies where the encoded text goes.
8033If nil, the region between START and END is replace by the encoded text.
8034If buffer, the encoded text is inserted in the buffer.
8035If t, the encoded text is returned.
8036
8037This function sets `last-coding-system-used' to the precise coding system
8038used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
8039not fully specified.)
8040It returns the length of the encoded text. */)
8041 (start, end, coding_system, destination)
8042 Lisp_Object start, end, coding_system, destination;
8043{
8044 return code_convert_region (start, end, coding_system, destination, 1, 0);
8045}
8046
8047Lisp_Object
8048code_convert_string (string, coding_system, dst_object,
8049 encodep, nocopy, norecord)
8050 Lisp_Object string, coding_system, dst_object;
8051 int encodep, nocopy, norecord;
8052{
8053 struct coding_system coding;
8054 EMACS_INT chars, bytes;
8055
8056 CHECK_STRING (string);
8057 if (NILP (coding_system))
8058 {
8059 if (! norecord)
8060 Vlast_coding_system_used = Qno_conversion;
8061 if (NILP (dst_object))
8062 return (nocopy ? Fcopy_sequence (string) : string);
8063 }
8064
8065 if (NILP (coding_system))
8066 coding_system = Qno_conversion;
8067 else
8068 CHECK_CODING_SYSTEM (coding_system);
8069 if (NILP (dst_object))
8070 dst_object = Qt;
8071 else if (! EQ (dst_object, Qt))
8072 CHECK_BUFFER (dst_object);
8073
8074 setup_coding_system (coding_system, &coding);
8075 coding.mode |= CODING_MODE_LAST_BLOCK;
8076 chars = SCHARS (string);
8077 bytes = SBYTES (string);
8078 if (encodep)
8079 encode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8080 else
8081 decode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
8082 if (! norecord)
8083 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
8084
8085 return (BUFFERP (dst_object)
8086 ? make_number (coding.produced_char)
8087 : coding.dst_object);
8088}
8089
8090
8091/* Encode or decode STRING according to CODING_SYSTEM.
8092 Do not set Vlast_coding_system_used.
8093
8094 This function is called only from macros DECODE_FILE and
8095 ENCODE_FILE, thus we ignore character composition. */
8096
8097Lisp_Object
8098code_convert_string_norecord (string, coding_system, encodep)
8099 Lisp_Object string, coding_system;
8100 int encodep;
8101{
8102 return code_convert_string (string, coding_system, Qt, encodep, 0, 1);
8103}
8104
8105
8106DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string,
8107 2, 4, 0,
8108 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
8109
8110Optional third arg NOCOPY non-nil means it is OK to return STRING itself
8111if the decoding operation is trivial.
8112
8113Optional fourth arg BUFFER non-nil meant that the decoded text is
8114inserted in BUFFER instead of returned as a string. In this case,
8115the return value is BUFFER.
8116
8117This function sets `last-coding-system-used' to the precise coding system
8118used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
8119not fully specified. */)
8120 (string, coding_system, nocopy, buffer)
8121 Lisp_Object string, coding_system, nocopy, buffer;
8122{
8123 return code_convert_string (string, coding_system, buffer,
8124 0, ! NILP (nocopy), 0);
8125}
8126
8127DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
8128 2, 4, 0,
8129 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
8130
8131Optional third arg NOCOPY non-nil means it is OK to return STRING
8132itself if the encoding operation is trivial.
8133
8134Optional fourth arg BUFFER non-nil meant that the encoded text is
8135inserted in BUFFER instead of returned as a string. In this case,
8136the return value is BUFFER.
8137
8138This function sets `last-coding-system-used' to the precise coding system
8139used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
8140not fully specified.) */)
8141 (string, coding_system, nocopy, buffer)
8142 Lisp_Object string, coding_system, nocopy, buffer;
8143{
8144 return code_convert_string (string, coding_system, buffer,
8145 1, ! NILP (nocopy), 1);
8146}
8147
8148\f
8149DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0,
8150 doc: /* Decode a Japanese character which has CODE in shift_jis encoding.
8151Return the corresponding character. */)
8152 (code)
8153 Lisp_Object code;
8154{
8155 Lisp_Object spec, attrs, val;
8156 struct charset *charset_roman, *charset_kanji, *charset_kana, *charset;
8157 int c;
8158
8159 CHECK_NATNUM (code);
8160 c = XFASTINT (code);
8161 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8162 attrs = AREF (spec, 0);
8163
8164 if (ASCII_BYTE_P (c)
8165 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8166 return code;
8167
8168 val = CODING_ATTR_CHARSET_LIST (attrs);
8169 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8170 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8171 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val)));
8172
8173 if (c <= 0x7F)
8174 charset = charset_roman;
8175 else if (c >= 0xA0 && c < 0xDF)
8176 {
8177 charset = charset_kana;
8178 c -= 0x80;
8179 }
8180 else
8181 {
8182 int s1 = c >> 8, s2 = c & 0xFF;
8183
8184 if (s1 < 0x81 || (s1 > 0x9F && s1 < 0xE0) || s1 > 0xEF
8185 || s2 < 0x40 || s2 == 0x7F || s2 > 0xFC)
8186 error ("Invalid code: %d", code);
8187 SJIS_TO_JIS (c);
8188 charset = charset_kanji;
8189 }
8190 c = DECODE_CHAR (charset, c);
8191 if (c < 0)
8192 error ("Invalid code: %d", code);
8193 return make_number (c);
8194}
8195
8196
8197DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0,
8198 doc: /* Encode a Japanese character CH to shift_jis encoding.
8199Return the corresponding code in SJIS. */)
8200 (ch)
8201 Lisp_Object ch;
8202{
8203 Lisp_Object spec, attrs, charset_list;
8204 int c;
8205 struct charset *charset;
8206 unsigned code;
8207
8208 CHECK_CHARACTER (ch);
8209 c = XFASTINT (ch);
8210 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
8211 attrs = AREF (spec, 0);
8212
8213 if (ASCII_CHAR_P (c)
8214 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8215 return ch;
8216
8217 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8218 charset = char_charset (c, charset_list, &code);
8219 if (code == CHARSET_INVALID_CODE (charset))
8220 error ("Can't encode by shift_jis encoding: %d", c);
8221 JIS_TO_SJIS (code);
8222
8223 return make_number (code);
8224}
8225
8226DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0,
8227 doc: /* Decode a Big5 character which has CODE in BIG5 coding system.
8228Return the corresponding character. */)
8229 (code)
8230 Lisp_Object code;
8231{
8232 Lisp_Object spec, attrs, val;
8233 struct charset *charset_roman, *charset_big5, *charset;
8234 int c;
8235
8236 CHECK_NATNUM (code);
8237 c = XFASTINT (code);
8238 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8239 attrs = AREF (spec, 0);
8240
8241 if (ASCII_BYTE_P (c)
8242 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8243 return code;
8244
8245 val = CODING_ATTR_CHARSET_LIST (attrs);
8246 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
8247 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
8248
8249 if (c <= 0x7F)
8250 charset = charset_roman;
8251 else
8252 {
8253 int b1 = c >> 8, b2 = c & 0x7F;
8254 if (b1 < 0xA1 || b1 > 0xFE
8255 || b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE)
8256 error ("Invalid code: %d", code);
8257 charset = charset_big5;
8258 }
8259 c = DECODE_CHAR (charset, (unsigned )c);
8260 if (c < 0)
8261 error ("Invalid code: %d", code);
8262 return make_number (c);
8263}
8264
8265DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0,
8266 doc: /* Encode the Big5 character CH to BIG5 coding system.
8267Return the corresponding character code in Big5. */)
8268 (ch)
8269 Lisp_Object ch;
8270{
8271 Lisp_Object spec, attrs, charset_list;
8272 struct charset *charset;
8273 int c;
8274 unsigned code;
8275
8276 CHECK_CHARACTER (ch);
8277 c = XFASTINT (ch);
8278 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
8279 attrs = AREF (spec, 0);
8280 if (ASCII_CHAR_P (c)
8281 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
8282 return ch;
8283
8284 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
8285 charset = char_charset (c, charset_list, &code);
8286 if (code == CHARSET_INVALID_CODE (charset))
8287 error ("Can't encode by Big5 encoding: %d", c);
8288
8289 return make_number (code);
8290}
8291
8292\f
8293DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal,
8294 Sset_terminal_coding_system_internal, 1, 2, 0,
8295 doc: /* Internal use only. */)
8296 (coding_system, terminal)
8297 Lisp_Object coding_system;
8298 Lisp_Object terminal;
8299{
8300 struct coding_system *terminal_coding = TERMINAL_TERMINAL_CODING (get_terminal (terminal, 1));
8301 CHECK_SYMBOL (coding_system);
8302 setup_coding_system (Fcheck_coding_system (coding_system), terminal_coding);
8303 /* We had better not send unsafe characters to terminal. */
8304 terminal_coding->mode |= CODING_MODE_SAFE_ENCODING;
8305 /* Characer composition should be disabled. */
8306 terminal_coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
8307 terminal_coding->src_multibyte = 1;
8308 terminal_coding->dst_multibyte = 0;
8309 return Qnil;
8310}
8311
8312DEFUN ("set-safe-terminal-coding-system-internal",
8313 Fset_safe_terminal_coding_system_internal,
8314 Sset_safe_terminal_coding_system_internal, 1, 1, 0,
8315 doc: /* Internal use only. */)
8316 (coding_system)
8317 Lisp_Object coding_system;
8318{
8319 CHECK_SYMBOL (coding_system);
8320 setup_coding_system (Fcheck_coding_system (coding_system),
8321 &safe_terminal_coding);
8322 /* Characer composition should be disabled. */
8323 safe_terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
8324 safe_terminal_coding.src_multibyte = 1;
8325 safe_terminal_coding.dst_multibyte = 0;
8326 return Qnil;
8327}
8328
8329DEFUN ("terminal-coding-system", Fterminal_coding_system,
8330 Sterminal_coding_system, 0, 1, 0,
8331 doc: /* Return coding system specified for terminal output on the given terminal.
8332TERMINAL may be a terminal id, a frame, or nil for the selected
8333frame's terminal device. */)
8334 (terminal)
8335 Lisp_Object terminal;
8336{
8337 struct coding_system *terminal_coding
8338 = TERMINAL_TERMINAL_CODING (get_terminal (terminal, 1));
8339 Lisp_Object coding_system = CODING_ID_NAME (terminal_coding->id);
8340
8341 /* For backward compatibility, return nil if it is `undecided'. */
8342 return (! EQ (coding_system, Qundecided) ? coding_system : Qnil);
8343}
8344
8345DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal,
8346 Sset_keyboard_coding_system_internal, 1, 2, 0,
8347 doc: /* Internal use only. */)
8348 (coding_system, terminal)
8349 Lisp_Object coding_system;
8350 Lisp_Object terminal;
8351{
8352 struct terminal *t = get_terminal (terminal, 1);
8353 CHECK_SYMBOL (coding_system);
8354 setup_coding_system (Fcheck_coding_system (coding_system),
8355 TERMINAL_KEYBOARD_CODING (t));
8356 /* Characer composition should be disabled. */
8357 TERMINAL_KEYBOARD_CODING (t)->common_flags
8358 &= ~CODING_ANNOTATE_COMPOSITION_MASK;
8359 return Qnil;
8360}
8361
8362DEFUN ("keyboard-coding-system",
8363 Fkeyboard_coding_system, Skeyboard_coding_system, 0, 1, 0,
8364 doc: /* Return coding system specified for decoding keyboard input. */)
8365 (terminal)
8366 Lisp_Object terminal;
8367{
8368 return CODING_ID_NAME (TERMINAL_KEYBOARD_CODING
8369 (get_terminal (terminal, 1))->id);
8370}
8371
8372\f
8373DEFUN ("find-operation-coding-system", Ffind_operation_coding_system,
8374 Sfind_operation_coding_system, 1, MANY, 0,
8375 doc: /* Choose a coding system for an operation based on the target name.
8376The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
8377DECODING-SYSTEM is the coding system to use for decoding
8378\(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
8379for encoding (in case OPERATION does encoding).
8380
8381The first argument OPERATION specifies an I/O primitive:
8382 For file I/O, `insert-file-contents' or `write-region'.
8383 For process I/O, `call-process', `call-process-region', or `start-process'.
8384 For network I/O, `open-network-stream'.
8385
8386The remaining arguments should be the same arguments that were passed
8387to the primitive. Depending on which primitive, one of those arguments
8388is selected as the TARGET. For example, if OPERATION does file I/O,
8389whichever argument specifies the file name is TARGET.
8390
8391TARGET has a meaning which depends on OPERATION:
8392 For file I/O, TARGET is a file name (except for the special case below).
8393 For process I/O, TARGET is a process name.
8394 For network I/O, TARGET is a service name or a port number
8395
8396This function looks up what specified for TARGET in,
8397`file-coding-system-alist', `process-coding-system-alist',
8398or `network-coding-system-alist' depending on OPERATION.
8399They may specify a coding system, a cons of coding systems,
8400or a function symbol to call.
8401In the last case, we call the function with one argument,
8402which is a list of all the arguments given to this function.
8403If the function can't decide a coding system, it can return
8404`undecided' so that the normal code-detection is performed.
8405
8406If OPERATION is `insert-file-contents', the argument corresponding to
8407TARGET may be a cons (FILENAME . BUFFER). In that case, FILENAME is a
8408file name to look up, and BUFFER is a buffer that contains the file's
8409contents (not yet decoded). If `file-coding-system-alist' specifies a
8410function to call for FILENAME, that function should examine the
8411contents of BUFFER instead of reading the file.
8412
8413usage: (find-operation-coding-system OPERATION ARGUMENTS...) */)
8414 (nargs, args)
8415 int nargs;
8416 Lisp_Object *args;
8417{
8418 Lisp_Object operation, target_idx, target, val;
8419 register Lisp_Object chain;
8420
8421 if (nargs < 2)
8422 error ("Too few arguments");
8423 operation = args[0];
8424 if (!SYMBOLP (operation)
8425 || !INTEGERP (target_idx = Fget (operation, Qtarget_idx)))
8426 error ("Invalid first arguement");
8427 if (nargs < 1 + XINT (target_idx))
8428 error ("Too few arguments for operation: %s",
8429 SDATA (SYMBOL_NAME (operation)));
8430 target = args[XINT (target_idx) + 1];
8431 if (!(STRINGP (target)
8432 || (EQ (operation, Qinsert_file_contents) && CONSP (target)
8433 && STRINGP (XCAR (target)) && BUFFERP (XCDR (target)))
8434 || (EQ (operation, Qopen_network_stream) && INTEGERP (target))))
8435 error ("Invalid %dth argument", XINT (target_idx) + 1);
8436 if (CONSP (target))
8437 target = XCAR (target);
8438
8439 chain = ((EQ (operation, Qinsert_file_contents)
8440 || EQ (operation, Qwrite_region))
8441 ? Vfile_coding_system_alist
8442 : (EQ (operation, Qopen_network_stream)
8443 ? Vnetwork_coding_system_alist
8444 : Vprocess_coding_system_alist));
8445 if (NILP (chain))
8446 return Qnil;
8447
8448 for (; CONSP (chain); chain = XCDR (chain))
8449 {
8450 Lisp_Object elt;
8451
8452 elt = XCAR (chain);
8453 if (CONSP (elt)
8454 && ((STRINGP (target)
8455 && STRINGP (XCAR (elt))
8456 && fast_string_match (XCAR (elt), target) >= 0)
8457 || (INTEGERP (target) && EQ (target, XCAR (elt)))))
8458 {
8459 val = XCDR (elt);
8460 /* Here, if VAL is both a valid coding system and a valid
8461 function symbol, we return VAL as a coding system. */
8462 if (CONSP (val))
8463 return val;
8464 if (! SYMBOLP (val))
8465 return Qnil;
8466 if (! NILP (Fcoding_system_p (val)))
8467 return Fcons (val, val);
8468 if (! NILP (Ffboundp (val)))
8469 {
8470 /* We use call1 rather than safe_call1
8471 so as to get bug reports about functions called here
8472 which don't handle the current interface. */
8473 val = call1 (val, Flist (nargs, args));
8474 if (CONSP (val))
8475 return val;
8476 if (SYMBOLP (val) && ! NILP (Fcoding_system_p (val)))
8477 return Fcons (val, val);
8478 }
8479 return Qnil;
8480 }
8481 }
8482 return Qnil;
8483}
8484
8485DEFUN ("set-coding-system-priority", Fset_coding_system_priority,
8486 Sset_coding_system_priority, 0, MANY, 0,
8487 doc: /* Assign higher priority to the coding systems given as arguments.
8488If multiple coding systems belongs to the same category,
8489all but the first one are ignored.
8490
8491usage: (set-coding-system-priority ...) */)
8492 (nargs, args)
8493 int nargs;
8494 Lisp_Object *args;
8495{
8496 int i, j;
8497 int changed[coding_category_max];
8498 enum coding_category priorities[coding_category_max];
8499
8500 bzero (changed, sizeof changed);
8501
8502 for (i = j = 0; i < nargs; i++)
8503 {
8504 enum coding_category category;
8505 Lisp_Object spec, attrs;
8506
8507 CHECK_CODING_SYSTEM_GET_SPEC (args[i], spec);
8508 attrs = AREF (spec, 0);
8509 category = XINT (CODING_ATTR_CATEGORY (attrs));
8510 if (changed[category])
8511 /* Ignore this coding system because a coding system of the
8512 same category already had a higher priority. */
8513 continue;
8514 changed[category] = 1;
8515 priorities[j++] = category;
8516 if (coding_categories[category].id >= 0
8517 && ! EQ (args[i], CODING_ID_NAME (coding_categories[category].id)))
8518 setup_coding_system (args[i], &coding_categories[category]);
8519 Fset (AREF (Vcoding_category_table, category), args[i]);
8520 }
8521
8522 /* Now we have decided top J priorities. Reflect the order of the
8523 original priorities to the remaining priorities. */
8524
8525 for (i = j, j = 0; i < coding_category_max; i++, j++)
8526 {
8527 while (j < coding_category_max
8528 && changed[coding_priorities[j]])
8529 j++;
8530 if (j == coding_category_max)
8531 abort ();
8532 priorities[i] = coding_priorities[j];
8533 }
8534
8535 bcopy (priorities, coding_priorities, sizeof priorities);
8536
8537 /* Update `coding-category-list'. */
8538 Vcoding_category_list = Qnil;
8539 for (i = coding_category_max - 1; i >= 0; i--)
8540 Vcoding_category_list
8541 = Fcons (AREF (Vcoding_category_table, priorities[i]),
8542 Vcoding_category_list);
8543
8544 return Qnil;
8545}
8546
8547DEFUN ("coding-system-priority-list", Fcoding_system_priority_list,
8548 Scoding_system_priority_list, 0, 1, 0,
8549 doc: /* Return a list of coding systems ordered by their priorities.
8550HIGHESTP non-nil means just return the highest priority one. */)
8551 (highestp)
8552 Lisp_Object highestp;
8553{
8554 int i;
8555 Lisp_Object val;
8556
8557 for (i = 0, val = Qnil; i < coding_category_max; i++)
8558 {
8559 enum coding_category category = coding_priorities[i];
8560 int id = coding_categories[category].id;
8561 Lisp_Object attrs;
8562
8563 if (id < 0)
8564 continue;
8565 attrs = CODING_ID_ATTRS (id);
8566 if (! NILP (highestp))
8567 return CODING_ATTR_BASE_NAME (attrs);
8568 val = Fcons (CODING_ATTR_BASE_NAME (attrs), val);
8569 }
8570 return Fnreverse (val);
8571}
8572
8573static char *suffixes[] = { "-unix", "-dos", "-mac" };
8574
8575static Lisp_Object
8576make_subsidiaries (base)
8577 Lisp_Object base;
8578{
8579 Lisp_Object subsidiaries;
8580 int base_name_len = SBYTES (SYMBOL_NAME (base));
8581 char *buf = (char *) alloca (base_name_len + 6);
8582 int i;
8583
8584 bcopy (SDATA (SYMBOL_NAME (base)), buf, base_name_len);
8585 subsidiaries = Fmake_vector (make_number (3), Qnil);
8586 for (i = 0; i < 3; i++)
8587 {
8588 bcopy (suffixes[i], buf + base_name_len, strlen (suffixes[i]) + 1);
8589 ASET (subsidiaries, i, intern (buf));
8590 }
8591 return subsidiaries;
8592}
8593
8594
8595DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal,
8596 Sdefine_coding_system_internal, coding_arg_max, MANY, 0,
8597 doc: /* For internal use only.
8598usage: (define-coding-system-internal ...) */)
8599 (nargs, args)
8600 int nargs;
8601 Lisp_Object *args;
8602{
8603 Lisp_Object name;
8604 Lisp_Object spec_vec; /* [ ATTRS ALIASE EOL_TYPE ] */
8605 Lisp_Object attrs; /* Vector of attributes. */
8606 Lisp_Object eol_type;
8607 Lisp_Object aliases;
8608 Lisp_Object coding_type, charset_list, safe_charsets;
8609 enum coding_category category;
8610 Lisp_Object tail, val;
8611 int max_charset_id = 0;
8612 int i;
8613
8614 if (nargs < coding_arg_max)
8615 goto short_args;
8616
8617 attrs = Fmake_vector (make_number (coding_attr_last_index), Qnil);
8618
8619 name = args[coding_arg_name];
8620 CHECK_SYMBOL (name);
8621 CODING_ATTR_BASE_NAME (attrs) = name;
8622
8623 val = args[coding_arg_mnemonic];
8624 if (! STRINGP (val))
8625 CHECK_CHARACTER (val);
8626 CODING_ATTR_MNEMONIC (attrs) = val;
8627
8628 coding_type = args[coding_arg_coding_type];
8629 CHECK_SYMBOL (coding_type);
8630 CODING_ATTR_TYPE (attrs) = coding_type;
8631
8632 charset_list = args[coding_arg_charset_list];
8633 if (SYMBOLP (charset_list))
8634 {
8635 if (EQ (charset_list, Qiso_2022))
8636 {
8637 if (! EQ (coding_type, Qiso_2022))
8638 error ("Invalid charset-list");
8639 charset_list = Viso_2022_charset_list;
8640 }
8641 else if (EQ (charset_list, Qemacs_mule))
8642 {
8643 if (! EQ (coding_type, Qemacs_mule))
8644 error ("Invalid charset-list");
8645 charset_list = Vemacs_mule_charset_list;
8646 }
8647 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8648 if (max_charset_id < XFASTINT (XCAR (tail)))
8649 max_charset_id = XFASTINT (XCAR (tail));
8650 }
8651 else
8652 {
8653 charset_list = Fcopy_sequence (charset_list);
8654 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8655 {
8656 struct charset *charset;
8657
8658 val = XCAR (tail);
8659 CHECK_CHARSET_GET_CHARSET (val, charset);
8660 if (EQ (coding_type, Qiso_2022)
8661 ? CHARSET_ISO_FINAL (charset) < 0
8662 : EQ (coding_type, Qemacs_mule)
8663 ? CHARSET_EMACS_MULE_ID (charset) < 0
8664 : 0)
8665 error ("Can't handle charset `%s'",
8666 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
8667
8668 XSETCAR (tail, make_number (charset->id));
8669 if (max_charset_id < charset->id)
8670 max_charset_id = charset->id;
8671 }
8672 }
8673 CODING_ATTR_CHARSET_LIST (attrs) = charset_list;
8674
8675 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
8676 make_number (255));
8677 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8678 SSET (safe_charsets, XFASTINT (XCAR (tail)), 0);
8679 CODING_ATTR_SAFE_CHARSETS (attrs) = safe_charsets;
8680
8681 CODING_ATTR_ASCII_COMPAT (attrs) = args[coding_arg_ascii_compatible_p];
8682
8683 val = args[coding_arg_decode_translation_table];
8684 if (! CHAR_TABLE_P (val) && ! CONSP (val))
8685 CHECK_SYMBOL (val);
8686 CODING_ATTR_DECODE_TBL (attrs) = val;
8687
8688 val = args[coding_arg_encode_translation_table];
8689 if (! CHAR_TABLE_P (val) && ! CONSP (val))
8690 CHECK_SYMBOL (val);
8691 CODING_ATTR_ENCODE_TBL (attrs) = val;
8692
8693 val = args[coding_arg_post_read_conversion];
8694 CHECK_SYMBOL (val);
8695 CODING_ATTR_POST_READ (attrs) = val;
8696
8697 val = args[coding_arg_pre_write_conversion];
8698 CHECK_SYMBOL (val);
8699 CODING_ATTR_PRE_WRITE (attrs) = val;
8700
8701 val = args[coding_arg_default_char];
8702 if (NILP (val))
8703 CODING_ATTR_DEFAULT_CHAR (attrs) = make_number (' ');
8704 else
8705 {
8706 CHECK_CHARACTER (val);
8707 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
8708 }
8709
8710 val = args[coding_arg_for_unibyte];
8711 CODING_ATTR_FOR_UNIBYTE (attrs) = NILP (val) ? Qnil : Qt;
8712
8713 val = args[coding_arg_plist];
8714 CHECK_LIST (val);
8715 CODING_ATTR_PLIST (attrs) = val;
8716
8717 if (EQ (coding_type, Qcharset))
8718 {
8719 /* Generate a lisp vector of 256 elements. Each element is nil,
8720 integer, or a list of charset IDs.
8721
8722 If Nth element is nil, the byte code N is invalid in this
8723 coding system.
8724
8725 If Nth element is a number NUM, N is the first byte of a
8726 charset whose ID is NUM.
8727
8728 If Nth element is a list of charset IDs, N is the first byte
8729 of one of them. The list is sorted by dimensions of the
8730 charsets. A charset of smaller dimension comes firtst. */
8731 val = Fmake_vector (make_number (256), Qnil);
8732
8733 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
8734 {
8735 struct charset *charset = CHARSET_FROM_ID (XFASTINT (XCAR (tail)));
8736 int dim = CHARSET_DIMENSION (charset);
8737 int idx = (dim - 1) * 4;
8738
8739 if (CHARSET_ASCII_COMPATIBLE_P (charset))
8740 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
8741
8742 for (i = charset->code_space[idx];
8743 i <= charset->code_space[idx + 1]; i++)
8744 {
8745 Lisp_Object tmp, tmp2;
8746 int dim2;
8747
8748 tmp = AREF (val, i);
8749 if (NILP (tmp))
8750 tmp = XCAR (tail);
8751 else if (NUMBERP (tmp))
8752 {
8753 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (tmp)));
8754 if (dim < dim2)
8755 tmp = Fcons (XCAR (tail), Fcons (tmp, Qnil));
8756 else
8757 tmp = Fcons (tmp, Fcons (XCAR (tail), Qnil));
8758 }
8759 else
8760 {
8761 for (tmp2 = tmp; CONSP (tmp2); tmp2 = XCDR (tmp2))
8762 {
8763 dim2 = CHARSET_DIMENSION (CHARSET_FROM_ID (XFASTINT (XCAR (tmp2))));
8764 if (dim < dim2)
8765 break;
8766 }
8767 if (NILP (tmp2))
8768 tmp = nconc2 (tmp, Fcons (XCAR (tail), Qnil));
8769 else
8770 {
8771 XSETCDR (tmp2, Fcons (XCAR (tmp2), XCDR (tmp2)));
8772 XSETCAR (tmp2, XCAR (tail));
8773 }
8774 }
8775 ASET (val, i, tmp);
8776 }
8777 }
8778 ASET (attrs, coding_attr_charset_valids, val);
8779 category = coding_category_charset;
8780 }
8781 else if (EQ (coding_type, Qccl))
8782 {
8783 Lisp_Object valids;
8784
8785 if (nargs < coding_arg_ccl_max)
8786 goto short_args;
8787
8788 val = args[coding_arg_ccl_decoder];
8789 CHECK_CCL_PROGRAM (val);
8790 if (VECTORP (val))
8791 val = Fcopy_sequence (val);
8792 ASET (attrs, coding_attr_ccl_decoder, val);
8793
8794 val = args[coding_arg_ccl_encoder];
8795 CHECK_CCL_PROGRAM (val);
8796 if (VECTORP (val))
8797 val = Fcopy_sequence (val);
8798 ASET (attrs, coding_attr_ccl_encoder, val);
8799
8800 val = args[coding_arg_ccl_valids];
8801 valids = Fmake_string (make_number (256), make_number (0));
8802 for (tail = val; !NILP (tail); tail = Fcdr (tail))
8803 {
8804 int from, to;
8805
8806 val = Fcar (tail);
8807 if (INTEGERP (val))
8808 {
8809 from = to = XINT (val);
8810 if (from < 0 || from > 255)
8811 args_out_of_range_3 (val, make_number (0), make_number (255));
8812 }
8813 else
8814 {
8815 CHECK_CONS (val);
8816 CHECK_NATNUM_CAR (val);
8817 CHECK_NATNUM_CDR (val);
8818 from = XINT (XCAR (val));
8819 if (from > 255)
8820 args_out_of_range_3 (XCAR (val),
8821 make_number (0), make_number (255));
8822 to = XINT (XCDR (val));
8823 if (to < from || to > 255)
8824 args_out_of_range_3 (XCDR (val),
8825 XCAR (val), make_number (255));
8826 }
8827 for (i = from; i <= to; i++)
8828 SSET (valids, i, 1);
8829 }
8830 ASET (attrs, coding_attr_ccl_valids, valids);
8831
8832 category = coding_category_ccl;
8833 }
8834 else if (EQ (coding_type, Qutf_16))
8835 {
8836 Lisp_Object bom, endian;
8837
8838 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
8839
8840 if (nargs < coding_arg_utf16_max)
8841 goto short_args;
8842
8843 bom = args[coding_arg_utf16_bom];
8844 if (! NILP (bom) && ! EQ (bom, Qt))
8845 {
8846 CHECK_CONS (bom);
8847 val = XCAR (bom);
8848 CHECK_CODING_SYSTEM (val);
8849 val = XCDR (bom);
8850 CHECK_CODING_SYSTEM (val);
8851 }
8852 ASET (attrs, coding_attr_utf_16_bom, bom);
8853
8854 endian = args[coding_arg_utf16_endian];
8855 CHECK_SYMBOL (endian);
8856 if (NILP (endian))
8857 endian = Qbig;
8858 else if (! EQ (endian, Qbig) && ! EQ (endian, Qlittle))
8859 error ("Invalid endian: %s", SDATA (SYMBOL_NAME (endian)));
8860 ASET (attrs, coding_attr_utf_16_endian, endian);
8861
8862 category = (CONSP (bom)
8863 ? coding_category_utf_16_auto
8864 : NILP (bom)
8865 ? (EQ (endian, Qbig)
8866 ? coding_category_utf_16_be_nosig
8867 : coding_category_utf_16_le_nosig)
8868 : (EQ (endian, Qbig)
8869 ? coding_category_utf_16_be
8870 : coding_category_utf_16_le));
8871 }
8872 else if (EQ (coding_type, Qiso_2022))
8873 {
8874 Lisp_Object initial, reg_usage, request, flags;
8875 int i;
8876
8877 if (nargs < coding_arg_iso2022_max)
8878 goto short_args;
8879
8880 initial = Fcopy_sequence (args[coding_arg_iso2022_initial]);
8881 CHECK_VECTOR (initial);
8882 for (i = 0; i < 4; i++)
8883 {
8884 val = Faref (initial, make_number (i));
8885 if (! NILP (val))
8886 {
8887 struct charset *charset;
8888
8889 CHECK_CHARSET_GET_CHARSET (val, charset);
8890 ASET (initial, i, make_number (CHARSET_ID (charset)));
8891 if (i == 0 && CHARSET_ASCII_COMPATIBLE_P (charset))
8892 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
8893 }
8894 else
8895 ASET (initial, i, make_number (-1));
8896 }
8897
8898 reg_usage = args[coding_arg_iso2022_reg_usage];
8899 CHECK_CONS (reg_usage);
8900 CHECK_NUMBER_CAR (reg_usage);
8901 CHECK_NUMBER_CDR (reg_usage);
8902
8903 request = Fcopy_sequence (args[coding_arg_iso2022_request]);
8904 for (tail = request; ! NILP (tail); tail = Fcdr (tail))
8905 {
8906 int id;
8907 Lisp_Object tmp;
8908
8909 val = Fcar (tail);
8910 CHECK_CONS (val);
8911 tmp = XCAR (val);
8912 CHECK_CHARSET_GET_ID (tmp, id);
8913 CHECK_NATNUM_CDR (val);
8914 if (XINT (XCDR (val)) >= 4)
8915 error ("Invalid graphic register number: %d", XINT (XCDR (val)));
8916 XSETCAR (val, make_number (id));
8917 }
8918
8919 flags = args[coding_arg_iso2022_flags];
8920 CHECK_NATNUM (flags);
8921 i = XINT (flags);
8922 if (EQ (args[coding_arg_charset_list], Qiso_2022))
8923 flags = make_number (i | CODING_ISO_FLAG_FULL_SUPPORT);
8924
8925 ASET (attrs, coding_attr_iso_initial, initial);
8926 ASET (attrs, coding_attr_iso_usage, reg_usage);
8927 ASET (attrs, coding_attr_iso_request, request);
8928 ASET (attrs, coding_attr_iso_flags, flags);
8929 setup_iso_safe_charsets (attrs);
8930
8931 if (i & CODING_ISO_FLAG_SEVEN_BITS)
8932 category = ((i & (CODING_ISO_FLAG_LOCKING_SHIFT
8933 | CODING_ISO_FLAG_SINGLE_SHIFT))
8934 ? coding_category_iso_7_else
8935 : EQ (args[coding_arg_charset_list], Qiso_2022)
8936 ? coding_category_iso_7
8937 : coding_category_iso_7_tight);
8938 else
8939 {
8940 int id = XINT (AREF (initial, 1));
8941
8942 category = (((i & CODING_ISO_FLAG_LOCKING_SHIFT)
8943 || EQ (args[coding_arg_charset_list], Qiso_2022)
8944 || id < 0)
8945 ? coding_category_iso_8_else
8946 : (CHARSET_DIMENSION (CHARSET_FROM_ID (id)) == 1)
8947 ? coding_category_iso_8_1
8948 : coding_category_iso_8_2);
8949 }
8950 if (category != coding_category_iso_8_1
8951 && category != coding_category_iso_8_2)
8952 CODING_ATTR_ASCII_COMPAT (attrs) = Qnil;
8953 }
8954 else if (EQ (coding_type, Qemacs_mule))
8955 {
8956 if (EQ (args[coding_arg_charset_list], Qemacs_mule))
8957 ASET (attrs, coding_attr_emacs_mule_full, Qt);
8958 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
8959 category = coding_category_emacs_mule;
8960 }
8961 else if (EQ (coding_type, Qshift_jis))
8962 {
8963
8964 struct charset *charset;
8965
8966 if (XINT (Flength (charset_list)) != 3
8967 && XINT (Flength (charset_list)) != 4)
8968 error ("There should be three or four charsets");
8969
8970 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8971 if (CHARSET_DIMENSION (charset) != 1)
8972 error ("Dimension of charset %s is not one",
8973 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
8974 if (CHARSET_ASCII_COMPATIBLE_P (charset))
8975 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
8976
8977 charset_list = XCDR (charset_list);
8978 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8979 if (CHARSET_DIMENSION (charset) != 1)
8980 error ("Dimension of charset %s is not one",
8981 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
8982
8983 charset_list = XCDR (charset_list);
8984 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8985 if (CHARSET_DIMENSION (charset) != 2)
8986 error ("Dimension of charset %s is not two",
8987 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
8988
8989 charset_list = XCDR (charset_list);
8990 if (! NILP (charset_list))
8991 {
8992 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
8993 if (CHARSET_DIMENSION (charset) != 2)
8994 error ("Dimension of charset %s is not two",
8995 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
8996 }
8997
8998 category = coding_category_sjis;
8999 Vsjis_coding_system = name;
9000 }
9001 else if (EQ (coding_type, Qbig5))
9002 {
9003 struct charset *charset;
9004
9005 if (XINT (Flength (charset_list)) != 2)
9006 error ("There should be just two charsets");
9007
9008 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9009 if (CHARSET_DIMENSION (charset) != 1)
9010 error ("Dimension of charset %s is not one",
9011 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9012 if (CHARSET_ASCII_COMPATIBLE_P (charset))
9013 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
9014
9015 charset_list = XCDR (charset_list);
9016 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
9017 if (CHARSET_DIMENSION (charset) != 2)
9018 error ("Dimension of charset %s is not two",
9019 SDATA (SYMBOL_NAME (CHARSET_NAME (charset))));
9020
9021 category = coding_category_big5;
9022 Vbig5_coding_system = name;
9023 }
9024 else if (EQ (coding_type, Qraw_text))
9025 {
9026 category = coding_category_raw_text;
9027 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
9028 }
9029 else if (EQ (coding_type, Qutf_8))
9030 {
9031 category = coding_category_utf_8;
9032 CODING_ATTR_ASCII_COMPAT (attrs) = Qt;
9033 }
9034 else if (EQ (coding_type, Qundecided))
9035 category = coding_category_undecided;
9036 else
9037 error ("Invalid coding system type: %s",
9038 SDATA (SYMBOL_NAME (coding_type)));
9039
9040 CODING_ATTR_CATEGORY (attrs) = make_number (category);
9041 CODING_ATTR_PLIST (attrs)
9042 = Fcons (QCcategory, Fcons (AREF (Vcoding_category_table, category),
9043 CODING_ATTR_PLIST (attrs)));
9044 CODING_ATTR_PLIST (attrs)
9045 = Fcons (QCascii_compatible_p,
9046 Fcons (CODING_ATTR_ASCII_COMPAT (attrs),
9047 CODING_ATTR_PLIST (attrs)));
9048
9049 eol_type = args[coding_arg_eol_type];
9050 if (! NILP (eol_type)
9051 && ! EQ (eol_type, Qunix)
9052 && ! EQ (eol_type, Qdos)
9053 && ! EQ (eol_type, Qmac))
9054 error ("Invalid eol-type");
9055
9056 aliases = Fcons (name, Qnil);
9057
9058 if (NILP (eol_type))
9059 {
9060 eol_type = make_subsidiaries (name);
9061 for (i = 0; i < 3; i++)
9062 {
9063 Lisp_Object this_spec, this_name, this_aliases, this_eol_type;
9064
9065 this_name = AREF (eol_type, i);
9066 this_aliases = Fcons (this_name, Qnil);
9067 this_eol_type = (i == 0 ? Qunix : i == 1 ? Qdos : Qmac);
9068 this_spec = Fmake_vector (make_number (3), attrs);
9069 ASET (this_spec, 1, this_aliases);
9070 ASET (this_spec, 2, this_eol_type);
9071 Fputhash (this_name, this_spec, Vcoding_system_hash_table);
9072 Vcoding_system_list = Fcons (this_name, Vcoding_system_list);
9073 val = Fassoc (Fsymbol_name (this_name), Vcoding_system_alist);
9074 if (NILP (val))
9075 Vcoding_system_alist
9076 = Fcons (Fcons (Fsymbol_name (this_name), Qnil),
9077 Vcoding_system_alist);
9078 }
9079 }
9080
9081 spec_vec = Fmake_vector (make_number (3), attrs);
9082 ASET (spec_vec, 1, aliases);
9083 ASET (spec_vec, 2, eol_type);
9084
9085 Fputhash (name, spec_vec, Vcoding_system_hash_table);
9086 Vcoding_system_list = Fcons (name, Vcoding_system_list);
9087 val = Fassoc (Fsymbol_name (name), Vcoding_system_alist);
9088 if (NILP (val))
9089 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (name), Qnil),
9090 Vcoding_system_alist);
9091
9092 {
9093 int id = coding_categories[category].id;
9094
9095 if (id < 0 || EQ (name, CODING_ID_NAME (id)))
9096 setup_coding_system (name, &coding_categories[category]);
9097 }
9098
9099 return Qnil;
9100
9101 short_args:
9102 return Fsignal (Qwrong_number_of_arguments,
9103 Fcons (intern ("define-coding-system-internal"),
9104 make_number (nargs)));
9105}
9106
9107
9108DEFUN ("coding-system-put", Fcoding_system_put, Scoding_system_put,
9109 3, 3, 0,
9110 doc: /* Change value in CODING-SYSTEM's property list PROP to VAL. */)
9111 (coding_system, prop, val)
9112 Lisp_Object coding_system, prop, val;
9113{
9114 Lisp_Object spec, attrs;
9115
9116 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9117 attrs = AREF (spec, 0);
9118 if (EQ (prop, QCmnemonic))
9119 {
9120 if (! STRINGP (val))
9121 CHECK_CHARACTER (val);
9122 CODING_ATTR_MNEMONIC (attrs) = val;
9123 }
9124 else if (EQ (prop, QCdefalut_char))
9125 {
9126 if (NILP (val))
9127 val = make_number (' ');
9128 else
9129 CHECK_CHARACTER (val);
9130 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
9131 }
9132 else if (EQ (prop, QCdecode_translation_table))
9133 {
9134 if (! CHAR_TABLE_P (val) && ! CONSP (val))
9135 CHECK_SYMBOL (val);
9136 CODING_ATTR_DECODE_TBL (attrs) = val;
9137 }
9138 else if (EQ (prop, QCencode_translation_table))
9139 {
9140 if (! CHAR_TABLE_P (val) && ! CONSP (val))
9141 CHECK_SYMBOL (val);
9142 CODING_ATTR_ENCODE_TBL (attrs) = val;
9143 }
9144 else if (EQ (prop, QCpost_read_conversion))
9145 {
9146 CHECK_SYMBOL (val);
9147 CODING_ATTR_POST_READ (attrs) = val;
9148 }
9149 else if (EQ (prop, QCpre_write_conversion))
9150 {
9151 CHECK_SYMBOL (val);
9152 CODING_ATTR_PRE_WRITE (attrs) = val;
9153 }
9154 else if (EQ (prop, QCascii_compatible_p))
9155 {
9156 CODING_ATTR_ASCII_COMPAT (attrs) = val;
9157 }
9158
9159 CODING_ATTR_PLIST (attrs)
9160 = Fplist_put (CODING_ATTR_PLIST (attrs), prop, val);
9161 return val;
9162}
9163
9164
9165DEFUN ("define-coding-system-alias", Fdefine_coding_system_alias,
9166 Sdefine_coding_system_alias, 2, 2, 0,
9167 doc: /* Define ALIAS as an alias for CODING-SYSTEM. */)
9168 (alias, coding_system)
9169 Lisp_Object alias, coding_system;
9170{
9171 Lisp_Object spec, aliases, eol_type, val;
9172
9173 CHECK_SYMBOL (alias);
9174 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9175 aliases = AREF (spec, 1);
9176 /* ALISES should be a list of length more than zero, and the first
9177 element is a base coding system. Append ALIAS at the tail of the
9178 list. */
9179 while (!NILP (XCDR (aliases)))
9180 aliases = XCDR (aliases);
9181 XSETCDR (aliases, Fcons (alias, Qnil));
9182
9183 eol_type = AREF (spec, 2);
9184 if (VECTORP (eol_type))
9185 {
9186 Lisp_Object subsidiaries;
9187 int i;
9188
9189 subsidiaries = make_subsidiaries (alias);
9190 for (i = 0; i < 3; i++)
9191 Fdefine_coding_system_alias (AREF (subsidiaries, i),
9192 AREF (eol_type, i));
9193 }
9194
9195 Fputhash (alias, spec, Vcoding_system_hash_table);
9196 Vcoding_system_list = Fcons (alias, Vcoding_system_list);
9197 val = Fassoc (Fsymbol_name (alias), Vcoding_system_alist);
9198 if (NILP (val))
9199 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (alias), Qnil),
9200 Vcoding_system_alist);
9201
9202 return Qnil;
9203}
9204
9205DEFUN ("coding-system-base", Fcoding_system_base, Scoding_system_base,
9206 1, 1, 0,
9207 doc: /* Return the base of CODING-SYSTEM.
9208Any alias or subsidiary coding system is not a base coding system. */)
9209 (coding_system)
9210 Lisp_Object coding_system;
9211{
9212 Lisp_Object spec, attrs;
9213
9214 if (NILP (coding_system))
9215 return (Qno_conversion);
9216 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9217 attrs = AREF (spec, 0);
9218 return CODING_ATTR_BASE_NAME (attrs);
9219}
9220
9221DEFUN ("coding-system-plist", Fcoding_system_plist, Scoding_system_plist,
9222 1, 1, 0,
9223 doc: "Return the property list of CODING-SYSTEM.")
9224 (coding_system)
9225 Lisp_Object coding_system;
9226{
9227 Lisp_Object spec, attrs;
9228
9229 if (NILP (coding_system))
9230 coding_system = Qno_conversion;
9231 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9232 attrs = AREF (spec, 0);
9233 return CODING_ATTR_PLIST (attrs);
9234}
9235
9236
9237DEFUN ("coding-system-aliases", Fcoding_system_aliases, Scoding_system_aliases,
9238 1, 1, 0,
9239 doc: /* Return the list of aliases of CODING-SYSTEM. */)
9240 (coding_system)
9241 Lisp_Object coding_system;
9242{
9243 Lisp_Object spec;
9244
9245 if (NILP (coding_system))
9246 coding_system = Qno_conversion;
9247 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
9248 return AREF (spec, 1);
9249}
9250
9251DEFUN ("coding-system-eol-type", Fcoding_system_eol_type,
9252 Scoding_system_eol_type, 1, 1, 0,
9253 doc: /* Return eol-type of CODING-SYSTEM.
9254An eol-type is integer 0, 1, 2, or a vector of coding systems.
9255
9256Integer values 0, 1, and 2 indicate a format of end-of-line; LF, CRLF,
9257and CR respectively.
9258
9259A vector value indicates that a format of end-of-line should be
9260detected automatically. Nth element of the vector is the subsidiary
9261coding system whose eol-type is N. */)
9262 (coding_system)
9263 Lisp_Object coding_system;
9264{
9265 Lisp_Object spec, eol_type;
9266 int n;
9267
9268 if (NILP (coding_system))
9269 coding_system = Qno_conversion;
9270 if (! CODING_SYSTEM_P (coding_system))
9271 return Qnil;
9272 spec = CODING_SYSTEM_SPEC (coding_system);
9273 eol_type = AREF (spec, 2);
9274 if (VECTORP (eol_type))
9275 return Fcopy_sequence (eol_type);
9276 n = EQ (eol_type, Qunix) ? 0 : EQ (eol_type, Qdos) ? 1 : 2;
9277 return make_number (n);
9278}
9279
9280#endif /* emacs */
9281
9282\f
9283/*** 9. Post-amble ***/
9284
9285void
9286init_coding_once ()
9287{
9288 int i;
9289
9290 for (i = 0; i < coding_category_max; i++)
9291 {
9292 coding_categories[i].id = -1;
9293 coding_priorities[i] = i;
9294 }
9295
9296 /* ISO2022 specific initialize routine. */
9297 for (i = 0; i < 0x20; i++)
9298 iso_code_class[i] = ISO_control_0;
9299 for (i = 0x21; i < 0x7F; i++)
9300 iso_code_class[i] = ISO_graphic_plane_0;
9301 for (i = 0x80; i < 0xA0; i++)
9302 iso_code_class[i] = ISO_control_1;
9303 for (i = 0xA1; i < 0xFF; i++)
9304 iso_code_class[i] = ISO_graphic_plane_1;
9305 iso_code_class[0x20] = iso_code_class[0x7F] = ISO_0x20_or_0x7F;
9306 iso_code_class[0xA0] = iso_code_class[0xFF] = ISO_0xA0_or_0xFF;
9307 iso_code_class[ISO_CODE_SO] = ISO_shift_out;
9308 iso_code_class[ISO_CODE_SI] = ISO_shift_in;
9309 iso_code_class[ISO_CODE_SS2_7] = ISO_single_shift_2_7;
9310 iso_code_class[ISO_CODE_ESC] = ISO_escape;
9311 iso_code_class[ISO_CODE_SS2] = ISO_single_shift_2;
9312 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3;
9313 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer;
9314
9315 for (i = 0; i < 256; i++)
9316 {
9317 emacs_mule_bytes[i] = 1;
9318 }
9319 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_11] = 3;
9320 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_12] = 3;
9321 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_21] = 4;
9322 emacs_mule_bytes[EMACS_MULE_LEADING_CODE_PRIVATE_22] = 4;
9323}
9324
9325#ifdef emacs
9326
9327void
9328syms_of_coding ()
9329{
9330 staticpro (&Vcoding_system_hash_table);
9331 {
9332 Lisp_Object args[2];
9333 args[0] = QCtest;
9334 args[1] = Qeq;
9335 Vcoding_system_hash_table = Fmake_hash_table (2, args);
9336 }
9337
9338 staticpro (&Vsjis_coding_system);
9339 Vsjis_coding_system = Qnil;
9340
9341 staticpro (&Vbig5_coding_system);
9342 Vbig5_coding_system = Qnil;
9343
9344 staticpro (&Vcode_conversion_reused_workbuf);
9345 Vcode_conversion_reused_workbuf = Qnil;
9346
9347 staticpro (&Vcode_conversion_workbuf_name);
9348 Vcode_conversion_workbuf_name = build_string (" *code-conversion-work*");
9349
9350 reused_workbuf_in_use = 0;
9351
9352 DEFSYM (Qcharset, "charset");
9353 DEFSYM (Qtarget_idx, "target-idx");
9354 DEFSYM (Qcoding_system_history, "coding-system-history");
9355 Fset (Qcoding_system_history, Qnil);
9356
9357 /* Target FILENAME is the first argument. */
9358 Fput (Qinsert_file_contents, Qtarget_idx, make_number (0));
9359 /* Target FILENAME is the third argument. */
9360 Fput (Qwrite_region, Qtarget_idx, make_number (2));
9361
9362 DEFSYM (Qcall_process, "call-process");
9363 /* Target PROGRAM is the first argument. */
9364 Fput (Qcall_process, Qtarget_idx, make_number (0));
9365
9366 DEFSYM (Qcall_process_region, "call-process-region");
9367 /* Target PROGRAM is the third argument. */
9368 Fput (Qcall_process_region, Qtarget_idx, make_number (2));
9369
9370 DEFSYM (Qstart_process, "start-process");
9371 /* Target PROGRAM is the third argument. */
9372 Fput (Qstart_process, Qtarget_idx, make_number (2));
9373
9374 DEFSYM (Qopen_network_stream, "open-network-stream");
9375 /* Target SERVICE is the fourth argument. */
9376 Fput (Qopen_network_stream, Qtarget_idx, make_number (3));
9377
9378 DEFSYM (Qcoding_system, "coding-system");
9379 DEFSYM (Qcoding_aliases, "coding-aliases");
9380
9381 DEFSYM (Qeol_type, "eol-type");
9382 DEFSYM (Qunix, "unix");
9383 DEFSYM (Qdos, "dos");
9384
9385 DEFSYM (Qbuffer_file_coding_system, "buffer-file-coding-system");
9386 DEFSYM (Qpost_read_conversion, "post-read-conversion");
9387 DEFSYM (Qpre_write_conversion, "pre-write-conversion");
9388 DEFSYM (Qdefault_char, "default-char");
9389 DEFSYM (Qundecided, "undecided");
9390 DEFSYM (Qno_conversion, "no-conversion");
9391 DEFSYM (Qraw_text, "raw-text");
9392
9393 DEFSYM (Qiso_2022, "iso-2022");
9394
9395 DEFSYM (Qutf_8, "utf-8");
9396 DEFSYM (Qutf_8_emacs, "utf-8-emacs");
9397
9398 DEFSYM (Qutf_16, "utf-16");
9399 DEFSYM (Qbig, "big");
9400 DEFSYM (Qlittle, "little");
9401
9402 DEFSYM (Qshift_jis, "shift-jis");
9403 DEFSYM (Qbig5, "big5");
9404
9405 DEFSYM (Qcoding_system_p, "coding-system-p");
9406
9407 DEFSYM (Qcoding_system_error, "coding-system-error");
9408 Fput (Qcoding_system_error, Qerror_conditions,
9409 Fcons (Qcoding_system_error, Fcons (Qerror, Qnil)));
9410 Fput (Qcoding_system_error, Qerror_message,
9411 build_string ("Invalid coding system"));
9412
9413 /* Intern this now in case it isn't already done.
9414 Setting this variable twice is harmless.
9415 But don't staticpro it here--that is done in alloc.c. */
9416 Qchar_table_extra_slots = intern ("char-table-extra-slots");
9417
9418 DEFSYM (Qtranslation_table, "translation-table");
9419 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (2));
9420 DEFSYM (Qtranslation_table_id, "translation-table-id");
9421 DEFSYM (Qtranslation_table_for_decode, "translation-table-for-decode");
9422 DEFSYM (Qtranslation_table_for_encode, "translation-table-for-encode");
9423
9424 DEFSYM (Qvalid_codes, "valid-codes");
9425
9426 DEFSYM (Qemacs_mule, "emacs-mule");
9427
9428 DEFSYM (QCcategory, ":category");
9429 DEFSYM (QCmnemonic, ":mnemonic");
9430 DEFSYM (QCdefalut_char, ":default-char");
9431 DEFSYM (QCdecode_translation_table, ":decode-translation-table");
9432 DEFSYM (QCencode_translation_table, ":encode-translation-table");
9433 DEFSYM (QCpost_read_conversion, ":post-read-conversion");
9434 DEFSYM (QCpre_write_conversion, ":pre-write-conversion");
9435 DEFSYM (QCascii_compatible_p, ":ascii-compatible-p");
9436
9437 Vcoding_category_table
9438 = Fmake_vector (make_number (coding_category_max), Qnil);
9439 staticpro (&Vcoding_category_table);
9440 /* Followings are target of code detection. */
9441 ASET (Vcoding_category_table, coding_category_iso_7,
9442 intern ("coding-category-iso-7"));
9443 ASET (Vcoding_category_table, coding_category_iso_7_tight,
9444 intern ("coding-category-iso-7-tight"));
9445 ASET (Vcoding_category_table, coding_category_iso_8_1,
9446 intern ("coding-category-iso-8-1"));
9447 ASET (Vcoding_category_table, coding_category_iso_8_2,
9448 intern ("coding-category-iso-8-2"));
9449 ASET (Vcoding_category_table, coding_category_iso_7_else,
9450 intern ("coding-category-iso-7-else"));
9451 ASET (Vcoding_category_table, coding_category_iso_8_else,
9452 intern ("coding-category-iso-8-else"));
9453 ASET (Vcoding_category_table, coding_category_utf_8,
9454 intern ("coding-category-utf-8"));
9455 ASET (Vcoding_category_table, coding_category_utf_16_be,
9456 intern ("coding-category-utf-16-be"));
9457 ASET (Vcoding_category_table, coding_category_utf_16_auto,
9458 intern ("coding-category-utf-16-auto"));
9459 ASET (Vcoding_category_table, coding_category_utf_16_le,
9460 intern ("coding-category-utf-16-le"));
9461 ASET (Vcoding_category_table, coding_category_utf_16_be_nosig,
9462 intern ("coding-category-utf-16-be-nosig"));
9463 ASET (Vcoding_category_table, coding_category_utf_16_le_nosig,
9464 intern ("coding-category-utf-16-le-nosig"));
9465 ASET (Vcoding_category_table, coding_category_charset,
9466 intern ("coding-category-charset"));
9467 ASET (Vcoding_category_table, coding_category_sjis,
9468 intern ("coding-category-sjis"));
9469 ASET (Vcoding_category_table, coding_category_big5,
9470 intern ("coding-category-big5"));
9471 ASET (Vcoding_category_table, coding_category_ccl,
9472 intern ("coding-category-ccl"));
9473 ASET (Vcoding_category_table, coding_category_emacs_mule,
9474 intern ("coding-category-emacs-mule"));
9475 /* Followings are NOT target of code detection. */
9476 ASET (Vcoding_category_table, coding_category_raw_text,
9477 intern ("coding-category-raw-text"));
9478 ASET (Vcoding_category_table, coding_category_undecided,
9479 intern ("coding-category-undecided"));
9480
9481 DEFSYM (Qinsufficient_source, "insufficient-source");
9482 DEFSYM (Qinconsistent_eol, "inconsistent-eol");
9483 DEFSYM (Qinvalid_source, "invalid-source");
9484 DEFSYM (Qinterrupted, "interrupted");
9485 DEFSYM (Qinsufficient_memory, "insufficient-memory");
9486 DEFSYM (Qcoding_system_define_form, "coding-system-define-form");
9487
9488 defsubr (&Scoding_system_p);
9489 defsubr (&Sread_coding_system);
9490 defsubr (&Sread_non_nil_coding_system);
9491 defsubr (&Scheck_coding_system);
9492 defsubr (&Sdetect_coding_region);
9493 defsubr (&Sdetect_coding_string);
9494 defsubr (&Sfind_coding_systems_region_internal);
9495 defsubr (&Sunencodable_char_position);
9496 defsubr (&Scheck_coding_systems_region);
9497 defsubr (&Sdecode_coding_region);
9498 defsubr (&Sencode_coding_region);
9499 defsubr (&Sdecode_coding_string);
9500 defsubr (&Sencode_coding_string);
9501 defsubr (&Sdecode_sjis_char);
9502 defsubr (&Sencode_sjis_char);
9503 defsubr (&Sdecode_big5_char);
9504 defsubr (&Sencode_big5_char);
9505 defsubr (&Sset_terminal_coding_system_internal);
9506 defsubr (&Sset_safe_terminal_coding_system_internal);
9507 defsubr (&Sterminal_coding_system);
9508 defsubr (&Sset_keyboard_coding_system_internal);
9509 defsubr (&Skeyboard_coding_system);
9510 defsubr (&Sfind_operation_coding_system);
9511 defsubr (&Sset_coding_system_priority);
9512 defsubr (&Sdefine_coding_system_internal);
9513 defsubr (&Sdefine_coding_system_alias);
9514 defsubr (&Scoding_system_put);
9515 defsubr (&Scoding_system_base);
9516 defsubr (&Scoding_system_plist);
9517 defsubr (&Scoding_system_aliases);
9518 defsubr (&Scoding_system_eol_type);
9519 defsubr (&Scoding_system_priority_list);
9520
9521 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list,
9522 doc: /* List of coding systems.
9523
9524Do not alter the value of this variable manually. This variable should be
9525updated by the functions `define-coding-system' and
9526`define-coding-system-alias'. */);
9527 Vcoding_system_list = Qnil;
9528
9529 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist,
9530 doc: /* Alist of coding system names.
9531Each element is one element list of coding system name.
9532This variable is given to `completing-read' as TABLE argument.
9533
9534Do not alter the value of this variable manually. This variable should be
9535updated by the functions `make-coding-system' and
9536`define-coding-system-alias'. */);
9537 Vcoding_system_alist = Qnil;
9538
9539 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list,
9540 doc: /* List of coding-categories (symbols) ordered by priority.
9541
9542On detecting a coding system, Emacs tries code detection algorithms
9543associated with each coding-category one by one in this order. When
9544one algorithm agrees with a byte sequence of source text, the coding
9545system bound to the corresponding coding-category is selected.
9546
9547Don't modify this variable directly, but use `set-coding-priority'. */);
9548 {
9549 int i;
9550
9551 Vcoding_category_list = Qnil;
9552 for (i = coding_category_max - 1; i >= 0; i--)
9553 Vcoding_category_list
9554 = Fcons (XVECTOR (Vcoding_category_table)->contents[i],
9555 Vcoding_category_list);
9556 }
9557
9558 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read,
9559 doc: /* Specify the coding system for read operations.
9560It is useful to bind this variable with `let', but do not set it globally.
9561If the value is a coding system, it is used for decoding on read operation.
9562If not, an appropriate element is used from one of the coding system alists:
9563There are three such tables, `file-coding-system-alist',
9564`process-coding-system-alist', and `network-coding-system-alist'. */);
9565 Vcoding_system_for_read = Qnil;
9566
9567 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write,
9568 doc: /* Specify the coding system for write operations.
9569Programs bind this variable with `let', but you should not set it globally.
9570If the value is a coding system, it is used for encoding of output,
9571when writing it to a file and when sending it to a file or subprocess.
9572
9573If this does not specify a coding system, an appropriate element
9574is used from one of the coding system alists:
9575There are three such tables, `file-coding-system-alist',
9576`process-coding-system-alist', and `network-coding-system-alist'.
9577For output to files, if the above procedure does not specify a coding system,
9578the value of `buffer-file-coding-system' is used. */);
9579 Vcoding_system_for_write = Qnil;
9580
9581 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used,
9582 doc: /*
9583Coding system used in the latest file or process I/O. */);
9584 Vlast_coding_system_used = Qnil;
9585
9586 DEFVAR_LISP ("last-code-conversion-error", &Vlast_code_conversion_error,
9587 doc: /*
9588Error status of the last code conversion.
9589
9590When an error was detected in the last code conversion, this variable
9591is set to one of the following symbols.
9592 `insufficient-source'
9593 `inconsistent-eol'
9594 `invalid-source'
9595 `interrupted'
9596 `insufficient-memory'
9597When no error was detected, the value doesn't change. So, to check
9598the error status of a code conversion by this variable, you must
9599explicitly set this variable to nil before performing code
9600conversion. */);
9601 Vlast_code_conversion_error = Qnil;
9602
9603 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion,
9604 doc: /*
9605*Non-nil means always inhibit code conversion of end-of-line format.
9606See info node `Coding Systems' and info node `Text and Binary' concerning
9607such conversion. */);
9608 inhibit_eol_conversion = 0;
9609
9610 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system,
9611 doc: /*
9612Non-nil means process buffer inherits coding system of process output.
9613Bind it to t if the process output is to be treated as if it were a file
9614read from some filesystem. */);
9615 inherit_process_coding_system = 0;
9616
9617 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist,
9618 doc: /*
9619Alist to decide a coding system to use for a file I/O operation.
9620The format is ((PATTERN . VAL) ...),
9621where PATTERN is a regular expression matching a file name,
9622VAL is a coding system, a cons of coding systems, or a function symbol.
9623If VAL is a coding system, it is used for both decoding and encoding
9624the file contents.
9625If VAL is a cons of coding systems, the car part is used for decoding,
9626and the cdr part is used for encoding.
9627If VAL is a function symbol, the function must return a coding system
9628or a cons of coding systems which are used as above. The function is
9629called with an argument that is a list of the arguments with which
9630`find-operation-coding-system' was called. If the function can't decide
9631a coding system, it can return `undecided' so that the normal
9632code-detection is performed.
9633
9634See also the function `find-operation-coding-system'
9635and the variable `auto-coding-alist'. */);
9636 Vfile_coding_system_alist = Qnil;
9637
9638 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist,
9639 doc: /*
9640Alist to decide a coding system to use for a process I/O operation.
9641The format is ((PATTERN . VAL) ...),
9642where PATTERN is a regular expression matching a program name,
9643VAL is a coding system, a cons of coding systems, or a function symbol.
9644If VAL is a coding system, it is used for both decoding what received
9645from the program and encoding what sent to the program.
9646If VAL is a cons of coding systems, the car part is used for decoding,
9647and the cdr part is used for encoding.
9648If VAL is a function symbol, the function must return a coding system
9649or a cons of coding systems which are used as above.
9650
9651See also the function `find-operation-coding-system'. */);
9652 Vprocess_coding_system_alist = Qnil;
9653
9654 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist,
9655 doc: /*
9656Alist to decide a coding system to use for a network I/O operation.
9657The format is ((PATTERN . VAL) ...),
9658where PATTERN is a regular expression matching a network service name
9659or is a port number to connect to,
9660VAL is a coding system, a cons of coding systems, or a function symbol.
9661If VAL is a coding system, it is used for both decoding what received
9662from the network stream and encoding what sent to the network stream.
9663If VAL is a cons of coding systems, the car part is used for decoding,
9664and the cdr part is used for encoding.
9665If VAL is a function symbol, the function must return a coding system
9666or a cons of coding systems which are used as above.
9667
9668See also the function `find-operation-coding-system'. */);
9669 Vnetwork_coding_system_alist = Qnil;
9670
9671 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system,
9672 doc: /* Coding system to use with system messages.
9673Also used for decoding keyboard input on X Window system. */);
9674 Vlocale_coding_system = Qnil;
9675
9676 /* The eol mnemonics are reset in startup.el system-dependently. */
9677 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix,
9678 doc: /*
9679*String displayed in mode line for UNIX-like (LF) end-of-line format. */);
9680 eol_mnemonic_unix = build_string (":");
9681
9682 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos,
9683 doc: /*
9684*String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
9685 eol_mnemonic_dos = build_string ("\\");
9686
9687 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac,
9688 doc: /*
9689*String displayed in mode line for MAC-like (CR) end-of-line format. */);
9690 eol_mnemonic_mac = build_string ("/");
9691
9692 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided,
9693 doc: /*
9694*String displayed in mode line when end-of-line format is not yet determined. */);
9695 eol_mnemonic_undecided = build_string (":");
9696
9697 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation,
9698 doc: /*
9699*Non-nil enables character translation while encoding and decoding. */);
9700 Venable_character_translation = Qt;
9701
9702 DEFVAR_LISP ("standard-translation-table-for-decode",
9703 &Vstandard_translation_table_for_decode,
9704 doc: /* Table for translating characters while decoding. */);
9705 Vstandard_translation_table_for_decode = Qnil;
9706
9707 DEFVAR_LISP ("standard-translation-table-for-encode",
9708 &Vstandard_translation_table_for_encode,
9709 doc: /* Table for translating characters while encoding. */);
9710 Vstandard_translation_table_for_encode = Qnil;
9711
9712 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_table,
9713 doc: /* Alist of charsets vs revision numbers.
9714While encoding, if a charset (car part of an element) is found,
9715designate it with the escape sequence identifying revision (cdr part
9716of the element). */);
9717 Vcharset_revision_table = Qnil;
9718
9719 DEFVAR_LISP ("default-process-coding-system",
9720 &Vdefault_process_coding_system,
9721 doc: /* Cons of coding systems used for process I/O by default.
9722The car part is used for decoding a process output,
9723the cdr part is used for encoding a text to be sent to a process. */);
9724 Vdefault_process_coding_system = Qnil;
9725
9726 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table,
9727 doc: /*
9728Table of extra Latin codes in the range 128..159 (inclusive).
9729This is a vector of length 256.
9730If Nth element is non-nil, the existence of code N in a file
9731\(or output of subprocess) doesn't prevent it to be detected as
9732a coding system of ISO 2022 variant which has a flag
9733`accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
9734or reading output of a subprocess.
9735Only 128th through 159th elements has a meaning. */);
9736 Vlatin_extra_code_table = Fmake_vector (make_number (256), Qnil);
9737
9738 DEFVAR_LISP ("select-safe-coding-system-function",
9739 &Vselect_safe_coding_system_function,
9740 doc: /*
9741Function to call to select safe coding system for encoding a text.
9742
9743If set, this function is called to force a user to select a proper
9744coding system which can encode the text in the case that a default
9745coding system used in each operation can't encode the text.
9746
9747The default value is `select-safe-coding-system' (which see). */);
9748 Vselect_safe_coding_system_function = Qnil;
9749
9750 DEFVAR_BOOL ("coding-system-require-warning",
9751 &coding_system_require_warning,
9752 doc: /* Internal use only.
9753If non-nil, on writing a file, `select-safe-coding-system-function' is
9754called even if `coding-system-for-write' is non-nil. The command
9755`universal-coding-system-argument' binds this variable to t temporarily. */);
9756 coding_system_require_warning = 0;
9757
9758
9759 DEFVAR_BOOL ("inhibit-iso-escape-detection",
9760 &inhibit_iso_escape_detection,
9761 doc: /*
9762If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
9763
9764By default, on reading a file, Emacs tries to detect how the text is
9765encoded. This code detection is sensitive to escape sequences. If
9766the sequence is valid as ISO2022, the code is determined as one of
9767the ISO2022 encodings, and the file is decoded by the corresponding
9768coding system (e.g. `iso-2022-7bit').
9769
9770However, there may be a case that you want to read escape sequences in
9771a file as is. In such a case, you can set this variable to non-nil.
9772Then, as the code detection ignores any escape sequences, no file is
9773detected as encoded in some ISO2022 encoding. The result is that all
9774escape sequences become visible in a buffer.
9775
9776The default value is nil, and it is strongly recommended not to change
9777it. That is because many Emacs Lisp source files that contain
9778non-ASCII characters are encoded by the coding system `iso-2022-7bit'
9779in Emacs's distribution, and they won't be decoded correctly on
9780reading if you suppress escape sequence detection.
9781
9782The other way to read escape sequences in a file without decoding is
9783to explicitly specify some coding system that doesn't use ISO2022's
9784escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
9785 inhibit_iso_escape_detection = 0;
9786
9787 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input,
9788 doc: /* Char table for translating self-inserting characters.
9789This is applied to the result of input methods, not their input. See also
9790`keyboard-translate-table'. */);
9791 Vtranslation_table_for_input = Qnil;
9792
9793 {
9794 Lisp_Object args[coding_arg_max];
9795 Lisp_Object plist[16];
9796 int i;
9797
9798 for (i = 0; i < coding_arg_max; i++)
9799 args[i] = Qnil;
9800
9801 plist[0] = intern (":name");
9802 plist[1] = args[coding_arg_name] = Qno_conversion;
9803 plist[2] = intern (":mnemonic");
9804 plist[3] = args[coding_arg_mnemonic] = make_number ('=');
9805 plist[4] = intern (":coding-type");
9806 plist[5] = args[coding_arg_coding_type] = Qraw_text;
9807 plist[6] = intern (":ascii-compatible-p");
9808 plist[7] = args[coding_arg_ascii_compatible_p] = Qt;
9809 plist[8] = intern (":default-char");
9810 plist[9] = args[coding_arg_default_char] = make_number (0);
9811 plist[10] = intern (":for-unibyte");
9812 plist[11] = args[coding_arg_for_unibyte] = Qt;
9813 plist[12] = intern (":docstring");
9814 plist[13] = build_string ("Do no conversion.\n\
9815\n\
9816When you visit a file with this coding, the file is read into a\n\
9817unibyte buffer as is, thus each byte of a file is treated as a\n\
9818character.");
9819 plist[14] = intern (":eol-type");
9820 plist[15] = args[coding_arg_eol_type] = Qunix;
9821 args[coding_arg_plist] = Flist (16, plist);
9822 Fdefine_coding_system_internal (coding_arg_max, args);
9823
9824 plist[1] = args[coding_arg_name] = Qundecided;
9825 plist[3] = args[coding_arg_mnemonic] = make_number ('-');
9826 plist[5] = args[coding_arg_coding_type] = Qundecided;
9827 /* This is already set.
9828 plist[7] = args[coding_arg_ascii_compatible_p] = Qt; */
9829 plist[8] = intern (":charset-list");
9830 plist[9] = args[coding_arg_charset_list] = Fcons (Qascii, Qnil);
9831 plist[11] = args[coding_arg_for_unibyte] = Qnil;
9832 plist[13] = build_string ("No conversion on encoding, automatic conversion on decoding.");
9833 plist[15] = args[coding_arg_eol_type] = Qnil;
9834 args[coding_arg_plist] = Flist (16, plist);
9835 Fdefine_coding_system_internal (coding_arg_max, args);
9836 }
9837
9838 setup_coding_system (Qno_conversion, &safe_terminal_coding);
9839
9840 {
9841 int i;
9842
9843 for (i = 0; i < coding_category_max; i++)
9844 Fset (AREF (Vcoding_category_table, i), Qno_conversion);
9845 }
9846#if defined (MSDOS) || defined (WINDOWSNT)
9847 system_eol_type = Qdos;
9848#else
9849 system_eol_type = Qunix;
9850#endif
9851 staticpro (&system_eol_type);
9852}
9853
9854char *
9855emacs_strerror (error_number)
9856 int error_number;
9857{
9858 char *str;
9859
9860 synchronize_system_messages_locale ();
9861 str = strerror (error_number);
9862
9863 if (! NILP (Vlocale_coding_system))
9864 {
9865 Lisp_Object dec = code_convert_string_norecord (build_string (str),
9866 Vlocale_coding_system,
9867 0);
9868 str = (char *) SDATA (dec);
9869 }
9870
9871 return str;
9872}
9873
9874#endif /* emacs */
9875
9876/* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
9877 (do not change this comment) */
Unnamed repository; edit this file 'description' to name the repository.