1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
3 2006, 2007, 2008 Free Software Foundation, Inc.
4 Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
6 National Institute of Advanced Industrial Science and Technology (AIST)
7 Registration Number H14PRO021
9 This file is part of GNU Emacs.
11 GNU Emacs is free software; you can redistribute it and/or modify
12 it under the terms of the GNU General Public License as published by
13 the Free Software Foundation; either version 3, or (at your option)
16 GNU Emacs is distributed in the hope that it will be useful,
17 but WITHOUT ANY WARRANTY; without even the implied warranty of
18 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19 GNU General Public License for more details.
21 You should have received a copy of the GNU General Public License
22 along with GNU Emacs; see the file COPYING. If not, write to
23 the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
24 Boston, MA 02110-1301, USA. */
26 /*** TABLE OF CONTENTS ***
30 2. Emacs' internal format (emacs-mule) handlers
32 4. Shift-JIS and BIG5 handlers
34 6. End-of-line handlers
35 7. C library functions
36 8. Emacs Lisp library functions
41 /*** 0. General comments ***/
44 /*** GENERAL NOTE on CODING SYSTEMS ***
46 A coding system is an encoding mechanism for one or more character
47 sets. Here's a list of coding systems which Emacs can handle. When
48 we say "decode", it means converting some other coding system to
49 Emacs' internal format (emacs-mule), and when we say "encode",
50 it means converting the coding system emacs-mule to some other
53 0. Emacs' internal format (emacs-mule)
55 Emacs itself holds a multi-lingual character in buffers and strings
56 in a special format. Details are described in section 2.
60 The most famous coding system for multiple character sets. X's
61 Compound Text, various EUCs (Extended Unix Code), and coding
62 systems used in Internet communication such as ISO-2022-JP are
63 all variants of ISO2022. Details are described in section 3.
65 2. SJIS (or Shift-JIS or MS-Kanji-Code)
67 A coding system to encode character sets: ASCII, JISX0201, and
68 JISX0208. Widely used for PC's in Japan. Details are described in
73 A coding system to encode the character sets ASCII and Big5. Widely
74 used for Chinese (mainly in Taiwan and Hong Kong). Details are
75 described in section 4. In this file, when we write "BIG5"
76 (all uppercase), we mean the coding system, and when we write
77 "Big5" (capitalized), we mean the character set.
81 A coding system for text containing random 8-bit code. Emacs does
82 no code conversion on such text except for end-of-line format.
86 If a user wants to read/write text encoded in a coding system not
87 listed above, he can supply a decoder and an encoder for it as CCL
88 (Code Conversion Language) programs. Emacs executes the CCL program
89 while reading/writing.
91 Emacs represents a coding system by a Lisp symbol that has a property
92 `coding-system'. But, before actually using the coding system, the
93 information about it is set in a structure of type `struct
94 coding_system' for rapid processing. See section 6 for more details.
98 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
100 How end-of-line of text is encoded depends on the operating system.
101 For instance, Unix's format is just one byte of `line-feed' code,
102 whereas DOS's format is two-byte sequence of `carriage-return' and
103 `line-feed' codes. MacOS's format is usually one byte of
106 Since text character encoding and end-of-line encoding are
107 independent, any coding system described above can have any
108 end-of-line format. So Emacs has information about end-of-line
109 format in each coding-system. See section 6 for more details.
113 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
115 These functions check if a text between SRC and SRC_END is encoded
116 in the coding system category XXX. Each returns an integer value in
117 which appropriate flag bits for the category XXX are set. The flag
118 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
119 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
120 of the range 0x80..0x9F are in multibyte form. */
123 detect_coding_emacs_mule (src
, src_end
, multibytep
)
124 unsigned char *src
, *src_end
;
131 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
133 These functions decode SRC_BYTES length of unibyte text at SOURCE
134 encoded in CODING to Emacs' internal format. The resulting
135 multibyte text goes to a place pointed to by DESTINATION, the length
136 of which should not exceed DST_BYTES.
138 These functions set the information about original and decoded texts
139 in the members `produced', `produced_char', `consumed', and
140 `consumed_char' of the structure *CODING. They also set the member
141 `result' to one of CODING_FINISH_XXX indicating how the decoding
144 DST_BYTES zero means that the source area and destination area are
145 overlapped, which means that we can produce a decoded text until it
146 reaches the head of the not-yet-decoded source text.
148 Below is a template for these functions. */
151 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
152 struct coding_system
*coding
;
153 const unsigned char *source
;
154 unsigned char *destination
;
155 int src_bytes
, dst_bytes
;
161 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
163 These functions encode SRC_BYTES length text at SOURCE from Emacs'
164 internal multibyte format to CODING. The resulting unibyte text
165 goes to a place pointed to by DESTINATION, the length of which
166 should not exceed DST_BYTES.
168 These functions set the information about original and encoded texts
169 in the members `produced', `produced_char', `consumed', and
170 `consumed_char' of the structure *CODING. They also set the member
171 `result' to one of CODING_FINISH_XXX indicating how the encoding
174 DST_BYTES zero means that the source area and destination area are
175 overlapped, which means that we can produce encoded text until it
176 reaches at the head of the not-yet-encoded source text.
178 Below is a template for these functions. */
181 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
182 struct coding_system
*coding
;
183 unsigned char *source
, *destination
;
184 int src_bytes
, dst_bytes
;
190 /*** COMMONLY USED MACROS ***/
192 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
193 get one, two, and three bytes from the source text respectively.
194 If there are not enough bytes in the source, they jump to
195 `label_end_of_loop'. The caller should set variables `coding',
196 `src' and `src_end' to appropriate pointer in advance. These
197 macros are called from decoding routines `decode_coding_XXX', thus
198 it is assumed that the source text is unibyte. */
200 #define ONE_MORE_BYTE(c1) \
202 if (src >= src_end) \
204 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
205 goto label_end_of_loop; \
210 #define TWO_MORE_BYTES(c1, c2) \
212 if (src + 1 >= src_end) \
214 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
215 goto label_end_of_loop; \
222 /* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
223 form if MULTIBYTEP is nonzero. In addition, if SRC is not less
224 than SRC_END, return with RET. */
226 #define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep, ret) \
228 if (src >= src_end) \
230 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
234 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
235 c1 = *src++ - 0x20; \
238 /* Set C to the next character at the source text pointed by `src'.
239 If there are not enough characters in the source, jump to
240 `label_end_of_loop'. The caller should set variables `coding'
241 `src', `src_end', and `translation_table' to appropriate pointers
242 in advance. This macro is used in encoding routines
243 `encode_coding_XXX', thus it assumes that the source text is in
244 multibyte form except for 8-bit characters. 8-bit characters are
245 in multibyte form if coding->src_multibyte is nonzero, else they
246 are represented by a single byte. */
248 #define ONE_MORE_CHAR(c) \
250 int len = src_end - src; \
254 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
255 goto label_end_of_loop; \
257 if (coding->src_multibyte \
258 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
259 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
261 c = *src, bytes = 1; \
262 if (!NILP (translation_table)) \
263 c = translate_char (translation_table, c, -1, 0, 0); \
268 /* Produce a multibyte form of character C to `dst'. Jump to
269 `label_end_of_loop' if there's not enough space at `dst'.
271 If we are now in the middle of a composition sequence, the decoded
272 character may be ALTCHAR (for the current composition). In that
273 case, the character goes to coding->cmp_data->data instead of
276 This macro is used in decoding routines. */
278 #define EMIT_CHAR(c) \
280 if (! COMPOSING_P (coding) \
281 || coding->composing == COMPOSITION_RELATIVE \
282 || coding->composing == COMPOSITION_WITH_RULE) \
284 int bytes = CHAR_BYTES (c); \
285 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
287 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
288 goto label_end_of_loop; \
290 dst += CHAR_STRING (c, dst); \
291 coding->produced_char++; \
294 if (COMPOSING_P (coding) \
295 && coding->composing != COMPOSITION_RELATIVE) \
297 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
298 coding->composition_rule_follows \
299 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
304 #define EMIT_ONE_BYTE(c) \
306 if (dst >= (dst_bytes ? dst_end : src)) \
308 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
309 goto label_end_of_loop; \
314 #define EMIT_TWO_BYTES(c1, c2) \
316 if (dst + 2 > (dst_bytes ? dst_end : src)) \
318 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
319 goto label_end_of_loop; \
321 *dst++ = c1, *dst++ = c2; \
324 #define EMIT_BYTES(from, to) \
326 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
328 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
329 goto label_end_of_loop; \
336 /*** 1. Preamble ***/
349 #include "composite.h"
353 #include "intervals.h"
355 #include "termhooks.h"
357 #else /* not emacs */
361 #endif /* not emacs */
363 Lisp_Object Qcoding_system
, Qeol_type
;
364 Lisp_Object Qbuffer_file_coding_system
;
365 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
366 Lisp_Object Qno_conversion
, Qundecided
;
367 Lisp_Object Qcoding_system_history
;
368 Lisp_Object Qsafe_chars
;
369 Lisp_Object Qvalid_codes
;
370 Lisp_Object Qascii_incompatible
;
372 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
373 Lisp_Object Qcall_process
, Qcall_process_region
;
374 Lisp_Object Qstart_process
, Qopen_network_stream
;
375 Lisp_Object Qtarget_idx
;
377 extern Lisp_Object Qcompletion_ignore_case
;
379 /* If a symbol has this property, evaluate the value to define the
380 symbol as a coding system. */
381 Lisp_Object Qcoding_system_define_form
;
383 Lisp_Object Vselect_safe_coding_system_function
;
385 int coding_system_require_warning
;
387 /* Mnemonic string for each format of end-of-line. */
388 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
389 /* Mnemonic string to indicate format of end-of-line is not yet
391 Lisp_Object eol_mnemonic_undecided
;
393 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
394 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac.
395 This has an effect only for external encoding (i.e. for output to
396 file and process), not for in-buffer or Lisp string encoding. */
401 /* Information about which coding system is safe for which chars.
402 The value has the form (GENERIC-LIST . NON-GENERIC-ALIST).
404 GENERIC-LIST is a list of generic coding systems which can encode
407 NON-GENERIC-ALIST is an alist of non generic coding systems vs the
408 corresponding char table that contains safe chars. */
409 Lisp_Object Vcoding_system_safe_chars
;
411 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
413 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
415 /* Coding system emacs-mule and raw-text are for converting only
416 end-of-line format. */
417 Lisp_Object Qemacs_mule
, Qraw_text
;
421 /* Coding-systems are handed between Emacs Lisp programs and C internal
422 routines by the following three variables. */
423 /* Coding-system for reading files and receiving data from process. */
424 Lisp_Object Vcoding_system_for_read
;
425 /* Coding-system for writing files and sending data to process. */
426 Lisp_Object Vcoding_system_for_write
;
427 /* Coding-system actually used in the latest I/O. */
428 Lisp_Object Vlast_coding_system_used
;
430 /* A vector of length 256 which contains information about special
431 Latin codes (especially for dealing with Microsoft codes). */
432 Lisp_Object Vlatin_extra_code_table
;
434 /* Flag to inhibit code conversion of end-of-line format. */
435 int inhibit_eol_conversion
;
437 /* Flag to inhibit ISO2022 escape sequence detection. */
438 int inhibit_iso_escape_detection
;
440 /* Flag to make buffer-file-coding-system inherit from process-coding. */
441 int inherit_process_coding_system
;
443 /* Coding system to be used to encode text for terminal display when
444 terminal coding system is nil. */
445 struct coding_system safe_terminal_coding
;
447 /* Default coding system to be used to write a file. */
448 struct coding_system default_buffer_file_coding
;
450 Lisp_Object Vfile_coding_system_alist
;
451 Lisp_Object Vprocess_coding_system_alist
;
452 Lisp_Object Vnetwork_coding_system_alist
;
454 Lisp_Object Vlocale_coding_system
;
458 Lisp_Object Qcoding_category
, Qcoding_category_index
;
460 /* List of symbols `coding-category-xxx' ordered by priority. */
461 Lisp_Object Vcoding_category_list
;
463 /* Table of coding categories (Lisp symbols). */
464 Lisp_Object Vcoding_category_table
;
466 /* Table of names of symbol for each coding-category. */
467 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
468 "coding-category-emacs-mule",
469 "coding-category-sjis",
470 "coding-category-iso-7",
471 "coding-category-iso-7-tight",
472 "coding-category-iso-8-1",
473 "coding-category-iso-8-2",
474 "coding-category-iso-7-else",
475 "coding-category-iso-8-else",
476 "coding-category-ccl",
477 "coding-category-big5",
478 "coding-category-utf-8",
479 "coding-category-utf-16-be",
480 "coding-category-utf-16-le",
481 "coding-category-raw-text",
482 "coding-category-binary"
485 /* Table of pointers to coding systems corresponding to each coding
487 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
489 /* Table of coding category masks. Nth element is a mask for a coding
490 category of which priority is Nth. */
492 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
494 /* Flag to tell if we look up translation table on character code
496 Lisp_Object Venable_character_translation
;
497 /* Standard translation table to look up on decoding (reading). */
498 Lisp_Object Vstandard_translation_table_for_decode
;
499 /* Standard translation table to look up on encoding (writing). */
500 Lisp_Object Vstandard_translation_table_for_encode
;
502 Lisp_Object Qtranslation_table
;
503 Lisp_Object Qtranslation_table_id
;
504 Lisp_Object Qtranslation_table_for_decode
;
505 Lisp_Object Qtranslation_table_for_encode
;
507 /* Alist of charsets vs revision number. */
508 Lisp_Object Vcharset_revision_alist
;
510 /* Default coding systems used for process I/O. */
511 Lisp_Object Vdefault_process_coding_system
;
513 /* Char table for translating Quail and self-inserting input. */
514 Lisp_Object Vtranslation_table_for_input
;
516 /* Global flag to tell that we can't call post-read-conversion and
517 pre-write-conversion functions. Usually the value is zero, but it
518 is set to 1 temporarily while such functions are running. This is
519 to avoid infinite recursive call. */
520 static int inhibit_pre_post_conversion
;
522 Lisp_Object Qchar_coding_system
;
524 /* Return `safe-chars' property of CODING_SYSTEM (symbol). Don't check
528 coding_safe_chars (coding_system
)
529 Lisp_Object coding_system
;
531 Lisp_Object coding_spec
, plist
, safe_chars
;
533 coding_spec
= Fget (coding_system
, Qcoding_system
);
534 plist
= XVECTOR (coding_spec
)->contents
[3];
535 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
536 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
539 #define CODING_SAFE_CHAR_P(safe_chars, c) \
540 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
543 /*** 2. Emacs internal format (emacs-mule) handlers ***/
545 /* Emacs' internal format for representation of multiple character
546 sets is a kind of multi-byte encoding, i.e. characters are
547 represented by variable-length sequences of one-byte codes.
549 ASCII characters and control characters (e.g. `tab', `newline') are
550 represented by one-byte sequences which are their ASCII codes, in
551 the range 0x00 through 0x7F.
553 8-bit characters of the range 0x80..0x9F are represented by
554 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
557 8-bit characters of the range 0xA0..0xFF are represented by
558 one-byte sequences which are their 8-bit code.
560 The other characters are represented by a sequence of `base
561 leading-code', optional `extended leading-code', and one or two
562 `position-code's. The length of the sequence is determined by the
563 base leading-code. Leading-code takes the range 0x81 through 0x9D,
564 whereas extended leading-code and position-code take the range 0xA0
565 through 0xFF. See `charset.h' for more details about leading-code
568 --- CODE RANGE of Emacs' internal format ---
572 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
573 eight-bit-graphic 0xA0..0xBF
574 ELSE 0x81..0x9D + [0xA0..0xFF]+
575 ---------------------------------------------
577 As this is the internal character representation, the format is
578 usually not used externally (i.e. in a file or in a data sent to a
579 process). But, it is possible to have a text externally in this
580 format (i.e. by encoding by the coding system `emacs-mule').
582 In that case, a sequence of one-byte codes has a slightly different
585 Firstly, all characters in eight-bit-control are represented by
586 one-byte sequences which are their 8-bit code.
588 Next, character composition data are represented by the byte
589 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
591 METHOD is 0xF0 plus one of composition method (enum
594 BYTES is 0xA0 plus the byte length of these composition data,
596 CHARS is 0xA0 plus the number of characters composed by these
599 COMPONENTs are characters of multibyte form or composition
600 rules encoded by two-byte of ASCII codes.
602 In addition, for backward compatibility, the following formats are
603 also recognized as composition data on decoding.
606 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
609 MSEQ is a multibyte form but in these special format:
610 ASCII: 0xA0 ASCII_CODE+0x80,
611 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
612 RULE is a one byte code of the range 0xA0..0xF0 that
613 represents a composition rule.
616 enum emacs_code_class_type emacs_code_class
[256];
618 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
619 Check if a text is encoded in Emacs' internal format. If it is,
620 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
623 detect_coding_emacs_mule (src
, src_end
, multibytep
)
624 unsigned char *src
, *src_end
;
629 /* Dummy for ONE_MORE_BYTE. */
630 struct coding_system dummy_coding
;
631 struct coding_system
*coding
= &dummy_coding
;
635 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
,
636 CODING_CATEGORY_MASK_EMACS_MULE
);
643 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
652 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
655 else if (c
>= 0x80 && c
< 0xA0)
658 /* Old leading code for a composite character. */
662 unsigned char *src_base
= src
- 1;
665 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
668 src
= src_base
+ bytes
;
675 /* Record the starting position START and METHOD of one composition. */
677 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
679 struct composition_data *cmp_data = coding->cmp_data; \
680 int *data = cmp_data->data + cmp_data->used; \
681 coding->cmp_data_start = cmp_data->used; \
683 data[1] = cmp_data->char_offset + start; \
684 data[3] = (int) method; \
685 cmp_data->used += 4; \
688 /* Record the ending position END of the current composition. */
690 #define CODING_ADD_COMPOSITION_END(coding, end) \
692 struct composition_data *cmp_data = coding->cmp_data; \
693 int *data = cmp_data->data + coding->cmp_data_start; \
694 data[0] = cmp_data->used - coding->cmp_data_start; \
695 data[2] = cmp_data->char_offset + end; \
698 /* Record one COMPONENT (alternate character or composition rule). */
700 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
702 coding->cmp_data->data[coding->cmp_data->used++] = component; \
703 if (coding->cmp_data->used - coding->cmp_data_start \
704 == COMPOSITION_DATA_MAX_BUNCH_LENGTH) \
706 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
707 coding->composing = COMPOSITION_NO; \
712 /* Get one byte from a data pointed by SRC and increment SRC. If SRC
713 is not less than SRC_END, return -1 without incrementing Src. */
715 #define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
718 /* Decode a character represented as a component of composition
719 sequence of Emacs 20 style at SRC. Set C to that character, store
720 its multibyte form sequence at P, and set P to the end of that
721 sequence. If no valid character is found, set C to -1. */
723 #define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
727 c = SAFE_ONE_MORE_BYTE (); \
730 if (CHAR_HEAD_P (c)) \
732 else if (c == 0xA0) \
734 c = SAFE_ONE_MORE_BYTE (); \
743 else if (BASE_LEADING_CODE_P (c - 0x20)) \
745 unsigned char *p0 = p; \
749 bytes = BYTES_BY_CHAR_HEAD (c); \
752 c = SAFE_ONE_MORE_BYTE (); \
757 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes) \
758 || (coding->flags /* We are recovering a file. */ \
759 && p0[0] == LEADING_CODE_8_BIT_CONTROL \
760 && ! CHAR_HEAD_P (p0[1]))) \
761 c = STRING_CHAR (p0, bytes); \
770 /* Decode a composition rule represented as a component of composition
771 sequence of Emacs 20 style at SRC. Set C to the rule. If not
772 valid rule is found, set C to -1. */
774 #define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
776 c = SAFE_ONE_MORE_BYTE (); \
778 if (c < 0 || c >= 81) \
782 gref = c / 9, nref = c % 9; \
783 c = COMPOSITION_ENCODE_RULE (gref, nref); \
788 /* Decode composition sequence encoded by `emacs-mule' at the source
789 pointed by SRC. SRC_END is the end of source. Store information
790 of the composition in CODING->cmp_data.
792 For backward compatibility, decode also a composition sequence of
793 Emacs 20 style. In that case, the composition sequence contains
794 characters that should be extracted into a buffer or string. Store
795 those characters at *DESTINATION in multibyte form.
797 If we encounter an invalid byte sequence, return 0.
798 If we encounter an insufficient source or destination, or
799 insufficient space in CODING->cmp_data, return 1.
800 Otherwise, return consumed bytes in the source.
804 decode_composition_emacs_mule (coding
, src
, src_end
,
805 destination
, dst_end
, dst_bytes
)
806 struct coding_system
*coding
;
807 const unsigned char *src
, *src_end
;
808 unsigned char **destination
, *dst_end
;
811 unsigned char *dst
= *destination
;
812 int method
, data_len
, nchars
;
813 const unsigned char *src_base
= src
++;
814 /* Store components of composition. */
815 int component
[COMPOSITION_DATA_MAX_BUNCH_LENGTH
];
817 /* Store multibyte form of characters to be composed. This is for
818 Emacs 20 style composition sequence. */
819 unsigned char buf
[MAX_COMPOSITION_COMPONENTS
* MAX_MULTIBYTE_LENGTH
];
820 unsigned char *bufp
= buf
;
821 int c
, i
, gref
, nref
;
823 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
824 >= COMPOSITION_DATA_SIZE
)
826 coding
->result
= CODING_FINISH_INSUFFICIENT_CMP
;
831 if (c
- 0xF0 >= COMPOSITION_RELATIVE
832 && c
- 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS
)
837 with_rule
= (method
== COMPOSITION_WITH_RULE
838 || method
== COMPOSITION_WITH_RULE_ALTCHARS
);
842 || src_base
+ data_len
> src_end
)
848 for (ncomponent
= 0; src
< src_base
+ data_len
; ncomponent
++)
850 /* If it is longer than this, it can't be valid. */
851 if (ncomponent
>= COMPOSITION_DATA_MAX_BUNCH_LENGTH
)
854 if (ncomponent
% 2 && with_rule
)
856 ONE_MORE_BYTE (gref
);
858 ONE_MORE_BYTE (nref
);
860 c
= COMPOSITION_ENCODE_RULE (gref
, nref
);
865 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
866 || (coding
->flags
/* We are recovering a file. */
867 && src
[0] == LEADING_CODE_8_BIT_CONTROL
868 && ! CHAR_HEAD_P (src
[1])))
869 c
= STRING_CHAR (src
, bytes
);
874 component
[ncomponent
] = c
;
879 /* This may be an old Emacs 20 style format. See the comment at
880 the section 2 of this file. */
881 while (src
< src_end
&& !CHAR_HEAD_P (*src
)) src
++;
883 && !(coding
->mode
& CODING_MODE_LAST_BLOCK
))
884 goto label_end_of_loop
;
890 method
= COMPOSITION_RELATIVE
;
891 for (ncomponent
= 0; ncomponent
< MAX_COMPOSITION_COMPONENTS
;)
893 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
896 component
[ncomponent
++] = c
;
904 method
= COMPOSITION_WITH_RULE
;
906 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
911 ncomponent
< MAX_COMPOSITION_COMPONENTS
* 2 - 1;)
913 DECODE_EMACS_MULE_COMPOSITION_RULE (c
);
916 component
[ncomponent
++] = c
;
917 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
920 component
[ncomponent
++] = c
;
924 nchars
= (ncomponent
+ 1) / 2;
932 if (buf
== bufp
|| dst
+ (bufp
- buf
) <= (dst_bytes
? dst_end
: src
))
934 CODING_ADD_COMPOSITION_START (coding
, coding
->produced_char
, method
);
935 for (i
= 0; i
< ncomponent
; i
++)
936 CODING_ADD_COMPOSITION_COMPONENT (coding
, component
[i
]);
937 CODING_ADD_COMPOSITION_END (coding
, coding
->produced_char
+ nchars
);
940 unsigned char *p
= buf
;
941 EMIT_BYTES (p
, bufp
);
942 *destination
+= bufp
- buf
;
943 coding
->produced_char
+= nchars
;
945 return (src
- src_base
);
951 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
954 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
955 struct coding_system
*coding
;
956 const unsigned char *source
;
957 unsigned char *destination
;
958 int src_bytes
, dst_bytes
;
960 const unsigned char *src
= source
;
961 const unsigned char *src_end
= source
+ src_bytes
;
962 unsigned char *dst
= destination
;
963 unsigned char *dst_end
= destination
+ dst_bytes
;
964 /* SRC_BASE remembers the start position in source in each loop.
965 The loop will be exited when there's not enough source code, or
966 when there's not enough destination area to produce a
968 const unsigned char *src_base
;
970 coding
->produced_char
= 0;
971 while ((src_base
= src
) < src_end
)
973 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
];
974 const unsigned char *p
;
981 if (coding
->eol_type
== CODING_EOL_CR
)
983 else if (coding
->eol_type
== CODING_EOL_CRLF
)
993 coding
->produced_char
++;
996 else if (*src
== '\n')
998 if ((coding
->eol_type
== CODING_EOL_CR
999 || coding
->eol_type
== CODING_EOL_CRLF
)
1000 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1002 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1003 goto label_end_of_loop
;
1006 coding
->produced_char
++;
1009 else if (*src
== 0x80 && coding
->cmp_data
)
1011 /* Start of composition data. */
1012 int consumed
= decode_composition_emacs_mule (coding
, src
, src_end
,
1016 goto label_end_of_loop
;
1017 else if (consumed
> 0)
1022 bytes
= CHAR_STRING (*src
, tmp
);
1026 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
1027 || (coding
->flags
/* We are recovering a file. */
1028 && src
[0] == LEADING_CODE_8_BIT_CONTROL
1029 && ! CHAR_HEAD_P (src
[1])))
1038 bytes
= BYTES_BY_CHAR_HEAD (*src
);
1040 for (i
= 1; i
< bytes
; i
++)
1043 if (CHAR_HEAD_P (c
))
1048 bytes
= CHAR_STRING (*src_base
, tmp
);
1057 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
1059 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
1062 while (bytes
--) *dst
++ = *p
++;
1063 coding
->produced_char
++;
1066 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1067 coding
->produced
= dst
- destination
;
1071 /* Encode composition data stored at DATA into a special byte sequence
1072 starting by 0x80. Update CODING->cmp_data_start and maybe
1073 CODING->cmp_data for the next call. */
1075 #define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1077 unsigned char buf[1024], *p0 = buf, *p; \
1078 int len = data[0]; \
1082 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1083 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1085 if (data[3] == COMPOSITION_WITH_RULE \
1086 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1088 p += CHAR_STRING (data[4], p); \
1089 for (i = 5; i < len; i += 2) \
1092 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1093 *p++ = 0x20 + gref; \
1094 *p++ = 0x20 + nref; \
1095 p += CHAR_STRING (data[i + 1], p); \
1100 for (i = 4; i < len; i++) \
1101 p += CHAR_STRING (data[i], p); \
1103 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1105 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1107 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1108 goto label_end_of_loop; \
1112 coding->cmp_data_start += data[0]; \
1113 if (coding->cmp_data_start == coding->cmp_data->used \
1114 && coding->cmp_data->next) \
1116 coding->cmp_data = coding->cmp_data->next; \
1117 coding->cmp_data_start = 0; \
1122 static void encode_eol
P_ ((struct coding_system
*, const unsigned char *,
1123 unsigned char *, int, int));
1126 encode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
1127 struct coding_system
*coding
;
1128 const unsigned char *source
;
1129 unsigned char *destination
;
1130 int src_bytes
, dst_bytes
;
1132 const unsigned char *src
= source
;
1133 const unsigned char *src_end
= source
+ src_bytes
;
1134 unsigned char *dst
= destination
;
1135 unsigned char *dst_end
= destination
+ dst_bytes
;
1136 const unsigned char *src_base
;
1141 Lisp_Object translation_table
;
1143 translation_table
= Qnil
;
1145 /* Optimization for the case that there's no composition. */
1146 if (!coding
->cmp_data
|| coding
->cmp_data
->used
== 0)
1148 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
1152 char_offset
= coding
->cmp_data
->char_offset
;
1153 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1158 /* If SRC starts a composition, encode the information about the
1159 composition in advance. */
1160 if (coding
->cmp_data_start
< coding
->cmp_data
->used
1161 && char_offset
+ coding
->consumed_char
== data
[1])
1163 ENCODE_COMPOSITION_EMACS_MULE (coding
, data
);
1164 char_offset
= coding
->cmp_data
->char_offset
;
1165 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1169 if (c
== '\n' && (coding
->eol_type
== CODING_EOL_CRLF
1170 || coding
->eol_type
== CODING_EOL_CR
))
1172 if (coding
->eol_type
== CODING_EOL_CRLF
)
1173 EMIT_TWO_BYTES ('\r', c
);
1175 EMIT_ONE_BYTE ('\r');
1177 else if (SINGLE_BYTE_CHAR_P (c
))
1179 if (coding
->flags
&& ! ASCII_BYTE_P (c
))
1181 /* As we are auto saving, retain the multibyte form for
1183 unsigned char buf
[MAX_MULTIBYTE_LENGTH
];
1184 int bytes
= CHAR_STRING (c
, buf
);
1187 EMIT_ONE_BYTE (buf
[0]);
1189 EMIT_TWO_BYTES (buf
[0], buf
[1]);
1195 EMIT_BYTES (src_base
, src
);
1196 coding
->consumed_char
++;
1199 coding
->consumed
= src_base
- source
;
1200 coding
->produced
= coding
->produced_char
= dst
- destination
;
1205 /*** 3. ISO2022 handlers ***/
1207 /* The following note describes the coding system ISO2022 briefly.
1208 Since the intention of this note is to help understand the
1209 functions in this file, some parts are NOT ACCURATE or are OVERLY
1210 SIMPLIFIED. For thorough understanding, please refer to the
1211 original document of ISO2022. This is equivalent to the standard
1212 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1214 ISO2022 provides many mechanisms to encode several character sets
1215 in 7-bit and 8-bit environments. For 7-bit environments, all text
1216 is encoded using bytes less than 128. This may make the encoded
1217 text a little bit longer, but the text passes more easily through
1218 several types of gateway, some of which strip off the MSB (Most
1221 There are two kinds of character sets: control character sets and
1222 graphic character sets. The former contain control characters such
1223 as `newline' and `escape' to provide control functions (control
1224 functions are also provided by escape sequences). The latter
1225 contain graphic characters such as 'A' and '-'. Emacs recognizes
1226 two control character sets and many graphic character sets.
1228 Graphic character sets are classified into one of the following
1229 four classes, according to the number of bytes (DIMENSION) and
1230 number of characters in one dimension (CHARS) of the set:
1231 - DIMENSION1_CHARS94
1232 - DIMENSION1_CHARS96
1233 - DIMENSION2_CHARS94
1234 - DIMENSION2_CHARS96
1236 In addition, each character set is assigned an identification tag,
1237 unique for each set, called the "final character" (denoted as <F>
1238 hereafter). The <F> of each character set is decided by ECMA(*)
1239 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1240 (0x30..0x3F are for private use only).
1242 Note (*): ECMA = European Computer Manufacturers Association
1244 Here are examples of graphic character sets [NAME(<F>)]:
1245 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1246 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1247 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1248 o DIMENSION2_CHARS96 -- none for the moment
1250 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
1251 C0 [0x00..0x1F] -- control character plane 0
1252 GL [0x20..0x7F] -- graphic character plane 0
1253 C1 [0x80..0x9F] -- control character plane 1
1254 GR [0xA0..0xFF] -- graphic character plane 1
1256 A control character set is directly designated and invoked to C0 or
1257 C1 by an escape sequence. The most common case is that:
1258 - ISO646's control character set is designated/invoked to C0, and
1259 - ISO6429's control character set is designated/invoked to C1,
1260 and usually these designations/invocations are omitted in encoded
1261 text. In a 7-bit environment, only C0 can be used, and a control
1262 character for C1 is encoded by an appropriate escape sequence to
1263 fit into the environment. All control characters for C1 are
1264 defined to have corresponding escape sequences.
1266 A graphic character set is at first designated to one of four
1267 graphic registers (G0 through G3), then these graphic registers are
1268 invoked to GL or GR. These designations and invocations can be
1269 done independently. The most common case is that G0 is invoked to
1270 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1271 these invocations and designations are omitted in encoded text.
1272 In a 7-bit environment, only GL can be used.
1274 When a graphic character set of CHARS94 is invoked to GL, codes
1275 0x20 and 0x7F of the GL area work as control characters SPACE and
1276 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1279 There are two ways of invocation: locking-shift and single-shift.
1280 With locking-shift, the invocation lasts until the next different
1281 invocation, whereas with single-shift, the invocation affects the
1282 following character only and doesn't affect the locking-shift
1283 state. Invocations are done by the following control characters or
1286 ----------------------------------------------------------------------
1287 abbrev function cntrl escape seq description
1288 ----------------------------------------------------------------------
1289 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1290 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1291 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1292 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1293 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1294 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1295 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1296 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1297 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
1298 ----------------------------------------------------------------------
1299 (*) These are not used by any known coding system.
1301 Control characters for these functions are defined by macros
1302 ISO_CODE_XXX in `coding.h'.
1304 Designations are done by the following escape sequences:
1305 ----------------------------------------------------------------------
1306 escape sequence description
1307 ----------------------------------------------------------------------
1308 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1309 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1310 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1311 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1312 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1313 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1314 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1315 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1316 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1317 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1318 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1319 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1320 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1321 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1322 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1323 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1324 ----------------------------------------------------------------------
1326 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
1327 of dimension 1, chars 94, and final character <F>, etc...
1329 Note (*): Although these designations are not allowed in ISO2022,
1330 Emacs accepts them on decoding, and produces them on encoding
1331 CHARS96 character sets in a coding system which is characterized as
1332 7-bit environment, non-locking-shift, and non-single-shift.
1334 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1335 '(' can be omitted. We refer to this as "short-form" hereafter.
1337 Now you may notice that there are a lot of ways of encoding the
1338 same multilingual text in ISO2022. Actually, there exist many
1339 coding systems such as Compound Text (used in X11's inter client
1340 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1341 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
1342 localized platforms), and all of these are variants of ISO2022.
1344 In addition to the above, Emacs handles two more kinds of escape
1345 sequences: ISO6429's direction specification and Emacs' private
1346 sequence for specifying character composition.
1348 ISO6429's direction specification takes the following form:
1349 o CSI ']' -- end of the current direction
1350 o CSI '0' ']' -- end of the current direction
1351 o CSI '1' ']' -- start of left-to-right text
1352 o CSI '2' ']' -- start of right-to-left text
1353 The control character CSI (0x9B: control sequence introducer) is
1354 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1356 Character composition specification takes the following form:
1357 o ESC '0' -- start relative composition
1358 o ESC '1' -- end composition
1359 o ESC '2' -- start rule-base composition (*)
1360 o ESC '3' -- start relative composition with alternate chars (**)
1361 o ESC '4' -- start rule-base composition with alternate chars (**)
1362 Since these are not standard escape sequences of any ISO standard,
1363 the use of them with these meanings is restricted to Emacs only.
1365 (*) This form is used only in Emacs 20.5 and older versions,
1366 but the newer versions can safely decode it.
1367 (**) This form is used only in Emacs 21.1 and newer versions,
1368 and the older versions can't decode it.
1370 Here's a list of example usages of these composition escape
1371 sequences (categorized by `enum composition_method').
1373 COMPOSITION_RELATIVE:
1374 ESC 0 CHAR [ CHAR ] ESC 1
1375 COMPOSITION_WITH_RULE:
1376 ESC 2 CHAR [ RULE CHAR ] ESC 1
1377 COMPOSITION_WITH_ALTCHARS:
1378 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
1379 COMPOSITION_WITH_RULE_ALTCHARS:
1380 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
1382 enum iso_code_class_type iso_code_class
[256];
1384 #define CHARSET_OK(idx, charset, c) \
1385 (coding_system_table[idx] \
1386 && (charset == CHARSET_ASCII \
1387 || (safe_chars = coding_safe_chars (coding_system_table[idx]->symbol), \
1388 CODING_SAFE_CHAR_P (safe_chars, c))) \
1389 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1391 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
1393 #define SHIFT_OUT_OK(idx) \
1394 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1396 #define COMPOSITION_OK(idx) \
1397 (coding_system_table[idx]->composing != COMPOSITION_DISABLED)
1399 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1400 Check if a text is encoded in ISO2022. If it is, return an
1401 integer in which appropriate flag bits any of:
1402 CODING_CATEGORY_MASK_ISO_7
1403 CODING_CATEGORY_MASK_ISO_7_TIGHT
1404 CODING_CATEGORY_MASK_ISO_8_1
1405 CODING_CATEGORY_MASK_ISO_8_2
1406 CODING_CATEGORY_MASK_ISO_7_ELSE
1407 CODING_CATEGORY_MASK_ISO_8_ELSE
1408 are set. If a code which should never appear in ISO2022 is found,
1411 If *latin_extra_code_state is zero and Latin extra codes are found,
1412 set *latin_extra_code_state to 1 and return 0. If it is nonzero,
1413 accept Latin extra codes. */
1416 detect_coding_iso2022 (src
, src_end
, multibytep
, latin_extra_code_state
)
1417 unsigned char *src
, *src_end
;
1419 int *latin_extra_code_state
;
1421 int mask
= CODING_CATEGORY_MASK_ISO
;
1423 int reg
[4], shift_out
= 0, single_shifting
= 0;
1425 /* Dummy for ONE_MORE_BYTE. */
1426 struct coding_system dummy_coding
;
1427 struct coding_system
*coding
= &dummy_coding
;
1428 Lisp_Object safe_chars
;
1430 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
1433 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, mask
& mask_found
);
1438 if (inhibit_iso_escape_detection
)
1440 single_shifting
= 0;
1441 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, mask
& mask_found
);
1442 if (c
>= '(' && c
<= '/')
1444 /* Designation sequence for a charset of dimension 1. */
1445 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
, mask
& mask_found
);
1446 if (c1
< ' ' || c1
>= 0x80
1447 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
1448 /* Invalid designation sequence. Just ignore. */
1450 reg
[(c
- '(') % 4] = charset
;
1454 /* Designation sequence for a charset of dimension 2. */
1455 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, mask
& mask_found
);
1456 if (c
>= '@' && c
<= 'B')
1457 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1458 reg
[0] = charset
= iso_charset_table
[1][0][c
];
1459 else if (c
>= '(' && c
<= '/')
1461 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
,
1463 if (c1
< ' ' || c1
>= 0x80
1464 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
1465 /* Invalid designation sequence. Just ignore. */
1467 reg
[(c
- '(') % 4] = charset
;
1470 /* Invalid designation sequence. Just ignore. */
1473 else if (c
== 'N' || c
== 'O')
1475 /* ESC <Fe> for SS2 or SS3. */
1476 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1479 else if (c
>= '0' && c
<= '4')
1481 /* ESC <Fp> for start/end composition. */
1482 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7
))
1483 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1485 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1486 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
))
1487 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1489 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1490 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_1
))
1491 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1493 mask
&= ~CODING_CATEGORY_MASK_ISO_8_1
;
1494 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_2
))
1495 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1497 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1498 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
))
1499 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1501 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1502 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
))
1503 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1505 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1509 /* Invalid escape sequence. Just ignore. */
1512 /* We found a valid designation sequence for CHARSET. */
1513 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
1514 c
= MAKE_CHAR (charset
, 0, 0);
1515 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
1516 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1518 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1519 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
1520 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1522 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1523 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
1524 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1526 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1527 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
1528 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1530 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1534 if (inhibit_iso_escape_detection
)
1536 single_shifting
= 0;
1539 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
1540 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
1542 /* Locking shift out. */
1543 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1544 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1549 if (inhibit_iso_escape_detection
)
1551 single_shifting
= 0;
1554 /* Locking shift in. */
1555 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1556 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1561 single_shifting
= 0;
1565 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1567 if (inhibit_iso_escape_detection
)
1569 if (c
!= ISO_CODE_CSI
)
1571 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1572 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1573 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1574 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1575 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1576 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1577 single_shifting
= 1;
1579 if (VECTORP (Vlatin_extra_code_table
)
1580 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1582 if (! *latin_extra_code_state
)
1584 *latin_extra_code_state
= 1;
1587 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1588 & CODING_FLAG_ISO_LATIN_EXTRA
)
1589 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1590 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1591 & CODING_FLAG_ISO_LATIN_EXTRA
)
1592 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1595 mask_found
|= newmask
;
1602 single_shifting
= 0;
1607 single_shifting
= 0;
1608 if (VECTORP (Vlatin_extra_code_table
)
1609 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1613 if (! *latin_extra_code_state
)
1615 *latin_extra_code_state
= 1;
1618 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1619 & CODING_FLAG_ISO_LATIN_EXTRA
)
1620 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1621 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1622 & CODING_FLAG_ISO_LATIN_EXTRA
)
1623 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1625 mask_found
|= newmask
;
1632 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1633 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1634 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1635 /* Check the length of succeeding codes of the range
1636 0xA0..0FF. If the byte length is odd, we exclude
1637 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1638 when we are not single shifting. */
1639 if (!single_shifting
1640 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1645 while (src
< src_end
)
1647 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
,
1654 if (i
& 1 && src
< src_end
)
1655 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1657 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1659 /* This means that we have read one extra byte. */
1666 return (mask
& mask_found
);
1669 /* Decode a character of which charset is CHARSET, the 1st position
1670 code is C1, the 2nd position code is C2, and return the decoded
1671 character code. If the variable `translation_table' is non-nil,
1672 returned the translated code. */
1674 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1675 (NILP (translation_table) \
1676 ? MAKE_CHAR (charset, c1, c2) \
1677 : translate_char (translation_table, -1, charset, c1, c2))
1679 /* Set designation state into CODING. */
1680 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1684 if (final_char < '0' || final_char >= 128) \
1685 goto label_invalid_code; \
1686 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1687 make_number (chars), \
1688 make_number (final_char)); \
1689 c = MAKE_CHAR (charset, 0, 0); \
1691 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1692 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1694 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1696 && charset == CHARSET_ASCII) \
1698 /* We should insert this designation sequence as is so \
1699 that it is surely written back to a file. */ \
1700 coding->spec.iso2022.last_invalid_designation_register = -1; \
1701 goto label_invalid_code; \
1703 coding->spec.iso2022.last_invalid_designation_register = -1; \
1704 if ((coding->mode & CODING_MODE_DIRECTION) \
1705 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1706 charset = CHARSET_REVERSE_CHARSET (charset); \
1707 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1711 coding->spec.iso2022.last_invalid_designation_register = reg; \
1712 goto label_invalid_code; \
1716 /* Allocate a memory block for storing information about compositions.
1717 The block is chained to the already allocated blocks. */
1720 coding_allocate_composition_data (coding
, char_offset
)
1721 struct coding_system
*coding
;
1724 struct composition_data
*cmp_data
1725 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1727 cmp_data
->char_offset
= char_offset
;
1729 cmp_data
->prev
= coding
->cmp_data
;
1730 cmp_data
->next
= NULL
;
1731 if (coding
->cmp_data
)
1732 coding
->cmp_data
->next
= cmp_data
;
1733 coding
->cmp_data
= cmp_data
;
1734 coding
->cmp_data_start
= 0;
1735 coding
->composing
= COMPOSITION_NO
;
1738 /* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1739 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1740 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1741 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1742 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1745 #define DECODE_COMPOSITION_START(c1) \
1747 if (coding->composing == COMPOSITION_DISABLED) \
1749 *dst++ = ISO_CODE_ESC; \
1750 *dst++ = c1 & 0x7f; \
1751 coding->produced_char += 2; \
1753 else if (!COMPOSING_P (coding)) \
1755 /* This is surely the start of a composition. We must be sure \
1756 that coding->cmp_data has enough space to store the \
1757 information about the composition. If not, terminate the \
1758 current decoding loop, allocate one more memory block for \
1759 coding->cmp_data in the caller, then start the decoding \
1760 loop again. We can't allocate memory here directly because \
1761 it may cause buffer/string relocation. */ \
1762 if (!coding->cmp_data \
1763 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1764 >= COMPOSITION_DATA_SIZE)) \
1766 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1767 goto label_end_of_loop; \
1769 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1770 : c1 == '2' ? COMPOSITION_WITH_RULE \
1771 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1772 : COMPOSITION_WITH_RULE_ALTCHARS); \
1773 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1774 coding->composing); \
1775 coding->composition_rule_follows = 0; \
1779 /* We are already handling a composition. If the method is \
1780 the following two, the codes following the current escape \
1781 sequence are actual characters stored in a buffer. */ \
1782 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1783 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1785 coding->composing = COMPOSITION_RELATIVE; \
1786 coding->composition_rule_follows = 0; \
1791 /* Handle composition end sequence ESC 1. */
1793 #define DECODE_COMPOSITION_END(c1) \
1795 if (! COMPOSING_P (coding)) \
1797 *dst++ = ISO_CODE_ESC; \
1799 coding->produced_char += 2; \
1803 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1804 coding->composing = COMPOSITION_NO; \
1808 /* Decode a composition rule from the byte C1 (and maybe one more byte
1809 from SRC) and store one encoded composition rule in
1810 coding->cmp_data. */
1812 #define DECODE_COMPOSITION_RULE(c1) \
1816 if (c1 < 81) /* old format (before ver.21) */ \
1818 int gref = (c1) / 9; \
1819 int nref = (c1) % 9; \
1820 if (gref == 4) gref = 10; \
1821 if (nref == 4) nref = 10; \
1822 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1824 else if (c1 < 93) /* new format (after ver.21) */ \
1826 ONE_MORE_BYTE (c2); \
1827 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1829 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1830 coding->composition_rule_follows = 0; \
1834 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1837 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1838 struct coding_system
*coding
;
1839 const unsigned char *source
;
1840 unsigned char *destination
;
1841 int src_bytes
, dst_bytes
;
1843 const unsigned char *src
= source
;
1844 const unsigned char *src_end
= source
+ src_bytes
;
1845 unsigned char *dst
= destination
;
1846 unsigned char *dst_end
= destination
+ dst_bytes
;
1847 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1848 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1849 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1850 /* SRC_BASE remembers the start position in source in each loop.
1851 The loop will be exited when there's not enough source code
1852 (within macro ONE_MORE_BYTE), or when there's not enough
1853 destination area to produce a character (within macro
1855 const unsigned char *src_base
;
1857 Lisp_Object translation_table
;
1858 Lisp_Object safe_chars
;
1860 safe_chars
= coding_safe_chars (coding
->symbol
);
1862 if (NILP (Venable_character_translation
))
1863 translation_table
= Qnil
;
1866 translation_table
= coding
->translation_table_for_decode
;
1867 if (NILP (translation_table
))
1868 translation_table
= Vstandard_translation_table_for_decode
;
1871 coding
->result
= CODING_FINISH_NORMAL
;
1880 /* We produce no character or one character. */
1881 switch (iso_code_class
[c1
])
1883 case ISO_0x20_or_0x7F
:
1884 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1886 DECODE_COMPOSITION_RULE (c1
);
1889 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1891 /* This is SPACE or DEL. */
1892 charset
= CHARSET_ASCII
;
1895 /* This is a graphic character, we fall down ... */
1897 case ISO_graphic_plane_0
:
1898 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1900 DECODE_COMPOSITION_RULE (c1
);
1906 case ISO_0xA0_or_0xFF
:
1907 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1908 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1909 goto label_invalid_code
;
1910 /* This is a graphic character, we fall down ... */
1912 case ISO_graphic_plane_1
:
1914 goto label_invalid_code
;
1919 if (COMPOSING_P (coding
))
1920 DECODE_COMPOSITION_END ('1');
1922 /* All ISO2022 control characters in this class have the
1923 same representation in Emacs internal format. */
1925 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1926 && (coding
->eol_type
== CODING_EOL_CR
1927 || coding
->eol_type
== CODING_EOL_CRLF
))
1929 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1930 goto label_end_of_loop
;
1932 charset
= CHARSET_ASCII
;
1936 if (COMPOSING_P (coding
))
1937 DECODE_COMPOSITION_END ('1');
1938 goto label_invalid_code
;
1940 case ISO_carriage_return
:
1941 if (COMPOSING_P (coding
))
1942 DECODE_COMPOSITION_END ('1');
1944 if (coding
->eol_type
== CODING_EOL_CR
)
1946 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1949 if (c1
!= ISO_CODE_LF
)
1955 charset
= CHARSET_ASCII
;
1959 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1960 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1961 goto label_invalid_code
;
1962 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1963 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1967 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1968 goto label_invalid_code
;
1969 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1970 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1973 case ISO_single_shift_2_7
:
1974 case ISO_single_shift_2
:
1975 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1976 goto label_invalid_code
;
1977 /* SS2 is handled as an escape sequence of ESC 'N' */
1979 goto label_escape_sequence
;
1981 case ISO_single_shift_3
:
1982 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1983 goto label_invalid_code
;
1984 /* SS2 is handled as an escape sequence of ESC 'O' */
1986 goto label_escape_sequence
;
1988 case ISO_control_sequence_introducer
:
1989 /* CSI is handled as an escape sequence of ESC '[' ... */
1991 goto label_escape_sequence
;
1995 label_escape_sequence
:
1996 /* Escape sequences handled by Emacs are invocation,
1997 designation, direction specification, and character
1998 composition specification. */
2001 case '&': /* revision of following character set */
2003 if (!(c1
>= '@' && c1
<= '~'))
2004 goto label_invalid_code
;
2006 if (c1
!= ISO_CODE_ESC
)
2007 goto label_invalid_code
;
2009 goto label_escape_sequence
;
2011 case '$': /* designation of 2-byte character set */
2012 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
2013 goto label_invalid_code
;
2015 if (c1
>= '@' && c1
<= 'B')
2016 { /* designation of JISX0208.1978, GB2312.1980,
2018 DECODE_DESIGNATION (0, 2, 94, c1
);
2020 else if (c1
>= 0x28 && c1
<= 0x2B)
2021 { /* designation of DIMENSION2_CHARS94 character set */
2023 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
2025 else if (c1
>= 0x2C && c1
<= 0x2F)
2026 { /* designation of DIMENSION2_CHARS96 character set */
2028 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
2031 goto label_invalid_code
;
2032 /* We must update these variables now. */
2033 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2034 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2037 case 'n': /* invocation of locking-shift-2 */
2038 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2039 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2040 goto label_invalid_code
;
2041 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
2042 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2045 case 'o': /* invocation of locking-shift-3 */
2046 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2047 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2048 goto label_invalid_code
;
2049 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
2050 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2053 case 'N': /* invocation of single-shift-2 */
2054 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2055 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2056 goto label_invalid_code
;
2057 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
2059 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2060 goto label_invalid_code
;
2063 case 'O': /* invocation of single-shift-3 */
2064 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2065 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2066 goto label_invalid_code
;
2067 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
2069 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2070 goto label_invalid_code
;
2073 case '0': case '2': case '3': case '4': /* start composition */
2074 DECODE_COMPOSITION_START (c1
);
2077 case '1': /* end composition */
2078 DECODE_COMPOSITION_END (c1
);
2081 case '[': /* specification of direction */
2082 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
2083 goto label_invalid_code
;
2084 /* For the moment, nested direction is not supported.
2085 So, `coding->mode & CODING_MODE_DIRECTION' zero means
2086 left-to-right, and nonzero means right-to-left. */
2090 case ']': /* end of the current direction */
2091 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2093 case '0': /* end of the current direction */
2094 case '1': /* start of left-to-right direction */
2097 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2099 goto label_invalid_code
;
2102 case '2': /* start of right-to-left direction */
2105 coding
->mode
|= CODING_MODE_DIRECTION
;
2107 goto label_invalid_code
;
2111 goto label_invalid_code
;
2116 if (COMPOSING_P (coding
))
2117 DECODE_COMPOSITION_END ('1');
2121 /* CTEXT extended segment:
2122 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
2123 We keep these bytes as is for the moment.
2124 They may be decoded by post-read-conversion. */
2129 ONE_MORE_BYTE (dim
);
2132 size
= ((M
- 128) * 128) + (L
- 128);
2133 required
= 8 + size
* 2;
2134 if (dst
+ required
> (dst_bytes
? dst_end
: src
))
2135 goto label_end_of_loop
;
2136 *dst
++ = ISO_CODE_ESC
;
2141 dst
+= CHAR_STRING (M
, dst
), produced_chars
++;
2142 dst
+= CHAR_STRING (L
, dst
), produced_chars
++;
2146 dst
+= CHAR_STRING (c1
, dst
), produced_chars
++;
2148 coding
->produced_char
+= produced_chars
;
2152 unsigned char *d
= dst
;
2155 /* XFree86 extension for embedding UTF-8 in CTEXT:
2156 ESC % G --UTF-8-BYTES-- ESC % @
2157 We keep these bytes as is for the moment.
2158 They may be decoded by post-read-conversion. */
2159 if (d
+ 6 > (dst_bytes
? dst_end
: src
))
2160 goto label_end_of_loop
;
2161 *d
++ = ISO_CODE_ESC
;
2165 while (d
+ 1 < (dst_bytes
? dst_end
: src
))
2168 if (c1
== ISO_CODE_ESC
2169 && src
+ 1 < src_end
2176 d
+= CHAR_STRING (c1
, d
), produced_chars
++;
2178 if (d
+ 3 > (dst_bytes
? dst_end
: src
))
2179 goto label_end_of_loop
;
2180 *d
++ = ISO_CODE_ESC
;
2184 coding
->produced_char
+= produced_chars
+ 3;
2187 goto label_invalid_code
;
2191 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
2192 goto label_invalid_code
;
2193 if (c1
>= 0x28 && c1
<= 0x2B)
2194 { /* designation of DIMENSION1_CHARS94 character set */
2196 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
2198 else if (c1
>= 0x2C && c1
<= 0x2F)
2199 { /* designation of DIMENSION1_CHARS96 character set */
2201 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
2204 goto label_invalid_code
;
2205 /* We must update these variables now. */
2206 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2207 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2212 /* Now we know CHARSET and 1st position code C1 of a character.
2213 Produce a multibyte sequence for that character while getting
2214 2nd position code C2 if necessary. */
2215 if (CHARSET_DIMENSION (charset
) == 2)
2218 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
2219 /* C2 is not in a valid range. */
2220 goto label_invalid_code
;
2222 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2228 if (COMPOSING_P (coding
))
2229 DECODE_COMPOSITION_END ('1');
2232 if (! NILP (translation_table
))
2233 c
= translate_char (translation_table
, c
, 0, 0, 0);
2238 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2239 coding
->produced
= dst
- destination
;
2244 /* ISO2022 encoding stuff. */
2247 It is not enough to say just "ISO2022" on encoding, we have to
2248 specify more details. In Emacs, each ISO2022 coding system
2249 variant has the following specifications:
2250 1. Initial designation to G0 through G3.
2251 2. Allows short-form designation?
2252 3. ASCII should be designated to G0 before control characters?
2253 4. ASCII should be designated to G0 at end of line?
2254 5. 7-bit environment or 8-bit environment?
2255 6. Use locking-shift?
2256 7. Use Single-shift?
2257 And the following two are only for Japanese:
2258 8. Use ASCII in place of JIS0201-1976-Roman?
2259 9. Use JISX0208-1983 in place of JISX0208-1978?
2260 These specifications are encoded in `coding->flags' as flag bits
2261 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
2265 /* Produce codes (escape sequence) for designating CHARSET to graphic
2266 register REG at DST, and increment DST. If <final-char> of CHARSET is
2267 '@', 'A', or 'B' and the coding system CODING allows, produce
2268 designation sequence of short-form. */
2270 #define ENCODE_DESIGNATION(charset, reg, coding) \
2272 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2273 char *intermediate_char_94 = "()*+"; \
2274 char *intermediate_char_96 = ",-./"; \
2275 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
2277 if (revision < 255) \
2279 *dst++ = ISO_CODE_ESC; \
2281 *dst++ = '@' + revision; \
2283 *dst++ = ISO_CODE_ESC; \
2284 if (CHARSET_DIMENSION (charset) == 1) \
2286 if (CHARSET_CHARS (charset) == 94) \
2287 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2289 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2294 if (CHARSET_CHARS (charset) == 94) \
2296 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2298 || final_char < '@' || final_char > 'B') \
2299 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2302 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2304 *dst++ = final_char; \
2305 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2308 /* The following two macros produce codes (control character or escape
2309 sequence) for ISO2022 single-shift functions (single-shift-2 and
2312 #define ENCODE_SINGLE_SHIFT_2 \
2314 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2315 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2317 *dst++ = ISO_CODE_SS2; \
2318 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2321 #define ENCODE_SINGLE_SHIFT_3 \
2323 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2324 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2326 *dst++ = ISO_CODE_SS3; \
2327 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2330 /* The following four macros produce codes (control character or
2331 escape sequence) for ISO2022 locking-shift functions (shift-in,
2332 shift-out, locking-shift-2, and locking-shift-3). */
2334 #define ENCODE_SHIFT_IN \
2336 *dst++ = ISO_CODE_SI; \
2337 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2340 #define ENCODE_SHIFT_OUT \
2342 *dst++ = ISO_CODE_SO; \
2343 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2346 #define ENCODE_LOCKING_SHIFT_2 \
2348 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2349 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2352 #define ENCODE_LOCKING_SHIFT_3 \
2354 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
2355 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2358 /* Produce codes for a DIMENSION1 character whose character set is
2359 CHARSET and whose position-code is C1. Designation and invocation
2360 sequences are also produced in advance if necessary. */
2362 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2364 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2366 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2367 *dst++ = c1 & 0x7F; \
2369 *dst++ = c1 | 0x80; \
2370 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2373 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2375 *dst++ = c1 & 0x7F; \
2378 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2380 *dst++ = c1 | 0x80; \
2384 /* Since CHARSET is not yet invoked to any graphic planes, we \
2385 must invoke it, or, at first, designate it to some graphic \
2386 register. Then repeat the loop to actually produce the \
2388 dst = encode_invocation_designation (charset, coding, dst); \
2391 /* Produce codes for a DIMENSION2 character whose character set is
2392 CHARSET and whose position-codes are C1 and C2. Designation and
2393 invocation codes are also produced in advance if necessary. */
2395 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2397 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2399 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2400 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2402 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2403 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2406 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2408 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2411 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2413 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2417 /* Since CHARSET is not yet invoked to any graphic planes, we \
2418 must invoke it, or, at first, designate it to some graphic \
2419 register. Then repeat the loop to actually produce the \
2421 dst = encode_invocation_designation (charset, coding, dst); \
2424 #define ENCODE_ISO_CHARACTER(c) \
2426 int charset, c1, c2; \
2428 SPLIT_CHAR (c, charset, c1, c2); \
2429 if (CHARSET_DEFINED_P (charset)) \
2431 if (CHARSET_DIMENSION (charset) == 1) \
2433 if (charset == CHARSET_ASCII \
2434 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2435 charset = charset_latin_jisx0201; \
2436 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2440 if (charset == charset_jisx0208 \
2441 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2442 charset = charset_jisx0208_1978; \
2443 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2455 /* Instead of encoding character C, produce one or two `?'s. */
2457 #define ENCODE_UNSAFE_CHARACTER(c) \
2459 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2460 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2461 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2465 /* Produce designation and invocation codes at a place pointed by DST
2466 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2470 encode_invocation_designation (charset
, coding
, dst
)
2472 struct coding_system
*coding
;
2475 int reg
; /* graphic register number */
2477 /* At first, check designations. */
2478 for (reg
= 0; reg
< 4; reg
++)
2479 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
2484 /* CHARSET is not yet designated to any graphic registers. */
2485 /* At first check the requested designation. */
2486 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2487 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
2488 /* Since CHARSET requests no special designation, designate it
2489 to graphic register 0. */
2492 ENCODE_DESIGNATION (charset
, reg
, coding
);
2495 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
2496 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
2498 /* Since the graphic register REG is not invoked to any graphic
2499 planes, invoke it to graphic plane 0. */
2502 case 0: /* graphic register 0 */
2506 case 1: /* graphic register 1 */
2510 case 2: /* graphic register 2 */
2511 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2512 ENCODE_SINGLE_SHIFT_2
;
2514 ENCODE_LOCKING_SHIFT_2
;
2517 case 3: /* graphic register 3 */
2518 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2519 ENCODE_SINGLE_SHIFT_3
;
2521 ENCODE_LOCKING_SHIFT_3
;
2529 /* Produce 2-byte codes for encoded composition rule RULE. */
2531 #define ENCODE_COMPOSITION_RULE(rule) \
2534 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2535 *dst++ = 32 + 81 + gref; \
2536 *dst++ = 32 + nref; \
2539 /* Produce codes for indicating the start of a composition sequence
2540 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2541 which specify information about the composition. See the comment
2542 in coding.h for the format of DATA. */
2544 #define ENCODE_COMPOSITION_START(coding, data) \
2546 coding->composing = data[3]; \
2547 *dst++ = ISO_CODE_ESC; \
2548 if (coding->composing == COMPOSITION_RELATIVE) \
2552 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2554 coding->cmp_data_index = coding->cmp_data_start + 4; \
2555 coding->composition_rule_follows = 0; \
2559 /* Produce codes for indicating the end of the current composition. */
2561 #define ENCODE_COMPOSITION_END(coding, data) \
2563 *dst++ = ISO_CODE_ESC; \
2565 coding->cmp_data_start += data[0]; \
2566 coding->composing = COMPOSITION_NO; \
2567 if (coding->cmp_data_start == coding->cmp_data->used \
2568 && coding->cmp_data->next) \
2570 coding->cmp_data = coding->cmp_data->next; \
2571 coding->cmp_data_start = 0; \
2575 /* Produce composition start sequence ESC 0. Here, this sequence
2576 doesn't mean the start of a new composition but means that we have
2577 just produced components (alternate chars and composition rules) of
2578 the composition and the actual text follows in SRC. */
2580 #define ENCODE_COMPOSITION_FAKE_START(coding) \
2582 *dst++ = ISO_CODE_ESC; \
2584 coding->composing = COMPOSITION_RELATIVE; \
2587 /* The following three macros produce codes for indicating direction
2589 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2591 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2592 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2594 *dst++ = ISO_CODE_CSI; \
2597 #define ENCODE_DIRECTION_R2L \
2598 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2600 #define ENCODE_DIRECTION_L2R \
2601 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2603 /* Produce codes for designation and invocation to reset the graphic
2604 planes and registers to initial state. */
2605 #define ENCODE_RESET_PLANE_AND_REGISTER \
2608 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2610 for (reg = 0; reg < 4; reg++) \
2611 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2612 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2613 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2614 ENCODE_DESIGNATION \
2615 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
2618 /* Produce designation sequences of charsets in the line started from
2619 SRC to a place pointed by DST, and return updated DST.
2621 If the current block ends before any end-of-line, we may fail to
2622 find all the necessary designations. */
2624 static unsigned char *
2625 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
2626 struct coding_system
*coding
;
2627 Lisp_Object translation_table
;
2628 const unsigned char *src
, *src_end
;
2631 int charset
, c
, found
= 0, reg
;
2632 /* Table of charsets to be designated to each graphic register. */
2635 for (reg
= 0; reg
< 4; reg
++)
2644 charset
= CHAR_CHARSET (c
);
2645 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2646 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
2656 for (reg
= 0; reg
< 4; reg
++)
2658 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
2659 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2665 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2668 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2669 struct coding_system
*coding
;
2670 const unsigned char *source
;
2671 unsigned char *destination
;
2672 int src_bytes
, dst_bytes
;
2674 const unsigned char *src
= source
;
2675 const unsigned char *src_end
= source
+ src_bytes
;
2676 unsigned char *dst
= destination
;
2677 unsigned char *dst_end
= destination
+ dst_bytes
;
2678 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2679 from DST_END to assure overflow checking is necessary only at the
2681 unsigned char *adjusted_dst_end
= dst_end
- 19;
2682 /* SRC_BASE remembers the start position in source in each loop.
2683 The loop will be exited when there's not enough source text to
2684 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2685 there's not enough destination area to produce encoded codes
2686 (within macro EMIT_BYTES). */
2687 const unsigned char *src_base
;
2689 Lisp_Object translation_table
;
2690 Lisp_Object safe_chars
;
2692 if (coding
->flags
& CODING_FLAG_ISO_SAFE
)
2693 coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
2695 safe_chars
= coding_safe_chars (coding
->symbol
);
2697 if (NILP (Venable_character_translation
))
2698 translation_table
= Qnil
;
2701 translation_table
= coding
->translation_table_for_encode
;
2702 if (NILP (translation_table
))
2703 translation_table
= Vstandard_translation_table_for_encode
;
2706 coding
->consumed_char
= 0;
2712 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2714 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2718 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2719 && CODING_SPEC_ISO_BOL (coding
))
2721 /* We have to produce designation sequences if any now. */
2722 dst
= encode_designation_at_bol (coding
, translation_table
,
2724 CODING_SPEC_ISO_BOL (coding
) = 0;
2727 /* Check composition start and end. */
2728 if (coding
->composing
!= COMPOSITION_DISABLED
2729 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2731 struct composition_data
*cmp_data
= coding
->cmp_data
;
2732 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2733 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2735 if (coding
->composing
== COMPOSITION_RELATIVE
)
2737 if (this_pos
== data
[2])
2739 ENCODE_COMPOSITION_END (coding
, data
);
2740 cmp_data
= coding
->cmp_data
;
2741 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2744 else if (COMPOSING_P (coding
))
2746 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2747 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2748 /* We have consumed components of the composition.
2749 What follows in SRC is the composition's base
2751 ENCODE_COMPOSITION_FAKE_START (coding
);
2754 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2755 if (coding
->composition_rule_follows
)
2757 ENCODE_COMPOSITION_RULE (c
);
2758 coding
->composition_rule_follows
= 0;
2762 if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2763 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2764 ENCODE_UNSAFE_CHARACTER (c
);
2766 ENCODE_ISO_CHARACTER (c
);
2767 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2768 coding
->composition_rule_follows
= 1;
2773 if (!COMPOSING_P (coding
))
2775 if (this_pos
== data
[1])
2777 ENCODE_COMPOSITION_START (coding
, data
);
2785 /* Now encode the character C. */
2786 if (c
< 0x20 || c
== 0x7F)
2790 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2792 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2793 ENCODE_RESET_PLANE_AND_REGISTER
;
2797 /* fall down to treat '\r' as '\n' ... */
2802 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2803 ENCODE_RESET_PLANE_AND_REGISTER
;
2804 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2805 bcopy (coding
->spec
.iso2022
.initial_designation
,
2806 coding
->spec
.iso2022
.current_designation
,
2807 sizeof coding
->spec
.iso2022
.initial_designation
);
2808 if (coding
->eol_type
== CODING_EOL_LF
2809 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2810 *dst
++ = ISO_CODE_LF
;
2811 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2812 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2814 *dst
++ = ISO_CODE_CR
;
2815 CODING_SPEC_ISO_BOL (coding
) = 1;
2819 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2820 ENCODE_RESET_PLANE_AND_REGISTER
;
2824 else if (ASCII_BYTE_P (c
))
2825 ENCODE_ISO_CHARACTER (c
);
2826 else if (SINGLE_BYTE_CHAR_P (c
))
2831 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2832 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2833 ENCODE_UNSAFE_CHARACTER (c
);
2835 ENCODE_ISO_CHARACTER (c
);
2837 coding
->consumed_char
++;
2841 coding
->consumed
= src_base
- source
;
2842 coding
->produced
= coding
->produced_char
= dst
- destination
;
2846 /*** 4. SJIS and BIG5 handlers ***/
2848 /* Although SJIS and BIG5 are not ISO coding systems, they are used
2849 quite widely. So, for the moment, Emacs supports them in the bare
2850 C code. But, in the future, they may be supported only by CCL. */
2852 /* SJIS is a coding system encoding three character sets: ASCII, right
2853 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2854 as is. A character of charset katakana-jisx0201 is encoded by
2855 "position-code + 0x80". A character of charset japanese-jisx0208
2856 is encoded in 2-byte but two position-codes are divided and shifted
2857 so that it fits in the range below.
2859 --- CODE RANGE of SJIS ---
2860 (character set) (range)
2862 KATAKANA-JISX0201 0xA1 .. 0xDF
2863 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2864 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2865 -------------------------------
2869 /* BIG5 is a coding system encoding two character sets: ASCII and
2870 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2871 character set and is encoded in two bytes.
2873 --- CODE RANGE of BIG5 ---
2874 (character set) (range)
2876 Big5 (1st byte) 0xA1 .. 0xFE
2877 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2878 --------------------------
2880 Since the number of characters in Big5 is larger than maximum
2881 characters in Emacs' charset (96x96), it can't be handled as one
2882 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2883 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2884 contains frequently used characters and the latter contains less
2885 frequently used characters. */
2887 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2888 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2889 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
2890 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2892 /* Number of Big5 characters which have the same code in 1st byte. */
2893 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2895 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2898 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2900 charset = charset_big5_1; \
2903 charset = charset_big5_2; \
2904 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2906 c1 = temp / (0xFF - 0xA1) + 0x21; \
2907 c2 = temp % (0xFF - 0xA1) + 0x21; \
2910 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2912 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2913 if (charset == charset_big5_2) \
2914 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2915 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2916 b2 = temp % BIG5_SAME_ROW; \
2917 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2920 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2921 Check if a text is encoded in SJIS. If it is, return
2922 CODING_CATEGORY_MASK_SJIS, else return 0. */
2925 detect_coding_sjis (src
, src_end
, multibytep
)
2926 unsigned char *src
, *src_end
;
2930 /* Dummy for ONE_MORE_BYTE. */
2931 struct coding_system dummy_coding
;
2932 struct coding_system
*coding
= &dummy_coding
;
2936 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_SJIS
);
2939 if (c
== 0x80 || c
== 0xA0 || c
> 0xEF)
2941 if (c
<= 0x9F || c
>= 0xE0)
2943 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
2944 if (c
< 0x40 || c
== 0x7F || c
> 0xFC)
2950 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2951 Check if a text is encoded in BIG5. If it is, return
2952 CODING_CATEGORY_MASK_BIG5, else return 0. */
2955 detect_coding_big5 (src
, src_end
, multibytep
)
2956 unsigned char *src
, *src_end
;
2960 /* Dummy for ONE_MORE_BYTE. */
2961 struct coding_system dummy_coding
;
2962 struct coding_system
*coding
= &dummy_coding
;
2966 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_BIG5
);
2969 if (c
< 0xA1 || c
> 0xFE)
2971 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
2972 if (c
< 0x40 || (c
> 0x7F && c
< 0xA1) || c
> 0xFE)
2977 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2978 Check if a text is encoded in UTF-8. If it is, return
2979 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2981 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2982 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2983 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2984 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2985 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2986 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2987 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2990 detect_coding_utf_8 (src
, src_end
, multibytep
)
2991 unsigned char *src
, *src_end
;
2995 int seq_maybe_bytes
;
2996 /* Dummy for ONE_MORE_BYTE. */
2997 struct coding_system dummy_coding
;
2998 struct coding_system
*coding
= &dummy_coding
;
3002 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_UTF_8
);
3003 if (UTF_8_1_OCTET_P (c
))
3005 else if (UTF_8_2_OCTET_LEADING_P (c
))
3006 seq_maybe_bytes
= 1;
3007 else if (UTF_8_3_OCTET_LEADING_P (c
))
3008 seq_maybe_bytes
= 2;
3009 else if (UTF_8_4_OCTET_LEADING_P (c
))
3010 seq_maybe_bytes
= 3;
3011 else if (UTF_8_5_OCTET_LEADING_P (c
))
3012 seq_maybe_bytes
= 4;
3013 else if (UTF_8_6_OCTET_LEADING_P (c
))
3014 seq_maybe_bytes
= 5;
3020 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
3021 if (!UTF_8_EXTRA_OCTET_P (c
))
3025 while (seq_maybe_bytes
> 0);
3029 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3030 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
3031 Little Endian (otherwise). If it is, return
3032 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
3035 #define UTF_16_INVALID_P(val) \
3036 (((val) == 0xFFFE) \
3037 || ((val) == 0xFFFF))
3039 #define UTF_16_HIGH_SURROGATE_P(val) \
3040 (((val) & 0xD800) == 0xD800)
3042 #define UTF_16_LOW_SURROGATE_P(val) \
3043 (((val) & 0xDC00) == 0xDC00)
3046 detect_coding_utf_16 (src
, src_end
, multibytep
)
3047 unsigned char *src
, *src_end
;
3050 unsigned char c1
, c2
;
3051 /* Dummy for ONE_MORE_BYTE_CHECK_MULTIBYTE. */
3052 struct coding_system dummy_coding
;
3053 struct coding_system
*coding
= &dummy_coding
;
3055 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
, 0);
3056 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2
, multibytep
, 0);
3058 if ((c1
== 0xFF) && (c2
== 0xFE))
3059 return CODING_CATEGORY_MASK_UTF_16_LE
;
3060 else if ((c1
== 0xFE) && (c2
== 0xFF))
3061 return CODING_CATEGORY_MASK_UTF_16_BE
;
3065 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
3066 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
3069 decode_coding_sjis_big5 (coding
, source
, destination
,
3070 src_bytes
, dst_bytes
, sjis_p
)
3071 struct coding_system
*coding
;
3072 const unsigned char *source
;
3073 unsigned char *destination
;
3074 int src_bytes
, dst_bytes
;
3077 const unsigned char *src
= source
;
3078 const unsigned char *src_end
= source
+ src_bytes
;
3079 unsigned char *dst
= destination
;
3080 unsigned char *dst_end
= destination
+ dst_bytes
;
3081 /* SRC_BASE remembers the start position in source in each loop.
3082 The loop will be exited when there's not enough source code
3083 (within macro ONE_MORE_BYTE), or when there's not enough
3084 destination area to produce a character (within macro
3086 const unsigned char *src_base
;
3087 Lisp_Object translation_table
;
3089 if (NILP (Venable_character_translation
))
3090 translation_table
= Qnil
;
3093 translation_table
= coding
->translation_table_for_decode
;
3094 if (NILP (translation_table
))
3095 translation_table
= Vstandard_translation_table_for_decode
;
3098 coding
->produced_char
= 0;
3101 int c
, charset
, c1
, c2
= 0;
3108 charset
= CHARSET_ASCII
;
3113 if (coding
->eol_type
== CODING_EOL_CRLF
)
3119 /* To process C2 again, SRC is subtracted by 1. */
3122 else if (coding
->eol_type
== CODING_EOL_CR
)
3126 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3127 && (coding
->eol_type
== CODING_EOL_CR
3128 || coding
->eol_type
== CODING_EOL_CRLF
))
3130 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3131 goto label_end_of_loop
;
3139 if (c1
== 0x80 || c1
== 0xA0 || c1
> 0xEF)
3140 goto label_invalid_code
;
3141 if (c1
<= 0x9F || c1
>= 0xE0)
3143 /* SJIS -> JISX0208 */
3145 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
3146 goto label_invalid_code
;
3147 DECODE_SJIS (c1
, c2
, c1
, c2
);
3148 charset
= charset_jisx0208
;
3151 /* SJIS -> JISX0201-Kana */
3152 charset
= charset_katakana_jisx0201
;
3157 if (c1
< 0xA0 || c1
> 0xFE)
3158 goto label_invalid_code
;
3160 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
3161 goto label_invalid_code
;
3162 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
3166 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
3178 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3179 coding
->produced
= dst
- destination
;
3183 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
3184 This function can encode charsets `ascii', `katakana-jisx0201',
3185 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
3186 are sure that all these charsets are registered as official charset
3187 (i.e. do not have extended leading-codes). Characters of other
3188 charsets are produced without any encoding. If SJIS_P is 1, encode
3189 SJIS text, else encode BIG5 text. */
3192 encode_coding_sjis_big5 (coding
, source
, destination
,
3193 src_bytes
, dst_bytes
, sjis_p
)
3194 struct coding_system
*coding
;
3195 unsigned char *source
, *destination
;
3196 int src_bytes
, dst_bytes
;
3199 unsigned char *src
= source
;
3200 unsigned char *src_end
= source
+ src_bytes
;
3201 unsigned char *dst
= destination
;
3202 unsigned char *dst_end
= destination
+ dst_bytes
;
3203 /* SRC_BASE remembers the start position in source in each loop.
3204 The loop will be exited when there's not enough source text to
3205 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3206 there's not enough destination area to produce encoded codes
3207 (within macro EMIT_BYTES). */
3208 unsigned char *src_base
;
3209 Lisp_Object translation_table
;
3211 if (NILP (Venable_character_translation
))
3212 translation_table
= Qnil
;
3215 translation_table
= coding
->translation_table_for_encode
;
3216 if (NILP (translation_table
))
3217 translation_table
= Vstandard_translation_table_for_encode
;
3222 int c
, charset
, c1
, c2
;
3227 /* Now encode the character C. */
3228 if (SINGLE_BYTE_CHAR_P (c
))
3233 if (!(coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
3240 if (coding
->eol_type
== CODING_EOL_CRLF
)
3242 EMIT_TWO_BYTES ('\r', c
);
3245 else if (coding
->eol_type
== CODING_EOL_CR
)
3253 SPLIT_CHAR (c
, charset
, c1
, c2
);
3256 if (charset
== charset_jisx0208
3257 || charset
== charset_jisx0208_1978
)
3259 ENCODE_SJIS (c1
, c2
, c1
, c2
);
3260 EMIT_TWO_BYTES (c1
, c2
);
3262 else if (charset
== charset_katakana_jisx0201
)
3263 EMIT_ONE_BYTE (c1
| 0x80);
3264 else if (charset
== charset_latin_jisx0201
)
3266 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3268 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3269 if (CHARSET_WIDTH (charset
) > 1)
3270 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3273 /* There's no way other than producing the internal
3275 EMIT_BYTES (src_base
, src
);
3279 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
3281 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
3282 EMIT_TWO_BYTES (c1
, c2
);
3284 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3286 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3287 if (CHARSET_WIDTH (charset
) > 1)
3288 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3291 /* There's no way other than producing the internal
3293 EMIT_BYTES (src_base
, src
);
3296 coding
->consumed_char
++;
3300 coding
->consumed
= src_base
- source
;
3301 coding
->produced
= coding
->produced_char
= dst
- destination
;
3305 /*** 5. CCL handlers ***/
3307 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3308 Check if a text is encoded in a coding system of which
3309 encoder/decoder are written in CCL program. If it is, return
3310 CODING_CATEGORY_MASK_CCL, else return 0. */
3313 detect_coding_ccl (src
, src_end
, multibytep
)
3314 unsigned char *src
, *src_end
;
3317 unsigned char *valid
;
3319 /* Dummy for ONE_MORE_BYTE. */
3320 struct coding_system dummy_coding
;
3321 struct coding_system
*coding
= &dummy_coding
;
3323 /* No coding system is assigned to coding-category-ccl. */
3324 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
3327 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
3330 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_CCL
);
3337 /*** 6. End-of-line handlers ***/
3339 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3342 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3343 struct coding_system
*coding
;
3344 const unsigned char *source
;
3345 unsigned char *destination
;
3346 int src_bytes
, dst_bytes
;
3348 const unsigned char *src
= source
;
3349 unsigned char *dst
= destination
;
3350 const unsigned char *src_end
= src
+ src_bytes
;
3351 unsigned char *dst_end
= dst
+ dst_bytes
;
3352 Lisp_Object translation_table
;
3353 /* SRC_BASE remembers the start position in source in each loop.
3354 The loop will be exited when there's not enough source code
3355 (within macro ONE_MORE_BYTE), or when there's not enough
3356 destination area to produce a character (within macro
3358 const unsigned char *src_base
;
3361 translation_table
= Qnil
;
3362 switch (coding
->eol_type
)
3364 case CODING_EOL_CRLF
:
3379 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
3381 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3382 goto label_end_of_loop
;
3395 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3397 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3398 goto label_end_of_loop
;
3407 default: /* no need for EOL handling */
3417 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3418 coding
->produced
= dst
- destination
;
3422 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
3423 format of end-of-line according to `coding->eol_type'. It also
3424 convert multibyte form 8-bit characters to unibyte if
3425 CODING->src_multibyte is nonzero. If `coding->mode &
3426 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3427 also means end-of-line. */
3430 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3431 struct coding_system
*coding
;
3432 const unsigned char *source
;
3433 unsigned char *destination
;
3434 int src_bytes
, dst_bytes
;
3436 const unsigned char *src
= source
;
3437 unsigned char *dst
= destination
;
3438 const unsigned char *src_end
= src
+ src_bytes
;
3439 unsigned char *dst_end
= dst
+ dst_bytes
;
3440 Lisp_Object translation_table
;
3441 /* SRC_BASE remembers the start position in source in each loop.
3442 The loop will be exited when there's not enough source text to
3443 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3444 there's not enough destination area to produce encoded codes
3445 (within macro EMIT_BYTES). */
3446 const unsigned char *src_base
;
3449 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
3451 translation_table
= Qnil
;
3452 if (coding
->src_multibyte
3453 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3457 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
3460 if (coding
->eol_type
== CODING_EOL_CRLF
)
3462 while (src
< src_end
)
3468 else if (c
== '\n' || (c
== '\r' && selective_display
))
3469 EMIT_TWO_BYTES ('\r', '\n');
3479 if (!dst_bytes
|| src_bytes
<= dst_bytes
)
3481 safe_bcopy (src
, dst
, src_bytes
);
3487 if (coding
->src_multibyte
3488 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3490 safe_bcopy (src
, dst
, dst_bytes
);
3491 src_base
= src
+ dst_bytes
;
3492 dst
= destination
+ dst_bytes
;
3493 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
3495 if (coding
->eol_type
== CODING_EOL_CR
)
3497 for (tmp
= destination
; tmp
< dst
; tmp
++)
3498 if (*tmp
== '\n') *tmp
= '\r';
3500 else if (selective_display
)
3502 for (tmp
= destination
; tmp
< dst
; tmp
++)
3503 if (*tmp
== '\r') *tmp
= '\n';
3506 if (coding
->src_multibyte
)
3507 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
3509 coding
->consumed
= src_base
- source
;
3510 coding
->produced
= dst
- destination
;
3511 coding
->produced_char
= coding
->produced
;
3515 /*** 7. C library functions ***/
3517 /* In Emacs Lisp, a coding system is represented by a Lisp symbol which
3518 has a property `coding-system'. The value of this property is a
3519 vector of length 5 (called the coding-vector). Among elements of
3520 this vector, the first (element[0]) and the fifth (element[4])
3521 carry important information for decoding/encoding. Before
3522 decoding/encoding, this information should be set in fields of a
3523 structure of type `coding_system'.
3525 The value of the property `coding-system' can be a symbol of another
3526 subsidiary coding-system. In that case, Emacs gets coding-vector
3529 `element[0]' contains information to be set in `coding->type'. The
3530 value and its meaning is as follows:
3532 0 -- coding_type_emacs_mule
3533 1 -- coding_type_sjis
3534 2 -- coding_type_iso2022
3535 3 -- coding_type_big5
3536 4 -- coding_type_ccl encoder/decoder written in CCL
3537 nil -- coding_type_no_conversion
3538 t -- coding_type_undecided (automatic conversion on decoding,
3539 no-conversion on encoding)
3541 `element[4]' contains information to be set in `coding->flags' and
3542 `coding->spec'. The meaning varies by `coding->type'.
3544 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3545 of length 32 (of which the first 13 sub-elements are used now).
3546 Meanings of these sub-elements are:
3548 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3549 If the value is an integer of valid charset, the charset is
3550 assumed to be designated to graphic register N initially.
3552 If the value is minus, it is a minus value of charset which
3553 reserves graphic register N, which means that the charset is
3554 not designated initially but should be designated to graphic
3555 register N just before encoding a character in that charset.
3557 If the value is nil, graphic register N is never used on
3560 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3561 Each value takes t or nil. See the section ISO2022 of
3562 `coding.h' for more information.
3564 If `coding->type' is `coding_type_big5', element[4] is t to denote
3565 BIG5-ETen or nil to denote BIG5-HKU.
3567 If `coding->type' takes the other value, element[4] is ignored.
3569 Emacs Lisp's coding systems also carry information about format of
3570 end-of-line in a value of property `eol-type'. If the value is
3571 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3572 means CODING_EOL_CR. If it is not integer, it should be a vector
3573 of subsidiary coding systems of which property `eol-type' has one
3574 of the above values.
3578 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3579 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3580 is setup so that no conversion is necessary and return -1, else
3584 setup_coding_system (coding_system
, coding
)
3585 Lisp_Object coding_system
;
3586 struct coding_system
*coding
;
3588 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
3591 /* At first, zero clear all members. */
3592 bzero (coding
, sizeof (struct coding_system
));
3594 /* Initialize some fields required for all kinds of coding systems. */
3595 coding
->symbol
= coding_system
;
3596 coding
->heading_ascii
= -1;
3597 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
3598 coding
->composing
= COMPOSITION_DISABLED
;
3599 coding
->cmp_data
= NULL
;
3601 if (NILP (coding_system
))
3602 goto label_invalid_coding_system
;
3604 coding_spec
= Fget (coding_system
, Qcoding_system
);
3606 if (!VECTORP (coding_spec
)
3607 || XVECTOR (coding_spec
)->size
!= 5
3608 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
3609 goto label_invalid_coding_system
;
3611 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
3612 if (VECTORP (eol_type
))
3614 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3615 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
3616 if (system_eol_type
!= CODING_EOL_LF
)
3617 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3619 else if (XFASTINT (eol_type
) == 1)
3621 coding
->eol_type
= CODING_EOL_CRLF
;
3622 coding
->common_flags
3623 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3625 else if (XFASTINT (eol_type
) == 2)
3627 coding
->eol_type
= CODING_EOL_CR
;
3628 coding
->common_flags
3629 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3633 coding
->common_flags
= 0;
3634 coding
->eol_type
= CODING_EOL_LF
;
3637 coding_type
= XVECTOR (coding_spec
)->contents
[0];
3638 /* Try short cut. */
3639 if (SYMBOLP (coding_type
))
3641 if (EQ (coding_type
, Qt
))
3643 coding
->type
= coding_type_undecided
;
3644 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
3647 coding
->type
= coding_type_no_conversion
;
3648 /* Initialize this member. Any thing other than
3649 CODING_CATEGORY_IDX_UTF_16_BE and
3650 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3651 special treatment in detect_eol. */
3652 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
3657 /* Get values of coding system properties:
3658 `post-read-conversion', `pre-write-conversion',
3659 `translation-table-for-decode', `translation-table-for-encode'. */
3660 plist
= XVECTOR (coding_spec
)->contents
[3];
3661 /* Pre & post conversion functions should be disabled if
3662 inhibit_eol_conversion is nonzero. This is the case that a code
3663 conversion function is called while those functions are running. */
3664 if (! inhibit_pre_post_conversion
)
3666 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
3667 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
3669 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
3671 val
= Fget (val
, Qtranslation_table_for_decode
);
3672 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3673 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
3675 val
= Fget (val
, Qtranslation_table_for_encode
);
3676 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3677 val
= Fplist_get (plist
, Qcoding_category
);
3680 val
= Fget (val
, Qcoding_category_index
);
3682 coding
->category_idx
= XINT (val
);
3684 goto label_invalid_coding_system
;
3687 goto label_invalid_coding_system
;
3689 /* If the coding system has non-nil `composition' property, enable
3690 composition handling. */
3691 val
= Fplist_get (plist
, Qcomposition
);
3693 coding
->composing
= COMPOSITION_NO
;
3695 /* If the coding system is ascii-incompatible, record it in
3697 val
= Fplist_get (plist
, Qascii_incompatible
);
3699 coding
->common_flags
|= CODING_ASCII_INCOMPATIBLE_MASK
;
3701 switch (XFASTINT (coding_type
))
3704 coding
->type
= coding_type_emacs_mule
;
3705 coding
->common_flags
3706 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3707 if (!NILP (coding
->post_read_conversion
))
3708 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3709 if (!NILP (coding
->pre_write_conversion
))
3710 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3714 coding
->type
= coding_type_sjis
;
3715 coding
->common_flags
3716 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3720 coding
->type
= coding_type_iso2022
;
3721 coding
->common_flags
3722 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3724 Lisp_Object val
, temp
;
3726 int i
, charset
, reg_bits
= 0;
3728 val
= XVECTOR (coding_spec
)->contents
[4];
3730 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3731 goto label_invalid_coding_system
;
3733 flags
= XVECTOR (val
)->contents
;
3735 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3736 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3737 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3738 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3739 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3740 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3741 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3742 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3743 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3744 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3745 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3746 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3747 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3750 /* Invoke graphic register 0 to plane 0. */
3751 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3752 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3753 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3754 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3755 /* Not single shifting at first. */
3756 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3757 /* Beginning of buffer should also be regarded as bol. */
3758 CODING_SPEC_ISO_BOL (coding
) = 1;
3760 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3761 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3762 val
= Vcharset_revision_alist
;
3765 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3767 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3768 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3769 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3773 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3774 FLAGS[REG] can be one of below:
3775 integer CHARSET: CHARSET occupies register I,
3776 t: designate nothing to REG initially, but can be used
3778 list of integer, nil, or t: designate the first
3779 element (if integer) to REG initially, the remaining
3780 elements (if integer) is designated to REG on request,
3781 if an element is t, REG can be used by any charsets,
3782 nil: REG is never used. */
3783 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3784 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3785 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3786 for (i
= 0; i
< 4; i
++)
3788 if ((INTEGERP (flags
[i
])
3789 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
)))
3790 || (charset
= get_charset_id (flags
[i
])) >= 0)
3792 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3793 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3795 else if (EQ (flags
[i
], Qt
))
3797 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3799 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3801 else if (CONSP (flags
[i
]))
3806 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3807 if ((INTEGERP (XCAR (tail
))
3808 && (charset
= XINT (XCAR (tail
)),
3809 CHARSET_VALID_P (charset
)))
3810 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3812 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3813 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3816 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3818 while (CONSP (tail
))
3820 if ((INTEGERP (XCAR (tail
))
3821 && (charset
= XINT (XCAR (tail
)),
3822 CHARSET_VALID_P (charset
)))
3823 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3824 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3826 else if (EQ (XCAR (tail
), Qt
))
3832 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3834 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3835 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3838 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3840 /* REG 1 can be used only by locking shift in 7-bit env. */
3841 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3843 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3844 /* Without any shifting, only REG 0 and 1 can be used. */
3849 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3851 if (CHARSET_DEFINED_P (charset
)
3852 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3853 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3855 /* There exist some default graphic registers to be
3858 /* We had better avoid designating a charset of
3859 CHARS96 to REG 0 as far as possible. */
3860 if (CHARSET_CHARS (charset
) == 96)
3861 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3863 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3865 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3867 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3871 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3872 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3876 coding
->type
= coding_type_big5
;
3877 coding
->common_flags
3878 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3880 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3881 ? CODING_FLAG_BIG5_HKU
3882 : CODING_FLAG_BIG5_ETEN
);
3886 coding
->type
= coding_type_ccl
;
3887 coding
->common_flags
3888 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3890 val
= XVECTOR (coding_spec
)->contents
[4];
3892 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3894 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3896 goto label_invalid_coding_system
;
3898 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3899 val
= Fplist_get (plist
, Qvalid_codes
);
3904 for (; CONSP (val
); val
= XCDR (val
))
3908 && XINT (this) >= 0 && XINT (this) < 256)
3909 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3910 else if (CONSP (this)
3911 && INTEGERP (XCAR (this))
3912 && INTEGERP (XCDR (this)))
3914 int start
= XINT (XCAR (this));
3915 int end
= XINT (XCDR (this));
3917 if (start
>= 0 && start
<= end
&& end
< 256)
3918 while (start
<= end
)
3919 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3924 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3925 coding
->spec
.ccl
.cr_carryover
= 0;
3926 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
3930 coding
->type
= coding_type_raw_text
;
3934 goto label_invalid_coding_system
;
3938 label_invalid_coding_system
:
3939 coding
->type
= coding_type_no_conversion
;
3940 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3941 coding
->common_flags
= 0;
3942 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3943 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3944 return NILP (coding_system
) ? 0 : -1;
3947 /* Free memory blocks allocated for storing composition information. */
3950 coding_free_composition_data (coding
)
3951 struct coding_system
*coding
;
3953 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3957 /* Memory blocks are chained. At first, rewind to the first, then,
3958 free blocks one by one. */
3959 while (cmp_data
->prev
)
3960 cmp_data
= cmp_data
->prev
;
3963 next
= cmp_data
->next
;
3967 coding
->cmp_data
= NULL
;
3970 /* Set `char_offset' member of all memory blocks pointed by
3971 coding->cmp_data to POS. */
3974 coding_adjust_composition_offset (coding
, pos
)
3975 struct coding_system
*coding
;
3978 struct composition_data
*cmp_data
;
3980 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3981 cmp_data
->char_offset
= pos
;
3984 /* Setup raw-text or one of its subsidiaries in the structure
3985 coding_system CODING according to the already setup value eol_type
3986 in CODING. CODING should be setup for some coding system in
3990 setup_raw_text_coding_system (coding
)
3991 struct coding_system
*coding
;
3993 if (coding
->type
!= coding_type_raw_text
)
3995 coding
->symbol
= Qraw_text
;
3996 coding
->type
= coding_type_raw_text
;
3997 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3999 Lisp_Object subsidiaries
;
4000 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
4002 if (VECTORP (subsidiaries
)
4003 && XVECTOR (subsidiaries
)->size
== 3)
4005 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
4007 setup_coding_system (coding
->symbol
, coding
);
4012 /* Emacs has a mechanism to automatically detect a coding system if it
4013 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
4014 it's impossible to distinguish some coding systems accurately
4015 because they use the same range of codes. So, at first, coding
4016 systems are categorized into 7, those are:
4018 o coding-category-emacs-mule
4020 The category for a coding system which has the same code range
4021 as Emacs' internal format. Assigned the coding-system (Lisp
4022 symbol) `emacs-mule' by default.
4024 o coding-category-sjis
4026 The category for a coding system which has the same code range
4027 as SJIS. Assigned the coding-system (Lisp
4028 symbol) `japanese-shift-jis' by default.
4030 o coding-category-iso-7
4032 The category for a coding system which has the same code range
4033 as ISO2022 of 7-bit environment. This doesn't use any locking
4034 shift and single shift functions. This can encode/decode all
4035 charsets. Assigned the coding-system (Lisp symbol)
4036 `iso-2022-7bit' by default.
4038 o coding-category-iso-7-tight
4040 Same as coding-category-iso-7 except that this can
4041 encode/decode only the specified charsets.
4043 o coding-category-iso-8-1
4045 The category for a coding system which has the same code range
4046 as ISO2022 of 8-bit environment and graphic plane 1 used only
4047 for DIMENSION1 charset. This doesn't use any locking shift
4048 and single shift functions. Assigned the coding-system (Lisp
4049 symbol) `iso-latin-1' by default.
4051 o coding-category-iso-8-2
4053 The category for a coding system which has the same code range
4054 as ISO2022 of 8-bit environment and graphic plane 1 used only
4055 for DIMENSION2 charset. This doesn't use any locking shift
4056 and single shift functions. Assigned the coding-system (Lisp
4057 symbol) `japanese-iso-8bit' by default.
4059 o coding-category-iso-7-else
4061 The category for a coding system which has the same code range
4062 as ISO2022 of 7-bit environment but uses locking shift or
4063 single shift functions. Assigned the coding-system (Lisp
4064 symbol) `iso-2022-7bit-lock' by default.
4066 o coding-category-iso-8-else
4068 The category for a coding system which has the same code range
4069 as ISO2022 of 8-bit environment but uses locking shift or
4070 single shift functions. Assigned the coding-system (Lisp
4071 symbol) `iso-2022-8bit-ss2' by default.
4073 o coding-category-big5
4075 The category for a coding system which has the same code range
4076 as BIG5. Assigned the coding-system (Lisp symbol)
4077 `cn-big5' by default.
4079 o coding-category-utf-8
4081 The category for a coding system which has the same code range
4082 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
4083 symbol) `utf-8' by default.
4085 o coding-category-utf-16-be
4087 The category for a coding system in which a text has an
4088 Unicode signature (cf. Unicode Standard) in the order of BIG
4089 endian at the head. Assigned the coding-system (Lisp symbol)
4090 `utf-16-be' by default.
4092 o coding-category-utf-16-le
4094 The category for a coding system in which a text has an
4095 Unicode signature (cf. Unicode Standard) in the order of
4096 LITTLE endian at the head. Assigned the coding-system (Lisp
4097 symbol) `utf-16-le' by default.
4099 o coding-category-ccl
4101 The category for a coding system of which encoder/decoder is
4102 written in CCL programs. The default value is nil, i.e., no
4103 coding system is assigned.
4105 o coding-category-binary
4107 The category for a coding system not categorized in any of the
4108 above. Assigned the coding-system (Lisp symbol)
4109 `no-conversion' by default.
4111 Each of them is a Lisp symbol and the value is an actual
4112 `coding-system' (this is also a Lisp symbol) assigned by a user.
4113 What Emacs does actually is to detect a category of coding system.
4114 Then, it uses a `coding-system' assigned to it. If Emacs can't
4115 decide a single possible category, it selects a category of the
4116 highest priority. Priorities of categories are also specified by a
4117 user in a Lisp variable `coding-category-list'.
4122 int ascii_skip_code
[256];
4124 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
4125 If it detects possible coding systems, return an integer in which
4126 appropriate flag bits are set. Flag bits are defined by macros
4127 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
4128 it should point the table `coding_priorities'. In that case, only
4129 the flag bit for a coding system of the highest priority is set in
4130 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
4131 range 0x80..0x9F are in multibyte form.
4133 How many ASCII characters are at the head is returned as *SKIP. */
4136 detect_coding_mask (source
, src_bytes
, priorities
, skip
, multibytep
)
4137 unsigned char *source
;
4138 int src_bytes
, *priorities
, *skip
;
4141 register unsigned char c
;
4142 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
4143 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
4145 int null_byte_found
;
4146 int latin_extra_code_state
= 1;
4148 /* At first, skip all ASCII characters and control characters except
4149 for three ISO2022 specific control characters. */
4150 ascii_skip_code
[ISO_CODE_SO
] = 0;
4151 ascii_skip_code
[ISO_CODE_SI
] = 0;
4152 ascii_skip_code
[ISO_CODE_ESC
] = 0;
4154 label_loop_detect_coding
:
4155 null_byte_found
= 0;
4156 /* We stop this loop before the last byte because it may be a NULL
4158 while (src
< src_end
- 1 && ascii_skip_code
[*src
])
4159 null_byte_found
|= (! *src
++);
4160 if (ascii_skip_code
[*src
])
4162 else if (! null_byte_found
)
4164 unsigned char *p
= src
+ 1;
4165 while (p
< src_end
- 1)
4166 null_byte_found
|= (! *p
++);
4168 *skip
= src
- source
;
4171 /* We found nothing other than ASCII (and NULL byte). There's
4176 /* The text seems to be encoded in some multilingual coding system.
4177 Now, try to find in which coding system the text is encoded. */
4178 if (! null_byte_found
&& c
< 0x80)
4180 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
4181 /* C is an ISO2022 specific control code of C0. */
4182 latin_extra_code_state
= 1;
4183 mask
= detect_coding_iso2022 (src
, src_end
, multibytep
,
4184 &latin_extra_code_state
);
4187 /* No valid ISO2022 code follows C. Try again. */
4189 if (c
== ISO_CODE_ESC
)
4190 ascii_skip_code
[ISO_CODE_ESC
] = 1;
4192 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
4193 goto label_loop_detect_coding
;
4197 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4199 if (mask
& priorities
[i
])
4200 return priorities
[i
];
4202 return CODING_CATEGORY_MASK_RAW_TEXT
;
4209 if (multibytep
&& c
== LEADING_CODE_8_BIT_CONTROL
)
4212 if (null_byte_found
)
4214 try = (CODING_CATEGORY_MASK_UTF_16_BE
4215 | CODING_CATEGORY_MASK_UTF_16_LE
);
4219 /* C is the first byte of SJIS character code,
4220 or a leading-code of Emacs' internal format (emacs-mule),
4221 or the first byte of UTF-16. */
4222 try = (CODING_CATEGORY_MASK_SJIS
4223 | CODING_CATEGORY_MASK_EMACS_MULE
4224 | CODING_CATEGORY_MASK_UTF_16_BE
4225 | CODING_CATEGORY_MASK_UTF_16_LE
);
4227 /* Or, if C is a special latin extra code,
4228 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4229 or is an ISO2022 control-sequence-introducer (CSI),
4230 we should also consider the possibility of ISO2022 codings. */
4231 if ((latin_extra_code_state
4232 && VECTORP (Vlatin_extra_code_table
)
4233 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
4234 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
4235 || (c
== ISO_CODE_CSI
4238 || ((*src
== '0' || *src
== '1' || *src
== '2')
4239 && src
+ 1 < src_end
4240 && src
[1] == ']')))))
4241 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4242 | CODING_CATEGORY_MASK_ISO_8BIT
);
4245 /* C is a character of ISO2022 in graphic plane right,
4246 or a SJIS's 1-byte character code (i.e. JISX0201),
4247 or the first byte of BIG5's 2-byte code,
4248 or the first byte of UTF-8/16. */
4249 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4250 | CODING_CATEGORY_MASK_ISO_8BIT
4251 | CODING_CATEGORY_MASK_SJIS
4252 | CODING_CATEGORY_MASK_BIG5
4253 | CODING_CATEGORY_MASK_UTF_8
4254 | CODING_CATEGORY_MASK_UTF_16_BE
4255 | CODING_CATEGORY_MASK_UTF_16_LE
);
4257 /* Or, we may have to consider the possibility of CCL. */
4258 if (! null_byte_found
4259 && coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4260 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4261 ->spec
.ccl
.valid_codes
)[c
])
4262 try |= CODING_CATEGORY_MASK_CCL
;
4267 /* At first try detection with Latin extra codes not-allowed.
4268 If no proper coding system is found because of Latin extra
4269 codes, try detection with Latin extra codes allowed. */
4270 latin_extra_code_state
= 0;
4272 utf16_examined_p
= iso2022_examined_p
= 0;
4273 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4275 if (!iso2022_examined_p
4276 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
4278 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
,
4279 &latin_extra_code_state
);
4280 iso2022_examined_p
= 1;
4282 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
4283 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4284 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
4285 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4286 else if (!utf16_examined_p
4287 && (priorities
[i
] & try &
4288 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
4290 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4291 utf16_examined_p
= 1;
4293 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
4294 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4295 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
4296 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4297 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
4298 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4299 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
4301 if (latin_extra_code_state
== 1)
4303 /* Detection of ISO-2022 based coding system
4304 failed because of Latin extra codes. Before
4305 falling back to raw-text, try again with
4306 Latin extra codes allowed. */
4307 latin_extra_code_state
= 2;
4308 try = (mask
| CODING_CATEGORY_MASK_ISO_8_ELSE
4309 | CODING_CATEGORY_MASK_ISO_8BIT
);
4312 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
4314 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
4316 if (latin_extra_code_state
== 1)
4318 /* See the above comment. */
4319 latin_extra_code_state
= 2;
4320 try = (mask
| CODING_CATEGORY_MASK_ISO_8_ELSE
4321 | CODING_CATEGORY_MASK_ISO_8BIT
);
4324 mask
|= CODING_CATEGORY_MASK_BINARY
;
4326 if (mask
& priorities
[i
])
4327 return priorities
[i
];
4329 return CODING_CATEGORY_MASK_RAW_TEXT
;
4331 if (try & CODING_CATEGORY_MASK_ISO
)
4332 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
,
4333 &latin_extra_code_state
);
4334 if (try & CODING_CATEGORY_MASK_SJIS
)
4335 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4336 if (try & CODING_CATEGORY_MASK_BIG5
)
4337 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4338 if (try & CODING_CATEGORY_MASK_UTF_8
)
4339 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4340 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
4341 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4342 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
4343 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4344 if (try & CODING_CATEGORY_MASK_CCL
)
4345 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4347 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
4350 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4351 The information of the detected coding system is set in CODING. */
4354 detect_coding (coding
, src
, src_bytes
)
4355 struct coding_system
*coding
;
4356 const unsigned char *src
;
4363 val
= Vcoding_category_list
;
4364 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
,
4365 coding
->src_multibyte
);
4366 coding
->heading_ascii
= skip
;
4370 /* We found a single coding system of the highest priority in MASK. */
4372 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
4374 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
4376 val
= find_symbol_value (XVECTOR (Vcoding_category_table
)->contents
[idx
]);
4378 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4382 tmp
= Fget (val
, Qeol_type
);
4384 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
4387 /* Setup this new coding system while preserving some slots. */
4389 int src_multibyte
= coding
->src_multibyte
;
4390 int dst_multibyte
= coding
->dst_multibyte
;
4392 setup_coding_system (val
, coding
);
4393 coding
->src_multibyte
= src_multibyte
;
4394 coding
->dst_multibyte
= dst_multibyte
;
4395 coding
->heading_ascii
= skip
;
4399 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
4400 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4401 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4403 How many non-eol characters are at the head is returned as *SKIP. */
4405 #define MAX_EOL_CHECK_COUNT 3
4408 detect_eol_type (source
, src_bytes
, skip
)
4409 const unsigned char *source
;
4410 int src_bytes
, *skip
;
4412 const unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4414 int total
= 0; /* How many end-of-lines are found so far. */
4415 int eol_type
= CODING_EOL_UNDECIDED
;
4420 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4423 if (c
== '\n' || c
== '\r')
4426 *skip
= src
- 1 - source
;
4429 this_eol_type
= CODING_EOL_LF
;
4430 else if (src
>= src_end
|| *src
!= '\n')
4431 this_eol_type
= CODING_EOL_CR
;
4433 this_eol_type
= CODING_EOL_CRLF
, src
++;
4435 if (eol_type
== CODING_EOL_UNDECIDED
)
4436 /* This is the first end-of-line. */
4437 eol_type
= this_eol_type
;
4438 else if (eol_type
!= this_eol_type
)
4440 /* The found type is different from what found before. */
4441 eol_type
= CODING_EOL_INCONSISTENT
;
4448 *skip
= src_end
- source
;
4452 /* Like detect_eol_type, but detect EOL type in 2-octet
4453 big-endian/little-endian format for coding systems utf-16-be and
4457 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
4458 const unsigned char *source
;
4459 int src_bytes
, *skip
, big_endian_p
;
4461 const unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4462 unsigned int c1
, c2
;
4463 int total
= 0; /* How many end-of-lines are found so far. */
4464 int eol_type
= CODING_EOL_UNDECIDED
;
4475 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4477 c1
= (src
[msb
] << 8) | (src
[lsb
]);
4480 if (c1
== '\n' || c1
== '\r')
4483 *skip
= src
- 2 - source
;
4487 this_eol_type
= CODING_EOL_LF
;
4491 if ((src
+ 1) >= src_end
)
4493 this_eol_type
= CODING_EOL_CR
;
4497 c2
= (src
[msb
] << 8) | (src
[lsb
]);
4499 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
4501 this_eol_type
= CODING_EOL_CR
;
4505 if (eol_type
== CODING_EOL_UNDECIDED
)
4506 /* This is the first end-of-line. */
4507 eol_type
= this_eol_type
;
4508 else if (eol_type
!= this_eol_type
)
4510 /* The found type is different from what found before. */
4511 eol_type
= CODING_EOL_INCONSISTENT
;
4518 *skip
= src_end
- source
;
4522 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4523 is encoded. If it detects an appropriate format of end-of-line, it
4524 sets the information in *CODING. */
4527 detect_eol (coding
, src
, src_bytes
)
4528 struct coding_system
*coding
;
4529 const unsigned char *src
;
4536 switch (coding
->category_idx
)
4538 case CODING_CATEGORY_IDX_UTF_16_BE
:
4539 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
4541 case CODING_CATEGORY_IDX_UTF_16_LE
:
4542 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
4545 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
4549 if (coding
->heading_ascii
> skip
)
4550 coding
->heading_ascii
= skip
;
4552 skip
= coding
->heading_ascii
;
4554 if (eol_type
== CODING_EOL_UNDECIDED
)
4556 if (eol_type
== CODING_EOL_INCONSISTENT
)
4559 /* This code is suppressed until we find a better way to
4560 distinguish raw text file and binary file. */
4562 /* If we have already detected that the coding is raw-text, the
4563 coding should actually be no-conversion. */
4564 if (coding
->type
== coding_type_raw_text
)
4566 setup_coding_system (Qno_conversion
, coding
);
4569 /* Else, let's decode only text code anyway. */
4571 eol_type
= CODING_EOL_LF
;
4574 val
= Fget (coding
->symbol
, Qeol_type
);
4575 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4577 int src_multibyte
= coding
->src_multibyte
;
4578 int dst_multibyte
= coding
->dst_multibyte
;
4579 struct composition_data
*cmp_data
= coding
->cmp_data
;
4581 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
4582 coding
->src_multibyte
= src_multibyte
;
4583 coding
->dst_multibyte
= dst_multibyte
;
4584 coding
->heading_ascii
= skip
;
4585 coding
->cmp_data
= cmp_data
;
4589 #define CONVERSION_BUFFER_EXTRA_ROOM 256
4591 #define DECODING_BUFFER_MAG(coding) \
4592 (coding->type == coding_type_iso2022 \
4594 : (coding->type == coding_type_ccl \
4595 ? coding->spec.ccl.decoder.buf_magnification \
4598 /* Return maximum size (bytes) of a buffer enough for decoding
4599 SRC_BYTES of text encoded in CODING. */
4602 decoding_buffer_size (coding
, src_bytes
)
4603 struct coding_system
*coding
;
4606 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
4607 + CONVERSION_BUFFER_EXTRA_ROOM
);
4610 /* Return maximum size (bytes) of a buffer enough for encoding
4611 SRC_BYTES of text to CODING. */
4614 encoding_buffer_size (coding
, src_bytes
)
4615 struct coding_system
*coding
;
4620 if (coding
->type
== coding_type_ccl
)
4622 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
4623 if (coding
->eol_type
== CODING_EOL_CRLF
)
4626 else if (CODING_REQUIRE_ENCODING (coding
))
4631 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
4634 /* Working buffer for code conversion. */
4635 struct conversion_buffer
4637 int size
; /* size of data. */
4638 int on_stack
; /* 1 if allocated by alloca. */
4639 unsigned char *data
;
4642 /* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4643 #define allocate_conversion_buffer(buf, len) \
4645 if (len < MAX_ALLOCA) \
4647 buf.data = (unsigned char *) alloca (len); \
4652 buf.data = (unsigned char *) xmalloc (len); \
4658 /* Double the allocated memory for *BUF. */
4660 extend_conversion_buffer (buf
)
4661 struct conversion_buffer
*buf
;
4665 unsigned char *save
= buf
->data
;
4666 buf
->data
= (unsigned char *) xmalloc (buf
->size
* 2);
4667 bcopy (save
, buf
->data
, buf
->size
);
4672 buf
->data
= (unsigned char *) xrealloc (buf
->data
, buf
->size
* 2);
4677 /* Free the allocated memory for BUF if it is not on stack. */
4679 free_conversion_buffer (buf
)
4680 struct conversion_buffer
*buf
;
4687 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
4688 struct coding_system
*coding
;
4689 unsigned char *source
, *destination
;
4690 int src_bytes
, dst_bytes
, encodep
;
4692 struct ccl_program
*ccl
4693 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
4694 unsigned char *dst
= destination
;
4696 ccl
->suppress_error
= coding
->suppress_error
;
4697 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
4700 /* On encoding, EOL format is converted within ccl_driver. For
4701 that, setup proper information in the structure CCL. */
4702 ccl
->eol_type
= coding
->eol_type
;
4703 if (ccl
->eol_type
==CODING_EOL_UNDECIDED
)
4704 ccl
->eol_type
= CODING_EOL_LF
;
4705 ccl
->cr_consumed
= coding
->spec
.ccl
.cr_carryover
;
4706 ccl
->eight_bit_control
= coding
->dst_multibyte
;
4709 ccl
->eight_bit_control
= 1;
4710 ccl
->multibyte
= coding
->src_multibyte
;
4711 if (coding
->spec
.ccl
.eight_bit_carryover
[0] != 0)
4713 /* Move carryover bytes to DESTINATION. */
4714 unsigned char *p
= coding
->spec
.ccl
.eight_bit_carryover
;
4717 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4719 dst_bytes
-= dst
- destination
;
4722 coding
->produced
= (ccl_driver (ccl
, source
, dst
, src_bytes
, dst_bytes
,
4723 &(coding
->consumed
))
4724 + dst
- destination
);
4728 coding
->produced_char
= coding
->produced
;
4729 coding
->spec
.ccl
.cr_carryover
= ccl
->cr_consumed
;
4731 else if (!ccl
->eight_bit_control
)
4733 /* The produced bytes forms a valid multibyte sequence. */
4734 coding
->produced_char
4735 = multibyte_chars_in_text (destination
, coding
->produced
);
4736 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4740 /* On decoding, the destination should always multibyte. But,
4741 CCL program might have been generated an invalid multibyte
4742 sequence. Here we make such a sequence valid as
4745 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
4747 if ((coding
->consumed
< src_bytes
4748 || !ccl
->last_block
)
4749 && coding
->produced
>= 1
4750 && destination
[coding
->produced
- 1] >= 0x80)
4752 /* We should not convert the tailing 8-bit codes to
4753 multibyte form even if they doesn't form a valid
4754 multibyte sequence. They may form a valid sequence in
4758 if (destination
[coding
->produced
- 1] < 0xA0)
4760 else if (coding
->produced
>= 2)
4762 if (destination
[coding
->produced
- 2] >= 0x80)
4764 if (destination
[coding
->produced
- 2] < 0xA0)
4766 else if (coding
->produced
>= 3
4767 && destination
[coding
->produced
- 3] >= 0x80
4768 && destination
[coding
->produced
- 3] < 0xA0)
4774 BCOPY_SHORT (destination
+ coding
->produced
- carryover
,
4775 coding
->spec
.ccl
.eight_bit_carryover
,
4777 coding
->spec
.ccl
.eight_bit_carryover
[carryover
] = 0;
4778 coding
->produced
-= carryover
;
4781 coding
->produced
= str_as_multibyte (destination
, bytes
,
4783 &(coding
->produced_char
));
4786 switch (ccl
->status
)
4788 case CCL_STAT_SUSPEND_BY_SRC
:
4789 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
4791 case CCL_STAT_SUSPEND_BY_DST
:
4792 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
4795 case CCL_STAT_INVALID_CMD
:
4796 coding
->result
= CODING_FINISH_INTERRUPT
;
4799 coding
->result
= CODING_FINISH_NORMAL
;
4802 return coding
->result
;
4805 /* Decode EOL format of the text at PTR of BYTES length destructively
4806 according to CODING->eol_type. This is called after the CCL
4807 program produced a decoded text at PTR. If we do CRLF->LF
4808 conversion, update CODING->produced and CODING->produced_char. */
4811 decode_eol_post_ccl (coding
, ptr
, bytes
)
4812 struct coding_system
*coding
;
4816 Lisp_Object val
, saved_coding_symbol
;
4817 unsigned char *pend
= ptr
+ bytes
;
4820 /* Remember the current coding system symbol. We set it back when
4821 an inconsistent EOL is found so that `last-coding-system-used' is
4822 set to the coding system that doesn't specify EOL conversion. */
4823 saved_coding_symbol
= coding
->symbol
;
4825 coding
->spec
.ccl
.cr_carryover
= 0;
4826 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4828 /* Here, to avoid the call of setup_coding_system, we directly
4829 call detect_eol_type. */
4830 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
4831 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
4832 coding
->eol_type
= CODING_EOL_LF
;
4833 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4835 val
= Fget (coding
->symbol
, Qeol_type
);
4836 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4837 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
4839 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4842 if (coding
->eol_type
== CODING_EOL_LF
4843 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
4845 /* We have nothing to do. */
4848 else if (coding
->eol_type
== CODING_EOL_CRLF
)
4850 unsigned char *pstart
= ptr
, *p
= ptr
;
4852 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4853 && *(pend
- 1) == '\r')
4855 /* If the last character is CR, we can't handle it here
4856 because LF will be in the not-yet-decoded source text.
4857 Record that the CR is not yet processed. */
4858 coding
->spec
.ccl
.cr_carryover
= 1;
4860 coding
->produced_char
--;
4867 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4874 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4875 goto undo_eol_conversion
;
4879 else if (*ptr
== '\n'
4880 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4881 goto undo_eol_conversion
;
4886 undo_eol_conversion
:
4887 /* We have faced with inconsistent EOL format at PTR.
4888 Convert all LFs before PTR back to CRLFs. */
4889 for (p
--, ptr
--; p
>= pstart
; p
--)
4892 *ptr
-- = '\n', *ptr
-- = '\r';
4896 /* If carryover is recorded, cancel it because we don't
4897 convert CRLF anymore. */
4898 if (coding
->spec
.ccl
.cr_carryover
)
4900 coding
->spec
.ccl
.cr_carryover
= 0;
4902 coding
->produced_char
++;
4906 coding
->eol_type
= CODING_EOL_LF
;
4907 coding
->symbol
= saved_coding_symbol
;
4911 /* As each two-byte sequence CRLF was converted to LF, (PEND
4912 - P) is the number of deleted characters. */
4913 coding
->produced
-= pend
- p
;
4914 coding
->produced_char
-= pend
- p
;
4917 else /* i.e. coding->eol_type == CODING_EOL_CR */
4919 unsigned char *p
= ptr
;
4921 for (; ptr
< pend
; ptr
++)
4925 else if (*ptr
== '\n'
4926 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4928 for (; p
< ptr
; p
++)
4934 coding
->eol_type
= CODING_EOL_LF
;
4935 coding
->symbol
= saved_coding_symbol
;
4941 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4942 decoding, it may detect coding system and format of end-of-line if
4943 those are not yet decided. The source should be unibyte, the
4944 result is multibyte if CODING->dst_multibyte is nonzero, else
4948 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4949 struct coding_system
*coding
;
4950 const unsigned char *source
;
4951 unsigned char *destination
;
4952 int src_bytes
, dst_bytes
;
4956 if (coding
->type
== coding_type_undecided
)
4957 detect_coding (coding
, source
, src_bytes
);
4959 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4960 && coding
->type
!= coding_type_ccl
)
4962 detect_eol (coding
, source
, src_bytes
);
4963 /* We had better recover the original eol format if we
4964 encounter an inconsistent eol format while decoding. */
4965 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4968 coding
->produced
= coding
->produced_char
= 0;
4969 coding
->consumed
= coding
->consumed_char
= 0;
4971 coding
->result
= CODING_FINISH_NORMAL
;
4973 switch (coding
->type
)
4975 case coding_type_sjis
:
4976 decode_coding_sjis_big5 (coding
, source
, destination
,
4977 src_bytes
, dst_bytes
, 1);
4980 case coding_type_iso2022
:
4981 decode_coding_iso2022 (coding
, source
, destination
,
4982 src_bytes
, dst_bytes
);
4985 case coding_type_big5
:
4986 decode_coding_sjis_big5 (coding
, source
, destination
,
4987 src_bytes
, dst_bytes
, 0);
4990 case coding_type_emacs_mule
:
4991 decode_coding_emacs_mule (coding
, source
, destination
,
4992 src_bytes
, dst_bytes
);
4995 case coding_type_ccl
:
4996 if (coding
->spec
.ccl
.cr_carryover
)
4998 /* Put the CR which was not processed by the previous call
4999 of decode_eol_post_ccl in DESTINATION. It will be
5000 decoded together with the following LF by the call to
5001 decode_eol_post_ccl below. */
5002 *destination
= '\r';
5004 coding
->produced_char
++;
5006 extra
= coding
->spec
.ccl
.cr_carryover
;
5008 ccl_coding_driver (coding
, source
, destination
+ extra
,
5009 src_bytes
, dst_bytes
, 0);
5010 if (coding
->eol_type
!= CODING_EOL_LF
)
5012 coding
->produced
+= extra
;
5013 coding
->produced_char
+= extra
;
5014 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
5019 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
5022 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
5023 && coding
->mode
& CODING_MODE_LAST_BLOCK
5024 && coding
->consumed
== src_bytes
)
5025 coding
->result
= CODING_FINISH_NORMAL
;
5027 if (coding
->mode
& CODING_MODE_LAST_BLOCK
5028 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
5030 const unsigned char *src
= source
+ coding
->consumed
;
5031 unsigned char *dst
= destination
+ coding
->produced
;
5033 src_bytes
-= coding
->consumed
;
5035 if (COMPOSING_P (coding
))
5036 DECODE_COMPOSITION_END ('1');
5040 dst
+= CHAR_STRING (c
, dst
);
5041 coding
->produced_char
++;
5043 coding
->consumed
= coding
->consumed_char
= src
- source
;
5044 coding
->produced
= dst
- destination
;
5045 coding
->result
= CODING_FINISH_NORMAL
;
5048 if (!coding
->dst_multibyte
)
5050 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
5051 coding
->produced_char
= coding
->produced
;
5054 return coding
->result
;
5057 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
5058 multibyteness of the source is CODING->src_multibyte, the
5059 multibyteness of the result is always unibyte. */
5062 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
5063 struct coding_system
*coding
;
5064 const unsigned char *source
;
5065 unsigned char *destination
;
5066 int src_bytes
, dst_bytes
;
5068 coding
->produced
= coding
->produced_char
= 0;
5069 coding
->consumed
= coding
->consumed_char
= 0;
5071 coding
->result
= CODING_FINISH_NORMAL
;
5072 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5073 coding
->eol_type
= CODING_EOL_LF
;
5075 switch (coding
->type
)
5077 case coding_type_sjis
:
5078 encode_coding_sjis_big5 (coding
, source
, destination
,
5079 src_bytes
, dst_bytes
, 1);
5082 case coding_type_iso2022
:
5083 encode_coding_iso2022 (coding
, source
, destination
,
5084 src_bytes
, dst_bytes
);
5087 case coding_type_big5
:
5088 encode_coding_sjis_big5 (coding
, source
, destination
,
5089 src_bytes
, dst_bytes
, 0);
5092 case coding_type_emacs_mule
:
5093 encode_coding_emacs_mule (coding
, source
, destination
,
5094 src_bytes
, dst_bytes
);
5097 case coding_type_ccl
:
5098 ccl_coding_driver (coding
, source
, destination
,
5099 src_bytes
, dst_bytes
, 1);
5103 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
5106 if (coding
->mode
& CODING_MODE_LAST_BLOCK
5107 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
5109 const unsigned char *src
= source
+ coding
->consumed
;
5110 unsigned char *dst
= destination
+ coding
->produced
;
5112 if (coding
->type
== coding_type_iso2022
)
5113 ENCODE_RESET_PLANE_AND_REGISTER
;
5114 if (COMPOSING_P (coding
))
5115 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
5116 if (coding
->consumed
< src_bytes
)
5118 int len
= src_bytes
- coding
->consumed
;
5120 BCOPY_SHORT (src
, dst
, len
);
5121 if (coding
->src_multibyte
)
5122 len
= str_as_unibyte (dst
, len
);
5124 coding
->consumed
= src_bytes
;
5126 coding
->produced
= coding
->produced_char
= dst
- destination
;
5127 coding
->result
= CODING_FINISH_NORMAL
;
5130 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
5131 && coding
->consumed
== src_bytes
)
5132 coding
->result
= CODING_FINISH_NORMAL
;
5134 return coding
->result
;
5137 /* Scan text in the region between *BEG and *END (byte positions),
5138 skip characters which we don't have to decode by coding system
5139 CODING at the head and tail, then set *BEG and *END to the region
5140 of the text we actually have to convert. The caller should move
5141 the gap out of the region in advance if the region is from a
5144 If STR is not NULL, *BEG and *END are indices into STR. */
5147 shrink_decoding_region (beg
, end
, coding
, str
)
5149 struct coding_system
*coding
;
5152 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
5154 Lisp_Object translation_table
;
5156 if (coding
->type
== coding_type_ccl
5157 || coding
->type
== coding_type_undecided
5158 || coding
->eol_type
!= CODING_EOL_LF
5159 || !NILP (coding
->post_read_conversion
)
5160 || coding
->composing
!= COMPOSITION_DISABLED
)
5162 /* We can't skip any data. */
5165 if (coding
->type
== coding_type_no_conversion
5166 || coding
->type
== coding_type_raw_text
5167 || coding
->type
== coding_type_emacs_mule
)
5169 /* We need no conversion, but don't have to skip any data here.
5170 Decoding routine handles them effectively anyway. */
5174 translation_table
= coding
->translation_table_for_decode
;
5175 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5176 translation_table
= Vstandard_translation_table_for_decode
;
5177 if (CHAR_TABLE_P (translation_table
))
5180 for (i
= 0; i
< 128; i
++)
5181 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5184 /* Some ASCII character should be translated. We give up
5189 if (coding
->heading_ascii
>= 0)
5190 /* Detection routine has already found how much we can skip at the
5192 *beg
+= coding
->heading_ascii
;
5196 begp_orig
= begp
= str
+ *beg
;
5197 endp_orig
= endp
= str
+ *end
;
5201 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5202 endp_orig
= endp
= begp
+ *end
- *beg
;
5205 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5206 || coding
->eol_type
== CODING_EOL_CRLF
);
5208 switch (coding
->type
)
5210 case coding_type_sjis
:
5211 case coding_type_big5
:
5212 /* We can skip all ASCII characters at the head. */
5213 if (coding
->heading_ascii
< 0)
5216 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
5218 while (begp
< endp
&& *begp
< 0x80) begp
++;
5220 /* We can skip all ASCII characters at the tail except for the
5221 second byte of SJIS or BIG5 code. */
5223 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
5225 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5226 /* Do not consider LF as ascii if preceded by CR, since that
5227 confuses eol decoding. */
5228 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5230 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
5234 case coding_type_iso2022
:
5235 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5236 /* We can't skip any data. */
5238 if (coding
->heading_ascii
< 0)
5240 /* We can skip all ASCII characters at the head except for a
5241 few control codes. */
5242 while (begp
< endp
&& (c
= *begp
) < 0x80
5243 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
5244 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
5245 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
5248 switch (coding
->category_idx
)
5250 case CODING_CATEGORY_IDX_ISO_8_1
:
5251 case CODING_CATEGORY_IDX_ISO_8_2
:
5252 /* We can skip all ASCII characters at the tail. */
5254 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
5256 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5257 /* Do not consider LF as ascii if preceded by CR, since that
5258 confuses eol decoding. */
5259 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5263 case CODING_CATEGORY_IDX_ISO_7
:
5264 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
5266 /* We can skip all characters at the tail except for 8-bit
5267 codes and ESC and the following 2-byte at the tail. */
5268 unsigned char *eight_bit
= NULL
;
5272 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
5274 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5279 && (c
= endp
[-1]) != ISO_CODE_ESC
)
5281 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5284 /* Do not consider LF as ascii if preceded by CR, since that
5285 confuses eol decoding. */
5286 if (begp
< endp
&& endp
< endp_orig
5287 && endp
[-1] == '\r' && endp
[0] == '\n')
5289 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
5291 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
5292 /* This is an ASCII designation sequence. We can
5293 surely skip the tail. But, if we have
5294 encountered an 8-bit code, skip only the codes
5296 endp
= eight_bit
? eight_bit
: endp
+ 2;
5298 /* Hmmm, we can't skip the tail. */
5310 *beg
+= begp
- begp_orig
;
5311 *end
+= endp
- endp_orig
;
5315 /* Like shrink_decoding_region but for encoding. */
5318 shrink_encoding_region (beg
, end
, coding
, str
)
5320 struct coding_system
*coding
;
5323 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
5325 Lisp_Object translation_table
;
5327 if (coding
->type
== coding_type_ccl
5328 || coding
->eol_type
== CODING_EOL_CRLF
5329 || coding
->eol_type
== CODING_EOL_CR
5330 || (coding
->cmp_data
&& coding
->cmp_data
->used
> 0))
5332 /* We can't skip any data. */
5335 if (coding
->type
== coding_type_no_conversion
5336 || coding
->type
== coding_type_raw_text
5337 || coding
->type
== coding_type_emacs_mule
5338 || coding
->type
== coding_type_undecided
)
5340 /* We need no conversion, but don't have to skip any data here.
5341 Encoding routine handles them effectively anyway. */
5345 translation_table
= coding
->translation_table_for_encode
;
5346 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5347 translation_table
= Vstandard_translation_table_for_encode
;
5348 if (CHAR_TABLE_P (translation_table
))
5351 for (i
= 0; i
< 128; i
++)
5352 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5355 /* Some ASCII character should be translated. We give up
5362 begp_orig
= begp
= str
+ *beg
;
5363 endp_orig
= endp
= str
+ *end
;
5367 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5368 endp_orig
= endp
= begp
+ *end
- *beg
;
5371 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5372 || coding
->eol_type
== CODING_EOL_CRLF
);
5374 /* Here, we don't have to check coding->pre_write_conversion because
5375 the caller is expected to have handled it already. */
5376 switch (coding
->type
)
5378 case coding_type_iso2022
:
5379 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5380 /* We can't skip any data. */
5382 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
5384 unsigned char *bol
= begp
;
5385 while (begp
< endp
&& *begp
< 0x80)
5388 if (begp
[-1] == '\n')
5392 goto label_skip_tail
;
5396 case coding_type_sjis
:
5397 case coding_type_big5
:
5398 /* We can skip all ASCII characters at the head and tail. */
5400 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
5402 while (begp
< endp
&& *begp
< 0x80) begp
++;
5405 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
5407 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
5414 *beg
+= begp
- begp_orig
;
5415 *end
+= endp
- endp_orig
;
5419 /* As shrinking conversion region requires some overhead, we don't try
5420 shrinking if the length of conversion region is less than this
5422 static int shrink_conversion_region_threshhold
= 1024;
5424 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5426 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5428 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5429 else shrink_decoding_region (beg, end, coding, str); \
5433 /* ARG is (CODING BUFFER ...) where CODING is what to be set in
5434 Vlast_coding_system_used and the remaining elements are buffers to
5437 code_convert_region_unwind (arg
)
5440 struct gcpro gcpro1
;
5443 inhibit_pre_post_conversion
= 0;
5444 Vlast_coding_system_used
= XCAR (arg
);
5445 for (arg
= XCDR (arg
); CONSP (arg
); arg
= XCDR (arg
))
5446 Fkill_buffer (XCAR (arg
));
5452 /* Store information about all compositions in the range FROM and TO
5453 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5454 buffer or a string, defaults to the current buffer. */
5457 coding_save_composition (coding
, from
, to
, obj
)
5458 struct coding_system
*coding
;
5465 if (coding
->composing
== COMPOSITION_DISABLED
)
5467 if (!coding
->cmp_data
)
5468 coding_allocate_composition_data (coding
, from
);
5469 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
5473 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
5476 coding
->composing
= COMPOSITION_NO
;
5479 if (COMPOSITION_VALID_P (start
, end
, prop
))
5481 enum composition_method method
= COMPOSITION_METHOD (prop
);
5482 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
5483 >= COMPOSITION_DATA_SIZE
)
5484 coding_allocate_composition_data (coding
, from
);
5485 /* For relative composition, we remember start and end
5486 positions, for the other compositions, we also remember
5488 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
5489 if (method
!= COMPOSITION_RELATIVE
)
5491 /* We must store a*/
5492 Lisp_Object val
, ch
;
5494 val
= COMPOSITION_COMPONENTS (prop
);
5498 ch
= XCAR (val
), val
= XCDR (val
);
5499 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5501 else if (VECTORP (val
) || STRINGP (val
))
5503 int len
= (VECTORP (val
)
5504 ? XVECTOR (val
)->size
: SCHARS (val
));
5506 for (i
= 0; i
< len
; i
++)
5509 ? Faref (val
, make_number (i
))
5510 : XVECTOR (val
)->contents
[i
]);
5511 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5514 else /* INTEGERP (val) */
5515 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
5517 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
5522 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
5525 /* Make coding->cmp_data point to the first memory block. */
5526 while (coding
->cmp_data
->prev
)
5527 coding
->cmp_data
= coding
->cmp_data
->prev
;
5528 coding
->cmp_data_start
= 0;
5531 /* Reflect the saved information about compositions to OBJ.
5532 CODING->cmp_data points to a memory block for the information. OBJ
5533 is a buffer or a string, defaults to the current buffer. */
5536 coding_restore_composition (coding
, obj
)
5537 struct coding_system
*coding
;
5540 struct composition_data
*cmp_data
= coding
->cmp_data
;
5545 while (cmp_data
->prev
)
5546 cmp_data
= cmp_data
->prev
;
5552 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
5553 i
+= cmp_data
->data
[i
])
5555 int *data
= cmp_data
->data
+ i
;
5556 enum composition_method method
= (enum composition_method
) data
[3];
5557 Lisp_Object components
;
5559 if (data
[0] < 0 || i
+ data
[0] > cmp_data
->used
)
5560 /* Invalid composition data. */
5563 if (method
== COMPOSITION_RELATIVE
)
5567 int len
= data
[0] - 4, j
;
5568 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
5570 if (method
== COMPOSITION_WITH_RULE_ALTCHARS
5574 /* Invalid composition data. */
5576 for (j
= 0; j
< len
; j
++)
5577 args
[j
] = make_number (data
[4 + j
]);
5578 components
= (method
== COMPOSITION_WITH_ALTCHARS
5579 ? Fstring (len
, args
)
5580 : Fvector (len
, args
));
5582 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
5584 cmp_data
= cmp_data
->next
;
5588 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
5589 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5590 coding system CODING, and return the status code of code conversion
5591 (currently, this value has no meaning).
5593 How many characters (and bytes) are converted to how many
5594 characters (and bytes) are recorded in members of the structure
5597 If REPLACE is nonzero, we do various things as if the original text
5598 is deleted and a new text is inserted. See the comments in
5599 replace_range (insdel.c) to know what we are doing.
5601 If REPLACE is zero, it is assumed that the source text is unibyte.
5602 Otherwise, it is assumed that the source text is multibyte. */
5605 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
5606 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
5607 struct coding_system
*coding
;
5609 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
5610 int nchars_del
= 0, nbytes_del
= 0;
5611 int require
, inserted
, inserted_byte
;
5612 int head_skip
, tail_skip
, total_skip
= 0;
5613 Lisp_Object saved_coding_symbol
;
5615 unsigned char *src
, *dst
;
5616 Lisp_Object deletion
;
5617 int orig_point
= PT
, orig_len
= len
;
5619 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
5622 saved_coding_symbol
= coding
->symbol
;
5624 if (from
< PT
&& PT
< to
)
5626 TEMP_SET_PT_BOTH (from
, from_byte
);
5632 int saved_from
= from
;
5633 int saved_inhibit_modification_hooks
;
5635 prepare_to_modify_buffer (from
, to
, &from
);
5636 if (saved_from
!= from
)
5639 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
5640 len_byte
= to_byte
- from_byte
;
5643 /* The code conversion routine can not preserve text properties
5644 for now. So, we must remove all text properties in the
5645 region. Here, we must suppress all modification hooks. */
5646 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
5647 inhibit_modification_hooks
= 1;
5648 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
5649 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
5652 coding
->heading_ascii
= 0;
5654 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
5656 /* We must detect encoding of text and eol format. */
5658 if (from
< GPT
&& to
> GPT
)
5659 move_gap_both (from
, from_byte
);
5660 if (coding
->type
== coding_type_undecided
)
5662 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5663 if (coding
->type
== coding_type_undecided
)
5665 /* It seems that the text contains only ASCII, but we
5666 should not leave it undecided because the deeper
5667 decoding routine (decode_coding) tries to detect the
5668 encodings again in vain. */
5669 coding
->type
= coding_type_emacs_mule
;
5670 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5671 /* As emacs-mule decoder will handle composition, we
5672 need this setting to allocate coding->cmp_data
5674 coding
->composing
= COMPOSITION_NO
;
5677 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5678 && coding
->type
!= coding_type_ccl
)
5680 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5681 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5682 coding
->eol_type
= CODING_EOL_LF
;
5683 /* We had better recover the original eol format if we
5684 encounter an inconsistent eol format while decoding. */
5685 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5689 /* Now we convert the text. */
5691 /* For encoding, we must process pre-write-conversion in advance. */
5692 if (! inhibit_pre_post_conversion
5694 && SYMBOLP (coding
->pre_write_conversion
)
5695 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
5697 /* The function in pre-write-conversion may put a new text in a
5699 struct buffer
*prev
= current_buffer
;
5702 record_unwind_protect (code_convert_region_unwind
,
5703 Fcons (Vlast_coding_system_used
, Qnil
));
5704 /* We should not call any more pre-write/post-read-conversion
5705 functions while this pre-write-conversion is running. */
5706 inhibit_pre_post_conversion
= 1;
5707 call2 (coding
->pre_write_conversion
,
5708 make_number (from
), make_number (to
));
5709 inhibit_pre_post_conversion
= 0;
5710 /* Discard the unwind protect. */
5713 if (current_buffer
!= prev
)
5716 new = Fcurrent_buffer ();
5717 set_buffer_internal_1 (prev
);
5718 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
5719 TEMP_SET_PT_BOTH (from
, from_byte
);
5720 insert_from_buffer (XBUFFER (new), 1, len
, 0);
5722 if (orig_point
>= to
)
5723 orig_point
+= len
- orig_len
;
5724 else if (orig_point
> from
)
5728 from_byte
= CHAR_TO_BYTE (from
);
5729 to_byte
= CHAR_TO_BYTE (to
);
5730 len_byte
= to_byte
- from_byte
;
5731 TEMP_SET_PT_BOTH (from
, from_byte
);
5737 if (! EQ (current_buffer
->undo_list
, Qt
))
5738 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
5741 nchars_del
= to
- from
;
5742 nbytes_del
= to_byte
- from_byte
;
5746 if (coding
->composing
!= COMPOSITION_DISABLED
)
5749 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
5751 coding_allocate_composition_data (coding
, from
);
5754 /* Try to skip the heading and tailing ASCIIs. We can't skip them
5755 if we must run CCL program or there are compositions to
5757 if (coding
->type
!= coding_type_ccl
5758 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
5760 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
5762 if (from
< GPT
&& GPT
< to
)
5763 move_gap_both (from
, from_byte
);
5764 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
5765 if (from_byte
== to_byte
5766 && (encodep
|| NILP (coding
->post_read_conversion
))
5767 && ! CODING_REQUIRE_FLUSHING (coding
))
5769 coding
->produced
= len_byte
;
5770 coding
->produced_char
= len
;
5772 /* We must record and adjust for this new text now. */
5773 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
5774 coding_free_composition_data (coding
);
5778 head_skip
= from_byte
- from_byte_orig
;
5779 tail_skip
= to_byte_orig
- to_byte
;
5780 total_skip
= head_skip
+ tail_skip
;
5783 len
-= total_skip
; len_byte
-= total_skip
;
5786 /* For conversion, we must put the gap before the text in addition to
5787 making the gap larger for efficient decoding. The required gap
5788 size starts from 2000 which is the magic number used in make_gap.
5789 But, after one batch of conversion, it will be incremented if we
5790 find that it is not enough . */
5793 if (GAP_SIZE
< require
)
5794 make_gap (require
- GAP_SIZE
);
5795 move_gap_both (from
, from_byte
);
5797 inserted
= inserted_byte
= 0;
5799 GAP_SIZE
+= len_byte
;
5802 ZV_BYTE
-= len_byte
;
5805 if (GPT
- BEG
< BEG_UNCHANGED
)
5806 BEG_UNCHANGED
= GPT
- BEG
;
5807 if (Z
- GPT
< END_UNCHANGED
)
5808 END_UNCHANGED
= Z
- GPT
;
5810 if (!encodep
&& coding
->src_multibyte
)
5812 /* Decoding routines expects that the source text is unibyte.
5813 We must convert 8-bit characters of multibyte form to
5815 int len_byte_orig
= len_byte
;
5816 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
5817 if (len_byte
< len_byte_orig
)
5818 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
5820 coding
->src_multibyte
= 0;
5827 /* The buffer memory is now:
5828 +--------+converted-text+---------+-------original-text-------+---+
5829 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5830 |<---------------------- GAP ----------------------->| */
5831 src
= GAP_END_ADDR
- len_byte
;
5832 dst
= GPT_ADDR
+ inserted_byte
;
5835 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
5838 if (coding
->composing
!= COMPOSITION_DISABLED
)
5839 coding
->cmp_data
->char_offset
= from
+ inserted
;
5840 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
5843 /* The buffer memory is now:
5844 +--------+-------converted-text----+--+------original-text----+---+
5845 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5846 |<---------------------- GAP ----------------------->| */
5848 inserted
+= coding
->produced_char
;
5849 inserted_byte
+= coding
->produced
;
5850 len_byte
-= coding
->consumed
;
5852 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5854 coding_allocate_composition_data (coding
, from
+ inserted
);
5858 src
+= coding
->consumed
;
5859 dst
+= coding
->produced
;
5861 if (result
== CODING_FINISH_NORMAL
)
5866 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
5868 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
5869 Lisp_Object eol_type
;
5871 /* Encode LFs back to the original eol format (CR or CRLF). */
5872 if (coding
->eol_type
== CODING_EOL_CR
)
5874 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
5880 while (p
< pend
) if (*p
++ == '\n') count
++;
5881 if (src
- dst
< count
)
5883 /* We don't have sufficient room for encoding LFs
5884 back to CRLF. We must record converted and
5885 not-yet-converted text back to the buffer
5886 content, enlarge the gap, then record them out of
5887 the buffer contents again. */
5888 int add
= len_byte
+ inserted_byte
;
5891 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5892 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5893 make_gap (count
- GAP_SIZE
);
5895 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5896 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5897 /* Don't forget to update SRC, DST, and PEND. */
5898 src
= GAP_END_ADDR
- len_byte
;
5899 dst
= GPT_ADDR
+ inserted_byte
;
5903 inserted_byte
+= count
;
5904 coding
->produced
+= count
;
5905 p
= dst
= pend
+ count
;
5909 if (*p
== '\n') count
--, *--p
= '\r';
5913 /* Suppress eol-format conversion in the further conversion. */
5914 coding
->eol_type
= CODING_EOL_LF
;
5916 /* Set the coding system symbol to that for Unix-like EOL. */
5917 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5918 if (VECTORP (eol_type
)
5919 && XVECTOR (eol_type
)->size
== 3
5920 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
5921 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5923 coding
->symbol
= saved_coding_symbol
;
5929 if (coding
->type
!= coding_type_ccl
5930 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5932 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5935 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5937 /* The source text ends in invalid codes. Let's just
5938 make them valid buffer contents, and finish conversion. */
5941 unsigned char *start
= dst
;
5943 inserted
+= len_byte
;
5947 dst
+= CHAR_STRING (c
, dst
);
5950 inserted_byte
+= dst
- start
;
5954 inserted
+= len_byte
;
5955 inserted_byte
+= len_byte
;
5961 if (result
== CODING_FINISH_INTERRUPT
)
5963 /* The conversion procedure was interrupted by a user. */
5966 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5967 if (coding
->consumed
< 1)
5969 /* It's quite strange to require more memory without
5970 consuming any bytes. Perhaps CCL program bug. */
5975 /* We have just done the first batch of conversion which was
5976 stopped because of insufficient gap. Let's reconsider the
5977 required gap size (i.e. SRT - DST) now.
5979 We have converted ORIG bytes (== coding->consumed) into
5980 NEW bytes (coding->produced). To convert the remaining
5981 LEN bytes, we may need REQUIRE bytes of gap, where:
5982 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5983 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5984 Here, we are sure that NEW >= ORIG. */
5986 if (coding
->produced
<= coding
->consumed
)
5988 /* This happens because of CCL-based coding system with
5994 float ratio
= coding
->produced
- coding
->consumed
;
5995 ratio
/= coding
->consumed
;
5996 require
= len_byte
* ratio
;
6000 if ((src
- dst
) < (require
+ 2000))
6002 /* See the comment above the previous call of make_gap. */
6003 int add
= len_byte
+ inserted_byte
;
6006 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
6007 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
6008 make_gap (require
+ 2000);
6010 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
6011 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
6014 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
6016 if (encodep
&& coding
->dst_multibyte
)
6018 /* The output is unibyte. We must convert 8-bit characters to
6020 if (inserted_byte
* 2 > GAP_SIZE
)
6022 GAP_SIZE
-= inserted_byte
;
6023 ZV
+= inserted_byte
; Z
+= inserted_byte
;
6024 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
6025 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
6026 make_gap (inserted_byte
- GAP_SIZE
);
6027 GAP_SIZE
+= inserted_byte
;
6028 ZV
-= inserted_byte
; Z
-= inserted_byte
;
6029 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
6030 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
6032 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
6035 /* If we shrank the conversion area, adjust it now. */
6039 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
6040 inserted
+= total_skip
; inserted_byte
+= total_skip
;
6041 GAP_SIZE
+= total_skip
;
6042 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
6043 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
6044 Z
-= total_skip
; Z_BYTE
-= total_skip
;
6045 from
-= head_skip
; from_byte
-= head_skip
;
6046 to
+= tail_skip
; to_byte
+= tail_skip
;
6050 if (! EQ (current_buffer
->undo_list
, Qt
))
6051 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
6053 adjust_after_replace_noundo (from
, from_byte
, nchars_del
, nbytes_del
,
6054 inserted
, inserted_byte
);
6055 inserted
= Z
- prev_Z
;
6057 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
6058 coding_restore_composition (coding
, Fcurrent_buffer ());
6059 coding_free_composition_data (coding
);
6061 if (! inhibit_pre_post_conversion
6062 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
6065 Lisp_Object saved_coding_system
;
6068 TEMP_SET_PT_BOTH (from
, from_byte
);
6070 record_unwind_protect (code_convert_region_unwind
,
6071 Fcons (Vlast_coding_system_used
, Qnil
));
6072 saved_coding_system
= Vlast_coding_system_used
;
6073 Vlast_coding_system_used
= coding
->symbol
;
6074 /* We should not call any more pre-write/post-read-conversion
6075 functions while this post-read-conversion is running. */
6076 inhibit_pre_post_conversion
= 1;
6077 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
6078 inhibit_pre_post_conversion
= 0;
6079 coding
->symbol
= Vlast_coding_system_used
;
6080 Vlast_coding_system_used
= saved_coding_system
;
6081 /* Discard the unwind protect. */
6084 inserted
+= Z
- prev_Z
;
6087 if (orig_point
>= from
)
6089 if (orig_point
>= from
+ orig_len
)
6090 orig_point
+= inserted
- orig_len
;
6093 TEMP_SET_PT (orig_point
);
6098 signal_after_change (from
, to
- from
, inserted
);
6099 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
6103 coding
->consumed
= to_byte
- from_byte
;
6104 coding
->consumed_char
= to
- from
;
6105 coding
->produced
= inserted_byte
;
6106 coding
->produced_char
= inserted
;
6112 /* Name (or base name) of work buffer for code conversion. */
6113 static Lisp_Object Vcode_conversion_workbuf_name
;
6115 /* Set the current buffer to the working buffer prepared for
6116 code-conversion. MULTIBYTE specifies the multibyteness of the
6117 buffer. Return the buffer we set if it must be killed after use.
6118 Otherwise return Qnil. */
6121 set_conversion_work_buffer (multibyte
)
6124 Lisp_Object buffer
, buffer_to_kill
;
6127 buffer
= Fget_buffer_create (Vcode_conversion_workbuf_name
);
6128 buf
= XBUFFER (buffer
);
6129 if (buf
== current_buffer
)
6131 /* As we are already in the work buffer, we must generate a new
6132 buffer for the work. */
6135 name
= Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name
, Qnil
);
6136 buffer
= buffer_to_kill
= Fget_buffer_create (name
);
6137 buf
= XBUFFER (buffer
);
6140 buffer_to_kill
= Qnil
;
6142 delete_all_overlays (buf
);
6143 buf
->directory
= current_buffer
->directory
;
6144 buf
->read_only
= Qnil
;
6145 buf
->filename
= Qnil
;
6146 buf
->undo_list
= Qt
;
6147 eassert (buf
->overlays_before
== NULL
);
6148 eassert (buf
->overlays_after
== NULL
);
6149 set_buffer_internal (buf
);
6150 if (BEG
!= BEGV
|| Z
!= ZV
)
6152 del_range_2 (BEG
, BEG_BYTE
, Z
, Z_BYTE
, 0);
6153 buf
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
6154 return buffer_to_kill
;
6158 run_pre_post_conversion_on_str (str
, coding
, encodep
)
6160 struct coding_system
*coding
;
6163 int count
= SPECPDL_INDEX ();
6164 struct gcpro gcpro1
, gcpro2
;
6165 int multibyte
= STRING_MULTIBYTE (str
);
6166 Lisp_Object old_deactivate_mark
;
6167 Lisp_Object buffer_to_kill
;
6168 Lisp_Object unwind_arg
;
6170 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
6171 /* It is not crucial to specbind this. */
6172 old_deactivate_mark
= Vdeactivate_mark
;
6173 GCPRO2 (str
, old_deactivate_mark
);
6175 /* We must insert the contents of STR as is without
6176 unibyte<->multibyte conversion. For that, we adjust the
6177 multibyteness of the working buffer to that of STR. */
6178 buffer_to_kill
= set_conversion_work_buffer (multibyte
);
6179 if (NILP (buffer_to_kill
))
6180 unwind_arg
= Fcons (Vlast_coding_system_used
, Qnil
);
6182 unwind_arg
= list2 (Vlast_coding_system_used
, buffer_to_kill
);
6183 record_unwind_protect (code_convert_region_unwind
, unwind_arg
);
6185 insert_from_string (str
, 0, 0,
6186 SCHARS (str
), SBYTES (str
), 0);
6188 inhibit_pre_post_conversion
= 1;
6191 struct buffer
*prev
= current_buffer
;
6193 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
6194 if (prev
!= current_buffer
)
6195 /* We must kill the current buffer too. */
6196 Fsetcdr (unwind_arg
, Fcons (Fcurrent_buffer (), XCDR (unwind_arg
)));
6200 Vlast_coding_system_used
= coding
->symbol
;
6201 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
6202 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
6203 coding
->symbol
= Vlast_coding_system_used
;
6205 inhibit_pre_post_conversion
= 0;
6206 Vdeactivate_mark
= old_deactivate_mark
;
6207 str
= make_buffer_string (BEG
, Z
, 1);
6208 return unbind_to (count
, str
);
6212 /* Run pre-write-conversion function of CODING on NCHARS/NBYTES
6213 text in *STR. *SIZE is the allocated bytes for STR. As it
6214 is intended that this function is called from encode_terminal_code,
6215 the pre-write-conversion function is run by safe_call and thus
6216 "Error during redisplay: ..." is logged when an error occurs.
6218 Store the resulting text in *STR and set CODING->produced_char and
6219 CODING->produced to the number of characters and bytes
6220 respectively. If the size of *STR is too small, enlarge it by
6221 xrealloc and update *STR and *SIZE. */
6224 run_pre_write_conversin_on_c_str (str
, size
, nchars
, nbytes
, coding
)
6225 unsigned char **str
;
6226 int *size
, nchars
, nbytes
;
6227 struct coding_system
*coding
;
6229 struct gcpro gcpro1
, gcpro2
;
6230 struct buffer
*cur
= current_buffer
;
6231 struct buffer
*prev
;
6232 Lisp_Object old_deactivate_mark
, old_last_coding_system_used
;
6233 Lisp_Object args
[3];
6234 Lisp_Object buffer_to_kill
;
6236 /* It is not crucial to specbind this. */
6237 old_deactivate_mark
= Vdeactivate_mark
;
6238 old_last_coding_system_used
= Vlast_coding_system_used
;
6239 GCPRO2 (old_deactivate_mark
, old_last_coding_system_used
);
6241 /* We must insert the contents of STR as is without
6242 unibyte<->multibyte conversion. For that, we adjust the
6243 multibyteness of the working buffer to that of STR. */
6244 buffer_to_kill
= set_conversion_work_buffer (coding
->src_multibyte
);
6245 insert_1_both (*str
, nchars
, nbytes
, 0, 0, 0);
6247 inhibit_pre_post_conversion
= 1;
6248 prev
= current_buffer
;
6249 args
[0] = coding
->pre_write_conversion
;
6250 args
[1] = make_number (BEG
);
6251 args
[2] = make_number (Z
);
6252 safe_call (3, args
);
6253 inhibit_pre_post_conversion
= 0;
6254 Vdeactivate_mark
= old_deactivate_mark
;
6255 Vlast_coding_system_used
= old_last_coding_system_used
;
6256 coding
->produced_char
= Z
- BEG
;
6257 coding
->produced
= Z_BYTE
- BEG_BYTE
;
6258 if (coding
->produced
> *size
)
6260 *size
= coding
->produced
;
6261 *str
= xrealloc (*str
, *size
);
6263 if (BEG
< GPT
&& GPT
< Z
)
6265 bcopy (BEG_ADDR
, *str
, coding
->produced
);
6266 coding
->src_multibyte
6267 = ! NILP (current_buffer
->enable_multibyte_characters
);
6268 if (prev
!= current_buffer
)
6269 Fkill_buffer (Fcurrent_buffer ());
6270 set_buffer_internal (cur
);
6271 if (! NILP (buffer_to_kill
))
6272 Fkill_buffer (buffer_to_kill
);
6277 decode_coding_string (str
, coding
, nocopy
)
6279 struct coding_system
*coding
;
6283 struct conversion_buffer buf
;
6285 Lisp_Object saved_coding_symbol
;
6287 int require_decoding
;
6288 int shrinked_bytes
= 0;
6290 int consumed
, consumed_char
, produced
, produced_char
;
6293 to_byte
= SBYTES (str
);
6295 saved_coding_symbol
= coding
->symbol
;
6296 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6297 coding
->dst_multibyte
= 1;
6298 coding
->heading_ascii
= 0;
6300 if (CODING_REQUIRE_DETECTION (coding
))
6302 /* See the comments in code_convert_region. */
6303 if (coding
->type
== coding_type_undecided
)
6305 detect_coding (coding
, SDATA (str
), to_byte
);
6306 if (coding
->type
== coding_type_undecided
)
6308 coding
->type
= coding_type_emacs_mule
;
6309 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
6310 /* As emacs-mule decoder will handle composition, we
6311 need this setting to allocate coding->cmp_data
6313 coding
->composing
= COMPOSITION_NO
;
6316 if (coding
->eol_type
== CODING_EOL_UNDECIDED
6317 && coding
->type
!= coding_type_ccl
)
6319 saved_coding_symbol
= coding
->symbol
;
6320 detect_eol (coding
, SDATA (str
), to_byte
);
6321 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
6322 coding
->eol_type
= CODING_EOL_LF
;
6323 /* We had better recover the original eol format if we
6324 encounter an inconsistent eol format while decoding. */
6325 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
6329 if (coding
->type
== coding_type_no_conversion
6330 || coding
->type
== coding_type_raw_text
)
6331 coding
->dst_multibyte
= 0;
6333 require_decoding
= CODING_REQUIRE_DECODING (coding
);
6335 if (STRING_MULTIBYTE (str
))
6337 /* Decoding routines expect the source text to be unibyte. */
6338 str
= Fstring_as_unibyte (str
);
6339 to_byte
= SBYTES (str
);
6341 coding
->src_multibyte
= 0;
6344 /* Try to skip the heading and tailing ASCIIs. */
6345 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
6347 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6349 if (from
== to_byte
)
6350 require_decoding
= 0;
6351 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6354 if (!require_decoding
6355 && !(SYMBOLP (coding
->post_read_conversion
)
6356 && !NILP (Ffboundp (coding
->post_read_conversion
))))
6358 coding
->consumed
= SBYTES (str
);
6359 coding
->consumed_char
= SCHARS (str
);
6360 if (coding
->dst_multibyte
)
6362 str
= Fstring_as_multibyte (str
);
6365 coding
->produced
= SBYTES (str
);
6366 coding
->produced_char
= SCHARS (str
);
6367 return (nocopy
? str
: Fcopy_sequence (str
));
6370 if (coding
->composing
!= COMPOSITION_DISABLED
)
6371 coding_allocate_composition_data (coding
, from
);
6372 len
= decoding_buffer_size (coding
, to_byte
- from
);
6373 allocate_conversion_buffer (buf
, len
);
6375 consumed
= consumed_char
= produced
= produced_char
= 0;
6378 result
= decode_coding (coding
, SDATA (str
) + from
+ consumed
,
6379 buf
.data
+ produced
, to_byte
- from
- consumed
,
6380 buf
.size
- produced
);
6381 consumed
+= coding
->consumed
;
6382 consumed_char
+= coding
->consumed_char
;
6383 produced
+= coding
->produced
;
6384 produced_char
+= coding
->produced_char
;
6385 if (result
== CODING_FINISH_NORMAL
6386 || result
== CODING_FINISH_INTERRUPT
6387 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6388 && coding
->consumed
== 0))
6390 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
6391 coding_allocate_composition_data (coding
, from
+ produced_char
);
6392 else if (result
== CODING_FINISH_INSUFFICIENT_DST
)
6393 extend_conversion_buffer (&buf
);
6394 else if (result
== CODING_FINISH_INCONSISTENT_EOL
)
6396 Lisp_Object eol_type
;
6398 /* Recover the original EOL format. */
6399 if (coding
->eol_type
== CODING_EOL_CR
)
6402 for (p
= buf
.data
; p
< buf
.data
+ produced
; p
++)
6403 if (*p
== '\n') *p
= '\r';
6405 else if (coding
->eol_type
== CODING_EOL_CRLF
)
6408 unsigned char *p0
, *p1
;
6409 for (p0
= buf
.data
, p1
= p0
+ produced
; p0
< p1
; p0
++)
6410 if (*p0
== '\n') num_eol
++;
6411 if (produced
+ num_eol
>= buf
.size
)
6412 extend_conversion_buffer (&buf
);
6413 for (p0
= buf
.data
+ produced
, p1
= p0
+ num_eol
; p0
> buf
.data
;)
6416 if (*p0
== '\n') *--p1
= '\r';
6418 produced
+= num_eol
;
6419 produced_char
+= num_eol
;
6421 /* Suppress eol-format conversion in the further conversion. */
6422 coding
->eol_type
= CODING_EOL_LF
;
6424 /* Set the coding system symbol to that for Unix-like EOL. */
6425 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
6426 if (VECTORP (eol_type
)
6427 && XVECTOR (eol_type
)->size
== 3
6428 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
6429 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
6431 coding
->symbol
= saved_coding_symbol
;
6437 coding
->consumed
= consumed
;
6438 coding
->consumed_char
= consumed_char
;
6439 coding
->produced
= produced
;
6440 coding
->produced_char
= produced_char
;
6442 if (coding
->dst_multibyte
)
6443 newstr
= make_uninit_multibyte_string (produced_char
+ shrinked_bytes
,
6444 produced
+ shrinked_bytes
);
6446 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6448 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6449 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6450 if (shrinked_bytes
> from
)
6451 STRING_COPYIN (newstr
, from
+ produced
,
6452 SDATA (str
) + to_byte
,
6453 shrinked_bytes
- from
);
6454 free_conversion_buffer (&buf
);
6456 coding
->consumed
+= shrinked_bytes
;
6457 coding
->consumed_char
+= shrinked_bytes
;
6458 coding
->produced
+= shrinked_bytes
;
6459 coding
->produced_char
+= shrinked_bytes
;
6461 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
6462 coding_restore_composition (coding
, newstr
);
6463 coding_free_composition_data (coding
);
6465 if (SYMBOLP (coding
->post_read_conversion
)
6466 && !NILP (Ffboundp (coding
->post_read_conversion
)))
6467 newstr
= run_pre_post_conversion_on_str (newstr
, coding
, 0);
6473 encode_coding_string (str
, coding
, nocopy
)
6475 struct coding_system
*coding
;
6479 struct conversion_buffer buf
;
6480 int from
, to
, to_byte
;
6482 int shrinked_bytes
= 0;
6484 int consumed
, consumed_char
, produced
, produced_char
;
6486 if (SYMBOLP (coding
->pre_write_conversion
)
6487 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
6489 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
6490 /* As STR is just newly generated, we don't have to copy it
6497 to_byte
= SBYTES (str
);
6499 /* Encoding routines determine the multibyteness of the source text
6500 by coding->src_multibyte. */
6501 coding
->src_multibyte
= SCHARS (str
) < SBYTES (str
);
6502 coding
->dst_multibyte
= 0;
6503 if (! CODING_REQUIRE_ENCODING (coding
))
6504 goto no_need_of_encoding
;
6506 if (coding
->composing
!= COMPOSITION_DISABLED
)
6507 coding_save_composition (coding
, from
, to
, str
);
6509 /* Try to skip the heading and tailing ASCIIs. We can't skip them
6510 if we must run CCL program or there are compositions to
6512 coding
->heading_ascii
= 0;
6513 if (coding
->type
!= coding_type_ccl
6514 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
6516 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6518 if (from
== to_byte
)
6520 coding_free_composition_data (coding
);
6521 goto no_need_of_encoding
;
6523 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6526 len
= encoding_buffer_size (coding
, to_byte
- from
);
6527 allocate_conversion_buffer (buf
, len
);
6529 consumed
= consumed_char
= produced
= produced_char
= 0;
6532 result
= encode_coding (coding
, SDATA (str
) + from
+ consumed
,
6533 buf
.data
+ produced
, to_byte
- from
- consumed
,
6534 buf
.size
- produced
);
6535 consumed
+= coding
->consumed
;
6536 consumed_char
+= coding
->consumed_char
;
6537 produced
+= coding
->produced
;
6538 produced_char
+= coding
->produced_char
;
6539 if (result
== CODING_FINISH_NORMAL
6540 || result
== CODING_FINISH_INTERRUPT
6541 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6542 && coding
->consumed
== 0))
6544 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6545 extend_conversion_buffer (&buf
);
6548 coding
->consumed
= consumed
;
6549 coding
->consumed_char
= consumed_char
;
6550 coding
->produced
= produced
;
6551 coding
->produced_char
= produced_char
;
6553 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6555 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6556 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6557 if (shrinked_bytes
> from
)
6558 STRING_COPYIN (newstr
, from
+ produced
,
6559 SDATA (str
) + to_byte
,
6560 shrinked_bytes
- from
);
6562 free_conversion_buffer (&buf
);
6563 coding_free_composition_data (coding
);
6567 no_need_of_encoding
:
6568 coding
->consumed
= SBYTES (str
);
6569 coding
->consumed_char
= SCHARS (str
);
6570 if (STRING_MULTIBYTE (str
))
6573 /* We are sure that STR doesn't contain a multibyte
6575 STRING_SET_UNIBYTE (str
);
6578 str
= Fstring_as_unibyte (str
);
6582 coding
->produced
= SBYTES (str
);
6583 coding
->produced_char
= SCHARS (str
);
6584 return (nocopy
? str
: Fcopy_sequence (str
));
6589 /*** 8. Emacs Lisp library functions ***/
6591 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
6592 doc
: /* Return t if OBJECT is nil or a coding-system.
6593 See the documentation of `make-coding-system' for information
6594 about coding-system objects. */)
6602 if (! NILP (Fget (obj
, Qcoding_system_define_form
)))
6604 /* Get coding-spec vector for OBJ. */
6605 obj
= Fget (obj
, Qcoding_system
);
6606 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
6610 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
6611 Sread_non_nil_coding_system
, 1, 1, 0,
6612 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6619 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6620 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
6622 while (SCHARS (val
) == 0);
6623 return (Fintern (val
, Qnil
));
6626 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
6627 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6628 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM.
6629 Ignores case when completing coding systems (all Emacs coding systems
6630 are lower-case). */)
6631 (prompt
, default_coding_system
)
6632 Lisp_Object prompt
, default_coding_system
;
6635 int count
= SPECPDL_INDEX ();
6637 if (SYMBOLP (default_coding_system
))
6638 default_coding_system
= SYMBOL_NAME (default_coding_system
);
6639 specbind (Qcompletion_ignore_case
, Qt
);
6640 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6641 Qt
, Qnil
, Qcoding_system_history
,
6642 default_coding_system
, Qnil
);
6643 unbind_to (count
, Qnil
);
6644 return (SCHARS (val
) == 0 ? Qnil
: Fintern (val
, Qnil
));
6647 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
6649 doc
: /* Check validity of CODING-SYSTEM.
6650 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6651 It is valid if it is nil or a symbol with a non-nil `coding-system' property.
6652 The value of this property should be a vector of length 5. */)
6654 Lisp_Object coding_system
;
6656 Lisp_Object define_form
;
6658 define_form
= Fget (coding_system
, Qcoding_system_define_form
);
6659 if (! NILP (define_form
))
6661 Fput (coding_system
, Qcoding_system_define_form
, Qnil
);
6662 safe_eval (define_form
);
6664 if (!NILP (Fcoding_system_p (coding_system
)))
6665 return coding_system
;
6666 xsignal1 (Qcoding_system_error
, coding_system
);
6670 detect_coding_system (src
, src_bytes
, highest
, multibytep
)
6671 const unsigned char *src
;
6672 int src_bytes
, highest
;
6675 int coding_mask
, eol_type
;
6676 Lisp_Object val
, tmp
;
6679 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
, multibytep
);
6680 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
6681 if (eol_type
== CODING_EOL_INCONSISTENT
)
6682 eol_type
= CODING_EOL_UNDECIDED
;
6687 if (eol_type
!= CODING_EOL_UNDECIDED
)
6690 val2
= Fget (Qundecided
, Qeol_type
);
6692 val
= XVECTOR (val2
)->contents
[eol_type
];
6694 return (highest
? val
: Fcons (val
, Qnil
));
6697 /* At first, gather possible coding systems in VAL. */
6699 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
6701 Lisp_Object category_val
, category_index
;
6703 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
6704 category_val
= Fsymbol_value (XCAR (tmp
));
6705 if (!NILP (category_val
)
6706 && NATNUMP (category_index
)
6707 && (coding_mask
& (1 << XFASTINT (category_index
))))
6709 val
= Fcons (category_val
, val
);
6715 val
= Fnreverse (val
);
6717 /* Then, replace the elements with subsidiary coding systems. */
6718 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
6720 if (eol_type
!= CODING_EOL_UNDECIDED
6721 && eol_type
!= CODING_EOL_INCONSISTENT
)
6724 eol
= Fget (XCAR (tmp
), Qeol_type
);
6726 XSETCAR (tmp
, XVECTOR (eol
)->contents
[eol_type
]);
6729 return (highest
? XCAR (val
) : val
);
6732 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
6734 doc
: /* Detect how the byte sequence in the region is encoded.
6735 Return a list of possible coding systems used on decoding a byte
6736 sequence containing the bytes in the region between START and END when
6737 the coding system `undecided' is specified. The list is ordered by
6738 priority decided in the current language environment.
6740 If only ASCII characters are found (except for such ISO-2022 control
6741 characters ISO-2022 as ESC), it returns a list of single element
6742 `undecided' or its subsidiary coding system according to a detected
6745 If optional argument HIGHEST is non-nil, return the coding system of
6746 highest priority. */)
6747 (start
, end
, highest
)
6748 Lisp_Object start
, end
, highest
;
6751 int from_byte
, to_byte
;
6752 int include_anchor_byte
= 0;
6754 CHECK_NUMBER_COERCE_MARKER (start
);
6755 CHECK_NUMBER_COERCE_MARKER (end
);
6757 validate_region (&start
, &end
);
6758 from
= XINT (start
), to
= XINT (end
);
6759 from_byte
= CHAR_TO_BYTE (from
);
6760 to_byte
= CHAR_TO_BYTE (to
);
6762 if (from
< GPT
&& to
>= GPT
)
6763 move_gap_both (to
, to_byte
);
6764 /* If we an anchor byte `\0' follows the region, we include it in
6765 the detecting source. Then code detectors can handle the tailing
6766 byte sequence more accurately.
6768 Fix me: This is not a perfect solution. It is better that we
6769 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6771 if (to
== Z
|| (to
== GPT
&& GAP_SIZE
> 0))
6772 include_anchor_byte
= 1;
6773 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
6774 to_byte
- from_byte
+ include_anchor_byte
,
6776 !NILP (current_buffer
6777 ->enable_multibyte_characters
));
6780 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
6782 doc
: /* Detect how the byte sequence in STRING is encoded.
6783 Return a list of possible coding systems used on decoding a byte
6784 sequence containing the bytes in STRING when the coding system
6785 `undecided' is specified. The list is ordered by priority decided in
6786 the current language environment.
6788 If only ASCII characters are found (except for such ISO-2022 control
6789 characters ISO-2022 as ESC), it returns a list of single element
6790 `undecided' or its subsidiary coding system according to a detected
6793 If optional argument HIGHEST is non-nil, return the coding system of
6794 highest priority. */)
6796 Lisp_Object string
, highest
;
6798 CHECK_STRING (string
);
6800 return detect_coding_system (SDATA (string
),
6801 /* "+ 1" is to include the anchor byte
6802 `\0'. With this, code detectors can
6803 handle the tailing bytes more
6805 SBYTES (string
) + 1,
6807 STRING_MULTIBYTE (string
));
6810 /* Subroutine for Ffind_coding_systems_region_internal.
6812 Return a list of coding systems that safely encode the multibyte
6813 text between P and PEND. SAFE_CODINGS, if non-nil, is an alist of
6814 possible coding systems. If it is nil, it means that we have not
6815 yet found any coding systems.
6817 WORK_TABLE a char-table of which element is set to t once the
6818 element is looked up.
6820 If a non-ASCII single byte char is found, set
6821 *single_byte_char_found to 1. */
6824 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
6825 unsigned char *p
, *pend
;
6826 Lisp_Object safe_codings
, work_table
;
6827 int *single_byte_char_found
;
6830 Lisp_Object val
, ch
;
6831 Lisp_Object prev
, tail
;
6833 if (NILP (safe_codings
))
6834 goto done_safe_codings
;
6837 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6839 if (ASCII_BYTE_P (c
))
6840 /* We can ignore ASCII characters here. */
6842 if (SINGLE_BYTE_CHAR_P (c
))
6843 *single_byte_char_found
= 1;
6844 /* Check the safe coding systems for C. */
6845 ch
= make_number (c
);
6846 val
= Faref (work_table
, ch
);
6848 /* This element was already checked. Ignore it. */
6850 /* Remember that we checked this element. */
6851 Faset (work_table
, ch
, Qt
);
6853 for (prev
= tail
= safe_codings
; CONSP (tail
); tail
= XCDR (tail
))
6855 Lisp_Object elt
, translation_table
, hash_table
, accept_latin_extra
;
6859 if (CONSP (XCDR (elt
)))
6861 /* This entry has this format now:
6862 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6863 ACCEPT-LATIN-EXTRA ) */
6865 encodable
= ! NILP (Faref (XCAR (val
), ch
));
6869 translation_table
= XCAR (val
);
6870 hash_table
= XCAR (XCDR (val
));
6871 accept_latin_extra
= XCAR (XCDR (XCDR (val
)));
6876 /* This entry has this format now: ( CODING . SAFE-CHARS) */
6877 encodable
= ! NILP (Faref (XCDR (elt
), ch
));
6880 /* Transform the format to:
6881 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6882 ACCEPT-LATIN-EXTRA ) */
6883 val
= Fget (XCAR (elt
), Qcoding_system
);
6885 = Fplist_get (AREF (val
, 3),
6886 Qtranslation_table_for_encode
);
6887 if (SYMBOLP (translation_table
))
6888 translation_table
= Fget (translation_table
,
6889 Qtranslation_table
);
6891 = (CHAR_TABLE_P (translation_table
)
6892 ? XCHAR_TABLE (translation_table
)->extras
[1]
6895 = ((EQ (AREF (val
, 0), make_number (2))
6896 && VECTORP (AREF (val
, 4)))
6897 ? AREF (AREF (val
, 4), 16)
6899 XSETCAR (tail
, list5 (XCAR (elt
), XCDR (elt
),
6900 translation_table
, hash_table
,
6901 accept_latin_extra
));
6906 && ((CHAR_TABLE_P (translation_table
)
6907 && ! NILP (Faref (translation_table
, ch
)))
6908 || (HASH_TABLE_P (hash_table
)
6909 && ! NILP (Fgethash (ch
, hash_table
, Qnil
)))
6910 || (SINGLE_BYTE_CHAR_P (c
)
6911 && ! NILP (accept_latin_extra
)
6912 && VECTORP (Vlatin_extra_code_table
)
6913 && ! NILP (AREF (Vlatin_extra_code_table
, c
)))))
6919 /* Exclude this coding system from SAFE_CODINGS. */
6920 if (EQ (tail
, safe_codings
))
6922 safe_codings
= XCDR (safe_codings
);
6923 if (NILP (safe_codings
))
6924 goto done_safe_codings
;
6927 XSETCDR (prev
, XCDR (tail
));
6933 /* If the above loop was terminated before P reaches PEND, it means
6934 SAFE_CODINGS was set to nil. If we have not yet found an
6935 non-ASCII single-byte char, check it now. */
6936 if (! *single_byte_char_found
)
6939 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6941 if (! ASCII_BYTE_P (c
)
6942 && SINGLE_BYTE_CHAR_P (c
))
6944 *single_byte_char_found
= 1;
6948 return safe_codings
;
6951 DEFUN ("find-coding-systems-region-internal",
6952 Ffind_coding_systems_region_internal
,
6953 Sfind_coding_systems_region_internal
, 2, 2, 0,
6954 doc
: /* Internal use only. */)
6956 Lisp_Object start
, end
;
6958 Lisp_Object work_table
, safe_codings
;
6959 int non_ascii_p
= 0;
6960 int single_byte_char_found
= 0;
6961 const unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
6963 if (STRINGP (start
))
6965 if (!STRING_MULTIBYTE (start
))
6967 p1
= SDATA (start
), p1end
= p1
+ SBYTES (start
);
6969 if (SCHARS (start
) != SBYTES (start
))
6976 CHECK_NUMBER_COERCE_MARKER (start
);
6977 CHECK_NUMBER_COERCE_MARKER (end
);
6978 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
6979 args_out_of_range (start
, end
);
6980 if (NILP (current_buffer
->enable_multibyte_characters
))
6982 from
= CHAR_TO_BYTE (XINT (start
));
6983 to
= CHAR_TO_BYTE (XINT (end
));
6984 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
6985 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
6989 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
6990 if (XINT (end
) - XINT (start
) != to
- from
)
6996 /* We are sure that the text contains no multibyte character.
6997 Check if it contains eight-bit-graphic. */
6999 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
7002 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
7008 /* The text contains non-ASCII characters. */
7010 work_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
7011 safe_codings
= Fcopy_sequence (XCDR (Vcoding_system_safe_chars
));
7013 safe_codings
= find_safe_codings (p1
, p1end
, safe_codings
, work_table
,
7014 &single_byte_char_found
);
7016 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
7017 &single_byte_char_found
);
7018 if (EQ (safe_codings
, XCDR (Vcoding_system_safe_chars
)))
7022 /* Turn safe_codings to a list of coding systems... */
7025 if (single_byte_char_found
)
7026 /* ... and append these for eight-bit chars. */
7027 val
= Fcons (Qraw_text
,
7028 Fcons (Qemacs_mule
, Fcons (Qno_conversion
, Qnil
)));
7030 /* ... and append generic coding systems. */
7031 val
= Fcopy_sequence (XCAR (Vcoding_system_safe_chars
));
7033 for (; CONSP (safe_codings
); safe_codings
= XCDR (safe_codings
))
7034 val
= Fcons (XCAR (XCAR (safe_codings
)), val
);
7038 return safe_codings
;
7042 /* Search from position POS for such characters that are unencodable
7043 accoding to SAFE_CHARS, and return a list of their positions. P
7044 points where in the memory the character at POS exists. Limit the
7045 search at PEND or when Nth unencodable characters are found.
7047 If SAFE_CHARS is a char table, an element for an unencodable
7050 If SAFE_CHARS is nil, all non-ASCII characters are unencodable.
7052 Otherwise, SAFE_CHARS is t, and only eight-bit-contrl and
7053 eight-bit-graphic characters are unencodable. */
7056 unencodable_char_position (safe_chars
, pos
, p
, pend
, n
)
7057 Lisp_Object safe_chars
;
7059 unsigned char *p
, *pend
;
7062 Lisp_Object pos_list
;
7068 int c
= STRING_CHAR_AND_LENGTH (p
, MAX_MULTIBYTE_LENGTH
, len
);
7071 && (CHAR_TABLE_P (safe_chars
)
7072 ? NILP (CHAR_TABLE_REF (safe_chars
, c
))
7073 : (NILP (safe_chars
) || c
< 256)))
7075 pos_list
= Fcons (make_number (pos
), pos_list
);
7082 return Fnreverse (pos_list
);
7086 DEFUN ("unencodable-char-position", Funencodable_char_position
,
7087 Sunencodable_char_position
, 3, 5, 0,
7089 Return position of first un-encodable character in a region.
7090 START and END specfiy the region and CODING-SYSTEM specifies the
7091 encoding to check. Return nil if CODING-SYSTEM does encode the region.
7093 If optional 4th argument COUNT is non-nil, it specifies at most how
7094 many un-encodable characters to search. In this case, the value is a
7097 If optional 5th argument STRING is non-nil, it is a string to search
7098 for un-encodable characters. In that case, START and END are indexes
7100 (start
, end
, coding_system
, count
, string
)
7101 Lisp_Object start
, end
, coding_system
, count
, string
;
7104 Lisp_Object safe_chars
;
7105 struct coding_system coding
;
7106 Lisp_Object positions
;
7108 unsigned char *p
, *pend
;
7112 validate_region (&start
, &end
);
7113 from
= XINT (start
);
7115 if (NILP (current_buffer
->enable_multibyte_characters
))
7117 p
= CHAR_POS_ADDR (from
);
7121 pend
= CHAR_POS_ADDR (to
);
7125 CHECK_STRING (string
);
7126 CHECK_NATNUM (start
);
7128 from
= XINT (start
);
7131 || to
> SCHARS (string
))
7132 args_out_of_range_3 (string
, start
, end
);
7133 if (! STRING_MULTIBYTE (string
))
7135 p
= SDATA (string
) + string_char_to_byte (string
, from
);
7136 pend
= SDATA (string
) + string_char_to_byte (string
, to
);
7139 setup_coding_system (Fcheck_coding_system (coding_system
), &coding
);
7145 CHECK_NATNUM (count
);
7149 if (coding
.type
== coding_type_no_conversion
7150 || coding
.type
== coding_type_raw_text
)
7153 if (coding
.type
== coding_type_undecided
)
7156 safe_chars
= coding_safe_chars (coding_system
);
7158 if (STRINGP (string
)
7159 || from
>= GPT
|| to
<= GPT
)
7160 positions
= unencodable_char_position (safe_chars
, from
, p
, pend
, n
);
7163 Lisp_Object args
[2];
7165 args
[0] = unencodable_char_position (safe_chars
, from
, p
, GPT_ADDR
, n
);
7166 n
-= XINT (Flength (args
[0]));
7168 positions
= args
[0];
7171 args
[1] = unencodable_char_position (safe_chars
, GPT
, GAP_END_ADDR
,
7173 positions
= Fappend (2, args
);
7177 return (NILP (count
) ? Fcar (positions
) : positions
);
7182 code_convert_region1 (start
, end
, coding_system
, encodep
)
7183 Lisp_Object start
, end
, coding_system
;
7186 struct coding_system coding
;
7189 CHECK_NUMBER_COERCE_MARKER (start
);
7190 CHECK_NUMBER_COERCE_MARKER (end
);
7191 CHECK_SYMBOL (coding_system
);
7193 validate_region (&start
, &end
);
7194 from
= XFASTINT (start
);
7195 to
= XFASTINT (end
);
7197 if (NILP (coding_system
))
7198 return make_number (to
- from
);
7200 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7201 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7203 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7204 coding
.src_multibyte
= coding
.dst_multibyte
7205 = !NILP (current_buffer
->enable_multibyte_characters
);
7206 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
7207 &coding
, encodep
, 1);
7208 Vlast_coding_system_used
= coding
.symbol
;
7209 return make_number (coding
.produced_char
);
7212 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
7213 3, 3, "r\nzCoding system: ",
7214 doc
: /* Decode the current region from the specified coding system.
7215 When called from a program, takes three arguments:
7216 START, END, and CODING-SYSTEM. START and END are buffer positions.
7217 This function sets `last-coding-system-used' to the precise coding system
7218 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7219 not fully specified.)
7220 It returns the length of the decoded text. */)
7221 (start
, end
, coding_system
)
7222 Lisp_Object start
, end
, coding_system
;
7224 return code_convert_region1 (start
, end
, coding_system
, 0);
7227 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
7228 3, 3, "r\nzCoding system: ",
7229 doc
: /* Encode the current region into the specified coding system.
7230 When called from a program, takes three arguments:
7231 START, END, and CODING-SYSTEM. START and END are buffer positions.
7232 This function sets `last-coding-system-used' to the precise coding system
7233 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7234 not fully specified.)
7235 It returns the length of the encoded text. */)
7236 (start
, end
, coding_system
)
7237 Lisp_Object start
, end
, coding_system
;
7239 return code_convert_region1 (start
, end
, coding_system
, 1);
7243 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
7244 Lisp_Object string
, coding_system
, nocopy
;
7247 struct coding_system coding
;
7249 CHECK_STRING (string
);
7250 CHECK_SYMBOL (coding_system
);
7252 if (NILP (coding_system
))
7253 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
7255 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7256 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7258 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7260 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
7261 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
7262 Vlast_coding_system_used
= coding
.symbol
;
7267 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
7269 doc
: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7270 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7271 if the decoding operation is trivial.
7272 This function sets `last-coding-system-used' to the precise coding system
7273 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7274 not fully specified.) */)
7275 (string
, coding_system
, nocopy
)
7276 Lisp_Object string
, coding_system
, nocopy
;
7278 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
7281 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
7283 doc
: /* Encode STRING to CODING-SYSTEM, and return the result.
7284 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7285 if the encoding operation is trivial.
7286 This function sets `last-coding-system-used' to the precise coding system
7287 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7288 not fully specified.) */)
7289 (string
, coding_system
, nocopy
)
7290 Lisp_Object string
, coding_system
, nocopy
;
7292 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
7295 /* Encode or decode STRING according to CODING_SYSTEM.
7296 Do not set Vlast_coding_system_used.
7298 This function is called only from macros DECODE_FILE and
7299 ENCODE_FILE, thus we ignore character composition. */
7302 code_convert_string_norecord (string
, coding_system
, encodep
)
7303 Lisp_Object string
, coding_system
;
7306 struct coding_system coding
;
7308 CHECK_STRING (string
);
7309 CHECK_SYMBOL (coding_system
);
7311 if (NILP (coding_system
))
7314 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7315 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7317 coding
.composing
= COMPOSITION_DISABLED
;
7318 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7320 ? encode_coding_string (string
, &coding
, 1)
7321 : decode_coding_string (string
, &coding
, 1));
7324 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
7325 doc
: /* Decode a Japanese character which has CODE in shift_jis encoding.
7326 Return the corresponding character. */)
7330 unsigned char c1
, c2
, s1
, s2
;
7333 CHECK_NUMBER (code
);
7334 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
7338 XSETFASTINT (val
, s2
);
7339 else if (s2
>= 0xA0 || s2
<= 0xDF)
7340 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
7342 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7346 if ((s1
< 0x80 || (s1
> 0x9F && s1
< 0xE0) || s1
> 0xEF)
7347 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
7348 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7349 DECODE_SJIS (s1
, s2
, c1
, c2
);
7350 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
7355 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
7356 doc
: /* Encode a Japanese character CH to shift_jis encoding.
7357 Return the corresponding code in SJIS. */)
7361 int charset
, c1
, c2
, s1
, s2
;
7365 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7366 if (charset
== CHARSET_ASCII
)
7370 else if (charset
== charset_jisx0208
7371 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
7373 ENCODE_SJIS (c1
, c2
, s1
, s2
);
7374 XSETFASTINT (val
, (s1
<< 8) | s2
);
7376 else if (charset
== charset_katakana_jisx0201
7377 && c1
> 0x20 && c2
< 0xE0)
7379 XSETFASTINT (val
, c1
| 0x80);
7382 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
7386 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
7387 doc
: /* Decode a Big5 character which has CODE in BIG5 coding system.
7388 Return the corresponding character. */)
7393 unsigned char b1
, b2
, c1
, c2
;
7396 CHECK_NUMBER (code
);
7397 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
7401 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7406 if ((b1
< 0xA1 || b1
> 0xFE)
7407 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
7408 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7409 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
7410 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
7415 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
7416 doc
: /* Encode the Big5 character CH to BIG5 coding system.
7417 Return the corresponding character code in Big5. */)
7421 int charset
, c1
, c2
, b1
, b2
;
7425 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7426 if (charset
== CHARSET_ASCII
)
7430 else if ((charset
== charset_big5_1
7431 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
7432 || (charset
== charset_big5_2
7433 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
7435 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
7436 XSETFASTINT (val
, (b1
<< 8) | b2
);
7439 error ("Can't encode to Big5: %d", XFASTINT (ch
));
7443 DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal
,
7444 Sset_terminal_coding_system_internal
, 1, 2, 0,
7445 doc
: /* Internal use only. */)
7446 (coding_system
, terminal
)
7447 Lisp_Object coding_system
;
7448 Lisp_Object terminal
;
7450 struct coding_system
*terminal_coding
= TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1));
7451 CHECK_SYMBOL (coding_system
);
7452 setup_coding_system (Fcheck_coding_system (coding_system
), terminal_coding
);
7453 /* We had better not send unsafe characters to terminal. */
7454 terminal_coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
7455 /* Character composition should be disabled. */
7456 terminal_coding
->composing
= COMPOSITION_DISABLED
;
7457 /* Error notification should be suppressed. */
7458 terminal_coding
->suppress_error
= 1;
7459 terminal_coding
->src_multibyte
= 1;
7460 terminal_coding
->dst_multibyte
= 0;
7464 DEFUN ("set-safe-terminal-coding-system-internal", Fset_safe_terminal_coding_system_internal
,
7465 Sset_safe_terminal_coding_system_internal
, 1, 1, 0,
7466 doc
: /* Internal use only. */)
7468 Lisp_Object coding_system
;
7470 CHECK_SYMBOL (coding_system
);
7471 setup_coding_system (Fcheck_coding_system (coding_system
),
7472 &safe_terminal_coding
);
7473 /* Character composition should be disabled. */
7474 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
7475 /* Error notification should be suppressed. */
7476 safe_terminal_coding
.suppress_error
= 1;
7477 safe_terminal_coding
.src_multibyte
= 1;
7478 safe_terminal_coding
.dst_multibyte
= 0;
7482 DEFUN ("terminal-coding-system", Fterminal_coding_system
,
7483 Sterminal_coding_system
, 0, 1, 0,
7484 doc
: /* Return coding system specified for terminal output on the given terminal.
7485 TERMINAL may be a terminal id, a frame, or nil for the selected
7486 frame's terminal device. */)
7488 Lisp_Object terminal
;
7490 return TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1))->symbol
;
7493 DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal
,
7494 Sset_keyboard_coding_system_internal
, 1, 2, 0,
7495 doc
: /* Internal use only. */)
7496 (coding_system
, terminal
)
7497 Lisp_Object coding_system
;
7498 Lisp_Object terminal
;
7500 struct terminal
*t
= get_terminal (terminal
, 1);
7501 CHECK_SYMBOL (coding_system
);
7503 setup_coding_system (Fcheck_coding_system (coding_system
),
7504 TERMINAL_KEYBOARD_CODING (t
));
7505 /* Character composition should be disabled. */
7506 TERMINAL_KEYBOARD_CODING (t
)->composing
= COMPOSITION_DISABLED
;
7510 DEFUN ("keyboard-coding-system", Fkeyboard_coding_system
,
7511 Skeyboard_coding_system
, 0, 1, 0,
7512 doc
: /* Return coding system for decoding keyboard input on TERMINAL.
7513 TERMINAL may be a terminal id, a frame, or nil for the selected
7514 frame's terminal device. */)
7516 Lisp_Object terminal
;
7518 return TERMINAL_KEYBOARD_CODING (get_terminal (terminal
, 1))->symbol
;
7522 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
7523 Sfind_operation_coding_system
, 1, MANY
, 0,
7524 doc
: /* Choose a coding system for an operation based on the target name.
7525 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
7526 DECODING-SYSTEM is the coding system to use for decoding
7527 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
7528 for encoding (in case OPERATION does encoding).
7530 The first argument OPERATION specifies an I/O primitive:
7531 For file I/O, `insert-file-contents' or `write-region'.
7532 For process I/O, `call-process', `call-process-region', or `start-process'.
7533 For network I/O, `open-network-stream'.
7535 The remaining arguments should be the same arguments that were passed
7536 to the primitive. Depending on which primitive, one of those arguments
7537 is selected as the TARGET. For example, if OPERATION does file I/O,
7538 whichever argument specifies the file name is TARGET.
7540 TARGET has a meaning which depends on OPERATION:
7541 For file I/O, TARGET is a file name (except for the special case below).
7542 For process I/O, TARGET is a process name.
7543 For network I/O, TARGET is a service name or a port number
7545 This function looks up what specified for TARGET in,
7546 `file-coding-system-alist', `process-coding-system-alist',
7547 or `network-coding-system-alist' depending on OPERATION.
7548 They may specify a coding system, a cons of coding systems,
7549 or a function symbol to call.
7550 In the last case, we call the function with one argument,
7551 which is a list of all the arguments given to this function.
7552 If the function can't decide a coding system, it can return
7553 `undecided' so that the normal code-detection is performed.
7555 If OPERATION is `insert-file-contents', the argument corresponding to
7556 TARGET may be a cons (FILENAME . BUFFER). In that case, FILENAME is a
7557 file name to look up, and BUFFER is a buffer that contains the file's
7558 contents (not yet decoded). If `file-coding-system-alist' specifies a
7559 function to call for FILENAME, that function should examine the
7560 contents of BUFFER instead of reading the file.
7562 usage: (find-operation-coding-system OPERATION ARGUMENTS...) */)
7567 Lisp_Object operation
, target_idx
, target
, val
;
7568 register Lisp_Object chain
;
7571 error ("Too few arguments");
7572 operation
= args
[0];
7573 if (!SYMBOLP (operation
)
7574 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
7575 error ("Invalid first argument");
7576 if (nargs
< 1 + XINT (target_idx
))
7577 error ("Too few arguments for operation: %s",
7578 SDATA (SYMBOL_NAME (operation
)));
7579 /* For write-region, if the 6th argument (i.e. VISIT, the 5th
7580 argument to write-region) is string, it must be treated as a
7581 target file name. */
7582 if (EQ (operation
, Qwrite_region
)
7584 && STRINGP (args
[5]))
7585 target_idx
= make_number (4);
7586 target
= args
[XINT (target_idx
) + 1];
7587 if (!(STRINGP (target
)
7588 || (EQ (operation
, Qinsert_file_contents
) && CONSP (target
)
7589 && STRINGP (XCAR (target
)) && BUFFERP (XCDR (target
)))
7590 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
7591 error ("Invalid argument %d", XINT (target_idx
) + 1);
7593 target
= XCAR (target
);
7595 chain
= ((EQ (operation
, Qinsert_file_contents
)
7596 || EQ (operation
, Qwrite_region
))
7597 ? Vfile_coding_system_alist
7598 : (EQ (operation
, Qopen_network_stream
)
7599 ? Vnetwork_coding_system_alist
7600 : Vprocess_coding_system_alist
));
7604 for (; CONSP (chain
); chain
= XCDR (chain
))
7610 && ((STRINGP (target
)
7611 && STRINGP (XCAR (elt
))
7612 && fast_string_match (XCAR (elt
), target
) >= 0)
7613 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
7616 /* Here, if VAL is both a valid coding system and a valid
7617 function symbol, we return VAL as a coding system. */
7620 if (! SYMBOLP (val
))
7622 if (! NILP (Fcoding_system_p (val
)))
7623 return Fcons (val
, val
);
7624 if (! NILP (Ffboundp (val
)))
7626 /* We use call1 rather than safe_call1
7627 so as to get bug reports about functions called here
7628 which don't handle the current interface. */
7629 val
= call1 (val
, Flist (nargs
, args
));
7632 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
7633 return Fcons (val
, val
);
7641 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
7642 Supdate_coding_systems_internal
, 0, 0, 0,
7643 doc
: /* Update internal database for ISO2022 and CCL based coding systems.
7644 When values of any coding categories are changed, you must
7645 call this function. */)
7650 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7654 val
= find_symbol_value (XVECTOR (Vcoding_category_table
)->contents
[i
]);
7657 if (! coding_system_table
[i
])
7658 coding_system_table
[i
] = ((struct coding_system
*)
7659 xmalloc (sizeof (struct coding_system
)));
7660 setup_coding_system (val
, coding_system_table
[i
]);
7662 else if (coding_system_table
[i
])
7664 xfree (coding_system_table
[i
]);
7665 coding_system_table
[i
] = NULL
;
7672 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
7673 Sset_coding_priority_internal
, 0, 0, 0,
7674 doc
: /* Update internal database for the current value of `coding-category-list'.
7675 This function is internal use only. */)
7681 val
= Vcoding_category_list
;
7683 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
7685 if (! SYMBOLP (XCAR (val
)))
7687 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
7688 if (idx
>= CODING_CATEGORY_IDX_MAX
)
7690 coding_priorities
[i
++] = (1 << idx
);
7693 /* If coding-category-list is valid and contains all coding
7694 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
7695 the following code saves Emacs from crashing. */
7696 while (i
< CODING_CATEGORY_IDX_MAX
)
7697 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
7702 DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal
,
7703 Sdefine_coding_system_internal
, 1, 1, 0,
7704 doc
: /* Register CODING-SYSTEM as a base coding system.
7705 This function is internal use only. */)
7707 Lisp_Object coding_system
;
7709 Lisp_Object safe_chars
, slot
;
7711 if (NILP (Fcheck_coding_system (coding_system
)))
7712 xsignal1 (Qcoding_system_error
, coding_system
);
7714 safe_chars
= coding_safe_chars (coding_system
);
7715 if (! EQ (safe_chars
, Qt
) && ! CHAR_TABLE_P (safe_chars
))
7716 error ("No valid safe-chars property for %s",
7717 SDATA (SYMBOL_NAME (coding_system
)));
7719 if (EQ (safe_chars
, Qt
))
7721 if (NILP (Fmemq (coding_system
, XCAR (Vcoding_system_safe_chars
))))
7722 XSETCAR (Vcoding_system_safe_chars
,
7723 Fcons (coding_system
, XCAR (Vcoding_system_safe_chars
)));
7727 slot
= Fassq (coding_system
, XCDR (Vcoding_system_safe_chars
));
7729 XSETCDR (Vcoding_system_safe_chars
,
7730 nconc2 (XCDR (Vcoding_system_safe_chars
),
7731 Fcons (Fcons (coding_system
, safe_chars
), Qnil
)));
7733 XSETCDR (slot
, safe_chars
);
7741 /*** 9. Post-amble ***/
7748 /* Emacs' internal format specific initialize routine. */
7749 for (i
= 0; i
<= 0x20; i
++)
7750 emacs_code_class
[i
] = EMACS_control_code
;
7751 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
7752 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
7753 for (i
= 0x21 ; i
< 0x7F; i
++)
7754 emacs_code_class
[i
] = EMACS_ascii_code
;
7755 emacs_code_class
[0x7F] = EMACS_control_code
;
7756 for (i
= 0x80; i
< 0xFF; i
++)
7757 emacs_code_class
[i
] = EMACS_invalid_code
;
7758 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
7759 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
7760 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
7761 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
7763 /* ISO2022 specific initialize routine. */
7764 for (i
= 0; i
< 0x20; i
++)
7765 iso_code_class
[i
] = ISO_control_0
;
7766 for (i
= 0x21; i
< 0x7F; i
++)
7767 iso_code_class
[i
] = ISO_graphic_plane_0
;
7768 for (i
= 0x80; i
< 0xA0; i
++)
7769 iso_code_class
[i
] = ISO_control_1
;
7770 for (i
= 0xA1; i
< 0xFF; i
++)
7771 iso_code_class
[i
] = ISO_graphic_plane_1
;
7772 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
7773 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
7774 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
7775 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
7776 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
7777 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
7778 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
7779 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
7780 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
7781 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
7783 setup_coding_system (Qnil
, &safe_terminal_coding
);
7784 setup_coding_system (Qnil
, &default_buffer_file_coding
);
7786 bzero (coding_system_table
, sizeof coding_system_table
);
7788 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
7789 for (i
= 0; i
< 128; i
++)
7790 ascii_skip_code
[i
] = 1;
7792 #if defined (MSDOS) || defined (WINDOWSNT)
7793 system_eol_type
= CODING_EOL_CRLF
;
7795 system_eol_type
= CODING_EOL_LF
;
7798 inhibit_pre_post_conversion
= 0;
7806 staticpro (&Vcode_conversion_workbuf_name
);
7807 Vcode_conversion_workbuf_name
= build_string (" *code-conversion-work*");
7809 Qtarget_idx
= intern ("target-idx");
7810 staticpro (&Qtarget_idx
);
7812 Qcoding_system_history
= intern ("coding-system-history");
7813 staticpro (&Qcoding_system_history
);
7814 Fset (Qcoding_system_history
, Qnil
);
7816 /* Target FILENAME is the first argument. */
7817 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
7818 /* Target FILENAME is the third argument. */
7819 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
7821 Qcall_process
= intern ("call-process");
7822 staticpro (&Qcall_process
);
7823 /* Target PROGRAM is the first argument. */
7824 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
7826 Qcall_process_region
= intern ("call-process-region");
7827 staticpro (&Qcall_process_region
);
7828 /* Target PROGRAM is the third argument. */
7829 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
7831 Qstart_process
= intern ("start-process");
7832 staticpro (&Qstart_process
);
7833 /* Target PROGRAM is the third argument. */
7834 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
7836 Qopen_network_stream
= intern ("open-network-stream");
7837 staticpro (&Qopen_network_stream
);
7838 /* Target SERVICE is the fourth argument. */
7839 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
7841 Qcoding_system
= intern ("coding-system");
7842 staticpro (&Qcoding_system
);
7844 Qeol_type
= intern ("eol-type");
7845 staticpro (&Qeol_type
);
7847 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
7848 staticpro (&Qbuffer_file_coding_system
);
7850 Qpost_read_conversion
= intern ("post-read-conversion");
7851 staticpro (&Qpost_read_conversion
);
7853 Qpre_write_conversion
= intern ("pre-write-conversion");
7854 staticpro (&Qpre_write_conversion
);
7856 Qno_conversion
= intern ("no-conversion");
7857 staticpro (&Qno_conversion
);
7859 Qundecided
= intern ("undecided");
7860 staticpro (&Qundecided
);
7862 Qcoding_system_p
= intern ("coding-system-p");
7863 staticpro (&Qcoding_system_p
);
7865 Qcoding_system_error
= intern ("coding-system-error");
7866 staticpro (&Qcoding_system_error
);
7868 Fput (Qcoding_system_error
, Qerror_conditions
,
7869 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
7870 Fput (Qcoding_system_error
, Qerror_message
,
7871 build_string ("Invalid coding system"));
7873 Qcoding_category
= intern ("coding-category");
7874 staticpro (&Qcoding_category
);
7875 Qcoding_category_index
= intern ("coding-category-index");
7876 staticpro (&Qcoding_category_index
);
7878 Vcoding_category_table
7879 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
7880 staticpro (&Vcoding_category_table
);
7883 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7885 XVECTOR (Vcoding_category_table
)->contents
[i
]
7886 = intern (coding_category_name
[i
]);
7887 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
7888 Qcoding_category_index
, make_number (i
));
7892 Vcoding_system_safe_chars
= Fcons (Qnil
, Qnil
);
7893 staticpro (&Vcoding_system_safe_chars
);
7895 Qtranslation_table
= intern ("translation-table");
7896 staticpro (&Qtranslation_table
);
7897 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (2));
7899 Qtranslation_table_id
= intern ("translation-table-id");
7900 staticpro (&Qtranslation_table_id
);
7902 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
7903 staticpro (&Qtranslation_table_for_decode
);
7905 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
7906 staticpro (&Qtranslation_table_for_encode
);
7908 Qsafe_chars
= intern ("safe-chars");
7909 staticpro (&Qsafe_chars
);
7911 Qchar_coding_system
= intern ("char-coding-system");
7912 staticpro (&Qchar_coding_system
);
7914 /* Intern this now in case it isn't already done.
7915 Setting this variable twice is harmless.
7916 But don't staticpro it here--that is done in alloc.c. */
7917 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
7918 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
7919 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (0));
7921 Qvalid_codes
= intern ("valid-codes");
7922 staticpro (&Qvalid_codes
);
7924 Qascii_incompatible
= intern ("ascii-incompatible");
7925 staticpro (&Qascii_incompatible
);
7927 Qemacs_mule
= intern ("emacs-mule");
7928 staticpro (&Qemacs_mule
);
7930 Qraw_text
= intern ("raw-text");
7931 staticpro (&Qraw_text
);
7933 Qutf_8
= intern ("utf-8");
7934 staticpro (&Qutf_8
);
7936 Qcoding_system_define_form
= intern ("coding-system-define-form");
7937 staticpro (&Qcoding_system_define_form
);
7939 defsubr (&Scoding_system_p
);
7940 defsubr (&Sread_coding_system
);
7941 defsubr (&Sread_non_nil_coding_system
);
7942 defsubr (&Scheck_coding_system
);
7943 defsubr (&Sdetect_coding_region
);
7944 defsubr (&Sdetect_coding_string
);
7945 defsubr (&Sfind_coding_systems_region_internal
);
7946 defsubr (&Sunencodable_char_position
);
7947 defsubr (&Sdecode_coding_region
);
7948 defsubr (&Sencode_coding_region
);
7949 defsubr (&Sdecode_coding_string
);
7950 defsubr (&Sencode_coding_string
);
7951 defsubr (&Sdecode_sjis_char
);
7952 defsubr (&Sencode_sjis_char
);
7953 defsubr (&Sdecode_big5_char
);
7954 defsubr (&Sencode_big5_char
);
7955 defsubr (&Sset_terminal_coding_system_internal
);
7956 defsubr (&Sset_safe_terminal_coding_system_internal
);
7957 defsubr (&Sterminal_coding_system
);
7958 defsubr (&Sset_keyboard_coding_system_internal
);
7959 defsubr (&Skeyboard_coding_system
);
7960 defsubr (&Sfind_operation_coding_system
);
7961 defsubr (&Supdate_coding_systems_internal
);
7962 defsubr (&Sset_coding_priority_internal
);
7963 defsubr (&Sdefine_coding_system_internal
);
7965 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
7966 doc
: /* List of coding systems.
7968 Do not alter the value of this variable manually. This variable should be
7969 updated by the functions `make-coding-system' and
7970 `define-coding-system-alias'. */);
7971 Vcoding_system_list
= Qnil
;
7973 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
7974 doc
: /* Alist of coding system names.
7975 Each element is one element list of coding system name.
7976 This variable is given to `completing-read' as TABLE argument.
7978 Do not alter the value of this variable manually. This variable should be
7979 updated by the functions `make-coding-system' and
7980 `define-coding-system-alias'. */);
7981 Vcoding_system_alist
= Qnil
;
7983 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
7984 doc
: /* List of coding-categories (symbols) ordered by priority.
7986 On detecting a coding system, Emacs tries code detection algorithms
7987 associated with each coding-category one by one in this order. When
7988 one algorithm agrees with a byte sequence of source text, the coding
7989 system bound to the corresponding coding-category is selected.
7991 Don't modify this variable directly, but use `set-coding-priority'. */);
7995 Vcoding_category_list
= Qnil
;
7996 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
7997 Vcoding_category_list
7998 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
7999 Vcoding_category_list
);
8002 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
8003 doc
: /* Specify the coding system for read operations.
8004 It is useful to bind this variable with `let', but do not set it globally.
8005 If the value is a coding system, it is used for decoding on read operation.
8006 If not, an appropriate element is used from one of the coding system alists:
8007 There are three such tables, `file-coding-system-alist',
8008 `process-coding-system-alist', and `network-coding-system-alist'. */);
8009 Vcoding_system_for_read
= Qnil
;
8011 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
8012 doc
: /* Specify the coding system for write operations.
8013 Programs bind this variable with `let', but you should not set it globally.
8014 If the value is a coding system, it is used for encoding of output,
8015 when writing it to a file and when sending it to a file or subprocess.
8017 If this does not specify a coding system, an appropriate element
8018 is used from one of the coding system alists:
8019 There are three such tables, `file-coding-system-alist',
8020 `process-coding-system-alist', and `network-coding-system-alist'.
8021 For output to files, if the above procedure does not specify a coding system,
8022 the value of `buffer-file-coding-system' is used. */);
8023 Vcoding_system_for_write
= Qnil
;
8025 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
8026 doc
: /* Coding system used in the latest file or process I/O.
8027 Also set by `encode-coding-region', `decode-coding-region',
8028 `encode-coding-string' and `decode-coding-string'. */);
8029 Vlast_coding_system_used
= Qnil
;
8031 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
8032 doc
: /* *Non-nil means always inhibit code conversion of end-of-line format.
8033 See info node `Coding Systems' and info node `Text and Binary' concerning
8034 such conversion. */);
8035 inhibit_eol_conversion
= 0;
8037 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
8038 doc
: /* Non-nil means process buffer inherits coding system of process output.
8039 Bind it to t if the process output is to be treated as if it were a file
8040 read from some filesystem. */);
8041 inherit_process_coding_system
= 0;
8043 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
8044 doc
: /* Alist to decide a coding system to use for a file I/O operation.
8045 The format is ((PATTERN . VAL) ...),
8046 where PATTERN is a regular expression matching a file name,
8047 VAL is a coding system, a cons of coding systems, or a function symbol.
8048 If VAL is a coding system, it is used for both decoding and encoding
8050 If VAL is a cons of coding systems, the car part is used for decoding,
8051 and the cdr part is used for encoding.
8052 If VAL is a function symbol, the function must return a coding system
8053 or a cons of coding systems which are used as above. The function is
8054 called with an argument that is a list of the arguments with which
8055 `find-operation-coding-system' was called. If the function can't decide
8056 a coding system, it can return `undecided' so that the normal
8057 code-detection is performed.
8059 See also the function `find-operation-coding-system'
8060 and the variable `auto-coding-alist'. */);
8061 Vfile_coding_system_alist
= Qnil
;
8063 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
8064 doc
: /* Alist to decide a coding system to use for a process I/O operation.
8065 The format is ((PATTERN . VAL) ...),
8066 where PATTERN is a regular expression matching a program name,
8067 VAL is a coding system, a cons of coding systems, or a function symbol.
8068 If VAL is a coding system, it is used for both decoding what received
8069 from the program and encoding what sent to the program.
8070 If VAL is a cons of coding systems, the car part is used for decoding,
8071 and the cdr part is used for encoding.
8072 If VAL is a function symbol, the function must return a coding system
8073 or a cons of coding systems which are used as above.
8075 See also the function `find-operation-coding-system'. */);
8076 Vprocess_coding_system_alist
= Qnil
;
8078 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
8079 doc
: /* Alist to decide a coding system to use for a network I/O operation.
8080 The format is ((PATTERN . VAL) ...),
8081 where PATTERN is a regular expression matching a network service name
8082 or is a port number to connect to,
8083 VAL is a coding system, a cons of coding systems, or a function symbol.
8084 If VAL is a coding system, it is used for both decoding what received
8085 from the network stream and encoding what sent to the network stream.
8086 If VAL is a cons of coding systems, the car part is used for decoding,
8087 and the cdr part is used for encoding.
8088 If VAL is a function symbol, the function must return a coding system
8089 or a cons of coding systems which are used as above.
8091 See also the function `find-operation-coding-system'. */);
8092 Vnetwork_coding_system_alist
= Qnil
;
8094 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
8095 doc
: /* Coding system to use with system messages.
8096 Also used for decoding keyboard input on X Window system. */);
8097 Vlocale_coding_system
= Qnil
;
8099 /* The eol mnemonics are reset in startup.el system-dependently. */
8100 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
8101 doc
: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
8102 eol_mnemonic_unix
= build_string (":");
8104 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
8105 doc
: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
8106 eol_mnemonic_dos
= build_string ("\\");
8108 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
8109 doc
: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
8110 eol_mnemonic_mac
= build_string ("/");
8112 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
8113 doc
: /* *String displayed in mode line when end-of-line format is not yet determined. */);
8114 eol_mnemonic_undecided
= build_string (":");
8116 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
8117 doc
: /* *Non-nil enables character translation while encoding and decoding. */);
8118 Venable_character_translation
= Qt
;
8120 DEFVAR_LISP ("standard-translation-table-for-decode",
8121 &Vstandard_translation_table_for_decode
,
8122 doc
: /* Table for translating characters while decoding. */);
8123 Vstandard_translation_table_for_decode
= Qnil
;
8125 DEFVAR_LISP ("standard-translation-table-for-encode",
8126 &Vstandard_translation_table_for_encode
,
8127 doc
: /* Table for translating characters while encoding. */);
8128 Vstandard_translation_table_for_encode
= Qnil
;
8130 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
8131 doc
: /* Alist of charsets vs revision numbers.
8132 While encoding, if a charset (car part of an element) is found,
8133 designate it with the escape sequence identifying revision (cdr part of the element). */);
8134 Vcharset_revision_alist
= Qnil
;
8136 DEFVAR_LISP ("default-process-coding-system",
8137 &Vdefault_process_coding_system
,
8138 doc
: /* Cons of coding systems used for process I/O by default.
8139 The car part is used for decoding a process output,
8140 the cdr part is used for encoding a text to be sent to a process. */);
8141 Vdefault_process_coding_system
= Qnil
;
8143 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
8144 doc
: /* Table of extra Latin codes in the range 128..159 (inclusive).
8145 This is a vector of length 256.
8146 If Nth element is non-nil, the existence of code N in a file
8147 \(or output of subprocess) doesn't prevent it to be detected as
8148 a coding system of ISO 2022 variant which has a flag
8149 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
8150 or reading output of a subprocess.
8151 Only 128th through 159th elements has a meaning. */);
8152 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
8154 DEFVAR_LISP ("select-safe-coding-system-function",
8155 &Vselect_safe_coding_system_function
,
8156 doc
: /* Function to call to select safe coding system for encoding a text.
8158 If set, this function is called to force a user to select a proper
8159 coding system which can encode the text in the case that a default
8160 coding system used in each operation can't encode the text. The
8161 function should take care that the buffer is not modified while
8162 the coding system is being selected.
8164 The default value is `select-safe-coding-system' (which see). */);
8165 Vselect_safe_coding_system_function
= Qnil
;
8167 DEFVAR_BOOL ("coding-system-require-warning",
8168 &coding_system_require_warning
,
8169 doc
: /* Internal use only.
8170 If non-nil, on writing a file, `select-safe-coding-system-function' is
8171 called even if `coding-system-for-write' is non-nil. The command
8172 `universal-coding-system-argument' binds this variable to t temporarily. */);
8173 coding_system_require_warning
= 0;
8176 DEFVAR_BOOL ("inhibit-iso-escape-detection",
8177 &inhibit_iso_escape_detection
,
8178 doc
: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
8180 By default, on reading a file, Emacs tries to detect how the text is
8181 encoded. This code detection is sensitive to escape sequences. If
8182 the sequence is valid as ISO2022, the code is determined as one of
8183 the ISO2022 encodings, and the file is decoded by the corresponding
8184 coding system (e.g. `iso-2022-7bit').
8186 However, there may be a case that you want to read escape sequences in
8187 a file as is. In such a case, you can set this variable to non-nil.
8188 Then, as the code detection ignores any escape sequences, no file is
8189 detected as encoded in some ISO2022 encoding. The result is that all
8190 escape sequences become visible in a buffer.
8192 The default value is nil, and it is strongly recommended not to change
8193 it. That is because many Emacs Lisp source files that contain
8194 non-ASCII characters are encoded by the coding system `iso-2022-7bit'
8195 in Emacs's distribution, and they won't be decoded correctly on
8196 reading if you suppress escape sequence detection.
8198 The other way to read escape sequences in a file without decoding is
8199 to explicitly specify some coding system that doesn't use ISO2022's
8200 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
8201 inhibit_iso_escape_detection
= 0;
8203 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input
,
8204 doc
: /* Char table for translating self-inserting characters.
8205 This is applied to the result of input methods, not their input. See also
8206 `keyboard-translate-table'. */);
8207 Vtranslation_table_for_input
= Qnil
;
8211 emacs_strerror (error_number
)
8216 synchronize_system_messages_locale ();
8217 str
= strerror (error_number
);
8219 if (! NILP (Vlocale_coding_system
))
8221 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
8222 Vlocale_coding_system
,
8224 str
= (char *) SDATA (dec
);
8232 /* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
8233 (do not change this comment) */