1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
3 2006 Free Software Foundation, Inc.
4 Copyright (C) 1995, 1997, 1998, 2002, 2003, 2004, 2005
5 National Institute of Advanced Industrial Science and Technology (AIST)
6 Registration Number H14PRO021
8 This file is part of GNU Emacs.
10 GNU Emacs is free software; you can redistribute it and/or modify
11 it under the terms of the GNU General Public License as published by
12 the Free Software Foundation; either version 2, or (at your option)
15 GNU Emacs is distributed in the hope that it will be useful,
16 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
18 GNU General Public License for more details.
20 You should have received a copy of the GNU General Public License
21 along with GNU Emacs; see the file COPYING. If not, write to
22 the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
23 Boston, MA 02110-1301, USA. */
25 /*** TABLE OF CONTENTS ***
29 2. Emacs' internal format (emacs-mule) handlers
31 4. Shift-JIS and BIG5 handlers
33 6. End-of-line handlers
34 7. C library functions
35 8. Emacs Lisp library functions
40 /*** 0. General comments ***/
43 /*** GENERAL NOTE on CODING SYSTEMS ***
45 A coding system is an encoding mechanism for one or more character
46 sets. Here's a list of coding systems which Emacs can handle. When
47 we say "decode", it means converting some other coding system to
48 Emacs' internal format (emacs-mule), and when we say "encode",
49 it means converting the coding system emacs-mule to some other
52 0. Emacs' internal format (emacs-mule)
54 Emacs itself holds a multi-lingual character in buffers and strings
55 in a special format. Details are described in section 2.
59 The most famous coding system for multiple character sets. X's
60 Compound Text, various EUCs (Extended Unix Code), and coding
61 systems used in Internet communication such as ISO-2022-JP are
62 all variants of ISO2022. Details are described in section 3.
64 2. SJIS (or Shift-JIS or MS-Kanji-Code)
66 A coding system to encode character sets: ASCII, JISX0201, and
67 JISX0208. Widely used for PC's in Japan. Details are described in
72 A coding system to encode the character sets ASCII and Big5. Widely
73 used for Chinese (mainly in Taiwan and Hong Kong). Details are
74 described in section 4. In this file, when we write "BIG5"
75 (all uppercase), we mean the coding system, and when we write
76 "Big5" (capitalized), we mean the character set.
80 A coding system for text containing random 8-bit code. Emacs does
81 no code conversion on such text except for end-of-line format.
85 If a user wants to read/write text encoded in a coding system not
86 listed above, he can supply a decoder and an encoder for it as CCL
87 (Code Conversion Language) programs. Emacs executes the CCL program
88 while reading/writing.
90 Emacs represents a coding system by a Lisp symbol that has a property
91 `coding-system'. But, before actually using the coding system, the
92 information about it is set in a structure of type `struct
93 coding_system' for rapid processing. See section 6 for more details.
97 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
99 How end-of-line of text is encoded depends on the operating system.
100 For instance, Unix's format is just one byte of `line-feed' code,
101 whereas DOS's format is two-byte sequence of `carriage-return' and
102 `line-feed' codes. MacOS's format is usually one byte of
105 Since text character encoding and end-of-line encoding are
106 independent, any coding system described above can have any
107 end-of-line format. So Emacs has information about end-of-line
108 format in each coding-system. See section 6 for more details.
112 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
114 These functions check if a text between SRC and SRC_END is encoded
115 in the coding system category XXX. Each returns an integer value in
116 which appropriate flag bits for the category XXX are set. The flag
117 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
118 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
119 of the range 0x80..0x9F are in multibyte form. */
122 detect_coding_emacs_mule (src
, src_end
, multibytep
)
123 unsigned char *src
, *src_end
;
130 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
132 These functions decode SRC_BYTES length of unibyte text at SOURCE
133 encoded in CODING to Emacs' internal format. The resulting
134 multibyte text goes to a place pointed to by DESTINATION, the length
135 of which should not exceed DST_BYTES.
137 These functions set the information about original and decoded texts
138 in the members `produced', `produced_char', `consumed', and
139 `consumed_char' of the structure *CODING. They also set the member
140 `result' to one of CODING_FINISH_XXX indicating how the decoding
143 DST_BYTES zero means that the source area and destination area are
144 overlapped, which means that we can produce a decoded text until it
145 reaches the head of the not-yet-decoded source text.
147 Below is a template for these functions. */
150 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
151 struct coding_system
*coding
;
152 const unsigned char *source
;
153 unsigned char *destination
;
154 int src_bytes
, dst_bytes
;
160 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
162 These functions encode SRC_BYTES length text at SOURCE from Emacs'
163 internal multibyte format to CODING. The resulting unibyte text
164 goes to a place pointed to by DESTINATION, the length of which
165 should not exceed DST_BYTES.
167 These functions set the information about original and encoded texts
168 in the members `produced', `produced_char', `consumed', and
169 `consumed_char' of the structure *CODING. They also set the member
170 `result' to one of CODING_FINISH_XXX indicating how the encoding
173 DST_BYTES zero means that the source area and destination area are
174 overlapped, which means that we can produce encoded text until it
175 reaches at the head of the not-yet-encoded source text.
177 Below is a template for these functions. */
180 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
181 struct coding_system
*coding
;
182 unsigned char *source
, *destination
;
183 int src_bytes
, dst_bytes
;
189 /*** COMMONLY USED MACROS ***/
191 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
192 get one, two, and three bytes from the source text respectively.
193 If there are not enough bytes in the source, they jump to
194 `label_end_of_loop'. The caller should set variables `coding',
195 `src' and `src_end' to appropriate pointer in advance. These
196 macros are called from decoding routines `decode_coding_XXX', thus
197 it is assumed that the source text is unibyte. */
199 #define ONE_MORE_BYTE(c1) \
201 if (src >= src_end) \
203 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
204 goto label_end_of_loop; \
209 #define TWO_MORE_BYTES(c1, c2) \
211 if (src + 1 >= src_end) \
213 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
214 goto label_end_of_loop; \
221 /* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
222 form if MULTIBYTEP is nonzero. */
224 #define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep) \
226 if (src >= src_end) \
228 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
229 goto label_end_of_loop; \
232 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
233 c1 = *src++ - 0x20; \
236 /* Set C to the next character at the source text pointed by `src'.
237 If there are not enough characters in the source, jump to
238 `label_end_of_loop'. The caller should set variables `coding'
239 `src', `src_end', and `translation_table' to appropriate pointers
240 in advance. This macro is used in encoding routines
241 `encode_coding_XXX', thus it assumes that the source text is in
242 multibyte form except for 8-bit characters. 8-bit characters are
243 in multibyte form if coding->src_multibyte is nonzero, else they
244 are represented by a single byte. */
246 #define ONE_MORE_CHAR(c) \
248 int len = src_end - src; \
252 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
253 goto label_end_of_loop; \
255 if (coding->src_multibyte \
256 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
257 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
259 c = *src, bytes = 1; \
260 if (!NILP (translation_table)) \
261 c = translate_char (translation_table, c, -1, 0, 0); \
266 /* Produce a multibyte form of character C to `dst'. Jump to
267 `label_end_of_loop' if there's not enough space at `dst'.
269 If we are now in the middle of a composition sequence, the decoded
270 character may be ALTCHAR (for the current composition). In that
271 case, the character goes to coding->cmp_data->data instead of
274 This macro is used in decoding routines. */
276 #define EMIT_CHAR(c) \
278 if (! COMPOSING_P (coding) \
279 || coding->composing == COMPOSITION_RELATIVE \
280 || coding->composing == COMPOSITION_WITH_RULE) \
282 int bytes = CHAR_BYTES (c); \
283 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
285 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
286 goto label_end_of_loop; \
288 dst += CHAR_STRING (c, dst); \
289 coding->produced_char++; \
292 if (COMPOSING_P (coding) \
293 && coding->composing != COMPOSITION_RELATIVE) \
295 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
296 coding->composition_rule_follows \
297 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
302 #define EMIT_ONE_BYTE(c) \
304 if (dst >= (dst_bytes ? dst_end : src)) \
306 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
307 goto label_end_of_loop; \
312 #define EMIT_TWO_BYTES(c1, c2) \
314 if (dst + 2 > (dst_bytes ? dst_end : src)) \
316 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
317 goto label_end_of_loop; \
319 *dst++ = c1, *dst++ = c2; \
322 #define EMIT_BYTES(from, to) \
324 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
326 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
327 goto label_end_of_loop; \
334 /*** 1. Preamble ***/
347 #include "composite.h"
351 #include "intervals.h"
353 #include "termhooks.h"
355 #else /* not emacs */
359 #endif /* not emacs */
361 Lisp_Object Qcoding_system
, Qeol_type
;
362 Lisp_Object Qbuffer_file_coding_system
;
363 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
364 Lisp_Object Qno_conversion
, Qundecided
;
365 Lisp_Object Qcoding_system_history
;
366 Lisp_Object Qsafe_chars
;
367 Lisp_Object Qvalid_codes
;
369 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
370 Lisp_Object Qcall_process
, Qcall_process_region
;
371 Lisp_Object Qstart_process
, Qopen_network_stream
;
372 Lisp_Object Qtarget_idx
;
374 /* If a symbol has this property, evaluate the value to define the
375 symbol as a coding system. */
376 Lisp_Object Qcoding_system_define_form
;
378 Lisp_Object Vselect_safe_coding_system_function
;
380 int coding_system_require_warning
;
382 /* Mnemonic string for each format of end-of-line. */
383 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
384 /* Mnemonic string to indicate format of end-of-line is not yet
386 Lisp_Object eol_mnemonic_undecided
;
388 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
389 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac.
390 This has an effect only for external encoding (i.e. for output to
391 file and process), not for in-buffer or Lisp string encoding. */
396 /* Information about which coding system is safe for which chars.
397 The value has the form (GENERIC-LIST . NON-GENERIC-ALIST).
399 GENERIC-LIST is a list of generic coding systems which can encode
402 NON-GENERIC-ALIST is an alist of non generic coding systems vs the
403 corresponding char table that contains safe chars. */
404 Lisp_Object Vcoding_system_safe_chars
;
406 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
408 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
410 /* Coding system emacs-mule and raw-text are for converting only
411 end-of-line format. */
412 Lisp_Object Qemacs_mule
, Qraw_text
;
416 /* Coding-systems are handed between Emacs Lisp programs and C internal
417 routines by the following three variables. */
418 /* Coding-system for reading files and receiving data from process. */
419 Lisp_Object Vcoding_system_for_read
;
420 /* Coding-system for writing files and sending data to process. */
421 Lisp_Object Vcoding_system_for_write
;
422 /* Coding-system actually used in the latest I/O. */
423 Lisp_Object Vlast_coding_system_used
;
425 /* A vector of length 256 which contains information about special
426 Latin codes (especially for dealing with Microsoft codes). */
427 Lisp_Object Vlatin_extra_code_table
;
429 /* Flag to inhibit code conversion of end-of-line format. */
430 int inhibit_eol_conversion
;
432 /* Flag to inhibit ISO2022 escape sequence detection. */
433 int inhibit_iso_escape_detection
;
435 /* Flag to make buffer-file-coding-system inherit from process-coding. */
436 int inherit_process_coding_system
;
438 /* Coding system to be used to encode text for terminal display when
439 terminal coding system is nil. */
440 struct coding_system safe_terminal_coding
;
442 /* Default coding system to be used to write a file. */
443 struct coding_system default_buffer_file_coding
;
445 Lisp_Object Vfile_coding_system_alist
;
446 Lisp_Object Vprocess_coding_system_alist
;
447 Lisp_Object Vnetwork_coding_system_alist
;
449 Lisp_Object Vlocale_coding_system
;
453 Lisp_Object Qcoding_category
, Qcoding_category_index
;
455 /* List of symbols `coding-category-xxx' ordered by priority. */
456 Lisp_Object Vcoding_category_list
;
458 /* Table of coding categories (Lisp symbols). */
459 Lisp_Object Vcoding_category_table
;
461 /* Table of names of symbol for each coding-category. */
462 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
463 "coding-category-emacs-mule",
464 "coding-category-sjis",
465 "coding-category-iso-7",
466 "coding-category-iso-7-tight",
467 "coding-category-iso-8-1",
468 "coding-category-iso-8-2",
469 "coding-category-iso-7-else",
470 "coding-category-iso-8-else",
471 "coding-category-ccl",
472 "coding-category-big5",
473 "coding-category-utf-8",
474 "coding-category-utf-16-be",
475 "coding-category-utf-16-le",
476 "coding-category-raw-text",
477 "coding-category-binary"
480 /* Table of pointers to coding systems corresponding to each coding
482 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
484 /* Table of coding category masks. Nth element is a mask for a coding
485 category of which priority is Nth. */
487 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
489 /* Flag to tell if we look up translation table on character code
491 Lisp_Object Venable_character_translation
;
492 /* Standard translation table to look up on decoding (reading). */
493 Lisp_Object Vstandard_translation_table_for_decode
;
494 /* Standard translation table to look up on encoding (writing). */
495 Lisp_Object Vstandard_translation_table_for_encode
;
497 Lisp_Object Qtranslation_table
;
498 Lisp_Object Qtranslation_table_id
;
499 Lisp_Object Qtranslation_table_for_decode
;
500 Lisp_Object Qtranslation_table_for_encode
;
502 /* Alist of charsets vs revision number. */
503 Lisp_Object Vcharset_revision_alist
;
505 /* Default coding systems used for process I/O. */
506 Lisp_Object Vdefault_process_coding_system
;
508 /* Char table for translating Quail and self-inserting input. */
509 Lisp_Object Vtranslation_table_for_input
;
511 /* Global flag to tell that we can't call post-read-conversion and
512 pre-write-conversion functions. Usually the value is zero, but it
513 is set to 1 temporarily while such functions are running. This is
514 to avoid infinite recursive call. */
515 static int inhibit_pre_post_conversion
;
517 Lisp_Object Qchar_coding_system
;
519 /* Return `safe-chars' property of CODING_SYSTEM (symbol). Don't check
523 coding_safe_chars (coding_system
)
524 Lisp_Object coding_system
;
526 Lisp_Object coding_spec
, plist
, safe_chars
;
528 coding_spec
= Fget (coding_system
, Qcoding_system
);
529 plist
= XVECTOR (coding_spec
)->contents
[3];
530 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
531 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
534 #define CODING_SAFE_CHAR_P(safe_chars, c) \
535 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
538 /*** 2. Emacs internal format (emacs-mule) handlers ***/
540 /* Emacs' internal format for representation of multiple character
541 sets is a kind of multi-byte encoding, i.e. characters are
542 represented by variable-length sequences of one-byte codes.
544 ASCII characters and control characters (e.g. `tab', `newline') are
545 represented by one-byte sequences which are their ASCII codes, in
546 the range 0x00 through 0x7F.
548 8-bit characters of the range 0x80..0x9F are represented by
549 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
552 8-bit characters of the range 0xA0..0xFF are represented by
553 one-byte sequences which are their 8-bit code.
555 The other characters are represented by a sequence of `base
556 leading-code', optional `extended leading-code', and one or two
557 `position-code's. The length of the sequence is determined by the
558 base leading-code. Leading-code takes the range 0x81 through 0x9D,
559 whereas extended leading-code and position-code take the range 0xA0
560 through 0xFF. See `charset.h' for more details about leading-code
563 --- CODE RANGE of Emacs' internal format ---
567 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
568 eight-bit-graphic 0xA0..0xBF
569 ELSE 0x81..0x9D + [0xA0..0xFF]+
570 ---------------------------------------------
572 As this is the internal character representation, the format is
573 usually not used externally (i.e. in a file or in a data sent to a
574 process). But, it is possible to have a text externally in this
575 format (i.e. by encoding by the coding system `emacs-mule').
577 In that case, a sequence of one-byte codes has a slightly different
580 Firstly, all characters in eight-bit-control are represented by
581 one-byte sequences which are their 8-bit code.
583 Next, character composition data are represented by the byte
584 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
586 METHOD is 0xF0 plus one of composition method (enum
589 BYTES is 0xA0 plus the byte length of these composition data,
591 CHARS is 0xA0 plus the number of characters composed by these
594 COMPONENTs are characters of multibyte form or composition
595 rules encoded by two-byte of ASCII codes.
597 In addition, for backward compatibility, the following formats are
598 also recognized as composition data on decoding.
601 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
604 MSEQ is a multibyte form but in these special format:
605 ASCII: 0xA0 ASCII_CODE+0x80,
606 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
607 RULE is a one byte code of the range 0xA0..0xF0 that
608 represents a composition rule.
611 enum emacs_code_class_type emacs_code_class
[256];
613 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
614 Check if a text is encoded in Emacs' internal format. If it is,
615 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
618 detect_coding_emacs_mule (src
, src_end
, multibytep
)
619 unsigned char *src
, *src_end
;
624 /* Dummy for ONE_MORE_BYTE. */
625 struct coding_system dummy_coding
;
626 struct coding_system
*coding
= &dummy_coding
;
630 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
638 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
647 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
650 else if (c
>= 0x80 && c
< 0xA0)
653 /* Old leading code for a composite character. */
657 unsigned char *src_base
= src
- 1;
660 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
663 src
= src_base
+ bytes
;
668 return CODING_CATEGORY_MASK_EMACS_MULE
;
672 /* Record the starting position START and METHOD of one composition. */
674 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
676 struct composition_data *cmp_data = coding->cmp_data; \
677 int *data = cmp_data->data + cmp_data->used; \
678 coding->cmp_data_start = cmp_data->used; \
680 data[1] = cmp_data->char_offset + start; \
681 data[3] = (int) method; \
682 cmp_data->used += 4; \
685 /* Record the ending position END of the current composition. */
687 #define CODING_ADD_COMPOSITION_END(coding, end) \
689 struct composition_data *cmp_data = coding->cmp_data; \
690 int *data = cmp_data->data + coding->cmp_data_start; \
691 data[0] = cmp_data->used - coding->cmp_data_start; \
692 data[2] = cmp_data->char_offset + end; \
695 /* Record one COMPONENT (alternate character or composition rule). */
697 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
699 coding->cmp_data->data[coding->cmp_data->used++] = component; \
700 if (coding->cmp_data->used - coding->cmp_data_start \
701 == COMPOSITION_DATA_MAX_BUNCH_LENGTH) \
703 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
704 coding->composing = COMPOSITION_NO; \
709 /* Get one byte from a data pointed by SRC and increment SRC. If SRC
710 is not less than SRC_END, return -1 without incrementing Src. */
712 #define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
715 /* Decode a character represented as a component of composition
716 sequence of Emacs 20 style at SRC. Set C to that character, store
717 its multibyte form sequence at P, and set P to the end of that
718 sequence. If no valid character is found, set C to -1. */
720 #define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
724 c = SAFE_ONE_MORE_BYTE (); \
727 if (CHAR_HEAD_P (c)) \
729 else if (c == 0xA0) \
731 c = SAFE_ONE_MORE_BYTE (); \
740 else if (BASE_LEADING_CODE_P (c - 0x20)) \
742 unsigned char *p0 = p; \
746 bytes = BYTES_BY_CHAR_HEAD (c); \
749 c = SAFE_ONE_MORE_BYTE (); \
754 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes) \
755 || (coding->flags /* We are recovering a file. */ \
756 && p0[0] == LEADING_CODE_8_BIT_CONTROL \
757 && ! CHAR_HEAD_P (p0[1]))) \
758 c = STRING_CHAR (p0, bytes); \
767 /* Decode a composition rule represented as a component of composition
768 sequence of Emacs 20 style at SRC. Set C to the rule. If not
769 valid rule is found, set C to -1. */
771 #define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
773 c = SAFE_ONE_MORE_BYTE (); \
775 if (c < 0 || c >= 81) \
779 gref = c / 9, nref = c % 9; \
780 c = COMPOSITION_ENCODE_RULE (gref, nref); \
785 /* Decode composition sequence encoded by `emacs-mule' at the source
786 pointed by SRC. SRC_END is the end of source. Store information
787 of the composition in CODING->cmp_data.
789 For backward compatibility, decode also a composition sequence of
790 Emacs 20 style. In that case, the composition sequence contains
791 characters that should be extracted into a buffer or string. Store
792 those characters at *DESTINATION in multibyte form.
794 If we encounter an invalid byte sequence, return 0.
795 If we encounter an insufficient source or destination, or
796 insufficient space in CODING->cmp_data, return 1.
797 Otherwise, return consumed bytes in the source.
801 decode_composition_emacs_mule (coding
, src
, src_end
,
802 destination
, dst_end
, dst_bytes
)
803 struct coding_system
*coding
;
804 const unsigned char *src
, *src_end
;
805 unsigned char **destination
, *dst_end
;
808 unsigned char *dst
= *destination
;
809 int method
, data_len
, nchars
;
810 const unsigned char *src_base
= src
++;
811 /* Store components of composition. */
812 int component
[COMPOSITION_DATA_MAX_BUNCH_LENGTH
];
814 /* Store multibyte form of characters to be composed. This is for
815 Emacs 20 style composition sequence. */
816 unsigned char buf
[MAX_COMPOSITION_COMPONENTS
* MAX_MULTIBYTE_LENGTH
];
817 unsigned char *bufp
= buf
;
818 int c
, i
, gref
, nref
;
820 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
821 >= COMPOSITION_DATA_SIZE
)
823 coding
->result
= CODING_FINISH_INSUFFICIENT_CMP
;
828 if (c
- 0xF0 >= COMPOSITION_RELATIVE
829 && c
- 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS
)
834 with_rule
= (method
== COMPOSITION_WITH_RULE
835 || method
== COMPOSITION_WITH_RULE_ALTCHARS
);
839 || src_base
+ data_len
> src_end
)
845 for (ncomponent
= 0; src
< src_base
+ data_len
; ncomponent
++)
847 /* If it is longer than this, it can't be valid. */
848 if (ncomponent
>= COMPOSITION_DATA_MAX_BUNCH_LENGTH
)
851 if (ncomponent
% 2 && with_rule
)
853 ONE_MORE_BYTE (gref
);
855 ONE_MORE_BYTE (nref
);
857 c
= COMPOSITION_ENCODE_RULE (gref
, nref
);
862 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
863 || (coding
->flags
/* We are recovering a file. */
864 && src
[0] == LEADING_CODE_8_BIT_CONTROL
865 && ! CHAR_HEAD_P (src
[1])))
866 c
= STRING_CHAR (src
, bytes
);
871 component
[ncomponent
] = c
;
876 /* This may be an old Emacs 20 style format. See the comment at
877 the section 2 of this file. */
878 while (src
< src_end
&& !CHAR_HEAD_P (*src
)) src
++;
880 && !(coding
->mode
& CODING_MODE_LAST_BLOCK
))
881 goto label_end_of_loop
;
887 method
= COMPOSITION_RELATIVE
;
888 for (ncomponent
= 0; ncomponent
< MAX_COMPOSITION_COMPONENTS
;)
890 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
893 component
[ncomponent
++] = c
;
901 method
= COMPOSITION_WITH_RULE
;
903 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
908 ncomponent
< MAX_COMPOSITION_COMPONENTS
* 2 - 1;)
910 DECODE_EMACS_MULE_COMPOSITION_RULE (c
);
913 component
[ncomponent
++] = c
;
914 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
917 component
[ncomponent
++] = c
;
921 nchars
= (ncomponent
+ 1) / 2;
929 if (buf
== bufp
|| dst
+ (bufp
- buf
) <= (dst_bytes
? dst_end
: src
))
931 CODING_ADD_COMPOSITION_START (coding
, coding
->produced_char
, method
);
932 for (i
= 0; i
< ncomponent
; i
++)
933 CODING_ADD_COMPOSITION_COMPONENT (coding
, component
[i
]);
934 CODING_ADD_COMPOSITION_END (coding
, coding
->produced_char
+ nchars
);
937 unsigned char *p
= buf
;
938 EMIT_BYTES (p
, bufp
);
939 *destination
+= bufp
- buf
;
940 coding
->produced_char
+= nchars
;
942 return (src
- src_base
);
948 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
951 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
952 struct coding_system
*coding
;
953 const unsigned char *source
;
954 unsigned char *destination
;
955 int src_bytes
, dst_bytes
;
957 const unsigned char *src
= source
;
958 const unsigned char *src_end
= source
+ src_bytes
;
959 unsigned char *dst
= destination
;
960 unsigned char *dst_end
= destination
+ dst_bytes
;
961 /* SRC_BASE remembers the start position in source in each loop.
962 The loop will be exited when there's not enough source code, or
963 when there's not enough destination area to produce a
965 const unsigned char *src_base
;
967 coding
->produced_char
= 0;
968 while ((src_base
= src
) < src_end
)
970 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
];
971 const unsigned char *p
;
978 if (coding
->eol_type
== CODING_EOL_CR
)
980 else if (coding
->eol_type
== CODING_EOL_CRLF
)
990 coding
->produced_char
++;
993 else if (*src
== '\n')
995 if ((coding
->eol_type
== CODING_EOL_CR
996 || coding
->eol_type
== CODING_EOL_CRLF
)
997 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
999 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1000 goto label_end_of_loop
;
1003 coding
->produced_char
++;
1006 else if (*src
== 0x80 && coding
->cmp_data
)
1008 /* Start of composition data. */
1009 int consumed
= decode_composition_emacs_mule (coding
, src
, src_end
,
1013 goto label_end_of_loop
;
1014 else if (consumed
> 0)
1019 bytes
= CHAR_STRING (*src
, tmp
);
1023 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
1024 || (coding
->flags
/* We are recovering a file. */
1025 && src
[0] == LEADING_CODE_8_BIT_CONTROL
1026 && ! CHAR_HEAD_P (src
[1])))
1035 bytes
= BYTES_BY_CHAR_HEAD (*src
);
1037 for (i
= 1; i
< bytes
; i
++)
1040 if (CHAR_HEAD_P (c
))
1045 bytes
= CHAR_STRING (*src_base
, tmp
);
1054 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
1056 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
1059 while (bytes
--) *dst
++ = *p
++;
1060 coding
->produced_char
++;
1063 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1064 coding
->produced
= dst
- destination
;
1068 /* Encode composition data stored at DATA into a special byte sequence
1069 starting by 0x80. Update CODING->cmp_data_start and maybe
1070 CODING->cmp_data for the next call. */
1072 #define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1074 unsigned char buf[1024], *p0 = buf, *p; \
1075 int len = data[0]; \
1079 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1080 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1082 if (data[3] == COMPOSITION_WITH_RULE \
1083 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1085 p += CHAR_STRING (data[4], p); \
1086 for (i = 5; i < len; i += 2) \
1089 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1090 *p++ = 0x20 + gref; \
1091 *p++ = 0x20 + nref; \
1092 p += CHAR_STRING (data[i + 1], p); \
1097 for (i = 4; i < len; i++) \
1098 p += CHAR_STRING (data[i], p); \
1100 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1102 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1104 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1105 goto label_end_of_loop; \
1109 coding->cmp_data_start += data[0]; \
1110 if (coding->cmp_data_start == coding->cmp_data->used \
1111 && coding->cmp_data->next) \
1113 coding->cmp_data = coding->cmp_data->next; \
1114 coding->cmp_data_start = 0; \
1119 static void encode_eol
P_ ((struct coding_system
*, const unsigned char *,
1120 unsigned char *, int, int));
1123 encode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
1124 struct coding_system
*coding
;
1125 const unsigned char *source
;
1126 unsigned char *destination
;
1127 int src_bytes
, dst_bytes
;
1129 const unsigned char *src
= source
;
1130 const unsigned char *src_end
= source
+ src_bytes
;
1131 unsigned char *dst
= destination
;
1132 unsigned char *dst_end
= destination
+ dst_bytes
;
1133 const unsigned char *src_base
;
1138 Lisp_Object translation_table
;
1140 translation_table
= Qnil
;
1142 /* Optimization for the case that there's no composition. */
1143 if (!coding
->cmp_data
|| coding
->cmp_data
->used
== 0)
1145 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
1149 char_offset
= coding
->cmp_data
->char_offset
;
1150 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1155 /* If SRC starts a composition, encode the information about the
1156 composition in advance. */
1157 if (coding
->cmp_data_start
< coding
->cmp_data
->used
1158 && char_offset
+ coding
->consumed_char
== data
[1])
1160 ENCODE_COMPOSITION_EMACS_MULE (coding
, data
);
1161 char_offset
= coding
->cmp_data
->char_offset
;
1162 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1166 if (c
== '\n' && (coding
->eol_type
== CODING_EOL_CRLF
1167 || coding
->eol_type
== CODING_EOL_CR
))
1169 if (coding
->eol_type
== CODING_EOL_CRLF
)
1170 EMIT_TWO_BYTES ('\r', c
);
1172 EMIT_ONE_BYTE ('\r');
1174 else if (SINGLE_BYTE_CHAR_P (c
))
1176 if (coding
->flags
&& ! ASCII_BYTE_P (c
))
1178 /* As we are auto saving, retain the multibyte form for
1180 unsigned char buf
[MAX_MULTIBYTE_LENGTH
];
1181 int bytes
= CHAR_STRING (c
, buf
);
1184 EMIT_ONE_BYTE (buf
[0]);
1186 EMIT_TWO_BYTES (buf
[0], buf
[1]);
1192 EMIT_BYTES (src_base
, src
);
1193 coding
->consumed_char
++;
1196 coding
->consumed
= src_base
- source
;
1197 coding
->produced
= coding
->produced_char
= dst
- destination
;
1202 /*** 3. ISO2022 handlers ***/
1204 /* The following note describes the coding system ISO2022 briefly.
1205 Since the intention of this note is to help understand the
1206 functions in this file, some parts are NOT ACCURATE or are OVERLY
1207 SIMPLIFIED. For thorough understanding, please refer to the
1208 original document of ISO2022. This is equivalent to the standard
1209 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1211 ISO2022 provides many mechanisms to encode several character sets
1212 in 7-bit and 8-bit environments. For 7-bit environments, all text
1213 is encoded using bytes less than 128. This may make the encoded
1214 text a little bit longer, but the text passes more easily through
1215 several types of gateway, some of which strip off the MSB (Most
1218 There are two kinds of character sets: control character sets and
1219 graphic character sets. The former contain control characters such
1220 as `newline' and `escape' to provide control functions (control
1221 functions are also provided by escape sequences). The latter
1222 contain graphic characters such as 'A' and '-'. Emacs recognizes
1223 two control character sets and many graphic character sets.
1225 Graphic character sets are classified into one of the following
1226 four classes, according to the number of bytes (DIMENSION) and
1227 number of characters in one dimension (CHARS) of the set:
1228 - DIMENSION1_CHARS94
1229 - DIMENSION1_CHARS96
1230 - DIMENSION2_CHARS94
1231 - DIMENSION2_CHARS96
1233 In addition, each character set is assigned an identification tag,
1234 unique for each set, called the "final character" (denoted as <F>
1235 hereafter). The <F> of each character set is decided by ECMA(*)
1236 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1237 (0x30..0x3F are for private use only).
1239 Note (*): ECMA = European Computer Manufacturers Association
1241 Here are examples of graphic character sets [NAME(<F>)]:
1242 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1243 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1244 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1245 o DIMENSION2_CHARS96 -- none for the moment
1247 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
1248 C0 [0x00..0x1F] -- control character plane 0
1249 GL [0x20..0x7F] -- graphic character plane 0
1250 C1 [0x80..0x9F] -- control character plane 1
1251 GR [0xA0..0xFF] -- graphic character plane 1
1253 A control character set is directly designated and invoked to C0 or
1254 C1 by an escape sequence. The most common case is that:
1255 - ISO646's control character set is designated/invoked to C0, and
1256 - ISO6429's control character set is designated/invoked to C1,
1257 and usually these designations/invocations are omitted in encoded
1258 text. In a 7-bit environment, only C0 can be used, and a control
1259 character for C1 is encoded by an appropriate escape sequence to
1260 fit into the environment. All control characters for C1 are
1261 defined to have corresponding escape sequences.
1263 A graphic character set is at first designated to one of four
1264 graphic registers (G0 through G3), then these graphic registers are
1265 invoked to GL or GR. These designations and invocations can be
1266 done independently. The most common case is that G0 is invoked to
1267 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1268 these invocations and designations are omitted in encoded text.
1269 In a 7-bit environment, only GL can be used.
1271 When a graphic character set of CHARS94 is invoked to GL, codes
1272 0x20 and 0x7F of the GL area work as control characters SPACE and
1273 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1276 There are two ways of invocation: locking-shift and single-shift.
1277 With locking-shift, the invocation lasts until the next different
1278 invocation, whereas with single-shift, the invocation affects the
1279 following character only and doesn't affect the locking-shift
1280 state. Invocations are done by the following control characters or
1283 ----------------------------------------------------------------------
1284 abbrev function cntrl escape seq description
1285 ----------------------------------------------------------------------
1286 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1287 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1288 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1289 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1290 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1291 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1292 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1293 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1294 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
1295 ----------------------------------------------------------------------
1296 (*) These are not used by any known coding system.
1298 Control characters for these functions are defined by macros
1299 ISO_CODE_XXX in `coding.h'.
1301 Designations are done by the following escape sequences:
1302 ----------------------------------------------------------------------
1303 escape sequence description
1304 ----------------------------------------------------------------------
1305 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1306 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1307 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1308 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1309 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1310 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1311 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1312 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1313 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1314 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1315 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1316 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1317 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1318 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1319 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1320 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1321 ----------------------------------------------------------------------
1323 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
1324 of dimension 1, chars 94, and final character <F>, etc...
1326 Note (*): Although these designations are not allowed in ISO2022,
1327 Emacs accepts them on decoding, and produces them on encoding
1328 CHARS96 character sets in a coding system which is characterized as
1329 7-bit environment, non-locking-shift, and non-single-shift.
1331 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1332 '(' can be omitted. We refer to this as "short-form" hereafter.
1334 Now you may notice that there are a lot of ways of encoding the
1335 same multilingual text in ISO2022. Actually, there exist many
1336 coding systems such as Compound Text (used in X11's inter client
1337 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1338 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
1339 localized platforms), and all of these are variants of ISO2022.
1341 In addition to the above, Emacs handles two more kinds of escape
1342 sequences: ISO6429's direction specification and Emacs' private
1343 sequence for specifying character composition.
1345 ISO6429's direction specification takes the following form:
1346 o CSI ']' -- end of the current direction
1347 o CSI '0' ']' -- end of the current direction
1348 o CSI '1' ']' -- start of left-to-right text
1349 o CSI '2' ']' -- start of right-to-left text
1350 The control character CSI (0x9B: control sequence introducer) is
1351 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1353 Character composition specification takes the following form:
1354 o ESC '0' -- start relative composition
1355 o ESC '1' -- end composition
1356 o ESC '2' -- start rule-base composition (*)
1357 o ESC '3' -- start relative composition with alternate chars (**)
1358 o ESC '4' -- start rule-base composition with alternate chars (**)
1359 Since these are not standard escape sequences of any ISO standard,
1360 the use of them with these meanings is restricted to Emacs only.
1362 (*) This form is used only in Emacs 20.5 and older versions,
1363 but the newer versions can safely decode it.
1364 (**) This form is used only in Emacs 21.1 and newer versions,
1365 and the older versions can't decode it.
1367 Here's a list of example usages of these composition escape
1368 sequences (categorized by `enum composition_method').
1370 COMPOSITION_RELATIVE:
1371 ESC 0 CHAR [ CHAR ] ESC 1
1372 COMPOSITION_WITH_RULE:
1373 ESC 2 CHAR [ RULE CHAR ] ESC 1
1374 COMPOSITION_WITH_ALTCHARS:
1375 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
1376 COMPOSITION_WITH_RULE_ALTCHARS:
1377 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
1379 enum iso_code_class_type iso_code_class
[256];
1381 #define CHARSET_OK(idx, charset, c) \
1382 (coding_system_table[idx] \
1383 && (charset == CHARSET_ASCII \
1384 || (safe_chars = coding_safe_chars (coding_system_table[idx]->symbol), \
1385 CODING_SAFE_CHAR_P (safe_chars, c))) \
1386 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1388 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
1390 #define SHIFT_OUT_OK(idx) \
1391 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1393 #define COMPOSITION_OK(idx) \
1394 (coding_system_table[idx]->composing != COMPOSITION_DISABLED)
1396 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1397 Check if a text is encoded in ISO2022. If it is, return an
1398 integer in which appropriate flag bits any of:
1399 CODING_CATEGORY_MASK_ISO_7
1400 CODING_CATEGORY_MASK_ISO_7_TIGHT
1401 CODING_CATEGORY_MASK_ISO_8_1
1402 CODING_CATEGORY_MASK_ISO_8_2
1403 CODING_CATEGORY_MASK_ISO_7_ELSE
1404 CODING_CATEGORY_MASK_ISO_8_ELSE
1405 are set. If a code which should never appear in ISO2022 is found,
1409 detect_coding_iso2022 (src
, src_end
, multibytep
)
1410 unsigned char *src
, *src_end
;
1413 int mask
= CODING_CATEGORY_MASK_ISO
;
1415 int reg
[4], shift_out
= 0, single_shifting
= 0;
1417 /* Dummy for ONE_MORE_BYTE. */
1418 struct coding_system dummy_coding
;
1419 struct coding_system
*coding
= &dummy_coding
;
1420 Lisp_Object safe_chars
;
1422 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
1423 while (mask
&& src
< src_end
)
1425 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1430 if (inhibit_iso_escape_detection
)
1432 single_shifting
= 0;
1433 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1434 if (c
>= '(' && c
<= '/')
1436 /* Designation sequence for a charset of dimension 1. */
1437 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1438 if (c1
< ' ' || c1
>= 0x80
1439 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
1440 /* Invalid designation sequence. Just ignore. */
1442 reg
[(c
- '(') % 4] = charset
;
1446 /* Designation sequence for a charset of dimension 2. */
1447 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1448 if (c
>= '@' && c
<= 'B')
1449 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1450 reg
[0] = charset
= iso_charset_table
[1][0][c
];
1451 else if (c
>= '(' && c
<= '/')
1453 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1454 if (c1
< ' ' || c1
>= 0x80
1455 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
1456 /* Invalid designation sequence. Just ignore. */
1458 reg
[(c
- '(') % 4] = charset
;
1461 /* Invalid designation sequence. Just ignore. */
1464 else if (c
== 'N' || c
== 'O')
1466 /* ESC <Fe> for SS2 or SS3. */
1467 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1470 else if (c
>= '0' && c
<= '4')
1472 /* ESC <Fp> for start/end composition. */
1473 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7
))
1474 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1476 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1477 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
))
1478 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1480 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1481 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_1
))
1482 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1484 mask
&= ~CODING_CATEGORY_MASK_ISO_8_1
;
1485 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_2
))
1486 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1488 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1489 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
))
1490 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1492 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1493 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
))
1494 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1496 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1500 /* Invalid escape sequence. Just ignore. */
1503 /* We found a valid designation sequence for CHARSET. */
1504 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
1505 c
= MAKE_CHAR (charset
, 0, 0);
1506 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
1507 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1509 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1510 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
1511 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1513 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1514 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
1515 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1517 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1518 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
1519 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1521 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1525 if (inhibit_iso_escape_detection
)
1527 single_shifting
= 0;
1530 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
1531 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
1533 /* Locking shift out. */
1534 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1535 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1540 if (inhibit_iso_escape_detection
)
1542 single_shifting
= 0;
1545 /* Locking shift in. */
1546 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1547 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1552 single_shifting
= 0;
1556 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1558 if (inhibit_iso_escape_detection
)
1560 if (c
!= ISO_CODE_CSI
)
1562 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1563 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1564 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1565 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1566 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1567 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1568 single_shifting
= 1;
1570 if (VECTORP (Vlatin_extra_code_table
)
1571 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1573 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1574 & CODING_FLAG_ISO_LATIN_EXTRA
)
1575 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1576 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1577 & CODING_FLAG_ISO_LATIN_EXTRA
)
1578 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1581 mask_found
|= newmask
;
1588 single_shifting
= 0;
1593 single_shifting
= 0;
1594 if (VECTORP (Vlatin_extra_code_table
)
1595 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1599 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1600 & CODING_FLAG_ISO_LATIN_EXTRA
)
1601 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1602 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1603 & CODING_FLAG_ISO_LATIN_EXTRA
)
1604 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1606 mask_found
|= newmask
;
1613 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1614 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1615 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1616 /* Check the length of succeeding codes of the range
1617 0xA0..0FF. If the byte length is odd, we exclude
1618 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1619 when we are not single shifting. */
1620 if (!single_shifting
1621 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1626 while (src
< src_end
)
1628 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1634 if (i
& 1 && src
< src_end
)
1635 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1637 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1639 /* This means that we have read one extra byte. */
1647 return (mask
& mask_found
);
1650 /* Decode a character of which charset is CHARSET, the 1st position
1651 code is C1, the 2nd position code is C2, and return the decoded
1652 character code. If the variable `translation_table' is non-nil,
1653 returned the translated code. */
1655 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1656 (NILP (translation_table) \
1657 ? MAKE_CHAR (charset, c1, c2) \
1658 : translate_char (translation_table, -1, charset, c1, c2))
1660 /* Set designation state into CODING. */
1661 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1665 if (final_char < '0' || final_char >= 128) \
1666 goto label_invalid_code; \
1667 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1668 make_number (chars), \
1669 make_number (final_char)); \
1670 c = MAKE_CHAR (charset, 0, 0); \
1672 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1673 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1675 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1677 && charset == CHARSET_ASCII) \
1679 /* We should insert this designation sequence as is so \
1680 that it is surely written back to a file. */ \
1681 coding->spec.iso2022.last_invalid_designation_register = -1; \
1682 goto label_invalid_code; \
1684 coding->spec.iso2022.last_invalid_designation_register = -1; \
1685 if ((coding->mode & CODING_MODE_DIRECTION) \
1686 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1687 charset = CHARSET_REVERSE_CHARSET (charset); \
1688 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1692 coding->spec.iso2022.last_invalid_designation_register = reg; \
1693 goto label_invalid_code; \
1697 /* Allocate a memory block for storing information about compositions.
1698 The block is chained to the already allocated blocks. */
1701 coding_allocate_composition_data (coding
, char_offset
)
1702 struct coding_system
*coding
;
1705 struct composition_data
*cmp_data
1706 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1708 cmp_data
->char_offset
= char_offset
;
1710 cmp_data
->prev
= coding
->cmp_data
;
1711 cmp_data
->next
= NULL
;
1712 if (coding
->cmp_data
)
1713 coding
->cmp_data
->next
= cmp_data
;
1714 coding
->cmp_data
= cmp_data
;
1715 coding
->cmp_data_start
= 0;
1716 coding
->composing
= COMPOSITION_NO
;
1719 /* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1720 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1721 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1722 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1723 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1726 #define DECODE_COMPOSITION_START(c1) \
1728 if (coding->composing == COMPOSITION_DISABLED) \
1730 *dst++ = ISO_CODE_ESC; \
1731 *dst++ = c1 & 0x7f; \
1732 coding->produced_char += 2; \
1734 else if (!COMPOSING_P (coding)) \
1736 /* This is surely the start of a composition. We must be sure \
1737 that coding->cmp_data has enough space to store the \
1738 information about the composition. If not, terminate the \
1739 current decoding loop, allocate one more memory block for \
1740 coding->cmp_data in the caller, then start the decoding \
1741 loop again. We can't allocate memory here directly because \
1742 it may cause buffer/string relocation. */ \
1743 if (!coding->cmp_data \
1744 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1745 >= COMPOSITION_DATA_SIZE)) \
1747 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1748 goto label_end_of_loop; \
1750 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1751 : c1 == '2' ? COMPOSITION_WITH_RULE \
1752 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1753 : COMPOSITION_WITH_RULE_ALTCHARS); \
1754 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1755 coding->composing); \
1756 coding->composition_rule_follows = 0; \
1760 /* We are already handling a composition. If the method is \
1761 the following two, the codes following the current escape \
1762 sequence are actual characters stored in a buffer. */ \
1763 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1764 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1766 coding->composing = COMPOSITION_RELATIVE; \
1767 coding->composition_rule_follows = 0; \
1772 /* Handle composition end sequence ESC 1. */
1774 #define DECODE_COMPOSITION_END(c1) \
1776 if (! COMPOSING_P (coding)) \
1778 *dst++ = ISO_CODE_ESC; \
1780 coding->produced_char += 2; \
1784 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1785 coding->composing = COMPOSITION_NO; \
1789 /* Decode a composition rule from the byte C1 (and maybe one more byte
1790 from SRC) and store one encoded composition rule in
1791 coding->cmp_data. */
1793 #define DECODE_COMPOSITION_RULE(c1) \
1797 if (c1 < 81) /* old format (before ver.21) */ \
1799 int gref = (c1) / 9; \
1800 int nref = (c1) % 9; \
1801 if (gref == 4) gref = 10; \
1802 if (nref == 4) nref = 10; \
1803 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1805 else if (c1 < 93) /* new format (after ver.21) */ \
1807 ONE_MORE_BYTE (c2); \
1808 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1810 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1811 coding->composition_rule_follows = 0; \
1815 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1818 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1819 struct coding_system
*coding
;
1820 const unsigned char *source
;
1821 unsigned char *destination
;
1822 int src_bytes
, dst_bytes
;
1824 const unsigned char *src
= source
;
1825 const unsigned char *src_end
= source
+ src_bytes
;
1826 unsigned char *dst
= destination
;
1827 unsigned char *dst_end
= destination
+ dst_bytes
;
1828 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1829 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1830 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1831 /* SRC_BASE remembers the start position in source in each loop.
1832 The loop will be exited when there's not enough source code
1833 (within macro ONE_MORE_BYTE), or when there's not enough
1834 destination area to produce a character (within macro
1836 const unsigned char *src_base
;
1838 Lisp_Object translation_table
;
1839 Lisp_Object safe_chars
;
1841 safe_chars
= coding_safe_chars (coding
->symbol
);
1843 if (NILP (Venable_character_translation
))
1844 translation_table
= Qnil
;
1847 translation_table
= coding
->translation_table_for_decode
;
1848 if (NILP (translation_table
))
1849 translation_table
= Vstandard_translation_table_for_decode
;
1852 coding
->result
= CODING_FINISH_NORMAL
;
1861 /* We produce no character or one character. */
1862 switch (iso_code_class
[c1
])
1864 case ISO_0x20_or_0x7F
:
1865 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1867 DECODE_COMPOSITION_RULE (c1
);
1870 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1872 /* This is SPACE or DEL. */
1873 charset
= CHARSET_ASCII
;
1876 /* This is a graphic character, we fall down ... */
1878 case ISO_graphic_plane_0
:
1879 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1881 DECODE_COMPOSITION_RULE (c1
);
1887 case ISO_0xA0_or_0xFF
:
1888 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1889 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1890 goto label_invalid_code
;
1891 /* This is a graphic character, we fall down ... */
1893 case ISO_graphic_plane_1
:
1895 goto label_invalid_code
;
1900 if (COMPOSING_P (coding
))
1901 DECODE_COMPOSITION_END ('1');
1903 /* All ISO2022 control characters in this class have the
1904 same representation in Emacs internal format. */
1906 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1907 && (coding
->eol_type
== CODING_EOL_CR
1908 || coding
->eol_type
== CODING_EOL_CRLF
))
1910 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1911 goto label_end_of_loop
;
1913 charset
= CHARSET_ASCII
;
1917 if (COMPOSING_P (coding
))
1918 DECODE_COMPOSITION_END ('1');
1919 goto label_invalid_code
;
1921 case ISO_carriage_return
:
1922 if (COMPOSING_P (coding
))
1923 DECODE_COMPOSITION_END ('1');
1925 if (coding
->eol_type
== CODING_EOL_CR
)
1927 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1930 if (c1
!= ISO_CODE_LF
)
1936 charset
= CHARSET_ASCII
;
1940 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1941 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1942 goto label_invalid_code
;
1943 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1944 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1948 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1949 goto label_invalid_code
;
1950 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1951 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1954 case ISO_single_shift_2_7
:
1955 case ISO_single_shift_2
:
1956 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1957 goto label_invalid_code
;
1958 /* SS2 is handled as an escape sequence of ESC 'N' */
1960 goto label_escape_sequence
;
1962 case ISO_single_shift_3
:
1963 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1964 goto label_invalid_code
;
1965 /* SS2 is handled as an escape sequence of ESC 'O' */
1967 goto label_escape_sequence
;
1969 case ISO_control_sequence_introducer
:
1970 /* CSI is handled as an escape sequence of ESC '[' ... */
1972 goto label_escape_sequence
;
1976 label_escape_sequence
:
1977 /* Escape sequences handled by Emacs are invocation,
1978 designation, direction specification, and character
1979 composition specification. */
1982 case '&': /* revision of following character set */
1984 if (!(c1
>= '@' && c1
<= '~'))
1985 goto label_invalid_code
;
1987 if (c1
!= ISO_CODE_ESC
)
1988 goto label_invalid_code
;
1990 goto label_escape_sequence
;
1992 case '$': /* designation of 2-byte character set */
1993 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1994 goto label_invalid_code
;
1996 if (c1
>= '@' && c1
<= 'B')
1997 { /* designation of JISX0208.1978, GB2312.1980,
1999 DECODE_DESIGNATION (0, 2, 94, c1
);
2001 else if (c1
>= 0x28 && c1
<= 0x2B)
2002 { /* designation of DIMENSION2_CHARS94 character set */
2004 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
2006 else if (c1
>= 0x2C && c1
<= 0x2F)
2007 { /* designation of DIMENSION2_CHARS96 character set */
2009 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
2012 goto label_invalid_code
;
2013 /* We must update these variables now. */
2014 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2015 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2018 case 'n': /* invocation of locking-shift-2 */
2019 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2020 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2021 goto label_invalid_code
;
2022 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
2023 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2026 case 'o': /* invocation of locking-shift-3 */
2027 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2028 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2029 goto label_invalid_code
;
2030 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
2031 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2034 case 'N': /* invocation of single-shift-2 */
2035 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2036 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2037 goto label_invalid_code
;
2038 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
2040 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2041 goto label_invalid_code
;
2044 case 'O': /* invocation of single-shift-3 */
2045 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2046 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2047 goto label_invalid_code
;
2048 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
2050 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2051 goto label_invalid_code
;
2054 case '0': case '2': case '3': case '4': /* start composition */
2055 DECODE_COMPOSITION_START (c1
);
2058 case '1': /* end composition */
2059 DECODE_COMPOSITION_END (c1
);
2062 case '[': /* specification of direction */
2063 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
2064 goto label_invalid_code
;
2065 /* For the moment, nested direction is not supported.
2066 So, `coding->mode & CODING_MODE_DIRECTION' zero means
2067 left-to-right, and nonzero means right-to-left. */
2071 case ']': /* end of the current direction */
2072 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2074 case '0': /* end of the current direction */
2075 case '1': /* start of left-to-right direction */
2078 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2080 goto label_invalid_code
;
2083 case '2': /* start of right-to-left direction */
2086 coding
->mode
|= CODING_MODE_DIRECTION
;
2088 goto label_invalid_code
;
2092 goto label_invalid_code
;
2097 if (COMPOSING_P (coding
))
2098 DECODE_COMPOSITION_END ('1');
2102 /* CTEXT extended segment:
2103 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
2104 We keep these bytes as is for the moment.
2105 They may be decoded by post-read-conversion. */
2110 ONE_MORE_BYTE (dim
);
2113 size
= ((M
- 128) * 128) + (L
- 128);
2114 required
= 8 + size
* 2;
2115 if (dst
+ required
> (dst_bytes
? dst_end
: src
))
2116 goto label_end_of_loop
;
2117 *dst
++ = ISO_CODE_ESC
;
2122 dst
+= CHAR_STRING (M
, dst
), produced_chars
++;
2123 dst
+= CHAR_STRING (L
, dst
), produced_chars
++;
2127 dst
+= CHAR_STRING (c1
, dst
), produced_chars
++;
2129 coding
->produced_char
+= produced_chars
;
2133 unsigned char *d
= dst
;
2136 /* XFree86 extension for embedding UTF-8 in CTEXT:
2137 ESC % G --UTF-8-BYTES-- ESC % @
2138 We keep these bytes as is for the moment.
2139 They may be decoded by post-read-conversion. */
2140 if (d
+ 6 > (dst_bytes
? dst_end
: src
))
2141 goto label_end_of_loop
;
2142 *d
++ = ISO_CODE_ESC
;
2146 while (d
+ 1 < (dst_bytes
? dst_end
: src
))
2149 if (c1
== ISO_CODE_ESC
2150 && src
+ 1 < src_end
2157 d
+= CHAR_STRING (c1
, d
), produced_chars
++;
2159 if (d
+ 3 > (dst_bytes
? dst_end
: src
))
2160 goto label_end_of_loop
;
2161 *d
++ = ISO_CODE_ESC
;
2165 coding
->produced_char
+= produced_chars
+ 3;
2168 goto label_invalid_code
;
2172 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
2173 goto label_invalid_code
;
2174 if (c1
>= 0x28 && c1
<= 0x2B)
2175 { /* designation of DIMENSION1_CHARS94 character set */
2177 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
2179 else if (c1
>= 0x2C && c1
<= 0x2F)
2180 { /* designation of DIMENSION1_CHARS96 character set */
2182 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
2185 goto label_invalid_code
;
2186 /* We must update these variables now. */
2187 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2188 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2193 /* Now we know CHARSET and 1st position code C1 of a character.
2194 Produce a multibyte sequence for that character while getting
2195 2nd position code C2 if necessary. */
2196 if (CHARSET_DIMENSION (charset
) == 2)
2199 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
2200 /* C2 is not in a valid range. */
2201 goto label_invalid_code
;
2203 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2209 if (COMPOSING_P (coding
))
2210 DECODE_COMPOSITION_END ('1');
2213 if (! NILP (translation_table
))
2214 c
= translate_char (translation_table
, c
, 0, 0, 0);
2219 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2220 coding
->produced
= dst
- destination
;
2225 /* ISO2022 encoding stuff. */
2228 It is not enough to say just "ISO2022" on encoding, we have to
2229 specify more details. In Emacs, each ISO2022 coding system
2230 variant has the following specifications:
2231 1. Initial designation to G0 through G3.
2232 2. Allows short-form designation?
2233 3. ASCII should be designated to G0 before control characters?
2234 4. ASCII should be designated to G0 at end of line?
2235 5. 7-bit environment or 8-bit environment?
2236 6. Use locking-shift?
2237 7. Use Single-shift?
2238 And the following two are only for Japanese:
2239 8. Use ASCII in place of JIS0201-1976-Roman?
2240 9. Use JISX0208-1983 in place of JISX0208-1978?
2241 These specifications are encoded in `coding->flags' as flag bits
2242 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
2246 /* Produce codes (escape sequence) for designating CHARSET to graphic
2247 register REG at DST, and increment DST. If <final-char> of CHARSET is
2248 '@', 'A', or 'B' and the coding system CODING allows, produce
2249 designation sequence of short-form. */
2251 #define ENCODE_DESIGNATION(charset, reg, coding) \
2253 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2254 char *intermediate_char_94 = "()*+"; \
2255 char *intermediate_char_96 = ",-./"; \
2256 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
2258 if (revision < 255) \
2260 *dst++ = ISO_CODE_ESC; \
2262 *dst++ = '@' + revision; \
2264 *dst++ = ISO_CODE_ESC; \
2265 if (CHARSET_DIMENSION (charset) == 1) \
2267 if (CHARSET_CHARS (charset) == 94) \
2268 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2270 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2275 if (CHARSET_CHARS (charset) == 94) \
2277 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2279 || final_char < '@' || final_char > 'B') \
2280 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2283 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2285 *dst++ = final_char; \
2286 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2289 /* The following two macros produce codes (control character or escape
2290 sequence) for ISO2022 single-shift functions (single-shift-2 and
2293 #define ENCODE_SINGLE_SHIFT_2 \
2295 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2296 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2298 *dst++ = ISO_CODE_SS2; \
2299 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2302 #define ENCODE_SINGLE_SHIFT_3 \
2304 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2305 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2307 *dst++ = ISO_CODE_SS3; \
2308 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2311 /* The following four macros produce codes (control character or
2312 escape sequence) for ISO2022 locking-shift functions (shift-in,
2313 shift-out, locking-shift-2, and locking-shift-3). */
2315 #define ENCODE_SHIFT_IN \
2317 *dst++ = ISO_CODE_SI; \
2318 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2321 #define ENCODE_SHIFT_OUT \
2323 *dst++ = ISO_CODE_SO; \
2324 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2327 #define ENCODE_LOCKING_SHIFT_2 \
2329 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2330 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2333 #define ENCODE_LOCKING_SHIFT_3 \
2335 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
2336 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2339 /* Produce codes for a DIMENSION1 character whose character set is
2340 CHARSET and whose position-code is C1. Designation and invocation
2341 sequences are also produced in advance if necessary. */
2343 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2345 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2347 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2348 *dst++ = c1 & 0x7F; \
2350 *dst++ = c1 | 0x80; \
2351 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2354 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2356 *dst++ = c1 & 0x7F; \
2359 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2361 *dst++ = c1 | 0x80; \
2365 /* Since CHARSET is not yet invoked to any graphic planes, we \
2366 must invoke it, or, at first, designate it to some graphic \
2367 register. Then repeat the loop to actually produce the \
2369 dst = encode_invocation_designation (charset, coding, dst); \
2372 /* Produce codes for a DIMENSION2 character whose character set is
2373 CHARSET and whose position-codes are C1 and C2. Designation and
2374 invocation codes are also produced in advance if necessary. */
2376 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2378 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2380 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2381 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2383 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2384 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2387 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2389 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2392 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2394 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2398 /* Since CHARSET is not yet invoked to any graphic planes, we \
2399 must invoke it, or, at first, designate it to some graphic \
2400 register. Then repeat the loop to actually produce the \
2402 dst = encode_invocation_designation (charset, coding, dst); \
2405 #define ENCODE_ISO_CHARACTER(c) \
2407 int charset, c1, c2; \
2409 SPLIT_CHAR (c, charset, c1, c2); \
2410 if (CHARSET_DEFINED_P (charset)) \
2412 if (CHARSET_DIMENSION (charset) == 1) \
2414 if (charset == CHARSET_ASCII \
2415 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2416 charset = charset_latin_jisx0201; \
2417 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2421 if (charset == charset_jisx0208 \
2422 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2423 charset = charset_jisx0208_1978; \
2424 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2436 /* Instead of encoding character C, produce one or two `?'s. */
2438 #define ENCODE_UNSAFE_CHARACTER(c) \
2440 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2441 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2442 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2446 /* Produce designation and invocation codes at a place pointed by DST
2447 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2451 encode_invocation_designation (charset
, coding
, dst
)
2453 struct coding_system
*coding
;
2456 int reg
; /* graphic register number */
2458 /* At first, check designations. */
2459 for (reg
= 0; reg
< 4; reg
++)
2460 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
2465 /* CHARSET is not yet designated to any graphic registers. */
2466 /* At first check the requested designation. */
2467 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2468 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
2469 /* Since CHARSET requests no special designation, designate it
2470 to graphic register 0. */
2473 ENCODE_DESIGNATION (charset
, reg
, coding
);
2476 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
2477 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
2479 /* Since the graphic register REG is not invoked to any graphic
2480 planes, invoke it to graphic plane 0. */
2483 case 0: /* graphic register 0 */
2487 case 1: /* graphic register 1 */
2491 case 2: /* graphic register 2 */
2492 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2493 ENCODE_SINGLE_SHIFT_2
;
2495 ENCODE_LOCKING_SHIFT_2
;
2498 case 3: /* graphic register 3 */
2499 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2500 ENCODE_SINGLE_SHIFT_3
;
2502 ENCODE_LOCKING_SHIFT_3
;
2510 /* Produce 2-byte codes for encoded composition rule RULE. */
2512 #define ENCODE_COMPOSITION_RULE(rule) \
2515 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2516 *dst++ = 32 + 81 + gref; \
2517 *dst++ = 32 + nref; \
2520 /* Produce codes for indicating the start of a composition sequence
2521 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2522 which specify information about the composition. See the comment
2523 in coding.h for the format of DATA. */
2525 #define ENCODE_COMPOSITION_START(coding, data) \
2527 coding->composing = data[3]; \
2528 *dst++ = ISO_CODE_ESC; \
2529 if (coding->composing == COMPOSITION_RELATIVE) \
2533 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2535 coding->cmp_data_index = coding->cmp_data_start + 4; \
2536 coding->composition_rule_follows = 0; \
2540 /* Produce codes for indicating the end of the current composition. */
2542 #define ENCODE_COMPOSITION_END(coding, data) \
2544 *dst++ = ISO_CODE_ESC; \
2546 coding->cmp_data_start += data[0]; \
2547 coding->composing = COMPOSITION_NO; \
2548 if (coding->cmp_data_start == coding->cmp_data->used \
2549 && coding->cmp_data->next) \
2551 coding->cmp_data = coding->cmp_data->next; \
2552 coding->cmp_data_start = 0; \
2556 /* Produce composition start sequence ESC 0. Here, this sequence
2557 doesn't mean the start of a new composition but means that we have
2558 just produced components (alternate chars and composition rules) of
2559 the composition and the actual text follows in SRC. */
2561 #define ENCODE_COMPOSITION_FAKE_START(coding) \
2563 *dst++ = ISO_CODE_ESC; \
2565 coding->composing = COMPOSITION_RELATIVE; \
2568 /* The following three macros produce codes for indicating direction
2570 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2572 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2573 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2575 *dst++ = ISO_CODE_CSI; \
2578 #define ENCODE_DIRECTION_R2L \
2579 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2581 #define ENCODE_DIRECTION_L2R \
2582 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2584 /* Produce codes for designation and invocation to reset the graphic
2585 planes and registers to initial state. */
2586 #define ENCODE_RESET_PLANE_AND_REGISTER \
2589 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2591 for (reg = 0; reg < 4; reg++) \
2592 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2593 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2594 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2595 ENCODE_DESIGNATION \
2596 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
2599 /* Produce designation sequences of charsets in the line started from
2600 SRC to a place pointed by DST, and return updated DST.
2602 If the current block ends before any end-of-line, we may fail to
2603 find all the necessary designations. */
2605 static unsigned char *
2606 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
2607 struct coding_system
*coding
;
2608 Lisp_Object translation_table
;
2609 const unsigned char *src
, *src_end
;
2612 int charset
, c
, found
= 0, reg
;
2613 /* Table of charsets to be designated to each graphic register. */
2616 for (reg
= 0; reg
< 4; reg
++)
2625 charset
= CHAR_CHARSET (c
);
2626 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2627 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
2637 for (reg
= 0; reg
< 4; reg
++)
2639 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
2640 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2646 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2649 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2650 struct coding_system
*coding
;
2651 const unsigned char *source
;
2652 unsigned char *destination
;
2653 int src_bytes
, dst_bytes
;
2655 const unsigned char *src
= source
;
2656 const unsigned char *src_end
= source
+ src_bytes
;
2657 unsigned char *dst
= destination
;
2658 unsigned char *dst_end
= destination
+ dst_bytes
;
2659 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2660 from DST_END to assure overflow checking is necessary only at the
2662 unsigned char *adjusted_dst_end
= dst_end
- 19;
2663 /* SRC_BASE remembers the start position in source in each loop.
2664 The loop will be exited when there's not enough source text to
2665 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2666 there's not enough destination area to produce encoded codes
2667 (within macro EMIT_BYTES). */
2668 const unsigned char *src_base
;
2670 Lisp_Object translation_table
;
2671 Lisp_Object safe_chars
;
2673 if (coding
->flags
& CODING_FLAG_ISO_SAFE
)
2674 coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
2676 safe_chars
= coding_safe_chars (coding
->symbol
);
2678 if (NILP (Venable_character_translation
))
2679 translation_table
= Qnil
;
2682 translation_table
= coding
->translation_table_for_encode
;
2683 if (NILP (translation_table
))
2684 translation_table
= Vstandard_translation_table_for_encode
;
2687 coding
->consumed_char
= 0;
2693 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2695 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2699 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2700 && CODING_SPEC_ISO_BOL (coding
))
2702 /* We have to produce designation sequences if any now. */
2703 dst
= encode_designation_at_bol (coding
, translation_table
,
2705 CODING_SPEC_ISO_BOL (coding
) = 0;
2708 /* Check composition start and end. */
2709 if (coding
->composing
!= COMPOSITION_DISABLED
2710 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2712 struct composition_data
*cmp_data
= coding
->cmp_data
;
2713 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2714 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2716 if (coding
->composing
== COMPOSITION_RELATIVE
)
2718 if (this_pos
== data
[2])
2720 ENCODE_COMPOSITION_END (coding
, data
);
2721 cmp_data
= coding
->cmp_data
;
2722 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2725 else if (COMPOSING_P (coding
))
2727 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2728 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2729 /* We have consumed components of the composition.
2730 What follows in SRC is the composition's base
2732 ENCODE_COMPOSITION_FAKE_START (coding
);
2735 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2736 if (coding
->composition_rule_follows
)
2738 ENCODE_COMPOSITION_RULE (c
);
2739 coding
->composition_rule_follows
= 0;
2743 if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2744 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2745 ENCODE_UNSAFE_CHARACTER (c
);
2747 ENCODE_ISO_CHARACTER (c
);
2748 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2749 coding
->composition_rule_follows
= 1;
2754 if (!COMPOSING_P (coding
))
2756 if (this_pos
== data
[1])
2758 ENCODE_COMPOSITION_START (coding
, data
);
2766 /* Now encode the character C. */
2767 if (c
< 0x20 || c
== 0x7F)
2771 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2773 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2774 ENCODE_RESET_PLANE_AND_REGISTER
;
2778 /* fall down to treat '\r' as '\n' ... */
2783 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2784 ENCODE_RESET_PLANE_AND_REGISTER
;
2785 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2786 bcopy (coding
->spec
.iso2022
.initial_designation
,
2787 coding
->spec
.iso2022
.current_designation
,
2788 sizeof coding
->spec
.iso2022
.initial_designation
);
2789 if (coding
->eol_type
== CODING_EOL_LF
2790 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2791 *dst
++ = ISO_CODE_LF
;
2792 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2793 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2795 *dst
++ = ISO_CODE_CR
;
2796 CODING_SPEC_ISO_BOL (coding
) = 1;
2800 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2801 ENCODE_RESET_PLANE_AND_REGISTER
;
2805 else if (ASCII_BYTE_P (c
))
2806 ENCODE_ISO_CHARACTER (c
);
2807 else if (SINGLE_BYTE_CHAR_P (c
))
2812 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2813 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2814 ENCODE_UNSAFE_CHARACTER (c
);
2816 ENCODE_ISO_CHARACTER (c
);
2818 coding
->consumed_char
++;
2822 coding
->consumed
= src_base
- source
;
2823 coding
->produced
= coding
->produced_char
= dst
- destination
;
2827 /*** 4. SJIS and BIG5 handlers ***/
2829 /* Although SJIS and BIG5 are not ISO coding systems, they are used
2830 quite widely. So, for the moment, Emacs supports them in the bare
2831 C code. But, in the future, they may be supported only by CCL. */
2833 /* SJIS is a coding system encoding three character sets: ASCII, right
2834 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2835 as is. A character of charset katakana-jisx0201 is encoded by
2836 "position-code + 0x80". A character of charset japanese-jisx0208
2837 is encoded in 2-byte but two position-codes are divided and shifted
2838 so that it fits in the range below.
2840 --- CODE RANGE of SJIS ---
2841 (character set) (range)
2843 KATAKANA-JISX0201 0xA1 .. 0xDF
2844 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2845 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2846 -------------------------------
2850 /* BIG5 is a coding system encoding two character sets: ASCII and
2851 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2852 character set and is encoded in two bytes.
2854 --- CODE RANGE of BIG5 ---
2855 (character set) (range)
2857 Big5 (1st byte) 0xA1 .. 0xFE
2858 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2859 --------------------------
2861 Since the number of characters in Big5 is larger than maximum
2862 characters in Emacs' charset (96x96), it can't be handled as one
2863 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2864 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2865 contains frequently used characters and the latter contains less
2866 frequently used characters. */
2868 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2869 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2870 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
2871 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2873 /* Number of Big5 characters which have the same code in 1st byte. */
2874 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2876 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2879 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2881 charset = charset_big5_1; \
2884 charset = charset_big5_2; \
2885 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2887 c1 = temp / (0xFF - 0xA1) + 0x21; \
2888 c2 = temp % (0xFF - 0xA1) + 0x21; \
2891 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2893 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2894 if (charset == charset_big5_2) \
2895 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2896 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2897 b2 = temp % BIG5_SAME_ROW; \
2898 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2901 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2902 Check if a text is encoded in SJIS. If it is, return
2903 CODING_CATEGORY_MASK_SJIS, else return 0. */
2906 detect_coding_sjis (src
, src_end
, multibytep
)
2907 unsigned char *src
, *src_end
;
2911 /* Dummy for ONE_MORE_BYTE. */
2912 struct coding_system dummy_coding
;
2913 struct coding_system
*coding
= &dummy_coding
;
2917 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2920 if (c
== 0x80 || c
== 0xA0 || c
> 0xEF)
2922 if (c
<= 0x9F || c
>= 0xE0)
2924 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2925 if (c
< 0x40 || c
== 0x7F || c
> 0xFC)
2930 return CODING_CATEGORY_MASK_SJIS
;
2933 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2934 Check if a text is encoded in BIG5. If it is, return
2935 CODING_CATEGORY_MASK_BIG5, else return 0. */
2938 detect_coding_big5 (src
, src_end
, multibytep
)
2939 unsigned char *src
, *src_end
;
2943 /* Dummy for ONE_MORE_BYTE. */
2944 struct coding_system dummy_coding
;
2945 struct coding_system
*coding
= &dummy_coding
;
2949 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2952 if (c
< 0xA1 || c
> 0xFE)
2954 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2955 if (c
< 0x40 || (c
> 0x7F && c
< 0xA1) || c
> 0xFE)
2959 return CODING_CATEGORY_MASK_BIG5
;
2962 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2963 Check if a text is encoded in UTF-8. If it is, return
2964 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2966 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2967 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2968 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2969 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2970 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2971 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2972 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2975 detect_coding_utf_8 (src
, src_end
, multibytep
)
2976 unsigned char *src
, *src_end
;
2980 int seq_maybe_bytes
;
2981 /* Dummy for ONE_MORE_BYTE. */
2982 struct coding_system dummy_coding
;
2983 struct coding_system
*coding
= &dummy_coding
;
2987 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2988 if (UTF_8_1_OCTET_P (c
))
2990 else if (UTF_8_2_OCTET_LEADING_P (c
))
2991 seq_maybe_bytes
= 1;
2992 else if (UTF_8_3_OCTET_LEADING_P (c
))
2993 seq_maybe_bytes
= 2;
2994 else if (UTF_8_4_OCTET_LEADING_P (c
))
2995 seq_maybe_bytes
= 3;
2996 else if (UTF_8_5_OCTET_LEADING_P (c
))
2997 seq_maybe_bytes
= 4;
2998 else if (UTF_8_6_OCTET_LEADING_P (c
))
2999 seq_maybe_bytes
= 5;
3005 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3006 if (!UTF_8_EXTRA_OCTET_P (c
))
3010 while (seq_maybe_bytes
> 0);
3014 return CODING_CATEGORY_MASK_UTF_8
;
3017 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3018 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
3019 Little Endian (otherwise). If it is, return
3020 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
3023 #define UTF_16_INVALID_P(val) \
3024 (((val) == 0xFFFE) \
3025 || ((val) == 0xFFFF))
3027 #define UTF_16_HIGH_SURROGATE_P(val) \
3028 (((val) & 0xD800) == 0xD800)
3030 #define UTF_16_LOW_SURROGATE_P(val) \
3031 (((val) & 0xDC00) == 0xDC00)
3034 detect_coding_utf_16 (src
, src_end
, multibytep
)
3035 unsigned char *src
, *src_end
;
3038 unsigned char c1
, c2
;
3039 /* Dummy for ONE_MORE_BYTE_CHECK_MULTIBYTE. */
3040 struct coding_system dummy_coding
;
3041 struct coding_system
*coding
= &dummy_coding
;
3043 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
3044 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2
, multibytep
);
3046 if ((c1
== 0xFF) && (c2
== 0xFE))
3047 return CODING_CATEGORY_MASK_UTF_16_LE
;
3048 else if ((c1
== 0xFE) && (c2
== 0xFF))
3049 return CODING_CATEGORY_MASK_UTF_16_BE
;
3055 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
3056 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
3059 decode_coding_sjis_big5 (coding
, source
, destination
,
3060 src_bytes
, dst_bytes
, sjis_p
)
3061 struct coding_system
*coding
;
3062 const unsigned char *source
;
3063 unsigned char *destination
;
3064 int src_bytes
, dst_bytes
;
3067 const unsigned char *src
= source
;
3068 const unsigned char *src_end
= source
+ src_bytes
;
3069 unsigned char *dst
= destination
;
3070 unsigned char *dst_end
= destination
+ dst_bytes
;
3071 /* SRC_BASE remembers the start position in source in each loop.
3072 The loop will be exited when there's not enough source code
3073 (within macro ONE_MORE_BYTE), or when there's not enough
3074 destination area to produce a character (within macro
3076 const unsigned char *src_base
;
3077 Lisp_Object translation_table
;
3079 if (NILP (Venable_character_translation
))
3080 translation_table
= Qnil
;
3083 translation_table
= coding
->translation_table_for_decode
;
3084 if (NILP (translation_table
))
3085 translation_table
= Vstandard_translation_table_for_decode
;
3088 coding
->produced_char
= 0;
3091 int c
, charset
, c1
, c2
= 0;
3098 charset
= CHARSET_ASCII
;
3103 if (coding
->eol_type
== CODING_EOL_CRLF
)
3109 /* To process C2 again, SRC is subtracted by 1. */
3112 else if (coding
->eol_type
== CODING_EOL_CR
)
3116 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3117 && (coding
->eol_type
== CODING_EOL_CR
3118 || coding
->eol_type
== CODING_EOL_CRLF
))
3120 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3121 goto label_end_of_loop
;
3129 if (c1
== 0x80 || c1
== 0xA0 || c1
> 0xEF)
3130 goto label_invalid_code
;
3131 if (c1
<= 0x9F || c1
>= 0xE0)
3133 /* SJIS -> JISX0208 */
3135 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
3136 goto label_invalid_code
;
3137 DECODE_SJIS (c1
, c2
, c1
, c2
);
3138 charset
= charset_jisx0208
;
3141 /* SJIS -> JISX0201-Kana */
3142 charset
= charset_katakana_jisx0201
;
3147 if (c1
< 0xA0 || c1
> 0xFE)
3148 goto label_invalid_code
;
3150 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
3151 goto label_invalid_code
;
3152 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
3156 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
3168 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3169 coding
->produced
= dst
- destination
;
3173 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
3174 This function can encode charsets `ascii', `katakana-jisx0201',
3175 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
3176 are sure that all these charsets are registered as official charset
3177 (i.e. do not have extended leading-codes). Characters of other
3178 charsets are produced without any encoding. If SJIS_P is 1, encode
3179 SJIS text, else encode BIG5 text. */
3182 encode_coding_sjis_big5 (coding
, source
, destination
,
3183 src_bytes
, dst_bytes
, sjis_p
)
3184 struct coding_system
*coding
;
3185 unsigned char *source
, *destination
;
3186 int src_bytes
, dst_bytes
;
3189 unsigned char *src
= source
;
3190 unsigned char *src_end
= source
+ src_bytes
;
3191 unsigned char *dst
= destination
;
3192 unsigned char *dst_end
= destination
+ dst_bytes
;
3193 /* SRC_BASE remembers the start position in source in each loop.
3194 The loop will be exited when there's not enough source text to
3195 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3196 there's not enough destination area to produce encoded codes
3197 (within macro EMIT_BYTES). */
3198 unsigned char *src_base
;
3199 Lisp_Object translation_table
;
3201 if (NILP (Venable_character_translation
))
3202 translation_table
= Qnil
;
3205 translation_table
= coding
->translation_table_for_encode
;
3206 if (NILP (translation_table
))
3207 translation_table
= Vstandard_translation_table_for_encode
;
3212 int c
, charset
, c1
, c2
;
3217 /* Now encode the character C. */
3218 if (SINGLE_BYTE_CHAR_P (c
))
3223 if (!(coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
3230 if (coding
->eol_type
== CODING_EOL_CRLF
)
3232 EMIT_TWO_BYTES ('\r', c
);
3235 else if (coding
->eol_type
== CODING_EOL_CR
)
3243 SPLIT_CHAR (c
, charset
, c1
, c2
);
3246 if (charset
== charset_jisx0208
3247 || charset
== charset_jisx0208_1978
)
3249 ENCODE_SJIS (c1
, c2
, c1
, c2
);
3250 EMIT_TWO_BYTES (c1
, c2
);
3252 else if (charset
== charset_katakana_jisx0201
)
3253 EMIT_ONE_BYTE (c1
| 0x80);
3254 else if (charset
== charset_latin_jisx0201
)
3256 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3258 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3259 if (CHARSET_WIDTH (charset
) > 1)
3260 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3263 /* There's no way other than producing the internal
3265 EMIT_BYTES (src_base
, src
);
3269 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
3271 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
3272 EMIT_TWO_BYTES (c1
, c2
);
3274 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3276 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3277 if (CHARSET_WIDTH (charset
) > 1)
3278 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3281 /* There's no way other than producing the internal
3283 EMIT_BYTES (src_base
, src
);
3286 coding
->consumed_char
++;
3290 coding
->consumed
= src_base
- source
;
3291 coding
->produced
= coding
->produced_char
= dst
- destination
;
3295 /*** 5. CCL handlers ***/
3297 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3298 Check if a text is encoded in a coding system of which
3299 encoder/decoder are written in CCL program. If it is, return
3300 CODING_CATEGORY_MASK_CCL, else return 0. */
3303 detect_coding_ccl (src
, src_end
, multibytep
)
3304 unsigned char *src
, *src_end
;
3307 unsigned char *valid
;
3309 /* Dummy for ONE_MORE_BYTE. */
3310 struct coding_system dummy_coding
;
3311 struct coding_system
*coding
= &dummy_coding
;
3313 /* No coding system is assigned to coding-category-ccl. */
3314 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
3317 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
3320 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3325 return CODING_CATEGORY_MASK_CCL
;
3329 /*** 6. End-of-line handlers ***/
3331 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3334 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3335 struct coding_system
*coding
;
3336 const unsigned char *source
;
3337 unsigned char *destination
;
3338 int src_bytes
, dst_bytes
;
3340 const unsigned char *src
= source
;
3341 unsigned char *dst
= destination
;
3342 const unsigned char *src_end
= src
+ src_bytes
;
3343 unsigned char *dst_end
= dst
+ dst_bytes
;
3344 Lisp_Object translation_table
;
3345 /* SRC_BASE remembers the start position in source in each loop.
3346 The loop will be exited when there's not enough source code
3347 (within macro ONE_MORE_BYTE), or when there's not enough
3348 destination area to produce a character (within macro
3350 const unsigned char *src_base
;
3353 translation_table
= Qnil
;
3354 switch (coding
->eol_type
)
3356 case CODING_EOL_CRLF
:
3371 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
3373 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3374 goto label_end_of_loop
;
3387 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3389 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3390 goto label_end_of_loop
;
3399 default: /* no need for EOL handling */
3409 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3410 coding
->produced
= dst
- destination
;
3414 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
3415 format of end-of-line according to `coding->eol_type'. It also
3416 convert multibyte form 8-bit characters to unibyte if
3417 CODING->src_multibyte is nonzero. If `coding->mode &
3418 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3419 also means end-of-line. */
3422 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3423 struct coding_system
*coding
;
3424 const unsigned char *source
;
3425 unsigned char *destination
;
3426 int src_bytes
, dst_bytes
;
3428 const unsigned char *src
= source
;
3429 unsigned char *dst
= destination
;
3430 const unsigned char *src_end
= src
+ src_bytes
;
3431 unsigned char *dst_end
= dst
+ dst_bytes
;
3432 Lisp_Object translation_table
;
3433 /* SRC_BASE remembers the start position in source in each loop.
3434 The loop will be exited when there's not enough source text to
3435 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3436 there's not enough destination area to produce encoded codes
3437 (within macro EMIT_BYTES). */
3438 const unsigned char *src_base
;
3441 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
3443 translation_table
= Qnil
;
3444 if (coding
->src_multibyte
3445 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3449 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
3452 if (coding
->eol_type
== CODING_EOL_CRLF
)
3454 while (src
< src_end
)
3460 else if (c
== '\n' || (c
== '\r' && selective_display
))
3461 EMIT_TWO_BYTES ('\r', '\n');
3471 if (!dst_bytes
|| src_bytes
<= dst_bytes
)
3473 safe_bcopy (src
, dst
, src_bytes
);
3479 if (coding
->src_multibyte
3480 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3482 safe_bcopy (src
, dst
, dst_bytes
);
3483 src_base
= src
+ dst_bytes
;
3484 dst
= destination
+ dst_bytes
;
3485 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
3487 if (coding
->eol_type
== CODING_EOL_CR
)
3489 for (tmp
= destination
; tmp
< dst
; tmp
++)
3490 if (*tmp
== '\n') *tmp
= '\r';
3492 else if (selective_display
)
3494 for (tmp
= destination
; tmp
< dst
; tmp
++)
3495 if (*tmp
== '\r') *tmp
= '\n';
3498 if (coding
->src_multibyte
)
3499 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
3501 coding
->consumed
= src_base
- source
;
3502 coding
->produced
= dst
- destination
;
3503 coding
->produced_char
= coding
->produced
;
3507 /*** 7. C library functions ***/
3509 /* In Emacs Lisp, a coding system is represented by a Lisp symbol which
3510 has a property `coding-system'. The value of this property is a
3511 vector of length 5 (called the coding-vector). Among elements of
3512 this vector, the first (element[0]) and the fifth (element[4])
3513 carry important information for decoding/encoding. Before
3514 decoding/encoding, this information should be set in fields of a
3515 structure of type `coding_system'.
3517 The value of the property `coding-system' can be a symbol of another
3518 subsidiary coding-system. In that case, Emacs gets coding-vector
3521 `element[0]' contains information to be set in `coding->type'. The
3522 value and its meaning is as follows:
3524 0 -- coding_type_emacs_mule
3525 1 -- coding_type_sjis
3526 2 -- coding_type_iso2022
3527 3 -- coding_type_big5
3528 4 -- coding_type_ccl encoder/decoder written in CCL
3529 nil -- coding_type_no_conversion
3530 t -- coding_type_undecided (automatic conversion on decoding,
3531 no-conversion on encoding)
3533 `element[4]' contains information to be set in `coding->flags' and
3534 `coding->spec'. The meaning varies by `coding->type'.
3536 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3537 of length 32 (of which the first 13 sub-elements are used now).
3538 Meanings of these sub-elements are:
3540 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3541 If the value is an integer of valid charset, the charset is
3542 assumed to be designated to graphic register N initially.
3544 If the value is minus, it is a minus value of charset which
3545 reserves graphic register N, which means that the charset is
3546 not designated initially but should be designated to graphic
3547 register N just before encoding a character in that charset.
3549 If the value is nil, graphic register N is never used on
3552 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3553 Each value takes t or nil. See the section ISO2022 of
3554 `coding.h' for more information.
3556 If `coding->type' is `coding_type_big5', element[4] is t to denote
3557 BIG5-ETen or nil to denote BIG5-HKU.
3559 If `coding->type' takes the other value, element[4] is ignored.
3561 Emacs Lisp's coding systems also carry information about format of
3562 end-of-line in a value of property `eol-type'. If the value is
3563 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3564 means CODING_EOL_CR. If it is not integer, it should be a vector
3565 of subsidiary coding systems of which property `eol-type' has one
3566 of the above values.
3570 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3571 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3572 is setup so that no conversion is necessary and return -1, else
3576 setup_coding_system (coding_system
, coding
)
3577 Lisp_Object coding_system
;
3578 struct coding_system
*coding
;
3580 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
3583 /* At first, zero clear all members. */
3584 bzero (coding
, sizeof (struct coding_system
));
3586 /* Initialize some fields required for all kinds of coding systems. */
3587 coding
->symbol
= coding_system
;
3588 coding
->heading_ascii
= -1;
3589 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
3590 coding
->composing
= COMPOSITION_DISABLED
;
3591 coding
->cmp_data
= NULL
;
3593 if (NILP (coding_system
))
3594 goto label_invalid_coding_system
;
3596 coding_spec
= Fget (coding_system
, Qcoding_system
);
3598 if (!VECTORP (coding_spec
)
3599 || XVECTOR (coding_spec
)->size
!= 5
3600 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
3601 goto label_invalid_coding_system
;
3603 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
3604 if (VECTORP (eol_type
))
3606 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3607 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
3608 if (system_eol_type
!= CODING_EOL_LF
)
3609 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3611 else if (XFASTINT (eol_type
) == 1)
3613 coding
->eol_type
= CODING_EOL_CRLF
;
3614 coding
->common_flags
3615 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3617 else if (XFASTINT (eol_type
) == 2)
3619 coding
->eol_type
= CODING_EOL_CR
;
3620 coding
->common_flags
3621 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3624 coding
->eol_type
= CODING_EOL_LF
;
3626 coding_type
= XVECTOR (coding_spec
)->contents
[0];
3627 /* Try short cut. */
3628 if (SYMBOLP (coding_type
))
3630 if (EQ (coding_type
, Qt
))
3632 coding
->type
= coding_type_undecided
;
3633 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
3636 coding
->type
= coding_type_no_conversion
;
3637 /* Initialize this member. Any thing other than
3638 CODING_CATEGORY_IDX_UTF_16_BE and
3639 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3640 special treatment in detect_eol. */
3641 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
3646 /* Get values of coding system properties:
3647 `post-read-conversion', `pre-write-conversion',
3648 `translation-table-for-decode', `translation-table-for-encode'. */
3649 plist
= XVECTOR (coding_spec
)->contents
[3];
3650 /* Pre & post conversion functions should be disabled if
3651 inhibit_eol_conversion is nonzero. This is the case that a code
3652 conversion function is called while those functions are running. */
3653 if (! inhibit_pre_post_conversion
)
3655 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
3656 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
3658 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
3660 val
= Fget (val
, Qtranslation_table_for_decode
);
3661 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3662 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
3664 val
= Fget (val
, Qtranslation_table_for_encode
);
3665 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3666 val
= Fplist_get (plist
, Qcoding_category
);
3669 val
= Fget (val
, Qcoding_category_index
);
3671 coding
->category_idx
= XINT (val
);
3673 goto label_invalid_coding_system
;
3676 goto label_invalid_coding_system
;
3678 /* If the coding system has non-nil `composition' property, enable
3679 composition handling. */
3680 val
= Fplist_get (plist
, Qcomposition
);
3682 coding
->composing
= COMPOSITION_NO
;
3684 switch (XFASTINT (coding_type
))
3687 coding
->type
= coding_type_emacs_mule
;
3688 coding
->common_flags
3689 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3690 if (!NILP (coding
->post_read_conversion
))
3691 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3692 if (!NILP (coding
->pre_write_conversion
))
3693 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3697 coding
->type
= coding_type_sjis
;
3698 coding
->common_flags
3699 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3703 coding
->type
= coding_type_iso2022
;
3704 coding
->common_flags
3705 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3707 Lisp_Object val
, temp
;
3709 int i
, charset
, reg_bits
= 0;
3711 val
= XVECTOR (coding_spec
)->contents
[4];
3713 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3714 goto label_invalid_coding_system
;
3716 flags
= XVECTOR (val
)->contents
;
3718 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3719 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3720 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3721 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3722 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3723 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3724 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3725 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3726 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3727 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3728 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3729 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3730 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3733 /* Invoke graphic register 0 to plane 0. */
3734 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3735 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3736 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3737 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3738 /* Not single shifting at first. */
3739 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3740 /* Beginning of buffer should also be regarded as bol. */
3741 CODING_SPEC_ISO_BOL (coding
) = 1;
3743 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3744 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3745 val
= Vcharset_revision_alist
;
3748 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3750 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3751 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3752 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3756 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3757 FLAGS[REG] can be one of below:
3758 integer CHARSET: CHARSET occupies register I,
3759 t: designate nothing to REG initially, but can be used
3761 list of integer, nil, or t: designate the first
3762 element (if integer) to REG initially, the remaining
3763 elements (if integer) is designated to REG on request,
3764 if an element is t, REG can be used by any charsets,
3765 nil: REG is never used. */
3766 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3767 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3768 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3769 for (i
= 0; i
< 4; i
++)
3771 if ((INTEGERP (flags
[i
])
3772 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
)))
3773 || (charset
= get_charset_id (flags
[i
])) >= 0)
3775 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3776 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3778 else if (EQ (flags
[i
], Qt
))
3780 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3782 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3784 else if (CONSP (flags
[i
]))
3789 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3790 if ((INTEGERP (XCAR (tail
))
3791 && (charset
= XINT (XCAR (tail
)),
3792 CHARSET_VALID_P (charset
)))
3793 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3795 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3796 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3799 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3801 while (CONSP (tail
))
3803 if ((INTEGERP (XCAR (tail
))
3804 && (charset
= XINT (XCAR (tail
)),
3805 CHARSET_VALID_P (charset
)))
3806 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3807 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3809 else if (EQ (XCAR (tail
), Qt
))
3815 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3817 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3818 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3821 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3823 /* REG 1 can be used only by locking shift in 7-bit env. */
3824 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3826 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3827 /* Without any shifting, only REG 0 and 1 can be used. */
3832 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3834 if (CHARSET_DEFINED_P (charset
)
3835 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3836 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3838 /* There exist some default graphic registers to be
3841 /* We had better avoid designating a charset of
3842 CHARS96 to REG 0 as far as possible. */
3843 if (CHARSET_CHARS (charset
) == 96)
3844 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3846 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3848 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3850 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3854 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3855 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3859 coding
->type
= coding_type_big5
;
3860 coding
->common_flags
3861 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3863 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3864 ? CODING_FLAG_BIG5_HKU
3865 : CODING_FLAG_BIG5_ETEN
);
3869 coding
->type
= coding_type_ccl
;
3870 coding
->common_flags
3871 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3873 val
= XVECTOR (coding_spec
)->contents
[4];
3875 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3877 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3879 goto label_invalid_coding_system
;
3881 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3882 val
= Fplist_get (plist
, Qvalid_codes
);
3887 for (; CONSP (val
); val
= XCDR (val
))
3891 && XINT (this) >= 0 && XINT (this) < 256)
3892 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3893 else if (CONSP (this)
3894 && INTEGERP (XCAR (this))
3895 && INTEGERP (XCDR (this)))
3897 int start
= XINT (XCAR (this));
3898 int end
= XINT (XCDR (this));
3900 if (start
>= 0 && start
<= end
&& end
< 256)
3901 while (start
<= end
)
3902 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3907 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3908 coding
->spec
.ccl
.cr_carryover
= 0;
3909 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
3913 coding
->type
= coding_type_raw_text
;
3917 goto label_invalid_coding_system
;
3921 label_invalid_coding_system
:
3922 coding
->type
= coding_type_no_conversion
;
3923 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3924 coding
->common_flags
= 0;
3925 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3926 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3927 return NILP (coding_system
) ? 0 : -1;
3930 /* Free memory blocks allocated for storing composition information. */
3933 coding_free_composition_data (coding
)
3934 struct coding_system
*coding
;
3936 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3940 /* Memory blocks are chained. At first, rewind to the first, then,
3941 free blocks one by one. */
3942 while (cmp_data
->prev
)
3943 cmp_data
= cmp_data
->prev
;
3946 next
= cmp_data
->next
;
3950 coding
->cmp_data
= NULL
;
3953 /* Set `char_offset' member of all memory blocks pointed by
3954 coding->cmp_data to POS. */
3957 coding_adjust_composition_offset (coding
, pos
)
3958 struct coding_system
*coding
;
3961 struct composition_data
*cmp_data
;
3963 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3964 cmp_data
->char_offset
= pos
;
3967 /* Setup raw-text or one of its subsidiaries in the structure
3968 coding_system CODING according to the already setup value eol_type
3969 in CODING. CODING should be setup for some coding system in
3973 setup_raw_text_coding_system (coding
)
3974 struct coding_system
*coding
;
3976 if (coding
->type
!= coding_type_raw_text
)
3978 coding
->symbol
= Qraw_text
;
3979 coding
->type
= coding_type_raw_text
;
3980 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3982 Lisp_Object subsidiaries
;
3983 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3985 if (VECTORP (subsidiaries
)
3986 && XVECTOR (subsidiaries
)->size
== 3)
3988 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3990 setup_coding_system (coding
->symbol
, coding
);
3995 /* Emacs has a mechanism to automatically detect a coding system if it
3996 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3997 it's impossible to distinguish some coding systems accurately
3998 because they use the same range of codes. So, at first, coding
3999 systems are categorized into 7, those are:
4001 o coding-category-emacs-mule
4003 The category for a coding system which has the same code range
4004 as Emacs' internal format. Assigned the coding-system (Lisp
4005 symbol) `emacs-mule' by default.
4007 o coding-category-sjis
4009 The category for a coding system which has the same code range
4010 as SJIS. Assigned the coding-system (Lisp
4011 symbol) `japanese-shift-jis' by default.
4013 o coding-category-iso-7
4015 The category for a coding system which has the same code range
4016 as ISO2022 of 7-bit environment. This doesn't use any locking
4017 shift and single shift functions. This can encode/decode all
4018 charsets. Assigned the coding-system (Lisp symbol)
4019 `iso-2022-7bit' by default.
4021 o coding-category-iso-7-tight
4023 Same as coding-category-iso-7 except that this can
4024 encode/decode only the specified charsets.
4026 o coding-category-iso-8-1
4028 The category for a coding system which has the same code range
4029 as ISO2022 of 8-bit environment and graphic plane 1 used only
4030 for DIMENSION1 charset. This doesn't use any locking shift
4031 and single shift functions. Assigned the coding-system (Lisp
4032 symbol) `iso-latin-1' by default.
4034 o coding-category-iso-8-2
4036 The category for a coding system which has the same code range
4037 as ISO2022 of 8-bit environment and graphic plane 1 used only
4038 for DIMENSION2 charset. This doesn't use any locking shift
4039 and single shift functions. Assigned the coding-system (Lisp
4040 symbol) `japanese-iso-8bit' by default.
4042 o coding-category-iso-7-else
4044 The category for a coding system which has the same code range
4045 as ISO2022 of 7-bit environment but uses locking shift or
4046 single shift functions. Assigned the coding-system (Lisp
4047 symbol) `iso-2022-7bit-lock' by default.
4049 o coding-category-iso-8-else
4051 The category for a coding system which has the same code range
4052 as ISO2022 of 8-bit environment but uses locking shift or
4053 single shift functions. Assigned the coding-system (Lisp
4054 symbol) `iso-2022-8bit-ss2' by default.
4056 o coding-category-big5
4058 The category for a coding system which has the same code range
4059 as BIG5. Assigned the coding-system (Lisp symbol)
4060 `cn-big5' by default.
4062 o coding-category-utf-8
4064 The category for a coding system which has the same code range
4065 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
4066 symbol) `utf-8' by default.
4068 o coding-category-utf-16-be
4070 The category for a coding system in which a text has an
4071 Unicode signature (cf. Unicode Standard) in the order of BIG
4072 endian at the head. Assigned the coding-system (Lisp symbol)
4073 `utf-16-be' by default.
4075 o coding-category-utf-16-le
4077 The category for a coding system in which a text has an
4078 Unicode signature (cf. Unicode Standard) in the order of
4079 LITTLE endian at the head. Assigned the coding-system (Lisp
4080 symbol) `utf-16-le' by default.
4082 o coding-category-ccl
4084 The category for a coding system of which encoder/decoder is
4085 written in CCL programs. The default value is nil, i.e., no
4086 coding system is assigned.
4088 o coding-category-binary
4090 The category for a coding system not categorized in any of the
4091 above. Assigned the coding-system (Lisp symbol)
4092 `no-conversion' by default.
4094 Each of them is a Lisp symbol and the value is an actual
4095 `coding-system' (this is also a Lisp symbol) assigned by a user.
4096 What Emacs does actually is to detect a category of coding system.
4097 Then, it uses a `coding-system' assigned to it. If Emacs can't
4098 decide a single possible category, it selects a category of the
4099 highest priority. Priorities of categories are also specified by a
4100 user in a Lisp variable `coding-category-list'.
4105 int ascii_skip_code
[256];
4107 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
4108 If it detects possible coding systems, return an integer in which
4109 appropriate flag bits are set. Flag bits are defined by macros
4110 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
4111 it should point the table `coding_priorities'. In that case, only
4112 the flag bit for a coding system of the highest priority is set in
4113 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
4114 range 0x80..0x9F are in multibyte form.
4116 How many ASCII characters are at the head is returned as *SKIP. */
4119 detect_coding_mask (source
, src_bytes
, priorities
, skip
, multibytep
)
4120 unsigned char *source
;
4121 int src_bytes
, *priorities
, *skip
;
4124 register unsigned char c
;
4125 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
4126 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
4129 /* At first, skip all ASCII characters and control characters except
4130 for three ISO2022 specific control characters. */
4131 ascii_skip_code
[ISO_CODE_SO
] = 0;
4132 ascii_skip_code
[ISO_CODE_SI
] = 0;
4133 ascii_skip_code
[ISO_CODE_ESC
] = 0;
4135 label_loop_detect_coding
:
4136 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
4137 *skip
= src
- source
;
4140 /* We found nothing other than ASCII. There's nothing to do. */
4144 /* The text seems to be encoded in some multilingual coding system.
4145 Now, try to find in which coding system the text is encoded. */
4148 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
4149 /* C is an ISO2022 specific control code of C0. */
4150 mask
= detect_coding_iso2022 (src
, src_end
, multibytep
);
4153 /* No valid ISO2022 code follows C. Try again. */
4155 if (c
== ISO_CODE_ESC
)
4156 ascii_skip_code
[ISO_CODE_ESC
] = 1;
4158 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
4159 goto label_loop_detect_coding
;
4163 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4165 if (mask
& priorities
[i
])
4166 return priorities
[i
];
4168 return CODING_CATEGORY_MASK_RAW_TEXT
;
4175 if (multibytep
&& c
== LEADING_CODE_8_BIT_CONTROL
)
4180 /* C is the first byte of SJIS character code,
4181 or a leading-code of Emacs' internal format (emacs-mule),
4182 or the first byte of UTF-16. */
4183 try = (CODING_CATEGORY_MASK_SJIS
4184 | CODING_CATEGORY_MASK_EMACS_MULE
4185 | CODING_CATEGORY_MASK_UTF_16_BE
4186 | CODING_CATEGORY_MASK_UTF_16_LE
);
4188 /* Or, if C is a special latin extra code,
4189 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4190 or is an ISO2022 control-sequence-introducer (CSI),
4191 we should also consider the possibility of ISO2022 codings. */
4192 if ((VECTORP (Vlatin_extra_code_table
)
4193 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
4194 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
4195 || (c
== ISO_CODE_CSI
4198 || ((*src
== '0' || *src
== '1' || *src
== '2')
4199 && src
+ 1 < src_end
4200 && src
[1] == ']')))))
4201 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4202 | CODING_CATEGORY_MASK_ISO_8BIT
);
4205 /* C is a character of ISO2022 in graphic plane right,
4206 or a SJIS's 1-byte character code (i.e. JISX0201),
4207 or the first byte of BIG5's 2-byte code,
4208 or the first byte of UTF-8/16. */
4209 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4210 | CODING_CATEGORY_MASK_ISO_8BIT
4211 | CODING_CATEGORY_MASK_SJIS
4212 | CODING_CATEGORY_MASK_BIG5
4213 | CODING_CATEGORY_MASK_UTF_8
4214 | CODING_CATEGORY_MASK_UTF_16_BE
4215 | CODING_CATEGORY_MASK_UTF_16_LE
);
4217 /* Or, we may have to consider the possibility of CCL. */
4218 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4219 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4220 ->spec
.ccl
.valid_codes
)[c
])
4221 try |= CODING_CATEGORY_MASK_CCL
;
4224 utf16_examined_p
= iso2022_examined_p
= 0;
4227 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4229 if (!iso2022_examined_p
4230 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
4232 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4233 iso2022_examined_p
= 1;
4235 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
4236 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4237 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
4238 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4239 else if (!utf16_examined_p
4240 && (priorities
[i
] & try &
4241 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
4243 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4244 utf16_examined_p
= 1;
4246 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
4247 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4248 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
4249 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4250 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
4251 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4252 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
4253 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
4254 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
4255 mask
|= CODING_CATEGORY_MASK_BINARY
;
4256 if (mask
& priorities
[i
])
4257 return priorities
[i
];
4259 return CODING_CATEGORY_MASK_RAW_TEXT
;
4261 if (try & CODING_CATEGORY_MASK_ISO
)
4262 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4263 if (try & CODING_CATEGORY_MASK_SJIS
)
4264 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4265 if (try & CODING_CATEGORY_MASK_BIG5
)
4266 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4267 if (try & CODING_CATEGORY_MASK_UTF_8
)
4268 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4269 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
4270 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4271 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
4272 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4273 if (try & CODING_CATEGORY_MASK_CCL
)
4274 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4276 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
4279 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4280 The information of the detected coding system is set in CODING. */
4283 detect_coding (coding
, src
, src_bytes
)
4284 struct coding_system
*coding
;
4285 const unsigned char *src
;
4292 val
= Vcoding_category_list
;
4293 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
,
4294 coding
->src_multibyte
);
4295 coding
->heading_ascii
= skip
;
4299 /* We found a single coding system of the highest priority in MASK. */
4301 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
4303 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
4305 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[idx
]);
4307 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4311 tmp
= Fget (val
, Qeol_type
);
4313 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
4316 /* Setup this new coding system while preserving some slots. */
4318 int src_multibyte
= coding
->src_multibyte
;
4319 int dst_multibyte
= coding
->dst_multibyte
;
4321 setup_coding_system (val
, coding
);
4322 coding
->src_multibyte
= src_multibyte
;
4323 coding
->dst_multibyte
= dst_multibyte
;
4324 coding
->heading_ascii
= skip
;
4328 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
4329 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4330 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4332 How many non-eol characters are at the head is returned as *SKIP. */
4334 #define MAX_EOL_CHECK_COUNT 3
4337 detect_eol_type (source
, src_bytes
, skip
)
4338 unsigned char *source
;
4339 int src_bytes
, *skip
;
4341 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4343 int total
= 0; /* How many end-of-lines are found so far. */
4344 int eol_type
= CODING_EOL_UNDECIDED
;
4349 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4352 if (c
== '\n' || c
== '\r')
4355 *skip
= src
- 1 - source
;
4358 this_eol_type
= CODING_EOL_LF
;
4359 else if (src
>= src_end
|| *src
!= '\n')
4360 this_eol_type
= CODING_EOL_CR
;
4362 this_eol_type
= CODING_EOL_CRLF
, src
++;
4364 if (eol_type
== CODING_EOL_UNDECIDED
)
4365 /* This is the first end-of-line. */
4366 eol_type
= this_eol_type
;
4367 else if (eol_type
!= this_eol_type
)
4369 /* The found type is different from what found before. */
4370 eol_type
= CODING_EOL_INCONSISTENT
;
4377 *skip
= src_end
- source
;
4381 /* Like detect_eol_type, but detect EOL type in 2-octet
4382 big-endian/little-endian format for coding systems utf-16-be and
4386 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
4387 unsigned char *source
;
4388 int src_bytes
, *skip
, big_endian_p
;
4390 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4391 unsigned int c1
, c2
;
4392 int total
= 0; /* How many end-of-lines are found so far. */
4393 int eol_type
= CODING_EOL_UNDECIDED
;
4404 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4406 c1
= (src
[msb
] << 8) | (src
[lsb
]);
4409 if (c1
== '\n' || c1
== '\r')
4412 *skip
= src
- 2 - source
;
4416 this_eol_type
= CODING_EOL_LF
;
4420 if ((src
+ 1) >= src_end
)
4422 this_eol_type
= CODING_EOL_CR
;
4426 c2
= (src
[msb
] << 8) | (src
[lsb
]);
4428 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
4430 this_eol_type
= CODING_EOL_CR
;
4434 if (eol_type
== CODING_EOL_UNDECIDED
)
4435 /* This is the first end-of-line. */
4436 eol_type
= this_eol_type
;
4437 else if (eol_type
!= this_eol_type
)
4439 /* The found type is different from what found before. */
4440 eol_type
= CODING_EOL_INCONSISTENT
;
4447 *skip
= src_end
- source
;
4451 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4452 is encoded. If it detects an appropriate format of end-of-line, it
4453 sets the information in *CODING. */
4456 detect_eol (coding
, src
, src_bytes
)
4457 struct coding_system
*coding
;
4458 const unsigned char *src
;
4465 switch (coding
->category_idx
)
4467 case CODING_CATEGORY_IDX_UTF_16_BE
:
4468 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
4470 case CODING_CATEGORY_IDX_UTF_16_LE
:
4471 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
4474 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
4478 if (coding
->heading_ascii
> skip
)
4479 coding
->heading_ascii
= skip
;
4481 skip
= coding
->heading_ascii
;
4483 if (eol_type
== CODING_EOL_UNDECIDED
)
4485 if (eol_type
== CODING_EOL_INCONSISTENT
)
4488 /* This code is suppressed until we find a better way to
4489 distinguish raw text file and binary file. */
4491 /* If we have already detected that the coding is raw-text, the
4492 coding should actually be no-conversion. */
4493 if (coding
->type
== coding_type_raw_text
)
4495 setup_coding_system (Qno_conversion
, coding
);
4498 /* Else, let's decode only text code anyway. */
4500 eol_type
= CODING_EOL_LF
;
4503 val
= Fget (coding
->symbol
, Qeol_type
);
4504 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4506 int src_multibyte
= coding
->src_multibyte
;
4507 int dst_multibyte
= coding
->dst_multibyte
;
4508 struct composition_data
*cmp_data
= coding
->cmp_data
;
4510 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
4511 coding
->src_multibyte
= src_multibyte
;
4512 coding
->dst_multibyte
= dst_multibyte
;
4513 coding
->heading_ascii
= skip
;
4514 coding
->cmp_data
= cmp_data
;
4518 #define CONVERSION_BUFFER_EXTRA_ROOM 256
4520 #define DECODING_BUFFER_MAG(coding) \
4521 (coding->type == coding_type_iso2022 \
4523 : (coding->type == coding_type_ccl \
4524 ? coding->spec.ccl.decoder.buf_magnification \
4527 /* Return maximum size (bytes) of a buffer enough for decoding
4528 SRC_BYTES of text encoded in CODING. */
4531 decoding_buffer_size (coding
, src_bytes
)
4532 struct coding_system
*coding
;
4535 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
4536 + CONVERSION_BUFFER_EXTRA_ROOM
);
4539 /* Return maximum size (bytes) of a buffer enough for encoding
4540 SRC_BYTES of text to CODING. */
4543 encoding_buffer_size (coding
, src_bytes
)
4544 struct coding_system
*coding
;
4549 if (coding
->type
== coding_type_ccl
)
4551 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
4552 if (coding
->eol_type
== CODING_EOL_CRLF
)
4555 else if (CODING_REQUIRE_ENCODING (coding
))
4560 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
4563 /* Working buffer for code conversion. */
4564 struct conversion_buffer
4566 int size
; /* size of data. */
4567 int on_stack
; /* 1 if allocated by alloca. */
4568 unsigned char *data
;
4571 /* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4572 #define allocate_conversion_buffer(buf, len) \
4574 if (len < MAX_ALLOCA) \
4576 buf.data = (unsigned char *) alloca (len); \
4581 buf.data = (unsigned char *) xmalloc (len); \
4587 /* Double the allocated memory for *BUF. */
4589 extend_conversion_buffer (buf
)
4590 struct conversion_buffer
*buf
;
4594 unsigned char *save
= buf
->data
;
4595 buf
->data
= (unsigned char *) xmalloc (buf
->size
* 2);
4596 bcopy (save
, buf
->data
, buf
->size
);
4601 buf
->data
= (unsigned char *) xrealloc (buf
->data
, buf
->size
* 2);
4606 /* Free the allocated memory for BUF if it is not on stack. */
4608 free_conversion_buffer (buf
)
4609 struct conversion_buffer
*buf
;
4616 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
4617 struct coding_system
*coding
;
4618 unsigned char *source
, *destination
;
4619 int src_bytes
, dst_bytes
, encodep
;
4621 struct ccl_program
*ccl
4622 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
4623 unsigned char *dst
= destination
;
4625 ccl
->suppress_error
= coding
->suppress_error
;
4626 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
4629 /* On encoding, EOL format is converted within ccl_driver. For
4630 that, setup proper information in the structure CCL. */
4631 ccl
->eol_type
= coding
->eol_type
;
4632 if (ccl
->eol_type
==CODING_EOL_UNDECIDED
)
4633 ccl
->eol_type
= CODING_EOL_LF
;
4634 ccl
->cr_consumed
= coding
->spec
.ccl
.cr_carryover
;
4635 ccl
->eight_bit_control
= coding
->dst_multibyte
;
4638 ccl
->eight_bit_control
= 1;
4639 ccl
->multibyte
= coding
->src_multibyte
;
4640 if (coding
->spec
.ccl
.eight_bit_carryover
[0] != 0)
4642 /* Move carryover bytes to DESTINATION. */
4643 unsigned char *p
= coding
->spec
.ccl
.eight_bit_carryover
;
4646 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4648 dst_bytes
-= dst
- destination
;
4651 coding
->produced
= (ccl_driver (ccl
, source
, dst
, src_bytes
, dst_bytes
,
4652 &(coding
->consumed
))
4653 + dst
- destination
);
4657 coding
->produced_char
= coding
->produced
;
4658 coding
->spec
.ccl
.cr_carryover
= ccl
->cr_consumed
;
4660 else if (!ccl
->eight_bit_control
)
4662 /* The produced bytes forms a valid multibyte sequence. */
4663 coding
->produced_char
4664 = multibyte_chars_in_text (destination
, coding
->produced
);
4665 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4669 /* On decoding, the destination should always multibyte. But,
4670 CCL program might have been generated an invalid multibyte
4671 sequence. Here we make such a sequence valid as
4674 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
4676 if ((coding
->consumed
< src_bytes
4677 || !ccl
->last_block
)
4678 && coding
->produced
>= 1
4679 && destination
[coding
->produced
- 1] >= 0x80)
4681 /* We should not convert the tailing 8-bit codes to
4682 multibyte form even if they doesn't form a valid
4683 multibyte sequence. They may form a valid sequence in
4687 if (destination
[coding
->produced
- 1] < 0xA0)
4689 else if (coding
->produced
>= 2)
4691 if (destination
[coding
->produced
- 2] >= 0x80)
4693 if (destination
[coding
->produced
- 2] < 0xA0)
4695 else if (coding
->produced
>= 3
4696 && destination
[coding
->produced
- 3] >= 0x80
4697 && destination
[coding
->produced
- 3] < 0xA0)
4703 BCOPY_SHORT (destination
+ coding
->produced
- carryover
,
4704 coding
->spec
.ccl
.eight_bit_carryover
,
4706 coding
->spec
.ccl
.eight_bit_carryover
[carryover
] = 0;
4707 coding
->produced
-= carryover
;
4710 coding
->produced
= str_as_multibyte (destination
, bytes
,
4712 &(coding
->produced_char
));
4715 switch (ccl
->status
)
4717 case CCL_STAT_SUSPEND_BY_SRC
:
4718 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
4720 case CCL_STAT_SUSPEND_BY_DST
:
4721 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
4724 case CCL_STAT_INVALID_CMD
:
4725 coding
->result
= CODING_FINISH_INTERRUPT
;
4728 coding
->result
= CODING_FINISH_NORMAL
;
4731 return coding
->result
;
4734 /* Decode EOL format of the text at PTR of BYTES length destructively
4735 according to CODING->eol_type. This is called after the CCL
4736 program produced a decoded text at PTR. If we do CRLF->LF
4737 conversion, update CODING->produced and CODING->produced_char. */
4740 decode_eol_post_ccl (coding
, ptr
, bytes
)
4741 struct coding_system
*coding
;
4745 Lisp_Object val
, saved_coding_symbol
;
4746 unsigned char *pend
= ptr
+ bytes
;
4749 /* Remember the current coding system symbol. We set it back when
4750 an inconsistent EOL is found so that `last-coding-system-used' is
4751 set to the coding system that doesn't specify EOL conversion. */
4752 saved_coding_symbol
= coding
->symbol
;
4754 coding
->spec
.ccl
.cr_carryover
= 0;
4755 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4757 /* Here, to avoid the call of setup_coding_system, we directly
4758 call detect_eol_type. */
4759 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
4760 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
4761 coding
->eol_type
= CODING_EOL_LF
;
4762 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4764 val
= Fget (coding
->symbol
, Qeol_type
);
4765 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4766 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
4768 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4771 if (coding
->eol_type
== CODING_EOL_LF
4772 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
4774 /* We have nothing to do. */
4777 else if (coding
->eol_type
== CODING_EOL_CRLF
)
4779 unsigned char *pstart
= ptr
, *p
= ptr
;
4781 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4782 && *(pend
- 1) == '\r')
4784 /* If the last character is CR, we can't handle it here
4785 because LF will be in the not-yet-decoded source text.
4786 Record that the CR is not yet processed. */
4787 coding
->spec
.ccl
.cr_carryover
= 1;
4789 coding
->produced_char
--;
4796 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4803 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4804 goto undo_eol_conversion
;
4808 else if (*ptr
== '\n'
4809 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4810 goto undo_eol_conversion
;
4815 undo_eol_conversion
:
4816 /* We have faced with inconsistent EOL format at PTR.
4817 Convert all LFs before PTR back to CRLFs. */
4818 for (p
--, ptr
--; p
>= pstart
; p
--)
4821 *ptr
-- = '\n', *ptr
-- = '\r';
4825 /* If carryover is recorded, cancel it because we don't
4826 convert CRLF anymore. */
4827 if (coding
->spec
.ccl
.cr_carryover
)
4829 coding
->spec
.ccl
.cr_carryover
= 0;
4831 coding
->produced_char
++;
4835 coding
->eol_type
= CODING_EOL_LF
;
4836 coding
->symbol
= saved_coding_symbol
;
4840 /* As each two-byte sequence CRLF was converted to LF, (PEND
4841 - P) is the number of deleted characters. */
4842 coding
->produced
-= pend
- p
;
4843 coding
->produced_char
-= pend
- p
;
4846 else /* i.e. coding->eol_type == CODING_EOL_CR */
4848 unsigned char *p
= ptr
;
4850 for (; ptr
< pend
; ptr
++)
4854 else if (*ptr
== '\n'
4855 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4857 for (; p
< ptr
; p
++)
4863 coding
->eol_type
= CODING_EOL_LF
;
4864 coding
->symbol
= saved_coding_symbol
;
4870 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4871 decoding, it may detect coding system and format of end-of-line if
4872 those are not yet decided. The source should be unibyte, the
4873 result is multibyte if CODING->dst_multibyte is nonzero, else
4877 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4878 struct coding_system
*coding
;
4879 const unsigned char *source
;
4880 unsigned char *destination
;
4881 int src_bytes
, dst_bytes
;
4885 if (coding
->type
== coding_type_undecided
)
4886 detect_coding (coding
, source
, src_bytes
);
4888 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4889 && coding
->type
!= coding_type_ccl
)
4891 detect_eol (coding
, source
, src_bytes
);
4892 /* We had better recover the original eol format if we
4893 encounter an inconsistent eol format while decoding. */
4894 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4897 coding
->produced
= coding
->produced_char
= 0;
4898 coding
->consumed
= coding
->consumed_char
= 0;
4900 coding
->result
= CODING_FINISH_NORMAL
;
4902 switch (coding
->type
)
4904 case coding_type_sjis
:
4905 decode_coding_sjis_big5 (coding
, source
, destination
,
4906 src_bytes
, dst_bytes
, 1);
4909 case coding_type_iso2022
:
4910 decode_coding_iso2022 (coding
, source
, destination
,
4911 src_bytes
, dst_bytes
);
4914 case coding_type_big5
:
4915 decode_coding_sjis_big5 (coding
, source
, destination
,
4916 src_bytes
, dst_bytes
, 0);
4919 case coding_type_emacs_mule
:
4920 decode_coding_emacs_mule (coding
, source
, destination
,
4921 src_bytes
, dst_bytes
);
4924 case coding_type_ccl
:
4925 if (coding
->spec
.ccl
.cr_carryover
)
4927 /* Put the CR which was not processed by the previous call
4928 of decode_eol_post_ccl in DESTINATION. It will be
4929 decoded together with the following LF by the call to
4930 decode_eol_post_ccl below. */
4931 *destination
= '\r';
4933 coding
->produced_char
++;
4935 extra
= coding
->spec
.ccl
.cr_carryover
;
4937 ccl_coding_driver (coding
, source
, destination
+ extra
,
4938 src_bytes
, dst_bytes
, 0);
4939 if (coding
->eol_type
!= CODING_EOL_LF
)
4941 coding
->produced
+= extra
;
4942 coding
->produced_char
+= extra
;
4943 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4948 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4951 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4952 && coding
->mode
& CODING_MODE_LAST_BLOCK
4953 && coding
->consumed
== src_bytes
)
4954 coding
->result
= CODING_FINISH_NORMAL
;
4956 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4957 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4959 const unsigned char *src
= source
+ coding
->consumed
;
4960 unsigned char *dst
= destination
+ coding
->produced
;
4962 src_bytes
-= coding
->consumed
;
4964 if (COMPOSING_P (coding
))
4965 DECODE_COMPOSITION_END ('1');
4969 dst
+= CHAR_STRING (c
, dst
);
4970 coding
->produced_char
++;
4972 coding
->consumed
= coding
->consumed_char
= src
- source
;
4973 coding
->produced
= dst
- destination
;
4974 coding
->result
= CODING_FINISH_NORMAL
;
4977 if (!coding
->dst_multibyte
)
4979 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4980 coding
->produced_char
= coding
->produced
;
4983 return coding
->result
;
4986 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4987 multibyteness of the source is CODING->src_multibyte, the
4988 multibyteness of the result is always unibyte. */
4991 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4992 struct coding_system
*coding
;
4993 const unsigned char *source
;
4994 unsigned char *destination
;
4995 int src_bytes
, dst_bytes
;
4997 coding
->produced
= coding
->produced_char
= 0;
4998 coding
->consumed
= coding
->consumed_char
= 0;
5000 coding
->result
= CODING_FINISH_NORMAL
;
5001 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5002 coding
->eol_type
= CODING_EOL_LF
;
5004 switch (coding
->type
)
5006 case coding_type_sjis
:
5007 encode_coding_sjis_big5 (coding
, source
, destination
,
5008 src_bytes
, dst_bytes
, 1);
5011 case coding_type_iso2022
:
5012 encode_coding_iso2022 (coding
, source
, destination
,
5013 src_bytes
, dst_bytes
);
5016 case coding_type_big5
:
5017 encode_coding_sjis_big5 (coding
, source
, destination
,
5018 src_bytes
, dst_bytes
, 0);
5021 case coding_type_emacs_mule
:
5022 encode_coding_emacs_mule (coding
, source
, destination
,
5023 src_bytes
, dst_bytes
);
5026 case coding_type_ccl
:
5027 ccl_coding_driver (coding
, source
, destination
,
5028 src_bytes
, dst_bytes
, 1);
5032 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
5035 if (coding
->mode
& CODING_MODE_LAST_BLOCK
5036 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
5038 const unsigned char *src
= source
+ coding
->consumed
;
5039 unsigned char *dst
= destination
+ coding
->produced
;
5041 if (coding
->type
== coding_type_iso2022
)
5042 ENCODE_RESET_PLANE_AND_REGISTER
;
5043 if (COMPOSING_P (coding
))
5044 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
5045 if (coding
->consumed
< src_bytes
)
5047 int len
= src_bytes
- coding
->consumed
;
5049 BCOPY_SHORT (src
, dst
, len
);
5050 if (coding
->src_multibyte
)
5051 len
= str_as_unibyte (dst
, len
);
5053 coding
->consumed
= src_bytes
;
5055 coding
->produced
= coding
->produced_char
= dst
- destination
;
5056 coding
->result
= CODING_FINISH_NORMAL
;
5059 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
5060 && coding
->consumed
== src_bytes
)
5061 coding
->result
= CODING_FINISH_NORMAL
;
5063 return coding
->result
;
5066 /* Scan text in the region between *BEG and *END (byte positions),
5067 skip characters which we don't have to decode by coding system
5068 CODING at the head and tail, then set *BEG and *END to the region
5069 of the text we actually have to convert. The caller should move
5070 the gap out of the region in advance if the region is from a
5073 If STR is not NULL, *BEG and *END are indices into STR. */
5076 shrink_decoding_region (beg
, end
, coding
, str
)
5078 struct coding_system
*coding
;
5081 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
5083 Lisp_Object translation_table
;
5085 if (coding
->type
== coding_type_ccl
5086 || coding
->type
== coding_type_undecided
5087 || coding
->eol_type
!= CODING_EOL_LF
5088 || !NILP (coding
->post_read_conversion
)
5089 || coding
->composing
!= COMPOSITION_DISABLED
)
5091 /* We can't skip any data. */
5094 if (coding
->type
== coding_type_no_conversion
5095 || coding
->type
== coding_type_raw_text
5096 || coding
->type
== coding_type_emacs_mule
)
5098 /* We need no conversion, but don't have to skip any data here.
5099 Decoding routine handles them effectively anyway. */
5103 translation_table
= coding
->translation_table_for_decode
;
5104 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5105 translation_table
= Vstandard_translation_table_for_decode
;
5106 if (CHAR_TABLE_P (translation_table
))
5109 for (i
= 0; i
< 128; i
++)
5110 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5113 /* Some ASCII character should be translated. We give up
5118 if (coding
->heading_ascii
>= 0)
5119 /* Detection routine has already found how much we can skip at the
5121 *beg
+= coding
->heading_ascii
;
5125 begp_orig
= begp
= str
+ *beg
;
5126 endp_orig
= endp
= str
+ *end
;
5130 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5131 endp_orig
= endp
= begp
+ *end
- *beg
;
5134 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5135 || coding
->eol_type
== CODING_EOL_CRLF
);
5137 switch (coding
->type
)
5139 case coding_type_sjis
:
5140 case coding_type_big5
:
5141 /* We can skip all ASCII characters at the head. */
5142 if (coding
->heading_ascii
< 0)
5145 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
5147 while (begp
< endp
&& *begp
< 0x80) begp
++;
5149 /* We can skip all ASCII characters at the tail except for the
5150 second byte of SJIS or BIG5 code. */
5152 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
5154 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5155 /* Do not consider LF as ascii if preceded by CR, since that
5156 confuses eol decoding. */
5157 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5159 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
5163 case coding_type_iso2022
:
5164 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5165 /* We can't skip any data. */
5167 if (coding
->heading_ascii
< 0)
5169 /* We can skip all ASCII characters at the head except for a
5170 few control codes. */
5171 while (begp
< endp
&& (c
= *begp
) < 0x80
5172 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
5173 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
5174 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
5177 switch (coding
->category_idx
)
5179 case CODING_CATEGORY_IDX_ISO_8_1
:
5180 case CODING_CATEGORY_IDX_ISO_8_2
:
5181 /* We can skip all ASCII characters at the tail. */
5183 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
5185 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5186 /* Do not consider LF as ascii if preceded by CR, since that
5187 confuses eol decoding. */
5188 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5192 case CODING_CATEGORY_IDX_ISO_7
:
5193 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
5195 /* We can skip all characters at the tail except for 8-bit
5196 codes and ESC and the following 2-byte at the tail. */
5197 unsigned char *eight_bit
= NULL
;
5201 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
5203 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5208 && (c
= endp
[-1]) != ISO_CODE_ESC
)
5210 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5213 /* Do not consider LF as ascii if preceded by CR, since that
5214 confuses eol decoding. */
5215 if (begp
< endp
&& endp
< endp_orig
5216 && endp
[-1] == '\r' && endp
[0] == '\n')
5218 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
5220 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
5221 /* This is an ASCII designation sequence. We can
5222 surely skip the tail. But, if we have
5223 encountered an 8-bit code, skip only the codes
5225 endp
= eight_bit
? eight_bit
: endp
+ 2;
5227 /* Hmmm, we can't skip the tail. */
5239 *beg
+= begp
- begp_orig
;
5240 *end
+= endp
- endp_orig
;
5244 /* Like shrink_decoding_region but for encoding. */
5247 shrink_encoding_region (beg
, end
, coding
, str
)
5249 struct coding_system
*coding
;
5252 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
5254 Lisp_Object translation_table
;
5256 if (coding
->type
== coding_type_ccl
5257 || coding
->eol_type
== CODING_EOL_CRLF
5258 || coding
->eol_type
== CODING_EOL_CR
5259 || (coding
->cmp_data
&& coding
->cmp_data
->used
> 0))
5261 /* We can't skip any data. */
5264 if (coding
->type
== coding_type_no_conversion
5265 || coding
->type
== coding_type_raw_text
5266 || coding
->type
== coding_type_emacs_mule
5267 || coding
->type
== coding_type_undecided
)
5269 /* We need no conversion, but don't have to skip any data here.
5270 Encoding routine handles them effectively anyway. */
5274 translation_table
= coding
->translation_table_for_encode
;
5275 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5276 translation_table
= Vstandard_translation_table_for_encode
;
5277 if (CHAR_TABLE_P (translation_table
))
5280 for (i
= 0; i
< 128; i
++)
5281 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5284 /* Some ASCII character should be translated. We give up
5291 begp_orig
= begp
= str
+ *beg
;
5292 endp_orig
= endp
= str
+ *end
;
5296 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5297 endp_orig
= endp
= begp
+ *end
- *beg
;
5300 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5301 || coding
->eol_type
== CODING_EOL_CRLF
);
5303 /* Here, we don't have to check coding->pre_write_conversion because
5304 the caller is expected to have handled it already. */
5305 switch (coding
->type
)
5307 case coding_type_iso2022
:
5308 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5309 /* We can't skip any data. */
5311 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
5313 unsigned char *bol
= begp
;
5314 while (begp
< endp
&& *begp
< 0x80)
5317 if (begp
[-1] == '\n')
5321 goto label_skip_tail
;
5325 case coding_type_sjis
:
5326 case coding_type_big5
:
5327 /* We can skip all ASCII characters at the head and tail. */
5329 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
5331 while (begp
< endp
&& *begp
< 0x80) begp
++;
5334 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
5336 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
5343 *beg
+= begp
- begp_orig
;
5344 *end
+= endp
- endp_orig
;
5348 /* As shrinking conversion region requires some overhead, we don't try
5349 shrinking if the length of conversion region is less than this
5351 static int shrink_conversion_region_threshhold
= 1024;
5353 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5355 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5357 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5358 else shrink_decoding_region (beg, end, coding, str); \
5362 /* ARG is (CODING BUFFER ...) where CODING is what to be set in
5363 Vlast_coding_system_used and the remaining elements are buffers to
5366 code_convert_region_unwind (arg
)
5369 struct gcpro gcpro1
;
5372 inhibit_pre_post_conversion
= 0;
5373 Vlast_coding_system_used
= XCAR (arg
);
5374 for (arg
= XCDR (arg
); ! NILP (arg
); arg
= XCDR (arg
))
5375 Fkill_buffer (XCAR (arg
));
5381 /* Store information about all compositions in the range FROM and TO
5382 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5383 buffer or a string, defaults to the current buffer. */
5386 coding_save_composition (coding
, from
, to
, obj
)
5387 struct coding_system
*coding
;
5394 if (coding
->composing
== COMPOSITION_DISABLED
)
5396 if (!coding
->cmp_data
)
5397 coding_allocate_composition_data (coding
, from
);
5398 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
5402 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
5405 coding
->composing
= COMPOSITION_NO
;
5408 if (COMPOSITION_VALID_P (start
, end
, prop
))
5410 enum composition_method method
= COMPOSITION_METHOD (prop
);
5411 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
5412 >= COMPOSITION_DATA_SIZE
)
5413 coding_allocate_composition_data (coding
, from
);
5414 /* For relative composition, we remember start and end
5415 positions, for the other compositions, we also remember
5417 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
5418 if (method
!= COMPOSITION_RELATIVE
)
5420 /* We must store a*/
5421 Lisp_Object val
, ch
;
5423 val
= COMPOSITION_COMPONENTS (prop
);
5427 ch
= XCAR (val
), val
= XCDR (val
);
5428 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5430 else if (VECTORP (val
) || STRINGP (val
))
5432 int len
= (VECTORP (val
)
5433 ? XVECTOR (val
)->size
: SCHARS (val
));
5435 for (i
= 0; i
< len
; i
++)
5438 ? Faref (val
, make_number (i
))
5439 : XVECTOR (val
)->contents
[i
]);
5440 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5443 else /* INTEGERP (val) */
5444 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
5446 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
5451 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
5454 /* Make coding->cmp_data point to the first memory block. */
5455 while (coding
->cmp_data
->prev
)
5456 coding
->cmp_data
= coding
->cmp_data
->prev
;
5457 coding
->cmp_data_start
= 0;
5460 /* Reflect the saved information about compositions to OBJ.
5461 CODING->cmp_data points to a memory block for the information. OBJ
5462 is a buffer or a string, defaults to the current buffer. */
5465 coding_restore_composition (coding
, obj
)
5466 struct coding_system
*coding
;
5469 struct composition_data
*cmp_data
= coding
->cmp_data
;
5474 while (cmp_data
->prev
)
5475 cmp_data
= cmp_data
->prev
;
5481 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
5482 i
+= cmp_data
->data
[i
])
5484 int *data
= cmp_data
->data
+ i
;
5485 enum composition_method method
= (enum composition_method
) data
[3];
5486 Lisp_Object components
;
5488 if (data
[0] < 0 || i
+ data
[0] > cmp_data
->used
)
5489 /* Invalid composition data. */
5492 if (method
== COMPOSITION_RELATIVE
)
5496 int len
= data
[0] - 4, j
;
5497 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
5499 if (method
== COMPOSITION_WITH_RULE_ALTCHARS
5503 /* Invalid composition data. */
5505 for (j
= 0; j
< len
; j
++)
5506 args
[j
] = make_number (data
[4 + j
]);
5507 components
= (method
== COMPOSITION_WITH_ALTCHARS
5508 ? Fstring (len
, args
)
5509 : Fvector (len
, args
));
5511 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
5513 cmp_data
= cmp_data
->next
;
5517 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
5518 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5519 coding system CODING, and return the status code of code conversion
5520 (currently, this value has no meaning).
5522 How many characters (and bytes) are converted to how many
5523 characters (and bytes) are recorded in members of the structure
5526 If REPLACE is nonzero, we do various things as if the original text
5527 is deleted and a new text is inserted. See the comments in
5528 replace_range (insdel.c) to know what we are doing.
5530 If REPLACE is zero, it is assumed that the source text is unibyte.
5531 Otherwise, it is assumed that the source text is multibyte. */
5534 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
5535 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
5536 struct coding_system
*coding
;
5538 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
5539 int nchars_del
= 0, nbytes_del
= 0;
5540 int require
, inserted
, inserted_byte
;
5541 int head_skip
, tail_skip
, total_skip
= 0;
5542 Lisp_Object saved_coding_symbol
;
5544 unsigned char *src
, *dst
;
5545 Lisp_Object deletion
;
5546 int orig_point
= PT
, orig_len
= len
;
5548 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
5551 saved_coding_symbol
= coding
->symbol
;
5553 if (from
< PT
&& PT
< to
)
5555 TEMP_SET_PT_BOTH (from
, from_byte
);
5561 int saved_from
= from
;
5562 int saved_inhibit_modification_hooks
;
5564 prepare_to_modify_buffer (from
, to
, &from
);
5565 if (saved_from
!= from
)
5568 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
5569 len_byte
= to_byte
- from_byte
;
5572 /* The code conversion routine can not preserve text properties
5573 for now. So, we must remove all text properties in the
5574 region. Here, we must suppress all modification hooks. */
5575 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
5576 inhibit_modification_hooks
= 1;
5577 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
5578 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
5581 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
5583 /* We must detect encoding of text and eol format. */
5585 if (from
< GPT
&& to
> GPT
)
5586 move_gap_both (from
, from_byte
);
5587 if (coding
->type
== coding_type_undecided
)
5589 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5590 if (coding
->type
== coding_type_undecided
)
5592 /* It seems that the text contains only ASCII, but we
5593 should not leave it undecided because the deeper
5594 decoding routine (decode_coding) tries to detect the
5595 encodings again in vain. */
5596 coding
->type
= coding_type_emacs_mule
;
5597 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5598 /* As emacs-mule decoder will handle composition, we
5599 need this setting to allocate coding->cmp_data
5601 coding
->composing
= COMPOSITION_NO
;
5604 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5605 && coding
->type
!= coding_type_ccl
)
5607 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5608 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5609 coding
->eol_type
= CODING_EOL_LF
;
5610 /* We had better recover the original eol format if we
5611 encounter an inconsistent eol format while decoding. */
5612 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5616 /* Now we convert the text. */
5618 /* For encoding, we must process pre-write-conversion in advance. */
5619 if (! inhibit_pre_post_conversion
5621 && SYMBOLP (coding
->pre_write_conversion
)
5622 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
5624 /* The function in pre-write-conversion may put a new text in a
5626 struct buffer
*prev
= current_buffer
;
5629 record_unwind_protect (code_convert_region_unwind
,
5630 Fcons (Vlast_coding_system_used
, Qnil
));
5631 /* We should not call any more pre-write/post-read-conversion
5632 functions while this pre-write-conversion is running. */
5633 inhibit_pre_post_conversion
= 1;
5634 call2 (coding
->pre_write_conversion
,
5635 make_number (from
), make_number (to
));
5636 inhibit_pre_post_conversion
= 0;
5637 /* Discard the unwind protect. */
5640 if (current_buffer
!= prev
)
5643 new = Fcurrent_buffer ();
5644 set_buffer_internal_1 (prev
);
5645 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
5646 TEMP_SET_PT_BOTH (from
, from_byte
);
5647 insert_from_buffer (XBUFFER (new), 1, len
, 0);
5649 if (orig_point
>= to
)
5650 orig_point
+= len
- orig_len
;
5651 else if (orig_point
> from
)
5655 from_byte
= CHAR_TO_BYTE (from
);
5656 to_byte
= CHAR_TO_BYTE (to
);
5657 len_byte
= to_byte
- from_byte
;
5658 TEMP_SET_PT_BOTH (from
, from_byte
);
5664 if (! EQ (current_buffer
->undo_list
, Qt
))
5665 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
5668 nchars_del
= to
- from
;
5669 nbytes_del
= to_byte
- from_byte
;
5673 if (coding
->composing
!= COMPOSITION_DISABLED
)
5676 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
5678 coding_allocate_composition_data (coding
, from
);
5681 /* Try to skip the heading and tailing ASCIIs. We can't skip them
5682 if we must run CCL program or there are compositions to
5684 if (coding
->type
!= coding_type_ccl
5685 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
5687 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
5689 if (from
< GPT
&& GPT
< to
)
5690 move_gap_both (from
, from_byte
);
5691 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
5692 if (from_byte
== to_byte
5693 && (encodep
|| NILP (coding
->post_read_conversion
))
5694 && ! CODING_REQUIRE_FLUSHING (coding
))
5696 coding
->produced
= len_byte
;
5697 coding
->produced_char
= len
;
5699 /* We must record and adjust for this new text now. */
5700 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
5701 coding_free_composition_data (coding
);
5705 head_skip
= from_byte
- from_byte_orig
;
5706 tail_skip
= to_byte_orig
- to_byte
;
5707 total_skip
= head_skip
+ tail_skip
;
5710 len
-= total_skip
; len_byte
-= total_skip
;
5713 /* For conversion, we must put the gap before the text in addition to
5714 making the gap larger for efficient decoding. The required gap
5715 size starts from 2000 which is the magic number used in make_gap.
5716 But, after one batch of conversion, it will be incremented if we
5717 find that it is not enough . */
5720 if (GAP_SIZE
< require
)
5721 make_gap (require
- GAP_SIZE
);
5722 move_gap_both (from
, from_byte
);
5724 inserted
= inserted_byte
= 0;
5726 GAP_SIZE
+= len_byte
;
5729 ZV_BYTE
-= len_byte
;
5732 if (GPT
- BEG
< BEG_UNCHANGED
)
5733 BEG_UNCHANGED
= GPT
- BEG
;
5734 if (Z
- GPT
< END_UNCHANGED
)
5735 END_UNCHANGED
= Z
- GPT
;
5737 if (!encodep
&& coding
->src_multibyte
)
5739 /* Decoding routines expects that the source text is unibyte.
5740 We must convert 8-bit characters of multibyte form to
5742 int len_byte_orig
= len_byte
;
5743 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
5744 if (len_byte
< len_byte_orig
)
5745 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
5747 coding
->src_multibyte
= 0;
5754 /* The buffer memory is now:
5755 +--------+converted-text+---------+-------original-text-------+---+
5756 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5757 |<---------------------- GAP ----------------------->| */
5758 src
= GAP_END_ADDR
- len_byte
;
5759 dst
= GPT_ADDR
+ inserted_byte
;
5762 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
5765 if (coding
->composing
!= COMPOSITION_DISABLED
)
5766 coding
->cmp_data
->char_offset
= from
+ inserted
;
5767 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
5770 /* The buffer memory is now:
5771 +--------+-------converted-text----+--+------original-text----+---+
5772 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5773 |<---------------------- GAP ----------------------->| */
5775 inserted
+= coding
->produced_char
;
5776 inserted_byte
+= coding
->produced
;
5777 len_byte
-= coding
->consumed
;
5779 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5781 coding_allocate_composition_data (coding
, from
+ inserted
);
5785 src
+= coding
->consumed
;
5786 dst
+= coding
->produced
;
5788 if (result
== CODING_FINISH_NORMAL
)
5793 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
5795 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
5796 Lisp_Object eol_type
;
5798 /* Encode LFs back to the original eol format (CR or CRLF). */
5799 if (coding
->eol_type
== CODING_EOL_CR
)
5801 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
5807 while (p
< pend
) if (*p
++ == '\n') count
++;
5808 if (src
- dst
< count
)
5810 /* We don't have sufficient room for encoding LFs
5811 back to CRLF. We must record converted and
5812 not-yet-converted text back to the buffer
5813 content, enlarge the gap, then record them out of
5814 the buffer contents again. */
5815 int add
= len_byte
+ inserted_byte
;
5818 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5819 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5820 make_gap (count
- GAP_SIZE
);
5822 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5823 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5824 /* Don't forget to update SRC, DST, and PEND. */
5825 src
= GAP_END_ADDR
- len_byte
;
5826 dst
= GPT_ADDR
+ inserted_byte
;
5830 inserted_byte
+= count
;
5831 coding
->produced
+= count
;
5832 p
= dst
= pend
+ count
;
5836 if (*p
== '\n') count
--, *--p
= '\r';
5840 /* Suppress eol-format conversion in the further conversion. */
5841 coding
->eol_type
= CODING_EOL_LF
;
5843 /* Set the coding system symbol to that for Unix-like EOL. */
5844 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5845 if (VECTORP (eol_type
)
5846 && XVECTOR (eol_type
)->size
== 3
5847 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
5848 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5850 coding
->symbol
= saved_coding_symbol
;
5856 if (coding
->type
!= coding_type_ccl
5857 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5859 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5862 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5864 /* The source text ends in invalid codes. Let's just
5865 make them valid buffer contents, and finish conversion. */
5868 unsigned char *start
= dst
;
5870 inserted
+= len_byte
;
5874 dst
+= CHAR_STRING (c
, dst
);
5877 inserted_byte
+= dst
- start
;
5881 inserted
+= len_byte
;
5882 inserted_byte
+= len_byte
;
5888 if (result
== CODING_FINISH_INTERRUPT
)
5890 /* The conversion procedure was interrupted by a user. */
5893 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5894 if (coding
->consumed
< 1)
5896 /* It's quite strange to require more memory without
5897 consuming any bytes. Perhaps CCL program bug. */
5902 /* We have just done the first batch of conversion which was
5903 stopped because of insufficient gap. Let's reconsider the
5904 required gap size (i.e. SRT - DST) now.
5906 We have converted ORIG bytes (== coding->consumed) into
5907 NEW bytes (coding->produced). To convert the remaining
5908 LEN bytes, we may need REQUIRE bytes of gap, where:
5909 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5910 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5911 Here, we are sure that NEW >= ORIG. */
5913 if (coding
->produced
<= coding
->consumed
)
5915 /* This happens because of CCL-based coding system with
5921 float ratio
= coding
->produced
- coding
->consumed
;
5922 ratio
/= coding
->consumed
;
5923 require
= len_byte
* ratio
;
5927 if ((src
- dst
) < (require
+ 2000))
5929 /* See the comment above the previous call of make_gap. */
5930 int add
= len_byte
+ inserted_byte
;
5933 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5934 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5935 make_gap (require
+ 2000);
5937 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5938 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5941 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5943 if (encodep
&& coding
->dst_multibyte
)
5945 /* The output is unibyte. We must convert 8-bit characters to
5947 if (inserted_byte
* 2 > GAP_SIZE
)
5949 GAP_SIZE
-= inserted_byte
;
5950 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5951 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5952 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5953 make_gap (inserted_byte
- GAP_SIZE
);
5954 GAP_SIZE
+= inserted_byte
;
5955 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5956 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5957 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5959 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5962 /* If we shrank the conversion area, adjust it now. */
5966 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5967 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5968 GAP_SIZE
+= total_skip
;
5969 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5970 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5971 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5972 from
-= head_skip
; from_byte
-= head_skip
;
5973 to
+= tail_skip
; to_byte
+= tail_skip
;
5977 if (! EQ (current_buffer
->undo_list
, Qt
))
5978 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5980 adjust_after_replace_noundo (from
, from_byte
, nchars_del
, nbytes_del
,
5981 inserted
, inserted_byte
);
5982 inserted
= Z
- prev_Z
;
5984 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5985 coding_restore_composition (coding
, Fcurrent_buffer ());
5986 coding_free_composition_data (coding
);
5988 if (! inhibit_pre_post_conversion
5989 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5992 Lisp_Object saved_coding_system
;
5995 TEMP_SET_PT_BOTH (from
, from_byte
);
5997 record_unwind_protect (code_convert_region_unwind
,
5998 Fcons (Vlast_coding_system_used
, Qnil
));
5999 saved_coding_system
= Vlast_coding_system_used
;
6000 Vlast_coding_system_used
= coding
->symbol
;
6001 /* We should not call any more pre-write/post-read-conversion
6002 functions while this post-read-conversion is running. */
6003 inhibit_pre_post_conversion
= 1;
6004 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
6005 inhibit_pre_post_conversion
= 0;
6006 coding
->symbol
= Vlast_coding_system_used
;
6007 Vlast_coding_system_used
= saved_coding_system
;
6008 /* Discard the unwind protect. */
6011 inserted
+= Z
- prev_Z
;
6014 if (orig_point
>= from
)
6016 if (orig_point
>= from
+ orig_len
)
6017 orig_point
+= inserted
- orig_len
;
6020 TEMP_SET_PT (orig_point
);
6025 signal_after_change (from
, to
- from
, inserted
);
6026 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
6030 coding
->consumed
= to_byte
- from_byte
;
6031 coding
->consumed_char
= to
- from
;
6032 coding
->produced
= inserted_byte
;
6033 coding
->produced_char
= inserted
;
6039 /* Name (or base name) of work buffer for code conversion. */
6040 static Lisp_Object Vcode_conversion_workbuf_name
;
6042 /* Set the current buffer to the working buffer prepared for
6043 code-conversion. MULTIBYTE specifies the multibyteness of the
6044 buffer. Return the buffer we set if it must be killed after use.
6045 Otherwise return Qnil. */
6048 set_conversion_work_buffer (multibyte
)
6051 Lisp_Object buffer
, buffer_to_kill
;
6054 buffer
= Fget_buffer_create (Vcode_conversion_workbuf_name
);
6055 buf
= XBUFFER (buffer
);
6056 if (buf
== current_buffer
)
6058 /* As we are already in the work buffer, we must generate a new
6059 buffer for the work. */
6062 name
= Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name
, Qnil
);
6063 buffer
= buffer_to_kill
= Fget_buffer_create (name
);
6064 buf
= XBUFFER (buffer
);
6067 buffer_to_kill
= Qnil
;
6069 delete_all_overlays (buf
);
6070 buf
->directory
= current_buffer
->directory
;
6071 buf
->read_only
= Qnil
;
6072 buf
->filename
= Qnil
;
6073 buf
->undo_list
= Qt
;
6074 eassert (buf
->overlays_before
== NULL
);
6075 eassert (buf
->overlays_after
== NULL
);
6076 set_buffer_internal (buf
);
6077 if (BEG
!= BEGV
|| Z
!= ZV
)
6079 del_range_2 (BEG
, BEG_BYTE
, Z
, Z_BYTE
, 0);
6080 buf
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
6081 return buffer_to_kill
;
6085 run_pre_post_conversion_on_str (str
, coding
, encodep
)
6087 struct coding_system
*coding
;
6090 int count
= SPECPDL_INDEX ();
6091 struct gcpro gcpro1
, gcpro2
;
6092 int multibyte
= STRING_MULTIBYTE (str
);
6093 Lisp_Object old_deactivate_mark
;
6094 Lisp_Object buffer_to_kill
;
6095 Lisp_Object unwind_arg
;
6097 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
6098 /* It is not crucial to specbind this. */
6099 old_deactivate_mark
= Vdeactivate_mark
;
6100 GCPRO2 (str
, old_deactivate_mark
);
6102 /* We must insert the contents of STR as is without
6103 unibyte<->multibyte conversion. For that, we adjust the
6104 multibyteness of the working buffer to that of STR. */
6105 buffer_to_kill
= set_conversion_work_buffer (multibyte
);
6106 if (NILP (buffer_to_kill
))
6107 unwind_arg
= Fcons (Vlast_coding_system_used
, Qnil
);
6109 unwind_arg
= list2 (Vlast_coding_system_used
, buffer_to_kill
);
6110 record_unwind_protect (code_convert_region_unwind
, unwind_arg
);
6112 insert_from_string (str
, 0, 0,
6113 SCHARS (str
), SBYTES (str
), 0);
6115 inhibit_pre_post_conversion
= 1;
6118 struct buffer
*prev
= current_buffer
;
6120 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
6121 if (prev
!= current_buffer
)
6122 /* We must kill the current buffer too. */
6123 Fsetcdr (unwind_arg
, Fcons (Fcurrent_buffer (), XCDR (unwind_arg
)));
6127 Vlast_coding_system_used
= coding
->symbol
;
6128 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
6129 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
6130 coding
->symbol
= Vlast_coding_system_used
;
6132 inhibit_pre_post_conversion
= 0;
6133 Vdeactivate_mark
= old_deactivate_mark
;
6134 str
= make_buffer_string (BEG
, Z
, 1);
6135 return unbind_to (count
, str
);
6139 /* Run pre-write-conversion function of CODING on NCHARS/NBYTES
6140 text in *STR. *SIZE is the allocated bytes for STR. As it
6141 is intended that this function is called from encode_terminal_code,
6142 the pre-write-conversion function is run by safe_call and thus
6143 "Error during redisplay: ..." is logged when an error occurs.
6145 Store the resulting text in *STR and set CODING->produced_char and
6146 CODING->produced to the number of characters and bytes
6147 respectively. If the size of *STR is too small, enlarge it by
6148 xrealloc and update *STR and *SIZE. */
6151 run_pre_write_conversin_on_c_str (str
, size
, nchars
, nbytes
, coding
)
6152 unsigned char **str
;
6153 int *size
, nchars
, nbytes
;
6154 struct coding_system
*coding
;
6156 struct gcpro gcpro1
, gcpro2
;
6157 struct buffer
*cur
= current_buffer
;
6158 struct buffer
*prev
;
6159 Lisp_Object old_deactivate_mark
, old_last_coding_system_used
;
6160 Lisp_Object args
[3];
6161 Lisp_Object buffer_to_kill
;
6163 /* It is not crucial to specbind this. */
6164 old_deactivate_mark
= Vdeactivate_mark
;
6165 old_last_coding_system_used
= Vlast_coding_system_used
;
6166 GCPRO2 (old_deactivate_mark
, old_last_coding_system_used
);
6168 /* We must insert the contents of STR as is without
6169 unibyte<->multibyte conversion. For that, we adjust the
6170 multibyteness of the working buffer to that of STR. */
6171 buffer_to_kill
= set_conversion_work_buffer (coding
->src_multibyte
);
6172 insert_1_both (*str
, nchars
, nbytes
, 0, 0, 0);
6174 inhibit_pre_post_conversion
= 1;
6175 prev
= current_buffer
;
6176 args
[0] = coding
->pre_write_conversion
;
6177 args
[1] = make_number (BEG
);
6178 args
[2] = make_number (Z
);
6179 safe_call (3, args
);
6180 inhibit_pre_post_conversion
= 0;
6181 Vdeactivate_mark
= old_deactivate_mark
;
6182 Vlast_coding_system_used
= old_last_coding_system_used
;
6183 coding
->produced_char
= Z
- BEG
;
6184 coding
->produced
= Z_BYTE
- BEG_BYTE
;
6185 if (coding
->produced
> *size
)
6187 *size
= coding
->produced
;
6188 *str
= xrealloc (*str
, *size
);
6190 if (BEG
< GPT
&& GPT
< Z
)
6192 bcopy (BEG_ADDR
, *str
, coding
->produced
);
6193 coding
->src_multibyte
6194 = ! NILP (current_buffer
->enable_multibyte_characters
);
6195 if (prev
!= current_buffer
)
6196 Fkill_buffer (Fcurrent_buffer ());
6197 set_buffer_internal (cur
);
6198 if (! NILP (buffer_to_kill
))
6199 Fkill_buffer (buffer_to_kill
);
6204 decode_coding_string (str
, coding
, nocopy
)
6206 struct coding_system
*coding
;
6210 struct conversion_buffer buf
;
6212 Lisp_Object saved_coding_symbol
;
6214 int require_decoding
;
6215 int shrinked_bytes
= 0;
6217 int consumed
, consumed_char
, produced
, produced_char
;
6220 to_byte
= SBYTES (str
);
6222 saved_coding_symbol
= coding
->symbol
;
6223 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6224 coding
->dst_multibyte
= 1;
6225 if (CODING_REQUIRE_DETECTION (coding
))
6227 /* See the comments in code_convert_region. */
6228 if (coding
->type
== coding_type_undecided
)
6230 detect_coding (coding
, SDATA (str
), to_byte
);
6231 if (coding
->type
== coding_type_undecided
)
6233 coding
->type
= coding_type_emacs_mule
;
6234 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
6235 /* As emacs-mule decoder will handle composition, we
6236 need this setting to allocate coding->cmp_data
6238 coding
->composing
= COMPOSITION_NO
;
6241 if (coding
->eol_type
== CODING_EOL_UNDECIDED
6242 && coding
->type
!= coding_type_ccl
)
6244 saved_coding_symbol
= coding
->symbol
;
6245 detect_eol (coding
, SDATA (str
), to_byte
);
6246 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
6247 coding
->eol_type
= CODING_EOL_LF
;
6248 /* We had better recover the original eol format if we
6249 encounter an inconsistent eol format while decoding. */
6250 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
6254 if (coding
->type
== coding_type_no_conversion
6255 || coding
->type
== coding_type_raw_text
)
6256 coding
->dst_multibyte
= 0;
6258 require_decoding
= CODING_REQUIRE_DECODING (coding
);
6260 if (STRING_MULTIBYTE (str
))
6262 /* Decoding routines expect the source text to be unibyte. */
6263 str
= Fstring_as_unibyte (str
);
6264 to_byte
= SBYTES (str
);
6266 coding
->src_multibyte
= 0;
6269 /* Try to skip the heading and tailing ASCIIs. */
6270 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
6272 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6274 if (from
== to_byte
)
6275 require_decoding
= 0;
6276 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6279 if (!require_decoding
6280 && !(SYMBOLP (coding
->post_read_conversion
)
6281 && !NILP (Ffboundp (coding
->post_read_conversion
))))
6283 coding
->consumed
= SBYTES (str
);
6284 coding
->consumed_char
= SCHARS (str
);
6285 if (coding
->dst_multibyte
)
6287 str
= Fstring_as_multibyte (str
);
6290 coding
->produced
= SBYTES (str
);
6291 coding
->produced_char
= SCHARS (str
);
6292 return (nocopy
? str
: Fcopy_sequence (str
));
6295 if (coding
->composing
!= COMPOSITION_DISABLED
)
6296 coding_allocate_composition_data (coding
, from
);
6297 len
= decoding_buffer_size (coding
, to_byte
- from
);
6298 allocate_conversion_buffer (buf
, len
);
6300 consumed
= consumed_char
= produced
= produced_char
= 0;
6303 result
= decode_coding (coding
, SDATA (str
) + from
+ consumed
,
6304 buf
.data
+ produced
, to_byte
- from
- consumed
,
6305 buf
.size
- produced
);
6306 consumed
+= coding
->consumed
;
6307 consumed_char
+= coding
->consumed_char
;
6308 produced
+= coding
->produced
;
6309 produced_char
+= coding
->produced_char
;
6310 if (result
== CODING_FINISH_NORMAL
6311 || result
== CODING_FINISH_INTERRUPT
6312 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6313 && coding
->consumed
== 0))
6315 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
6316 coding_allocate_composition_data (coding
, from
+ produced_char
);
6317 else if (result
== CODING_FINISH_INSUFFICIENT_DST
)
6318 extend_conversion_buffer (&buf
);
6319 else if (result
== CODING_FINISH_INCONSISTENT_EOL
)
6321 Lisp_Object eol_type
;
6323 /* Recover the original EOL format. */
6324 if (coding
->eol_type
== CODING_EOL_CR
)
6327 for (p
= buf
.data
; p
< buf
.data
+ produced
; p
++)
6328 if (*p
== '\n') *p
= '\r';
6330 else if (coding
->eol_type
== CODING_EOL_CRLF
)
6333 unsigned char *p0
, *p1
;
6334 for (p0
= buf
.data
, p1
= p0
+ produced
; p0
< p1
; p0
++)
6335 if (*p0
== '\n') num_eol
++;
6336 if (produced
+ num_eol
>= buf
.size
)
6337 extend_conversion_buffer (&buf
);
6338 for (p0
= buf
.data
+ produced
, p1
= p0
+ num_eol
; p0
> buf
.data
;)
6341 if (*p0
== '\n') *--p1
= '\r';
6343 produced
+= num_eol
;
6344 produced_char
+= num_eol
;
6346 /* Suppress eol-format conversion in the further conversion. */
6347 coding
->eol_type
= CODING_EOL_LF
;
6349 /* Set the coding system symbol to that for Unix-like EOL. */
6350 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
6351 if (VECTORP (eol_type
)
6352 && XVECTOR (eol_type
)->size
== 3
6353 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
6354 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
6356 coding
->symbol
= saved_coding_symbol
;
6362 coding
->consumed
= consumed
;
6363 coding
->consumed_char
= consumed_char
;
6364 coding
->produced
= produced
;
6365 coding
->produced_char
= produced_char
;
6367 if (coding
->dst_multibyte
)
6368 newstr
= make_uninit_multibyte_string (produced_char
+ shrinked_bytes
,
6369 produced
+ shrinked_bytes
);
6371 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6373 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6374 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6375 if (shrinked_bytes
> from
)
6376 STRING_COPYIN (newstr
, from
+ produced
,
6377 SDATA (str
) + to_byte
,
6378 shrinked_bytes
- from
);
6379 free_conversion_buffer (&buf
);
6381 coding
->consumed
+= shrinked_bytes
;
6382 coding
->consumed_char
+= shrinked_bytes
;
6383 coding
->produced
+= shrinked_bytes
;
6384 coding
->produced_char
+= shrinked_bytes
;
6386 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
6387 coding_restore_composition (coding
, newstr
);
6388 coding_free_composition_data (coding
);
6390 if (SYMBOLP (coding
->post_read_conversion
)
6391 && !NILP (Ffboundp (coding
->post_read_conversion
)))
6392 newstr
= run_pre_post_conversion_on_str (newstr
, coding
, 0);
6398 encode_coding_string (str
, coding
, nocopy
)
6400 struct coding_system
*coding
;
6404 struct conversion_buffer buf
;
6405 int from
, to
, to_byte
;
6407 int shrinked_bytes
= 0;
6409 int consumed
, consumed_char
, produced
, produced_char
;
6411 if (SYMBOLP (coding
->pre_write_conversion
)
6412 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
6414 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
6415 /* As STR is just newly generated, we don't have to copy it
6422 to_byte
= SBYTES (str
);
6424 /* Encoding routines determine the multibyteness of the source text
6425 by coding->src_multibyte. */
6426 coding
->src_multibyte
= SCHARS (str
) < SBYTES (str
);
6427 coding
->dst_multibyte
= 0;
6428 if (! CODING_REQUIRE_ENCODING (coding
))
6429 goto no_need_of_encoding
;
6431 if (coding
->composing
!= COMPOSITION_DISABLED
)
6432 coding_save_composition (coding
, from
, to
, str
);
6434 /* Try to skip the heading and tailing ASCIIs. We can't skip them
6435 if we must run CCL program or there are compositions to
6437 if (coding
->type
!= coding_type_ccl
6438 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
6440 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6442 if (from
== to_byte
)
6444 coding_free_composition_data (coding
);
6445 goto no_need_of_encoding
;
6447 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6450 len
= encoding_buffer_size (coding
, to_byte
- from
);
6451 allocate_conversion_buffer (buf
, len
);
6453 consumed
= consumed_char
= produced
= produced_char
= 0;
6456 result
= encode_coding (coding
, SDATA (str
) + from
+ consumed
,
6457 buf
.data
+ produced
, to_byte
- from
- consumed
,
6458 buf
.size
- produced
);
6459 consumed
+= coding
->consumed
;
6460 consumed_char
+= coding
->consumed_char
;
6461 produced
+= coding
->produced
;
6462 produced_char
+= coding
->produced_char
;
6463 if (result
== CODING_FINISH_NORMAL
6464 || result
== CODING_FINISH_INTERRUPT
6465 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6466 && coding
->consumed
== 0))
6468 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6469 extend_conversion_buffer (&buf
);
6472 coding
->consumed
= consumed
;
6473 coding
->consumed_char
= consumed_char
;
6474 coding
->produced
= produced
;
6475 coding
->produced_char
= produced_char
;
6477 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6479 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6480 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6481 if (shrinked_bytes
> from
)
6482 STRING_COPYIN (newstr
, from
+ produced
,
6483 SDATA (str
) + to_byte
,
6484 shrinked_bytes
- from
);
6486 free_conversion_buffer (&buf
);
6487 coding_free_composition_data (coding
);
6491 no_need_of_encoding
:
6492 coding
->consumed
= SBYTES (str
);
6493 coding
->consumed_char
= SCHARS (str
);
6494 if (STRING_MULTIBYTE (str
))
6497 /* We are sure that STR doesn't contain a multibyte
6499 STRING_SET_UNIBYTE (str
);
6502 str
= Fstring_as_unibyte (str
);
6506 coding
->produced
= SBYTES (str
);
6507 coding
->produced_char
= SCHARS (str
);
6508 return (nocopy
? str
: Fcopy_sequence (str
));
6513 /*** 8. Emacs Lisp library functions ***/
6515 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
6516 doc
: /* Return t if OBJECT is nil or a coding-system.
6517 See the documentation of `make-coding-system' for information
6518 about coding-system objects. */)
6526 if (! NILP (Fget (obj
, Qcoding_system_define_form
)))
6528 /* Get coding-spec vector for OBJ. */
6529 obj
= Fget (obj
, Qcoding_system
);
6530 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
6534 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
6535 Sread_non_nil_coding_system
, 1, 1, 0,
6536 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6543 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6544 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
6546 while (SCHARS (val
) == 0);
6547 return (Fintern (val
, Qnil
));
6550 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
6551 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6552 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
6553 (prompt
, default_coding_system
)
6554 Lisp_Object prompt
, default_coding_system
;
6557 if (SYMBOLP (default_coding_system
))
6558 default_coding_system
= SYMBOL_NAME (default_coding_system
);
6559 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6560 Qt
, Qnil
, Qcoding_system_history
,
6561 default_coding_system
, Qnil
);
6562 return (SCHARS (val
) == 0 ? Qnil
: Fintern (val
, Qnil
));
6565 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
6567 doc
: /* Check validity of CODING-SYSTEM.
6568 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6569 It is valid if it is nil or a symbol with a non-nil `coding-system' property.
6570 The value of this property should be a vector of length 5. */)
6572 Lisp_Object coding_system
;
6574 Lisp_Object define_form
;
6576 define_form
= Fget (coding_system
, Qcoding_system_define_form
);
6577 if (! NILP (define_form
))
6579 Fput (coding_system
, Qcoding_system_define_form
, Qnil
);
6580 safe_eval (define_form
);
6582 if (!NILP (Fcoding_system_p (coding_system
)))
6583 return coding_system
;
6585 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
6589 detect_coding_system (src
, src_bytes
, highest
, multibytep
)
6590 const unsigned char *src
;
6591 int src_bytes
, highest
;
6594 int coding_mask
, eol_type
;
6595 Lisp_Object val
, tmp
;
6598 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
, multibytep
);
6599 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
6600 if (eol_type
== CODING_EOL_INCONSISTENT
)
6601 eol_type
= CODING_EOL_UNDECIDED
;
6606 if (eol_type
!= CODING_EOL_UNDECIDED
)
6609 val2
= Fget (Qundecided
, Qeol_type
);
6611 val
= XVECTOR (val2
)->contents
[eol_type
];
6613 return (highest
? val
: Fcons (val
, Qnil
));
6616 /* At first, gather possible coding systems in VAL. */
6618 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
6620 Lisp_Object category_val
, category_index
;
6622 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
6623 category_val
= Fsymbol_value (XCAR (tmp
));
6624 if (!NILP (category_val
)
6625 && NATNUMP (category_index
)
6626 && (coding_mask
& (1 << XFASTINT (category_index
))))
6628 val
= Fcons (category_val
, val
);
6634 val
= Fnreverse (val
);
6636 /* Then, replace the elements with subsidiary coding systems. */
6637 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
6639 if (eol_type
!= CODING_EOL_UNDECIDED
6640 && eol_type
!= CODING_EOL_INCONSISTENT
)
6643 eol
= Fget (XCAR (tmp
), Qeol_type
);
6645 XSETCAR (tmp
, XVECTOR (eol
)->contents
[eol_type
]);
6648 return (highest
? XCAR (val
) : val
);
6651 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
6653 doc
: /* Detect how the byte sequence in the region is encoded.
6654 Return a list of possible coding systems used on decoding a byte
6655 sequence containing the bytes in the region between START and END when
6656 the coding system `undecided' is specified. The list is ordered by
6657 priority decided in the current language environment.
6659 If only ASCII characters are found, it returns a list of single element
6660 `undecided' or its subsidiary coding system according to a detected
6663 If optional argument HIGHEST is non-nil, return the coding system of
6664 highest priority. */)
6665 (start
, end
, highest
)
6666 Lisp_Object start
, end
, highest
;
6669 int from_byte
, to_byte
;
6670 int include_anchor_byte
= 0;
6672 CHECK_NUMBER_COERCE_MARKER (start
);
6673 CHECK_NUMBER_COERCE_MARKER (end
);
6675 validate_region (&start
, &end
);
6676 from
= XINT (start
), to
= XINT (end
);
6677 from_byte
= CHAR_TO_BYTE (from
);
6678 to_byte
= CHAR_TO_BYTE (to
);
6680 if (from
< GPT
&& to
>= GPT
)
6681 move_gap_both (to
, to_byte
);
6682 /* If we an anchor byte `\0' follows the region, we include it in
6683 the detecting source. Then code detectors can handle the tailing
6684 byte sequence more accurately.
6686 Fix me: This is not a perfect solution. It is better that we
6687 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6689 if (to
== Z
|| (to
== GPT
&& GAP_SIZE
> 0))
6690 include_anchor_byte
= 1;
6691 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
6692 to_byte
- from_byte
+ include_anchor_byte
,
6694 !NILP (current_buffer
6695 ->enable_multibyte_characters
));
6698 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
6700 doc
: /* Detect how the byte sequence in STRING is encoded.
6701 Return a list of possible coding systems used on decoding a byte
6702 sequence containing the bytes in STRING when the coding system
6703 `undecided' is specified. The list is ordered by priority decided in
6704 the current language environment.
6706 If only ASCII characters are found, it returns a list of single element
6707 `undecided' or its subsidiary coding system according to a detected
6710 If optional argument HIGHEST is non-nil, return the coding system of
6711 highest priority. */)
6713 Lisp_Object string
, highest
;
6715 CHECK_STRING (string
);
6717 return detect_coding_system (SDATA (string
),
6718 /* "+ 1" is to include the anchor byte
6719 `\0'. With this, code detectors can
6720 handle the tailing bytes more
6722 SBYTES (string
) + 1,
6724 STRING_MULTIBYTE (string
));
6727 /* Subroutine for Ffind_coding_systems_region_internal.
6729 Return a list of coding systems that safely encode the multibyte
6730 text between P and PEND. SAFE_CODINGS, if non-nil, is an alist of
6731 possible coding systems. If it is nil, it means that we have not
6732 yet found any coding systems.
6734 WORK_TABLE a char-table of which element is set to t once the
6735 element is looked up.
6737 If a non-ASCII single byte char is found, set
6738 *single_byte_char_found to 1. */
6741 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
6742 unsigned char *p
, *pend
;
6743 Lisp_Object safe_codings
, work_table
;
6744 int *single_byte_char_found
;
6747 Lisp_Object val
, ch
;
6748 Lisp_Object prev
, tail
;
6750 if (NILP (safe_codings
))
6751 goto done_safe_codings
;
6754 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6756 if (ASCII_BYTE_P (c
))
6757 /* We can ignore ASCII characters here. */
6759 if (SINGLE_BYTE_CHAR_P (c
))
6760 *single_byte_char_found
= 1;
6761 /* Check the safe coding systems for C. */
6762 ch
= make_number (c
);
6763 val
= Faref (work_table
, ch
);
6765 /* This element was already checked. Ignore it. */
6767 /* Remember that we checked this element. */
6768 Faset (work_table
, ch
, Qt
);
6770 for (prev
= tail
= safe_codings
; CONSP (tail
); tail
= XCDR (tail
))
6772 Lisp_Object elt
, translation_table
, hash_table
, accept_latin_extra
;
6776 if (CONSP (XCDR (elt
)))
6778 /* This entry has this format now:
6779 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6780 ACCEPT-LATIN-EXTRA ) */
6782 encodable
= ! NILP (Faref (XCAR (val
), ch
));
6786 translation_table
= XCAR (val
);
6787 hash_table
= XCAR (XCDR (val
));
6788 accept_latin_extra
= XCAR (XCDR (XCDR (val
)));
6793 /* This entry has this format now: ( CODING . SAFE-CHARS) */
6794 encodable
= ! NILP (Faref (XCDR (elt
), ch
));
6797 /* Transform the format to:
6798 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6799 ACCEPT-LATIN-EXTRA ) */
6800 val
= Fget (XCAR (elt
), Qcoding_system
);
6802 = Fplist_get (AREF (val
, 3),
6803 Qtranslation_table_for_encode
);
6804 if (SYMBOLP (translation_table
))
6805 translation_table
= Fget (translation_table
,
6806 Qtranslation_table
);
6808 = (CHAR_TABLE_P (translation_table
)
6809 ? XCHAR_TABLE (translation_table
)->extras
[1]
6812 = ((EQ (AREF (val
, 0), make_number (2))
6813 && VECTORP (AREF (val
, 4)))
6814 ? AREF (AREF (val
, 4), 16)
6816 XSETCAR (tail
, list5 (XCAR (elt
), XCDR (elt
),
6817 translation_table
, hash_table
,
6818 accept_latin_extra
));
6823 && ((CHAR_TABLE_P (translation_table
)
6824 && ! NILP (Faref (translation_table
, ch
)))
6825 || (HASH_TABLE_P (hash_table
)
6826 && ! NILP (Fgethash (ch
, hash_table
, Qnil
)))
6827 || (SINGLE_BYTE_CHAR_P (c
)
6828 && ! NILP (accept_latin_extra
)
6829 && VECTORP (Vlatin_extra_code_table
)
6830 && ! NILP (AREF (Vlatin_extra_code_table
, c
)))))
6836 /* Exclude this coding system from SAFE_CODINGS. */
6837 if (EQ (tail
, safe_codings
))
6839 safe_codings
= XCDR (safe_codings
);
6840 if (NILP (safe_codings
))
6841 goto done_safe_codings
;
6844 XSETCDR (prev
, XCDR (tail
));
6850 /* If the above loop was terminated before P reaches PEND, it means
6851 SAFE_CODINGS was set to nil. If we have not yet found an
6852 non-ASCII single-byte char, check it now. */
6853 if (! *single_byte_char_found
)
6856 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6858 if (! ASCII_BYTE_P (c
)
6859 && SINGLE_BYTE_CHAR_P (c
))
6861 *single_byte_char_found
= 1;
6865 return safe_codings
;
6868 DEFUN ("find-coding-systems-region-internal",
6869 Ffind_coding_systems_region_internal
,
6870 Sfind_coding_systems_region_internal
, 2, 2, 0,
6871 doc
: /* Internal use only. */)
6873 Lisp_Object start
, end
;
6875 Lisp_Object work_table
, safe_codings
;
6876 int non_ascii_p
= 0;
6877 int single_byte_char_found
= 0;
6878 const unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
6880 if (STRINGP (start
))
6882 if (!STRING_MULTIBYTE (start
))
6884 p1
= SDATA (start
), p1end
= p1
+ SBYTES (start
);
6886 if (SCHARS (start
) != SBYTES (start
))
6893 CHECK_NUMBER_COERCE_MARKER (start
);
6894 CHECK_NUMBER_COERCE_MARKER (end
);
6895 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
6896 args_out_of_range (start
, end
);
6897 if (NILP (current_buffer
->enable_multibyte_characters
))
6899 from
= CHAR_TO_BYTE (XINT (start
));
6900 to
= CHAR_TO_BYTE (XINT (end
));
6901 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
6902 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
6906 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
6907 if (XINT (end
) - XINT (start
) != to
- from
)
6913 /* We are sure that the text contains no multibyte character.
6914 Check if it contains eight-bit-graphic. */
6916 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
6919 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
6925 /* The text contains non-ASCII characters. */
6927 work_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
6928 safe_codings
= Fcopy_sequence (XCDR (Vcoding_system_safe_chars
));
6930 safe_codings
= find_safe_codings (p1
, p1end
, safe_codings
, work_table
,
6931 &single_byte_char_found
);
6933 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
6934 &single_byte_char_found
);
6935 if (EQ (safe_codings
, XCDR (Vcoding_system_safe_chars
)))
6939 /* Turn safe_codings to a list of coding systems... */
6942 if (single_byte_char_found
)
6943 /* ... and append these for eight-bit chars. */
6944 val
= Fcons (Qraw_text
,
6945 Fcons (Qemacs_mule
, Fcons (Qno_conversion
, Qnil
)));
6947 /* ... and append generic coding systems. */
6948 val
= Fcopy_sequence (XCAR (Vcoding_system_safe_chars
));
6950 for (; CONSP (safe_codings
); safe_codings
= XCDR (safe_codings
))
6951 val
= Fcons (XCAR (XCAR (safe_codings
)), val
);
6955 return safe_codings
;
6959 /* Search from position POS for such characters that are unencodable
6960 accoding to SAFE_CHARS, and return a list of their positions. P
6961 points where in the memory the character at POS exists. Limit the
6962 search at PEND or when Nth unencodable characters are found.
6964 If SAFE_CHARS is a char table, an element for an unencodable
6967 If SAFE_CHARS is nil, all non-ASCII characters are unencodable.
6969 Otherwise, SAFE_CHARS is t, and only eight-bit-contrl and
6970 eight-bit-graphic characters are unencodable. */
6973 unencodable_char_position (safe_chars
, pos
, p
, pend
, n
)
6974 Lisp_Object safe_chars
;
6976 unsigned char *p
, *pend
;
6979 Lisp_Object pos_list
;
6985 int c
= STRING_CHAR_AND_LENGTH (p
, MAX_MULTIBYTE_LENGTH
, len
);
6988 && (CHAR_TABLE_P (safe_chars
)
6989 ? NILP (CHAR_TABLE_REF (safe_chars
, c
))
6990 : (NILP (safe_chars
) || c
< 256)))
6992 pos_list
= Fcons (make_number (pos
), pos_list
);
6999 return Fnreverse (pos_list
);
7003 DEFUN ("unencodable-char-position", Funencodable_char_position
,
7004 Sunencodable_char_position
, 3, 5, 0,
7006 Return position of first un-encodable character in a region.
7007 START and END specfiy the region and CODING-SYSTEM specifies the
7008 encoding to check. Return nil if CODING-SYSTEM does encode the region.
7010 If optional 4th argument COUNT is non-nil, it specifies at most how
7011 many un-encodable characters to search. In this case, the value is a
7014 If optional 5th argument STRING is non-nil, it is a string to search
7015 for un-encodable characters. In that case, START and END are indexes
7017 (start
, end
, coding_system
, count
, string
)
7018 Lisp_Object start
, end
, coding_system
, count
, string
;
7021 Lisp_Object safe_chars
;
7022 struct coding_system coding
;
7023 Lisp_Object positions
;
7025 unsigned char *p
, *pend
;
7029 validate_region (&start
, &end
);
7030 from
= XINT (start
);
7032 if (NILP (current_buffer
->enable_multibyte_characters
))
7034 p
= CHAR_POS_ADDR (from
);
7038 pend
= CHAR_POS_ADDR (to
);
7042 CHECK_STRING (string
);
7043 CHECK_NATNUM (start
);
7045 from
= XINT (start
);
7048 || to
> SCHARS (string
))
7049 args_out_of_range_3 (string
, start
, end
);
7050 if (! STRING_MULTIBYTE (string
))
7052 p
= SDATA (string
) + string_char_to_byte (string
, from
);
7053 pend
= SDATA (string
) + string_char_to_byte (string
, to
);
7056 setup_coding_system (Fcheck_coding_system (coding_system
), &coding
);
7062 CHECK_NATNUM (count
);
7066 if (coding
.type
== coding_type_no_conversion
7067 || coding
.type
== coding_type_raw_text
)
7070 if (coding
.type
== coding_type_undecided
)
7073 safe_chars
= coding_safe_chars (coding_system
);
7075 if (STRINGP (string
)
7076 || from
>= GPT
|| to
<= GPT
)
7077 positions
= unencodable_char_position (safe_chars
, from
, p
, pend
, n
);
7080 Lisp_Object args
[2];
7082 args
[0] = unencodable_char_position (safe_chars
, from
, p
, GPT_ADDR
, n
);
7083 n
-= XINT (Flength (args
[0]));
7085 positions
= args
[0];
7088 args
[1] = unencodable_char_position (safe_chars
, GPT
, GAP_END_ADDR
,
7090 positions
= Fappend (2, args
);
7094 return (NILP (count
) ? Fcar (positions
) : positions
);
7099 code_convert_region1 (start
, end
, coding_system
, encodep
)
7100 Lisp_Object start
, end
, coding_system
;
7103 struct coding_system coding
;
7106 CHECK_NUMBER_COERCE_MARKER (start
);
7107 CHECK_NUMBER_COERCE_MARKER (end
);
7108 CHECK_SYMBOL (coding_system
);
7110 validate_region (&start
, &end
);
7111 from
= XFASTINT (start
);
7112 to
= XFASTINT (end
);
7114 if (NILP (coding_system
))
7115 return make_number (to
- from
);
7117 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7118 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7120 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7121 coding
.src_multibyte
= coding
.dst_multibyte
7122 = !NILP (current_buffer
->enable_multibyte_characters
);
7123 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
7124 &coding
, encodep
, 1);
7125 Vlast_coding_system_used
= coding
.symbol
;
7126 return make_number (coding
.produced_char
);
7129 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
7130 3, 3, "r\nzCoding system: ",
7131 doc
: /* Decode the current region from the specified coding system.
7132 When called from a program, takes three arguments:
7133 START, END, and CODING-SYSTEM. START and END are buffer positions.
7134 This function sets `last-coding-system-used' to the precise coding system
7135 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7136 not fully specified.)
7137 It returns the length of the decoded text. */)
7138 (start
, end
, coding_system
)
7139 Lisp_Object start
, end
, coding_system
;
7141 return code_convert_region1 (start
, end
, coding_system
, 0);
7144 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
7145 3, 3, "r\nzCoding system: ",
7146 doc
: /* Encode the current region into the specified coding system.
7147 When called from a program, takes three arguments:
7148 START, END, and CODING-SYSTEM. START and END are buffer positions.
7149 This function sets `last-coding-system-used' to the precise coding system
7150 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7151 not fully specified.)
7152 It returns the length of the encoded text. */)
7153 (start
, end
, coding_system
)
7154 Lisp_Object start
, end
, coding_system
;
7156 return code_convert_region1 (start
, end
, coding_system
, 1);
7160 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
7161 Lisp_Object string
, coding_system
, nocopy
;
7164 struct coding_system coding
;
7166 CHECK_STRING (string
);
7167 CHECK_SYMBOL (coding_system
);
7169 if (NILP (coding_system
))
7170 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
7172 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7173 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7175 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7177 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
7178 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
7179 Vlast_coding_system_used
= coding
.symbol
;
7184 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
7186 doc
: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7187 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7188 if the decoding operation is trivial.
7189 This function sets `last-coding-system-used' to the precise coding system
7190 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7191 not fully specified.) */)
7192 (string
, coding_system
, nocopy
)
7193 Lisp_Object string
, coding_system
, nocopy
;
7195 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
7198 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
7200 doc
: /* Encode STRING to CODING-SYSTEM, and return the result.
7201 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7202 if the encoding operation is trivial.
7203 This function sets `last-coding-system-used' to the precise coding system
7204 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7205 not fully specified.) */)
7206 (string
, coding_system
, nocopy
)
7207 Lisp_Object string
, coding_system
, nocopy
;
7209 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
7212 /* Encode or decode STRING according to CODING_SYSTEM.
7213 Do not set Vlast_coding_system_used.
7215 This function is called only from macros DECODE_FILE and
7216 ENCODE_FILE, thus we ignore character composition. */
7219 code_convert_string_norecord (string
, coding_system
, encodep
)
7220 Lisp_Object string
, coding_system
;
7223 struct coding_system coding
;
7225 CHECK_STRING (string
);
7226 CHECK_SYMBOL (coding_system
);
7228 if (NILP (coding_system
))
7231 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7232 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7234 coding
.composing
= COMPOSITION_DISABLED
;
7235 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7237 ? encode_coding_string (string
, &coding
, 1)
7238 : decode_coding_string (string
, &coding
, 1));
7241 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
7242 doc
: /* Decode a Japanese character which has CODE in shift_jis encoding.
7243 Return the corresponding character. */)
7247 unsigned char c1
, c2
, s1
, s2
;
7250 CHECK_NUMBER (code
);
7251 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
7255 XSETFASTINT (val
, s2
);
7256 else if (s2
>= 0xA0 || s2
<= 0xDF)
7257 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
7259 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7263 if ((s1
< 0x80 || (s1
> 0x9F && s1
< 0xE0) || s1
> 0xEF)
7264 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
7265 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7266 DECODE_SJIS (s1
, s2
, c1
, c2
);
7267 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
7272 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
7273 doc
: /* Encode a Japanese character CHAR to shift_jis encoding.
7274 Return the corresponding code in SJIS. */)
7278 int charset
, c1
, c2
, s1
, s2
;
7282 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7283 if (charset
== CHARSET_ASCII
)
7287 else if (charset
== charset_jisx0208
7288 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
7290 ENCODE_SJIS (c1
, c2
, s1
, s2
);
7291 XSETFASTINT (val
, (s1
<< 8) | s2
);
7293 else if (charset
== charset_katakana_jisx0201
7294 && c1
> 0x20 && c2
< 0xE0)
7296 XSETFASTINT (val
, c1
| 0x80);
7299 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
7303 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
7304 doc
: /* Decode a Big5 character which has CODE in BIG5 coding system.
7305 Return the corresponding character. */)
7310 unsigned char b1
, b2
, c1
, c2
;
7313 CHECK_NUMBER (code
);
7314 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
7318 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7323 if ((b1
< 0xA1 || b1
> 0xFE)
7324 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
7325 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7326 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
7327 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
7332 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
7333 doc
: /* Encode the Big5 character CHAR to BIG5 coding system.
7334 Return the corresponding character code in Big5. */)
7338 int charset
, c1
, c2
, b1
, b2
;
7342 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7343 if (charset
== CHARSET_ASCII
)
7347 else if ((charset
== charset_big5_1
7348 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
7349 || (charset
== charset_big5_2
7350 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
7352 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
7353 XSETFASTINT (val
, (b1
<< 8) | b2
);
7356 error ("Can't encode to Big5: %d", XFASTINT (ch
));
7360 DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal
,
7361 Sset_terminal_coding_system_internal
, 1, 2, 0,
7362 doc
: /* Internal use only. */)
7363 (coding_system
, terminal
)
7364 Lisp_Object coding_system
;
7365 Lisp_Object terminal
;
7367 struct coding_system
*terminal_coding
= TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1));
7368 CHECK_SYMBOL (coding_system
);
7369 setup_coding_system (Fcheck_coding_system (coding_system
), terminal_coding
);
7370 /* We had better not send unsafe characters to terminal. */
7371 terminal_coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
7372 /* Character composition should be disabled. */
7373 terminal_coding
->composing
= COMPOSITION_DISABLED
;
7374 /* Error notification should be suppressed. */
7375 terminal_coding
->suppress_error
= 1;
7376 terminal_coding
->src_multibyte
= 1;
7377 terminal_coding
->dst_multibyte
= 0;
7381 DEFUN ("set-safe-terminal-coding-system-internal", Fset_safe_terminal_coding_system_internal
,
7382 Sset_safe_terminal_coding_system_internal
, 1, 1, 0,
7383 doc
: /* Internal use only. */)
7385 Lisp_Object coding_system
;
7387 CHECK_SYMBOL (coding_system
);
7388 setup_coding_system (Fcheck_coding_system (coding_system
),
7389 &safe_terminal_coding
);
7390 /* Character composition should be disabled. */
7391 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
7392 /* Error notification should be suppressed. */
7393 safe_terminal_coding
.suppress_error
= 1;
7394 safe_terminal_coding
.src_multibyte
= 1;
7395 safe_terminal_coding
.dst_multibyte
= 0;
7399 DEFUN ("terminal-coding-system", Fterminal_coding_system
,
7400 Sterminal_coding_system
, 0, 1, 0,
7401 doc
: /* Return coding system specified for terminal output on the given terminal.
7402 TERMINAL may be a terminal id, a frame, or nil for the selected
7403 frame's terminal device. */)
7405 Lisp_Object terminal
;
7407 return TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1))->symbol
;
7410 DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal
,
7411 Sset_keyboard_coding_system_internal
, 1, 2, 0,
7412 doc
: /* Internal use only. */)
7413 (coding_system
, terminal
)
7414 Lisp_Object coding_system
;
7415 Lisp_Object terminal
;
7417 struct terminal
*t
= get_terminal (terminal
, 1);
7418 CHECK_SYMBOL (coding_system
);
7420 setup_coding_system (Fcheck_coding_system (coding_system
),
7421 TERMINAL_KEYBOARD_CODING (t
));
7422 /* Character composition should be disabled. */
7423 TERMINAL_KEYBOARD_CODING (t
)->composing
= COMPOSITION_DISABLED
;
7427 DEFUN ("keyboard-coding-system", Fkeyboard_coding_system
,
7428 Skeyboard_coding_system
, 0, 1, 0,
7429 doc
: /* Return coding system for decoding keyboard input on TERMINAL.
7430 TERMINAL may be a terminal id, a frame, or nil for the selected
7431 frame's terminal device. */)
7433 Lisp_Object terminal
;
7435 return TERMINAL_KEYBOARD_CODING (get_terminal (terminal
, 1))->symbol
;
7439 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
7440 Sfind_operation_coding_system
, 1, MANY
, 0,
7441 doc
: /* Choose a coding system for an operation based on the target name.
7442 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
7443 DECODING-SYSTEM is the coding system to use for decoding
7444 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
7445 for encoding (in case OPERATION does encoding).
7447 The first argument OPERATION specifies an I/O primitive:
7448 For file I/O, `insert-file-contents' or `write-region'.
7449 For process I/O, `call-process', `call-process-region', or `start-process'.
7450 For network I/O, `open-network-stream'.
7452 The remaining arguments should be the same arguments that were passed
7453 to the primitive. Depending on which primitive, one of those arguments
7454 is selected as the TARGET. For example, if OPERATION does file I/O,
7455 whichever argument specifies the file name is TARGET.
7457 TARGET has a meaning which depends on OPERATION:
7458 For file I/O, TARGET is a file name.
7459 For process I/O, TARGET is a process name.
7460 For network I/O, TARGET is a service name or a port number
7462 This function looks up what specified for TARGET in,
7463 `file-coding-system-alist', `process-coding-system-alist',
7464 or `network-coding-system-alist' depending on OPERATION.
7465 They may specify a coding system, a cons of coding systems,
7466 or a function symbol to call.
7467 In the last case, we call the function with one argument,
7468 which is a list of all the arguments given to this function.
7470 usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
7475 Lisp_Object operation
, target_idx
, target
, val
;
7476 register Lisp_Object chain
;
7479 error ("Too few arguments");
7480 operation
= args
[0];
7481 if (!SYMBOLP (operation
)
7482 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
7483 error ("Invalid first argument");
7484 if (nargs
< 1 + XINT (target_idx
))
7485 error ("Too few arguments for operation: %s",
7486 SDATA (SYMBOL_NAME (operation
)));
7487 /* For write-region, if the 6th argument (i.e. VISIT, the 5th
7488 argument to write-region) is string, it must be treated as a
7489 target file name. */
7490 if (EQ (operation
, Qwrite_region
)
7492 && STRINGP (args
[5]))
7493 target_idx
= make_number (4);
7494 target
= args
[XINT (target_idx
) + 1];
7495 if (!(STRINGP (target
)
7496 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
7497 error ("Invalid argument %d", XINT (target_idx
) + 1);
7499 chain
= ((EQ (operation
, Qinsert_file_contents
)
7500 || EQ (operation
, Qwrite_region
))
7501 ? Vfile_coding_system_alist
7502 : (EQ (operation
, Qopen_network_stream
)
7503 ? Vnetwork_coding_system_alist
7504 : Vprocess_coding_system_alist
));
7508 for (; CONSP (chain
); chain
= XCDR (chain
))
7514 && ((STRINGP (target
)
7515 && STRINGP (XCAR (elt
))
7516 && fast_string_match (XCAR (elt
), target
) >= 0)
7517 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
7520 /* Here, if VAL is both a valid coding system and a valid
7521 function symbol, we return VAL as a coding system. */
7524 if (! SYMBOLP (val
))
7526 if (! NILP (Fcoding_system_p (val
)))
7527 return Fcons (val
, val
);
7528 if (! NILP (Ffboundp (val
)))
7530 val
= call1 (val
, Flist (nargs
, args
));
7533 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
7534 return Fcons (val
, val
);
7542 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
7543 Supdate_coding_systems_internal
, 0, 0, 0,
7544 doc
: /* Update internal database for ISO2022 and CCL based coding systems.
7545 When values of any coding categories are changed, you must
7546 call this function. */)
7551 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7555 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[i
]);
7558 if (! coding_system_table
[i
])
7559 coding_system_table
[i
] = ((struct coding_system
*)
7560 xmalloc (sizeof (struct coding_system
)));
7561 setup_coding_system (val
, coding_system_table
[i
]);
7563 else if (coding_system_table
[i
])
7565 xfree (coding_system_table
[i
]);
7566 coding_system_table
[i
] = NULL
;
7573 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
7574 Sset_coding_priority_internal
, 0, 0, 0,
7575 doc
: /* Update internal database for the current value of `coding-category-list'.
7576 This function is internal use only. */)
7582 val
= Vcoding_category_list
;
7584 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
7586 if (! SYMBOLP (XCAR (val
)))
7588 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
7589 if (idx
>= CODING_CATEGORY_IDX_MAX
)
7591 coding_priorities
[i
++] = (1 << idx
);
7594 /* If coding-category-list is valid and contains all coding
7595 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
7596 the following code saves Emacs from crashing. */
7597 while (i
< CODING_CATEGORY_IDX_MAX
)
7598 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
7603 DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal
,
7604 Sdefine_coding_system_internal
, 1, 1, 0,
7605 doc
: /* Register CODING-SYSTEM as a base coding system.
7606 This function is internal use only. */)
7608 Lisp_Object coding_system
;
7610 Lisp_Object safe_chars
, slot
;
7612 if (NILP (Fcheck_coding_system (coding_system
)))
7613 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
7614 safe_chars
= coding_safe_chars (coding_system
);
7615 if (! EQ (safe_chars
, Qt
) && ! CHAR_TABLE_P (safe_chars
))
7616 error ("No valid safe-chars property for %s",
7617 SDATA (SYMBOL_NAME (coding_system
)));
7618 if (EQ (safe_chars
, Qt
))
7620 if (NILP (Fmemq (coding_system
, XCAR (Vcoding_system_safe_chars
))))
7621 XSETCAR (Vcoding_system_safe_chars
,
7622 Fcons (coding_system
, XCAR (Vcoding_system_safe_chars
)));
7626 slot
= Fassq (coding_system
, XCDR (Vcoding_system_safe_chars
));
7628 XSETCDR (Vcoding_system_safe_chars
,
7629 nconc2 (XCDR (Vcoding_system_safe_chars
),
7630 Fcons (Fcons (coding_system
, safe_chars
), Qnil
)));
7632 XSETCDR (slot
, safe_chars
);
7640 /*** 9. Post-amble ***/
7647 /* Emacs' internal format specific initialize routine. */
7648 for (i
= 0; i
<= 0x20; i
++)
7649 emacs_code_class
[i
] = EMACS_control_code
;
7650 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
7651 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
7652 for (i
= 0x21 ; i
< 0x7F; i
++)
7653 emacs_code_class
[i
] = EMACS_ascii_code
;
7654 emacs_code_class
[0x7F] = EMACS_control_code
;
7655 for (i
= 0x80; i
< 0xFF; i
++)
7656 emacs_code_class
[i
] = EMACS_invalid_code
;
7657 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
7658 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
7659 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
7660 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
7662 /* ISO2022 specific initialize routine. */
7663 for (i
= 0; i
< 0x20; i
++)
7664 iso_code_class
[i
] = ISO_control_0
;
7665 for (i
= 0x21; i
< 0x7F; i
++)
7666 iso_code_class
[i
] = ISO_graphic_plane_0
;
7667 for (i
= 0x80; i
< 0xA0; i
++)
7668 iso_code_class
[i
] = ISO_control_1
;
7669 for (i
= 0xA1; i
< 0xFF; i
++)
7670 iso_code_class
[i
] = ISO_graphic_plane_1
;
7671 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
7672 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
7673 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
7674 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
7675 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
7676 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
7677 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
7678 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
7679 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
7680 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
7682 setup_coding_system (Qnil
, &safe_terminal_coding
);
7683 setup_coding_system (Qnil
, &default_buffer_file_coding
);
7685 bzero (coding_system_table
, sizeof coding_system_table
);
7687 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
7688 for (i
= 0; i
< 128; i
++)
7689 ascii_skip_code
[i
] = 1;
7691 #if defined (MSDOS) || defined (WINDOWSNT)
7692 system_eol_type
= CODING_EOL_CRLF
;
7694 system_eol_type
= CODING_EOL_LF
;
7697 inhibit_pre_post_conversion
= 0;
7705 staticpro (&Vcode_conversion_workbuf_name
);
7706 Vcode_conversion_workbuf_name
= build_string (" *code-conversion-work*");
7708 Qtarget_idx
= intern ("target-idx");
7709 staticpro (&Qtarget_idx
);
7711 Qcoding_system_history
= intern ("coding-system-history");
7712 staticpro (&Qcoding_system_history
);
7713 Fset (Qcoding_system_history
, Qnil
);
7715 /* Target FILENAME is the first argument. */
7716 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
7717 /* Target FILENAME is the third argument. */
7718 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
7720 Qcall_process
= intern ("call-process");
7721 staticpro (&Qcall_process
);
7722 /* Target PROGRAM is the first argument. */
7723 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
7725 Qcall_process_region
= intern ("call-process-region");
7726 staticpro (&Qcall_process_region
);
7727 /* Target PROGRAM is the third argument. */
7728 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
7730 Qstart_process
= intern ("start-process");
7731 staticpro (&Qstart_process
);
7732 /* Target PROGRAM is the third argument. */
7733 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
7735 Qopen_network_stream
= intern ("open-network-stream");
7736 staticpro (&Qopen_network_stream
);
7737 /* Target SERVICE is the fourth argument. */
7738 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
7740 Qcoding_system
= intern ("coding-system");
7741 staticpro (&Qcoding_system
);
7743 Qeol_type
= intern ("eol-type");
7744 staticpro (&Qeol_type
);
7746 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
7747 staticpro (&Qbuffer_file_coding_system
);
7749 Qpost_read_conversion
= intern ("post-read-conversion");
7750 staticpro (&Qpost_read_conversion
);
7752 Qpre_write_conversion
= intern ("pre-write-conversion");
7753 staticpro (&Qpre_write_conversion
);
7755 Qno_conversion
= intern ("no-conversion");
7756 staticpro (&Qno_conversion
);
7758 Qundecided
= intern ("undecided");
7759 staticpro (&Qundecided
);
7761 Qcoding_system_p
= intern ("coding-system-p");
7762 staticpro (&Qcoding_system_p
);
7764 Qcoding_system_error
= intern ("coding-system-error");
7765 staticpro (&Qcoding_system_error
);
7767 Fput (Qcoding_system_error
, Qerror_conditions
,
7768 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
7769 Fput (Qcoding_system_error
, Qerror_message
,
7770 build_string ("Invalid coding system"));
7772 Qcoding_category
= intern ("coding-category");
7773 staticpro (&Qcoding_category
);
7774 Qcoding_category_index
= intern ("coding-category-index");
7775 staticpro (&Qcoding_category_index
);
7777 Vcoding_category_table
7778 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
7779 staticpro (&Vcoding_category_table
);
7782 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7784 XVECTOR (Vcoding_category_table
)->contents
[i
]
7785 = intern (coding_category_name
[i
]);
7786 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
7787 Qcoding_category_index
, make_number (i
));
7791 Vcoding_system_safe_chars
= Fcons (Qnil
, Qnil
);
7792 staticpro (&Vcoding_system_safe_chars
);
7794 Qtranslation_table
= intern ("translation-table");
7795 staticpro (&Qtranslation_table
);
7796 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (2));
7798 Qtranslation_table_id
= intern ("translation-table-id");
7799 staticpro (&Qtranslation_table_id
);
7801 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
7802 staticpro (&Qtranslation_table_for_decode
);
7804 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
7805 staticpro (&Qtranslation_table_for_encode
);
7807 Qsafe_chars
= intern ("safe-chars");
7808 staticpro (&Qsafe_chars
);
7810 Qchar_coding_system
= intern ("char-coding-system");
7811 staticpro (&Qchar_coding_system
);
7813 /* Intern this now in case it isn't already done.
7814 Setting this variable twice is harmless.
7815 But don't staticpro it here--that is done in alloc.c. */
7816 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
7817 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
7818 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (0));
7820 Qvalid_codes
= intern ("valid-codes");
7821 staticpro (&Qvalid_codes
);
7823 Qemacs_mule
= intern ("emacs-mule");
7824 staticpro (&Qemacs_mule
);
7826 Qraw_text
= intern ("raw-text");
7827 staticpro (&Qraw_text
);
7829 Qutf_8
= intern ("utf-8");
7830 staticpro (&Qutf_8
);
7832 Qcoding_system_define_form
= intern ("coding-system-define-form");
7833 staticpro (&Qcoding_system_define_form
);
7835 defsubr (&Scoding_system_p
);
7836 defsubr (&Sread_coding_system
);
7837 defsubr (&Sread_non_nil_coding_system
);
7838 defsubr (&Scheck_coding_system
);
7839 defsubr (&Sdetect_coding_region
);
7840 defsubr (&Sdetect_coding_string
);
7841 defsubr (&Sfind_coding_systems_region_internal
);
7842 defsubr (&Sunencodable_char_position
);
7843 defsubr (&Sdecode_coding_region
);
7844 defsubr (&Sencode_coding_region
);
7845 defsubr (&Sdecode_coding_string
);
7846 defsubr (&Sencode_coding_string
);
7847 defsubr (&Sdecode_sjis_char
);
7848 defsubr (&Sencode_sjis_char
);
7849 defsubr (&Sdecode_big5_char
);
7850 defsubr (&Sencode_big5_char
);
7851 defsubr (&Sset_terminal_coding_system_internal
);
7852 defsubr (&Sset_safe_terminal_coding_system_internal
);
7853 defsubr (&Sterminal_coding_system
);
7854 defsubr (&Sset_keyboard_coding_system_internal
);
7855 defsubr (&Skeyboard_coding_system
);
7856 defsubr (&Sfind_operation_coding_system
);
7857 defsubr (&Supdate_coding_systems_internal
);
7858 defsubr (&Sset_coding_priority_internal
);
7859 defsubr (&Sdefine_coding_system_internal
);
7861 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
7862 doc
: /* List of coding systems.
7864 Do not alter the value of this variable manually. This variable should be
7865 updated by the functions `make-coding-system' and
7866 `define-coding-system-alias'. */);
7867 Vcoding_system_list
= Qnil
;
7869 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
7870 doc
: /* Alist of coding system names.
7871 Each element is one element list of coding system name.
7872 This variable is given to `completing-read' as TABLE argument.
7874 Do not alter the value of this variable manually. This variable should be
7875 updated by the functions `make-coding-system' and
7876 `define-coding-system-alias'. */);
7877 Vcoding_system_alist
= Qnil
;
7879 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
7880 doc
: /* List of coding-categories (symbols) ordered by priority.
7882 On detecting a coding system, Emacs tries code detection algorithms
7883 associated with each coding-category one by one in this order. When
7884 one algorithm agrees with a byte sequence of source text, the coding
7885 system bound to the corresponding coding-category is selected.
7887 Don't modify this variable directly, but use `set-coding-priority'. */);
7891 Vcoding_category_list
= Qnil
;
7892 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
7893 Vcoding_category_list
7894 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
7895 Vcoding_category_list
);
7898 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
7899 doc
: /* Specify the coding system for read operations.
7900 It is useful to bind this variable with `let', but do not set it globally.
7901 If the value is a coding system, it is used for decoding on read operation.
7902 If not, an appropriate element is used from one of the coding system alists:
7903 There are three such tables, `file-coding-system-alist',
7904 `process-coding-system-alist', and `network-coding-system-alist'. */);
7905 Vcoding_system_for_read
= Qnil
;
7907 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
7908 doc
: /* Specify the coding system for write operations.
7909 Programs bind this variable with `let', but you should not set it globally.
7910 If the value is a coding system, it is used for encoding of output,
7911 when writing it to a file and when sending it to a file or subprocess.
7913 If this does not specify a coding system, an appropriate element
7914 is used from one of the coding system alists:
7915 There are three such tables, `file-coding-system-alist',
7916 `process-coding-system-alist', and `network-coding-system-alist'.
7917 For output to files, if the above procedure does not specify a coding system,
7918 the value of `buffer-file-coding-system' is used. */);
7919 Vcoding_system_for_write
= Qnil
;
7921 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
7922 doc
: /* Coding system used in the latest file or process I/O.
7923 Also set by `encode-coding-region', `decode-coding-region',
7924 `encode-coding-string' and `decode-coding-string'. */);
7925 Vlast_coding_system_used
= Qnil
;
7927 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
7928 doc
: /* *Non-nil means always inhibit code conversion of end-of-line format.
7929 See info node `Coding Systems' and info node `Text and Binary' concerning
7930 such conversion. */);
7931 inhibit_eol_conversion
= 0;
7933 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
7934 doc
: /* Non-nil means process buffer inherits coding system of process output.
7935 Bind it to t if the process output is to be treated as if it were a file
7936 read from some filesystem. */);
7937 inherit_process_coding_system
= 0;
7939 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
7940 doc
: /* Alist to decide a coding system to use for a file I/O operation.
7941 The format is ((PATTERN . VAL) ...),
7942 where PATTERN is a regular expression matching a file name,
7943 VAL is a coding system, a cons of coding systems, or a function symbol.
7944 If VAL is a coding system, it is used for both decoding and encoding
7946 If VAL is a cons of coding systems, the car part is used for decoding,
7947 and the cdr part is used for encoding.
7948 If VAL is a function symbol, the function must return a coding system
7949 or a cons of coding systems which are used as above. The function gets
7950 the arguments with which `find-operation-coding-system' was called.
7952 See also the function `find-operation-coding-system'
7953 and the variable `auto-coding-alist'. */);
7954 Vfile_coding_system_alist
= Qnil
;
7956 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
7957 doc
: /* Alist to decide a coding system to use for a process I/O operation.
7958 The format is ((PATTERN . VAL) ...),
7959 where PATTERN is a regular expression matching a program name,
7960 VAL is a coding system, a cons of coding systems, or a function symbol.
7961 If VAL is a coding system, it is used for both decoding what received
7962 from the program and encoding what sent to the program.
7963 If VAL is a cons of coding systems, the car part is used for decoding,
7964 and the cdr part is used for encoding.
7965 If VAL is a function symbol, the function must return a coding system
7966 or a cons of coding systems which are used as above.
7968 See also the function `find-operation-coding-system'. */);
7969 Vprocess_coding_system_alist
= Qnil
;
7971 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
7972 doc
: /* Alist to decide a coding system to use for a network I/O operation.
7973 The format is ((PATTERN . VAL) ...),
7974 where PATTERN is a regular expression matching a network service name
7975 or is a port number to connect to,
7976 VAL is a coding system, a cons of coding systems, or a function symbol.
7977 If VAL is a coding system, it is used for both decoding what received
7978 from the network stream and encoding what sent to the network stream.
7979 If VAL is a cons of coding systems, the car part is used for decoding,
7980 and the cdr part is used for encoding.
7981 If VAL is a function symbol, the function must return a coding system
7982 or a cons of coding systems which are used as above.
7984 See also the function `find-operation-coding-system'. */);
7985 Vnetwork_coding_system_alist
= Qnil
;
7987 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
7988 doc
: /* Coding system to use with system messages.
7989 Also used for decoding keyboard input on X Window system. */);
7990 Vlocale_coding_system
= Qnil
;
7992 /* The eol mnemonics are reset in startup.el system-dependently. */
7993 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
7994 doc
: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7995 eol_mnemonic_unix
= build_string (":");
7997 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
7998 doc
: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7999 eol_mnemonic_dos
= build_string ("\\");
8001 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
8002 doc
: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
8003 eol_mnemonic_mac
= build_string ("/");
8005 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
8006 doc
: /* *String displayed in mode line when end-of-line format is not yet determined. */);
8007 eol_mnemonic_undecided
= build_string (":");
8009 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
8010 doc
: /* *Non-nil enables character translation while encoding and decoding. */);
8011 Venable_character_translation
= Qt
;
8013 DEFVAR_LISP ("standard-translation-table-for-decode",
8014 &Vstandard_translation_table_for_decode
,
8015 doc
: /* Table for translating characters while decoding. */);
8016 Vstandard_translation_table_for_decode
= Qnil
;
8018 DEFVAR_LISP ("standard-translation-table-for-encode",
8019 &Vstandard_translation_table_for_encode
,
8020 doc
: /* Table for translating characters while encoding. */);
8021 Vstandard_translation_table_for_encode
= Qnil
;
8023 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
8024 doc
: /* Alist of charsets vs revision numbers.
8025 While encoding, if a charset (car part of an element) is found,
8026 designate it with the escape sequence identifying revision (cdr part of the element). */);
8027 Vcharset_revision_alist
= Qnil
;
8029 DEFVAR_LISP ("default-process-coding-system",
8030 &Vdefault_process_coding_system
,
8031 doc
: /* Cons of coding systems used for process I/O by default.
8032 The car part is used for decoding a process output,
8033 the cdr part is used for encoding a text to be sent to a process. */);
8034 Vdefault_process_coding_system
= Qnil
;
8036 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
8037 doc
: /* Table of extra Latin codes in the range 128..159 (inclusive).
8038 This is a vector of length 256.
8039 If Nth element is non-nil, the existence of code N in a file
8040 \(or output of subprocess) doesn't prevent it to be detected as
8041 a coding system of ISO 2022 variant which has a flag
8042 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
8043 or reading output of a subprocess.
8044 Only 128th through 159th elements has a meaning. */);
8045 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
8047 DEFVAR_LISP ("select-safe-coding-system-function",
8048 &Vselect_safe_coding_system_function
,
8049 doc
: /* Function to call to select safe coding system for encoding a text.
8051 If set, this function is called to force a user to select a proper
8052 coding system which can encode the text in the case that a default
8053 coding system used in each operation can't encode the text.
8055 The default value is `select-safe-coding-system' (which see). */);
8056 Vselect_safe_coding_system_function
= Qnil
;
8058 DEFVAR_BOOL ("coding-system-require-warning",
8059 &coding_system_require_warning
,
8060 doc
: /* Internal use only.
8061 If non-nil, on writing a file, `select-safe-coding-system-function' is
8062 called even if `coding-system-for-write' is non-nil. The command
8063 `universal-coding-system-argument' binds this variable to t temporarily. */);
8064 coding_system_require_warning
= 0;
8067 DEFVAR_BOOL ("inhibit-iso-escape-detection",
8068 &inhibit_iso_escape_detection
,
8069 doc
: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
8071 By default, on reading a file, Emacs tries to detect how the text is
8072 encoded. This code detection is sensitive to escape sequences. If
8073 the sequence is valid as ISO2022, the code is determined as one of
8074 the ISO2022 encodings, and the file is decoded by the corresponding
8075 coding system (e.g. `iso-2022-7bit').
8077 However, there may be a case that you want to read escape sequences in
8078 a file as is. In such a case, you can set this variable to non-nil.
8079 Then, as the code detection ignores any escape sequences, no file is
8080 detected as encoded in some ISO2022 encoding. The result is that all
8081 escape sequences become visible in a buffer.
8083 The default value is nil, and it is strongly recommended not to change
8084 it. That is because many Emacs Lisp source files that contain
8085 non-ASCII characters are encoded by the coding system `iso-2022-7bit'
8086 in Emacs's distribution, and they won't be decoded correctly on
8087 reading if you suppress escape sequence detection.
8089 The other way to read escape sequences in a file without decoding is
8090 to explicitly specify some coding system that doesn't use ISO2022's
8091 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
8092 inhibit_iso_escape_detection
= 0;
8094 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input
,
8095 doc
: /* Char table for translating self-inserting characters.
8096 This is applied to the result of input methods, not their input. See also
8097 `keyboard-translate-table'. */);
8098 Vtranslation_table_for_input
= Qnil
;
8102 emacs_strerror (error_number
)
8107 synchronize_system_messages_locale ();
8108 str
= strerror (error_number
);
8110 if (! NILP (Vlocale_coding_system
))
8112 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
8113 Vlocale_coding_system
,
8115 str
= (char *) SDATA (dec
);
8123 /* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
8124 (do not change this comment) */