1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 1995, 1997, 1998 Electrotechnical Laboratory, JAPAN.
3 Licensed to the Free Software Foundation.
5 This file is part of GNU Emacs.
7 GNU Emacs is free software; you can redistribute it and/or modify
8 it under the terms of the GNU General Public License as published by
9 the Free Software Foundation; either version 2, or (at your option)
12 GNU Emacs is distributed in the hope that it will be useful,
13 but WITHOUT ANY WARRANTY; without even the implied warranty of
14 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15 GNU General Public License for more details.
17 You should have received a copy of the GNU General Public License
18 along with GNU Emacs; see the file COPYING. If not, write to
19 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
20 Boston, MA 02111-1307, USA. */
22 /*** TABLE OF CONTENTS ***
26 2. Emacs' internal format (emacs-mule) handlers
28 4. Shift-JIS and BIG5 handlers
30 6. End-of-line handlers
31 7. C library functions
32 8. Emacs Lisp library functions
37 /*** 0. General comments ***/
40 /*** GENERAL NOTE on CODING SYSTEM ***
42 Coding system is an encoding mechanism of one or more character
43 sets. Here's a list of coding systems which Emacs can handle. When
44 we say "decode", it means converting some other coding system to
45 Emacs' internal format (emacs-internal), and when we say "encode",
46 it means converting the coding system emacs-mule to some other
49 0. Emacs' internal format (emacs-mule)
51 Emacs itself holds a multi-lingual character in a buffer and a string
52 in a special format. Details are described in section 2.
56 The most famous coding system for multiple character sets. X's
57 Compound Text, various EUCs (Extended Unix Code), and coding
58 systems used in Internet communication such as ISO-2022-JP are
59 all variants of ISO2022. Details are described in section 3.
61 2. SJIS (or Shift-JIS or MS-Kanji-Code)
63 A coding system to encode character sets: ASCII, JISX0201, and
64 JISX0208. Widely used for PC's in Japan. Details are described in
69 A coding system to encode character sets: ASCII and Big5. Widely
70 used by Chinese (mainly in Taiwan and Hong Kong). Details are
71 described in section 4. In this file, when we write "BIG5"
72 (all uppercase), we mean the coding system, and when we write
73 "Big5" (capitalized), we mean the character set.
77 A coding system for a text containing random 8-bit code. Emacs does
78 no code conversion on such a text except for end-of-line format.
82 If a user wants to read/write a text encoded in a coding system not
83 listed above, he can supply a decoder and an encoder for it in CCL
84 (Code Conversion Language) programs. Emacs executes the CCL program
85 while reading/writing.
87 Emacs represents a coding system by a Lisp symbol that has a property
88 `coding-system'. But, before actually using the coding system, the
89 information about it is set in a structure of type `struct
90 coding_system' for rapid processing. See section 6 for more details.
94 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
96 How end-of-line of a text is encoded depends on a system. For
97 instance, Unix's format is just one byte of `line-feed' code,
98 whereas DOS's format is two-byte sequence of `carriage-return' and
99 `line-feed' codes. MacOS's format is usually one byte of
102 Since text characters encoding and end-of-line encoding are
103 independent, any coding system described above can take
104 any format of end-of-line. So, Emacs has information of format of
105 end-of-line in each coding-system. See section 6 for more details.
109 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
111 These functions check if a text between SRC and SRC_END is encoded
112 in the coding system category XXX. Each returns an integer value in
113 which appropriate flag bits for the category XXX is set. The flag
114 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
115 template of these functions. */
118 detect_coding_emacs_mule (src
, src_end
)
119 unsigned char *src
, *src_end
;
125 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
127 These functions decode SRC_BYTES length of unibyte text at SOURCE
128 encoded in CODING to Emacs' internal format. The resulting
129 multibyte text goes to a place pointed to by DESTINATION, the length
130 of which should not exceed DST_BYTES.
132 These functions set the information of original and decoded texts in
133 the members produced, produced_char, consumed, and consumed_char of
134 the structure *CODING. They also set the member result to one of
135 CODING_FINISH_XXX indicating how the decoding finished.
137 DST_BYTES zero means that source area and destination area are
138 overlapped, which means that we can produce a decoded text until it
139 reaches at the head of not-yet-decoded source text.
141 Below is a template of these functions. */
144 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
145 struct coding_system
*coding
;
146 unsigned char *source
, *destination
;
147 int src_bytes
, dst_bytes
;
153 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
155 These functions encode SRC_BYTES length text at SOURCE of Emacs'
156 internal multibyte format to CODING. The resulting unibyte text
157 goes to a place pointed to by DESTINATION, the length of which
158 should not exceed DST_BYTES.
160 These functions set the information of original and encoded texts in
161 the members produced, produced_char, consumed, and consumed_char of
162 the structure *CODING. They also set the member result to one of
163 CODING_FINISH_XXX indicating how the encoding finished.
165 DST_BYTES zero means that source area and destination area are
166 overlapped, which means that we can produce a encoded text until it
167 reaches at the head of not-yet-encoded source text.
169 Below is a template of these functions. */
172 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
173 struct coding_system
*coding
;
174 unsigned char *source
, *destination
;
175 int src_bytes
, dst_bytes
;
181 /*** COMMONLY USED MACROS ***/
183 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
184 get one, two, and three bytes from the source text respectively.
185 If there are not enough bytes in the source, they jump to
186 `label_end_of_loop'. The caller should set variables `coding',
187 `src' and `src_end' to appropriate pointer in advance. These
188 macros are called from decoding routines `decode_coding_XXX', thus
189 it is assumed that the source text is unibyte. */
191 #define ONE_MORE_BYTE(c1) \
193 if (src >= src_end) \
195 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
196 goto label_end_of_loop; \
201 #define TWO_MORE_BYTES(c1, c2) \
203 if (src + 1 >= src_end) \
205 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
206 goto label_end_of_loop; \
213 /* Set C to the next character at the source text pointed by `src'.
214 If there are not enough characters in the source, jump to
215 `label_end_of_loop'. The caller should set variables `coding'
216 `src', `src_end', and `translation_table' to appropriate pointers
217 in advance. This macro is used in encoding routines
218 `encode_coding_XXX', thus it assumes that the source text is in
219 multibyte form except for 8-bit characters. 8-bit characters are
220 in multibyte form if coding->src_multibyte is nonzero, else they
221 are represented by a single byte. */
223 #define ONE_MORE_CHAR(c) \
225 int len = src_end - src; \
229 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
230 goto label_end_of_loop; \
232 if (coding->src_multibyte \
233 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
234 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
236 c = *src, bytes = 1; \
237 if (!NILP (translation_table)) \
238 c = translate_char (translation_table, c, 0, 0, 0); \
243 /* Produce a multibyte form of characater C to `dst'. Jump to
244 `label_end_of_loop' if there's not enough space at `dst'.
246 If we are now in the middle of composition sequence, the decoded
247 character may be ALTCHAR (for the current composition). In that
248 case, the character goes to coding->cmp_data->data instead of
251 This macro is used in decoding routines. */
253 #define EMIT_CHAR(c) \
255 if (! COMPOSING_P (coding) \
256 || coding->composing == COMPOSITION_RELATIVE \
257 || coding->composing == COMPOSITION_WITH_RULE) \
259 int bytes = CHAR_BYTES (c); \
260 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
262 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
263 goto label_end_of_loop; \
265 dst += CHAR_STRING (c, dst); \
266 coding->produced_char++; \
269 if (COMPOSING_P (coding) \
270 && coding->composing != COMPOSITION_RELATIVE) \
272 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
273 coding->composition_rule_follows \
274 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
279 #define EMIT_ONE_BYTE(c) \
281 if (dst >= (dst_bytes ? dst_end : src)) \
283 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
284 goto label_end_of_loop; \
289 #define EMIT_TWO_BYTES(c1, c2) \
291 if (dst + 2 > (dst_bytes ? dst_end : src)) \
293 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
294 goto label_end_of_loop; \
296 *dst++ = c1, *dst++ = c2; \
299 #define EMIT_BYTES(from, to) \
301 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
303 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
304 goto label_end_of_loop; \
311 /*** 1. Preamble ***/
324 #include "composite.h"
329 #else /* not emacs */
333 #endif /* not emacs */
335 Lisp_Object Qcoding_system
, Qeol_type
;
336 Lisp_Object Qbuffer_file_coding_system
;
337 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
338 Lisp_Object Qno_conversion
, Qundecided
;
339 Lisp_Object Qcoding_system_history
;
340 Lisp_Object Qsafe_chars
;
341 Lisp_Object Qvalid_codes
;
343 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
344 Lisp_Object Qcall_process
, Qcall_process_region
, Qprocess_argument
;
345 Lisp_Object Qstart_process
, Qopen_network_stream
;
346 Lisp_Object Qtarget_idx
;
348 Lisp_Object Vselect_safe_coding_system_function
;
350 /* Mnemonic string for each format of end-of-line. */
351 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
352 /* Mnemonic string to indicate format of end-of-line is not yet
354 Lisp_Object eol_mnemonic_undecided
;
356 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
357 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
362 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
364 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
366 /* Coding system emacs-mule and raw-text are for converting only
367 end-of-line format. */
368 Lisp_Object Qemacs_mule
, Qraw_text
;
370 /* Coding-systems are handed between Emacs Lisp programs and C internal
371 routines by the following three variables. */
372 /* Coding-system for reading files and receiving data from process. */
373 Lisp_Object Vcoding_system_for_read
;
374 /* Coding-system for writing files and sending data to process. */
375 Lisp_Object Vcoding_system_for_write
;
376 /* Coding-system actually used in the latest I/O. */
377 Lisp_Object Vlast_coding_system_used
;
379 /* A vector of length 256 which contains information about special
380 Latin codes (especially for dealing with Microsoft codes). */
381 Lisp_Object Vlatin_extra_code_table
;
383 /* Flag to inhibit code conversion of end-of-line format. */
384 int inhibit_eol_conversion
;
386 /* Flag to inhibit ISO2022 escape sequence detection. */
387 int inhibit_iso_escape_detection
;
389 /* Flag to make buffer-file-coding-system inherit from process-coding. */
390 int inherit_process_coding_system
;
392 /* Coding system to be used to encode text for terminal display. */
393 struct coding_system terminal_coding
;
395 /* Coding system to be used to encode text for terminal display when
396 terminal coding system is nil. */
397 struct coding_system safe_terminal_coding
;
399 /* Coding system of what is sent from terminal keyboard. */
400 struct coding_system keyboard_coding
;
402 /* Default coding system to be used to write a file. */
403 struct coding_system default_buffer_file_coding
;
405 Lisp_Object Vfile_coding_system_alist
;
406 Lisp_Object Vprocess_coding_system_alist
;
407 Lisp_Object Vnetwork_coding_system_alist
;
409 Lisp_Object Vlocale_coding_system
;
413 Lisp_Object Qcoding_category
, Qcoding_category_index
;
415 /* List of symbols `coding-category-xxx' ordered by priority. */
416 Lisp_Object Vcoding_category_list
;
418 /* Table of coding categories (Lisp symbols). */
419 Lisp_Object Vcoding_category_table
;
421 /* Table of names of symbol for each coding-category. */
422 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
423 "coding-category-emacs-mule",
424 "coding-category-sjis",
425 "coding-category-iso-7",
426 "coding-category-iso-7-tight",
427 "coding-category-iso-8-1",
428 "coding-category-iso-8-2",
429 "coding-category-iso-7-else",
430 "coding-category-iso-8-else",
431 "coding-category-ccl",
432 "coding-category-big5",
433 "coding-category-utf-8",
434 "coding-category-utf-16-be",
435 "coding-category-utf-16-le",
436 "coding-category-raw-text",
437 "coding-category-binary"
440 /* Table of pointers to coding systems corresponding to each coding
442 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
444 /* Table of coding category masks. Nth element is a mask for a coding
445 cateogry of which priority is Nth. */
447 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
449 /* Flag to tell if we look up translation table on character code
451 Lisp_Object Venable_character_translation
;
452 /* Standard translation table to look up on decoding (reading). */
453 Lisp_Object Vstandard_translation_table_for_decode
;
454 /* Standard translation table to look up on encoding (writing). */
455 Lisp_Object Vstandard_translation_table_for_encode
;
457 Lisp_Object Qtranslation_table
;
458 Lisp_Object Qtranslation_table_id
;
459 Lisp_Object Qtranslation_table_for_decode
;
460 Lisp_Object Qtranslation_table_for_encode
;
462 /* Alist of charsets vs revision number. */
463 Lisp_Object Vcharset_revision_alist
;
465 /* Default coding systems used for process I/O. */
466 Lisp_Object Vdefault_process_coding_system
;
468 /* Global flag to tell that we can't call post-read-conversion and
469 pre-write-conversion functions. Usually the value is zero, but it
470 is set to 1 temporarily while such functions are running. This is
471 to avoid infinite recursive call. */
472 static int inhibit_pre_post_conversion
;
474 /* Char-table containing safe coding systems of each character. */
475 Lisp_Object Vchar_coding_system_table
;
476 Lisp_Object Qchar_coding_system
;
478 /* Return `safe-chars' property of coding system CODING. Don't check
479 validity of CODING. */
482 coding_safe_chars (coding
)
483 struct coding_system
*coding
;
485 Lisp_Object coding_spec
, plist
, safe_chars
;
487 coding_spec
= Fget (coding
->symbol
, Qcoding_system
);
488 plist
= XVECTOR (coding_spec
)->contents
[3];
489 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
490 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
493 #define CODING_SAFE_CHAR_P(safe_chars, c) \
494 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
497 /*** 2. Emacs internal format (emacs-mule) handlers ***/
499 /* Emacs' internal format for encoding multiple character sets is a
500 kind of multi-byte encoding, i.e. characters are encoded by
501 variable-length sequences of one-byte codes.
503 ASCII characters and control characters (e.g. `tab', `newline') are
504 represented by one-byte sequences which are their ASCII codes, in
505 the range 0x00 through 0x7F.
507 8-bit characters of the range 0x80..0x9F are represented by
508 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
511 8-bit characters of the range 0xA0..0xFF are represented by
512 one-byte sequences which are their 8-bit code.
514 The other characters are represented by a sequence of `base
515 leading-code', optional `extended leading-code', and one or two
516 `position-code's. The length of the sequence is determined by the
517 base leading-code. Leading-code takes the range 0x80 through 0x9F,
518 whereas extended leading-code and position-code take the range 0xA0
519 through 0xFF. See `charset.h' for more details about leading-code
522 --- CODE RANGE of Emacs' internal format ---
526 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
527 eight-bit-graphic 0xA0..0xBF
528 ELSE 0x81..0x9F + [0xA0..0xFF]+
529 ---------------------------------------------
533 enum emacs_code_class_type emacs_code_class
[256];
535 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
536 Check if a text is encoded in Emacs' internal format. If it is,
537 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
540 detect_coding_emacs_mule (src
, src_end
)
541 unsigned char *src
, *src_end
;
545 /* Dummy for ONE_MORE_BYTE. */
546 struct coding_system dummy_coding
;
547 struct coding_system
*coding
= &dummy_coding
;
568 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
571 else if (c
>= 0x80 && c
< 0xA0)
574 /* Old leading code for a composite character. */
578 unsigned char *src_base
= src
- 1;
581 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
584 src
= src_base
+ bytes
;
589 return CODING_CATEGORY_MASK_EMACS_MULE
;
593 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
596 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
597 struct coding_system
*coding
;
598 unsigned char *source
, *destination
;
599 int src_bytes
, dst_bytes
;
601 unsigned char *src
= source
;
602 unsigned char *src_end
= source
+ src_bytes
;
603 unsigned char *dst
= destination
;
604 unsigned char *dst_end
= destination
+ dst_bytes
;
605 /* SRC_BASE remembers the start position in source in each loop.
606 The loop will be exited when there's not enough source code, or
607 when there's not enough destination area to produce a
609 unsigned char *src_base
;
611 coding
->produced_char
= 0;
612 while ((src_base
= src
) < src_end
)
614 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
], *p
;
617 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
))
624 bytes
= CHAR_STRING (*src
, tmp
);
628 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
630 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
633 while (bytes
--) *dst
++ = *p
++;
634 coding
->produced_char
++;
636 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
637 coding
->produced
= dst
- destination
;
640 #define encode_coding_emacs_mule(coding, source, destination, src_bytes, dst_bytes) \
641 encode_eol (coding, source, destination, src_bytes, dst_bytes)
645 /*** 3. ISO2022 handlers ***/
647 /* The following note describes the coding system ISO2022 briefly.
648 Since the intention of this note is to help understand the
649 functions in this file, some parts are NOT ACCURATE or OVERLY
650 SIMPLIFIED. For thorough understanding, please refer to the
651 original document of ISO2022.
653 ISO2022 provides many mechanisms to encode several character sets
654 in 7-bit and 8-bit environments. For 7-bite environments, all text
655 is encoded using bytes less than 128. This may make the encoded
656 text a little bit longer, but the text passes more easily through
657 several gateways, some of which strip off MSB (Most Signigant Bit).
659 There are two kinds of character sets: control character set and
660 graphic character set. The former contains control characters such
661 as `newline' and `escape' to provide control functions (control
662 functions are also provided by escape sequences). The latter
663 contains graphic characters such as 'A' and '-'. Emacs recognizes
664 two control character sets and many graphic character sets.
666 Graphic character sets are classified into one of the following
667 four classes, according to the number of bytes (DIMENSION) and
668 number of characters in one dimension (CHARS) of the set:
674 In addition, each character set is assigned an identification tag,
675 unique for each set, called "final character" (denoted as <F>
676 hereafter). The <F> of each character set is decided by ECMA(*)
677 when it is registered in ISO. The code range of <F> is 0x30..0x7F
678 (0x30..0x3F are for private use only).
680 Note (*): ECMA = European Computer Manufacturers Association
682 Here are examples of graphic character set [NAME(<F>)]:
683 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
684 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
685 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
686 o DIMENSION2_CHARS96 -- none for the moment
688 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
689 C0 [0x00..0x1F] -- control character plane 0
690 GL [0x20..0x7F] -- graphic character plane 0
691 C1 [0x80..0x9F] -- control character plane 1
692 GR [0xA0..0xFF] -- graphic character plane 1
694 A control character set is directly designated and invoked to C0 or
695 C1 by an escape sequence. The most common case is that:
696 - ISO646's control character set is designated/invoked to C0, and
697 - ISO6429's control character set is designated/invoked to C1,
698 and usually these designations/invocations are omitted in encoded
699 text. In a 7-bit environment, only C0 can be used, and a control
700 character for C1 is encoded by an appropriate escape sequence to
701 fit into the environment. All control characters for C1 are
702 defined to have corresponding escape sequences.
704 A graphic character set is at first designated to one of four
705 graphic registers (G0 through G3), then these graphic registers are
706 invoked to GL or GR. These designations and invocations can be
707 done independently. The most common case is that G0 is invoked to
708 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
709 these invocations and designations are omitted in encoded text.
710 In a 7-bit environment, only GL can be used.
712 When a graphic character set of CHARS94 is invoked to GL, codes
713 0x20 and 0x7F of the GL area work as control characters SPACE and
714 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
717 There are two ways of invocation: locking-shift and single-shift.
718 With locking-shift, the invocation lasts until the next different
719 invocation, whereas with single-shift, the invocation affects the
720 following character only and doesn't affect the locking-shift
721 state. Invocations are done by the following control characters or
724 ----------------------------------------------------------------------
725 abbrev function cntrl escape seq description
726 ----------------------------------------------------------------------
727 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
728 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
729 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
730 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
731 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
732 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
733 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
734 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
735 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
736 ----------------------------------------------------------------------
737 (*) These are not used by any known coding system.
739 Control characters for these functions are defined by macros
740 ISO_CODE_XXX in `coding.h'.
742 Designations are done by the following escape sequences:
743 ----------------------------------------------------------------------
744 escape sequence description
745 ----------------------------------------------------------------------
746 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
747 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
748 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
749 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
750 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
751 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
752 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
753 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
754 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
755 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
756 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
757 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
758 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
759 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
760 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
761 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
762 ----------------------------------------------------------------------
764 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
765 of dimension 1, chars 94, and final character <F>, etc...
767 Note (*): Although these designations are not allowed in ISO2022,
768 Emacs accepts them on decoding, and produces them on encoding
769 CHARS96 character sets in a coding system which is characterized as
770 7-bit environment, non-locking-shift, and non-single-shift.
772 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
773 '(' can be omitted. We refer to this as "short-form" hereafter.
775 Now you may notice that there are a lot of ways for encoding the
776 same multilingual text in ISO2022. Actually, there exist many
777 coding systems such as Compound Text (used in X11's inter client
778 communication, ISO-2022-JP (used in Japanese internet), ISO-2022-KR
779 (used in Korean internet), EUC (Extended UNIX Code, used in Asian
780 localized platforms), and all of these are variants of ISO2022.
782 In addition to the above, Emacs handles two more kinds of escape
783 sequences: ISO6429's direction specification and Emacs' private
784 sequence for specifying character composition.
786 ISO6429's direction specification takes the following form:
787 o CSI ']' -- end of the current direction
788 o CSI '0' ']' -- end of the current direction
789 o CSI '1' ']' -- start of left-to-right text
790 o CSI '2' ']' -- start of right-to-left text
791 The control character CSI (0x9B: control sequence introducer) is
792 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
794 Character composition specification takes the following form:
795 o ESC '0' -- start relative composition
796 o ESC '1' -- end composition
797 o ESC '2' -- start rule-base composition (*)
798 o ESC '3' -- start relative composition with alternate chars (**)
799 o ESC '4' -- start rule-base composition with alternate chars (**)
800 Since these are not standard escape sequences of any ISO standard,
801 the use of them for these meaning is restricted to Emacs only.
803 (*) This form is used only in Emacs 20.5 and the older versions,
804 but the newer versions can safely decode it.
805 (**) This form is used only in Emacs 21.1 and the newer versions,
806 and the older versions can't decode it.
808 Here's a list of examples usages of these composition escape
809 sequences (categorized by `enum composition_method').
811 COMPOSITION_RELATIVE:
812 ESC 0 CHAR [ CHAR ] ESC 1
813 COMPOSITOIN_WITH_RULE:
814 ESC 2 CHAR [ RULE CHAR ] ESC 1
815 COMPOSITION_WITH_ALTCHARS:
816 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
817 COMPOSITION_WITH_RULE_ALTCHARS:
818 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
820 enum iso_code_class_type iso_code_class
[256];
822 #define CHARSET_OK(idx, charset, c) \
823 (coding_system_table[idx] \
824 && (charset == CHARSET_ASCII \
825 || (safe_chars = coding_safe_chars (coding_system_table[idx]), \
826 CODING_SAFE_CHAR_P (safe_chars, c))) \
827 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
829 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
831 #define SHIFT_OUT_OK(idx) \
832 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
834 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
835 Check if a text is encoded in ISO2022. If it is, returns an
836 integer in which appropriate flag bits any of:
837 CODING_CATEGORY_MASK_ISO_7
838 CODING_CATEGORY_MASK_ISO_7_TIGHT
839 CODING_CATEGORY_MASK_ISO_8_1
840 CODING_CATEGORY_MASK_ISO_8_2
841 CODING_CATEGORY_MASK_ISO_7_ELSE
842 CODING_CATEGORY_MASK_ISO_8_ELSE
843 are set. If a code which should never appear in ISO2022 is found,
847 detect_coding_iso2022 (src
, src_end
)
848 unsigned char *src
, *src_end
;
850 int mask
= CODING_CATEGORY_MASK_ISO
;
852 int reg
[4], shift_out
= 0, single_shifting
= 0;
853 int c
, c1
, i
, charset
;
854 /* Dummy for ONE_MORE_BYTE. */
855 struct coding_system dummy_coding
;
856 struct coding_system
*coding
= &dummy_coding
;
857 Lisp_Object safe_chars
;
859 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
860 while (mask
&& src
< src_end
)
866 if (inhibit_iso_escape_detection
)
870 if (c
>= '(' && c
<= '/')
872 /* Designation sequence for a charset of dimension 1. */
874 if (c1
< ' ' || c1
>= 0x80
875 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
876 /* Invalid designation sequence. Just ignore. */
878 reg
[(c
- '(') % 4] = charset
;
882 /* Designation sequence for a charset of dimension 2. */
884 if (c
>= '@' && c
<= 'B')
885 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
886 reg
[0] = charset
= iso_charset_table
[1][0][c
];
887 else if (c
>= '(' && c
<= '/')
890 if (c1
< ' ' || c1
>= 0x80
891 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
892 /* Invalid designation sequence. Just ignore. */
894 reg
[(c
- '(') % 4] = charset
;
897 /* Invalid designation sequence. Just ignore. */
900 else if (c
== 'N' || c
== 'O')
902 /* ESC <Fe> for SS2 or SS3. */
903 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
906 else if (c
>= '0' && c
<= '4')
908 /* ESC <Fp> for start/end composition. */
909 mask_found
|= CODING_CATEGORY_MASK_ISO
;
913 /* Invalid escape sequence. Just ignore. */
916 /* We found a valid designation sequence for CHARSET. */
917 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
918 c
= MAKE_CHAR (charset
, 0, 0);
919 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
920 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
922 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
923 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
924 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
926 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
927 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
928 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
930 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
931 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
932 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
934 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
938 if (inhibit_iso_escape_detection
)
943 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
944 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
946 /* Locking shift out. */
947 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
948 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
953 if (inhibit_iso_escape_detection
)
958 /* Locking shift in. */
959 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
960 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
969 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
971 if (inhibit_iso_escape_detection
)
973 if (c
!= ISO_CODE_CSI
)
975 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
976 & CODING_FLAG_ISO_SINGLE_SHIFT
)
977 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
978 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
979 & CODING_FLAG_ISO_SINGLE_SHIFT
)
980 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
983 if (VECTORP (Vlatin_extra_code_table
)
984 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
986 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
987 & CODING_FLAG_ISO_LATIN_EXTRA
)
988 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
989 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
990 & CODING_FLAG_ISO_LATIN_EXTRA
)
991 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
994 mask_found
|= newmask
;
1001 single_shifting
= 0;
1006 single_shifting
= 0;
1007 if (VECTORP (Vlatin_extra_code_table
)
1008 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1012 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1013 & CODING_FLAG_ISO_LATIN_EXTRA
)
1014 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1015 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1016 & CODING_FLAG_ISO_LATIN_EXTRA
)
1017 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1019 mask_found
|= newmask
;
1026 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1027 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1028 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1029 /* Check the length of succeeding codes of the range
1030 0xA0..0FF. If the byte length is odd, we exclude
1031 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1032 when we are not single shifting. */
1033 if (!single_shifting
1034 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1037 while (src
< src_end
)
1045 if (i
& 1 && src
< src_end
)
1046 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1048 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1055 return (mask
& mask_found
);
1058 /* Decode a character of which charset is CHARSET, the 1st position
1059 code is C1, the 2nd position code is C2, and return the decoded
1060 character code. If the variable `translation_table' is non-nil,
1061 returned the translated code. */
1063 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1064 (NILP (translation_table) \
1065 ? MAKE_CHAR (charset, c1, c2) \
1066 : translate_char (translation_table, -1, charset, c1, c2))
1068 /* Set designation state into CODING. */
1069 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1073 if (final_char < '0' || final_char >= 128) \
1074 goto label_invalid_code; \
1075 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1076 make_number (chars), \
1077 make_number (final_char)); \
1078 c = MAKE_CHAR (charset, 0, 0); \
1080 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1081 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1083 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1085 && charset == CHARSET_ASCII) \
1087 /* We should insert this designation sequence as is so \
1088 that it is surely written back to a file. */ \
1089 coding->spec.iso2022.last_invalid_designation_register = -1; \
1090 goto label_invalid_code; \
1092 coding->spec.iso2022.last_invalid_designation_register = -1; \
1093 if ((coding->mode & CODING_MODE_DIRECTION) \
1094 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1095 charset = CHARSET_REVERSE_CHARSET (charset); \
1096 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1100 coding->spec.iso2022.last_invalid_designation_register = reg; \
1101 goto label_invalid_code; \
1105 /* Allocate a memory block for storing information about compositions.
1106 The block is chained to the already allocated blocks. */
1109 coding_allocate_composition_data (coding
, char_offset
)
1110 struct coding_system
*coding
;
1113 struct composition_data
*cmp_data
1114 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1116 cmp_data
->char_offset
= char_offset
;
1118 cmp_data
->prev
= coding
->cmp_data
;
1119 cmp_data
->next
= NULL
;
1120 if (coding
->cmp_data
)
1121 coding
->cmp_data
->next
= cmp_data
;
1122 coding
->cmp_data
= cmp_data
;
1123 coding
->cmp_data_start
= 0;
1126 /* Record the starting position START and METHOD of one composition. */
1128 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
1130 struct composition_data *cmp_data = coding->cmp_data; \
1131 int *data = cmp_data->data + cmp_data->used; \
1132 coding->cmp_data_start = cmp_data->used; \
1134 data[1] = cmp_data->char_offset + start; \
1135 data[3] = (int) method; \
1136 cmp_data->used += 4; \
1139 /* Record the ending position END of the current composition. */
1141 #define CODING_ADD_COMPOSITION_END(coding, end) \
1143 struct composition_data *cmp_data = coding->cmp_data; \
1144 int *data = cmp_data->data + coding->cmp_data_start; \
1145 data[0] = cmp_data->used - coding->cmp_data_start; \
1146 data[2] = cmp_data->char_offset + end; \
1149 /* Record one COMPONENT (alternate character or composition rule). */
1151 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
1152 (coding->cmp_data->data[coding->cmp_data->used++] = component)
1154 /* Handle compositoin start sequence ESC 0, ESC 2, ESC 3, or ESC 4. */
1156 #define DECODE_COMPOSITION_START(c1) \
1158 if (coding->composing == COMPOSITION_DISABLED) \
1160 *dst++ = ISO_CODE_ESC; \
1161 *dst++ = c1 & 0x7f; \
1162 coding->produced_char += 2; \
1164 else if (!COMPOSING_P (coding)) \
1166 /* This is surely the start of a composition. We must be sure \
1167 that coding->cmp_data has enough space to store the \
1168 information about the composition. If not, terminate the \
1169 current decoding loop, allocate one more memory block for \
1170 coding->cmp_data in the calller, then start the decoding \
1171 loop again. We can't allocate memory here directly because \
1172 it may cause buffer/string relocation. */ \
1173 if (!coding->cmp_data \
1174 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1175 >= COMPOSITION_DATA_SIZE)) \
1177 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1178 goto label_end_of_loop; \
1180 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1181 : c1 == '2' ? COMPOSITION_WITH_RULE \
1182 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1183 : COMPOSITION_WITH_RULE_ALTCHARS); \
1184 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1185 coding->composing); \
1186 coding->composition_rule_follows = 0; \
1190 /* We are already handling a composition. If the method is \
1191 the following two, the codes following the current escape \
1192 sequence are actual characters stored in a buffer. */ \
1193 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1194 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1196 coding->composing = COMPOSITION_RELATIVE; \
1197 coding->composition_rule_follows = 0; \
1202 /* Handle compositoin end sequence ESC 1. */
1204 #define DECODE_COMPOSITION_END(c1) \
1206 if (coding->composing == COMPOSITION_DISABLED) \
1208 *dst++ = ISO_CODE_ESC; \
1210 coding->produced_char += 2; \
1214 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1215 coding->composing = COMPOSITION_NO; \
1219 /* Decode a composition rule from the byte C1 (and maybe one more byte
1220 from SRC) and store one encoded composition rule in
1221 coding->cmp_data. */
1223 #define DECODE_COMPOSITION_RULE(c1) \
1227 if (c1 < 81) /* old format (before ver.21) */ \
1229 int gref = (c1) / 9; \
1230 int nref = (c1) % 9; \
1231 if (gref == 4) gref = 10; \
1232 if (nref == 4) nref = 10; \
1233 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1235 else if (c1 < 93) /* new format (after ver.21) */ \
1237 ONE_MORE_BYTE (c2); \
1238 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1240 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1241 coding->composition_rule_follows = 0; \
1245 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1248 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1249 struct coding_system
*coding
;
1250 unsigned char *source
, *destination
;
1251 int src_bytes
, dst_bytes
;
1253 unsigned char *src
= source
;
1254 unsigned char *src_end
= source
+ src_bytes
;
1255 unsigned char *dst
= destination
;
1256 unsigned char *dst_end
= destination
+ dst_bytes
;
1257 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1258 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1259 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1260 /* SRC_BASE remembers the start position in source in each loop.
1261 The loop will be exited when there's not enough source code
1262 (within macro ONE_MORE_BYTE), or when there's not enough
1263 destination area to produce a character (within macro
1265 unsigned char *src_base
;
1267 Lisp_Object translation_table
;
1268 Lisp_Object safe_chars
;
1270 safe_chars
= coding_safe_chars (coding
);
1272 if (NILP (Venable_character_translation
))
1273 translation_table
= Qnil
;
1276 translation_table
= coding
->translation_table_for_decode
;
1277 if (NILP (translation_table
))
1278 translation_table
= Vstandard_translation_table_for_decode
;
1281 coding
->result
= CODING_FINISH_NORMAL
;
1290 /* We produce no character or one character. */
1291 switch (iso_code_class
[c1
])
1293 case ISO_0x20_or_0x7F
:
1294 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1296 DECODE_COMPOSITION_RULE (c1
);
1299 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1301 /* This is SPACE or DEL. */
1302 charset
= CHARSET_ASCII
;
1305 /* This is a graphic character, we fall down ... */
1307 case ISO_graphic_plane_0
:
1308 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1310 DECODE_COMPOSITION_RULE (c1
);
1316 case ISO_0xA0_or_0xFF
:
1317 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1318 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1319 goto label_invalid_code
;
1320 /* This is a graphic character, we fall down ... */
1322 case ISO_graphic_plane_1
:
1324 goto label_invalid_code
;
1329 if (COMPOSING_P (coding
))
1330 DECODE_COMPOSITION_END ('1');
1332 /* All ISO2022 control characters in this class have the
1333 same representation in Emacs internal format. */
1335 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1336 && (coding
->eol_type
== CODING_EOL_CR
1337 || coding
->eol_type
== CODING_EOL_CRLF
))
1339 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1340 goto label_end_of_loop
;
1342 charset
= CHARSET_ASCII
;
1346 if (COMPOSING_P (coding
))
1347 DECODE_COMPOSITION_END ('1');
1348 goto label_invalid_code
;
1350 case ISO_carriage_return
:
1351 if (COMPOSING_P (coding
))
1352 DECODE_COMPOSITION_END ('1');
1354 if (coding
->eol_type
== CODING_EOL_CR
)
1356 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1359 if (c1
!= ISO_CODE_LF
)
1361 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1363 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1364 goto label_end_of_loop
;
1370 charset
= CHARSET_ASCII
;
1374 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1375 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1376 goto label_invalid_code
;
1377 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1378 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1382 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1383 goto label_invalid_code
;
1384 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1385 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1388 case ISO_single_shift_2_7
:
1389 case ISO_single_shift_2
:
1390 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1391 goto label_invalid_code
;
1392 /* SS2 is handled as an escape sequence of ESC 'N' */
1394 goto label_escape_sequence
;
1396 case ISO_single_shift_3
:
1397 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1398 goto label_invalid_code
;
1399 /* SS2 is handled as an escape sequence of ESC 'O' */
1401 goto label_escape_sequence
;
1403 case ISO_control_sequence_introducer
:
1404 /* CSI is handled as an escape sequence of ESC '[' ... */
1406 goto label_escape_sequence
;
1410 label_escape_sequence
:
1411 /* Escape sequences handled by Emacs are invocation,
1412 designation, direction specification, and character
1413 composition specification. */
1416 case '&': /* revision of following character set */
1418 if (!(c1
>= '@' && c1
<= '~'))
1419 goto label_invalid_code
;
1421 if (c1
!= ISO_CODE_ESC
)
1422 goto label_invalid_code
;
1424 goto label_escape_sequence
;
1426 case '$': /* designation of 2-byte character set */
1427 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1428 goto label_invalid_code
;
1430 if (c1
>= '@' && c1
<= 'B')
1431 { /* designation of JISX0208.1978, GB2312.1980,
1433 DECODE_DESIGNATION (0, 2, 94, c1
);
1435 else if (c1
>= 0x28 && c1
<= 0x2B)
1436 { /* designation of DIMENSION2_CHARS94 character set */
1438 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
1440 else if (c1
>= 0x2C && c1
<= 0x2F)
1441 { /* designation of DIMENSION2_CHARS96 character set */
1443 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
1446 goto label_invalid_code
;
1447 /* We must update these variables now. */
1448 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1449 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1452 case 'n': /* invocation of locking-shift-2 */
1453 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1454 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
1455 goto label_invalid_code
;
1456 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
1457 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1460 case 'o': /* invocation of locking-shift-3 */
1461 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1462 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
1463 goto label_invalid_code
;
1464 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
1465 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1468 case 'N': /* invocation of single-shift-2 */
1469 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1470 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
1471 goto label_invalid_code
;
1472 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
1474 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
1475 goto label_invalid_code
;
1478 case 'O': /* invocation of single-shift-3 */
1479 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1480 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
1481 goto label_invalid_code
;
1482 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
1484 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
1485 goto label_invalid_code
;
1488 case '0': case '2': case '3': case '4': /* start composition */
1489 DECODE_COMPOSITION_START (c1
);
1492 case '1': /* end composition */
1493 DECODE_COMPOSITION_END (c1
);
1496 case '[': /* specification of direction */
1497 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
1498 goto label_invalid_code
;
1499 /* For the moment, nested direction is not supported.
1500 So, `coding->mode & CODING_MODE_DIRECTION' zero means
1501 left-to-right, and nozero means right-to-left. */
1505 case ']': /* end of the current direction */
1506 coding
->mode
&= ~CODING_MODE_DIRECTION
;
1508 case '0': /* end of the current direction */
1509 case '1': /* start of left-to-right direction */
1512 coding
->mode
&= ~CODING_MODE_DIRECTION
;
1514 goto label_invalid_code
;
1517 case '2': /* start of right-to-left direction */
1520 coding
->mode
|= CODING_MODE_DIRECTION
;
1522 goto label_invalid_code
;
1526 goto label_invalid_code
;
1531 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1532 goto label_invalid_code
;
1533 if (c1
>= 0x28 && c1
<= 0x2B)
1534 { /* designation of DIMENSION1_CHARS94 character set */
1536 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
1538 else if (c1
>= 0x2C && c1
<= 0x2F)
1539 { /* designation of DIMENSION1_CHARS96 character set */
1541 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
1544 goto label_invalid_code
;
1545 /* We must update these variables now. */
1546 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1547 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1552 /* Now we know CHARSET and 1st position code C1 of a character.
1553 Produce a multibyte sequence for that character while getting
1554 2nd position code C2 if necessary. */
1555 if (CHARSET_DIMENSION (charset
) == 2)
1558 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
1559 /* C2 is not in a valid range. */
1560 goto label_invalid_code
;
1562 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
1568 if (COMPOSING_P (coding
))
1569 DECODE_COMPOSITION_END ('1');
1576 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1577 coding
->produced
= dst
- destination
;
1582 /* ISO2022 encoding stuff. */
1585 It is not enough to say just "ISO2022" on encoding, we have to
1586 specify more details. In Emacs, each coding system of ISO2022
1587 variant has the following specifications:
1588 1. Initial designation to G0 thru G3.
1589 2. Allows short-form designation?
1590 3. ASCII should be designated to G0 before control characters?
1591 4. ASCII should be designated to G0 at end of line?
1592 5. 7-bit environment or 8-bit environment?
1593 6. Use locking-shift?
1594 7. Use Single-shift?
1595 And the following two are only for Japanese:
1596 8. Use ASCII in place of JIS0201-1976-Roman?
1597 9. Use JISX0208-1983 in place of JISX0208-1978?
1598 These specifications are encoded in `coding->flags' as flag bits
1599 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
1603 /* Produce codes (escape sequence) for designating CHARSET to graphic
1604 register REG at DST, and increment DST. If <final-char> of CHARSET is
1605 '@', 'A', or 'B' and the coding system CODING allows, produce
1606 designation sequence of short-form. */
1608 #define ENCODE_DESIGNATION(charset, reg, coding) \
1610 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
1611 char *intermediate_char_94 = "()*+"; \
1612 char *intermediate_char_96 = ",-./"; \
1613 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
1615 if (revision < 255) \
1617 *dst++ = ISO_CODE_ESC; \
1619 *dst++ = '@' + revision; \
1621 *dst++ = ISO_CODE_ESC; \
1622 if (CHARSET_DIMENSION (charset) == 1) \
1624 if (CHARSET_CHARS (charset) == 94) \
1625 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
1627 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
1632 if (CHARSET_CHARS (charset) == 94) \
1634 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
1636 || final_char < '@' || final_char > 'B') \
1637 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
1640 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
1642 *dst++ = final_char; \
1643 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1646 /* The following two macros produce codes (control character or escape
1647 sequence) for ISO2022 single-shift functions (single-shift-2 and
1650 #define ENCODE_SINGLE_SHIFT_2 \
1652 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1653 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
1655 *dst++ = ISO_CODE_SS2; \
1656 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
1659 #define ENCODE_SINGLE_SHIFT_3 \
1661 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1662 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
1664 *dst++ = ISO_CODE_SS3; \
1665 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
1668 /* The following four macros produce codes (control character or
1669 escape sequence) for ISO2022 locking-shift functions (shift-in,
1670 shift-out, locking-shift-2, and locking-shift-3). */
1672 #define ENCODE_SHIFT_IN \
1674 *dst++ = ISO_CODE_SI; \
1675 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
1678 #define ENCODE_SHIFT_OUT \
1680 *dst++ = ISO_CODE_SO; \
1681 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
1684 #define ENCODE_LOCKING_SHIFT_2 \
1686 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
1687 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
1690 #define ENCODE_LOCKING_SHIFT_3 \
1692 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
1693 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
1696 /* Produce codes for a DIMENSION1 character whose character set is
1697 CHARSET and whose position-code is C1. Designation and invocation
1698 sequences are also produced in advance if necessary. */
1700 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
1702 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
1704 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1705 *dst++ = c1 & 0x7F; \
1707 *dst++ = c1 | 0x80; \
1708 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
1711 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
1713 *dst++ = c1 & 0x7F; \
1716 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
1718 *dst++ = c1 | 0x80; \
1722 /* Since CHARSET is not yet invoked to any graphic planes, we \
1723 must invoke it, or, at first, designate it to some graphic \
1724 register. Then repeat the loop to actually produce the \
1726 dst = encode_invocation_designation (charset, coding, dst); \
1729 /* Produce codes for a DIMENSION2 character whose character set is
1730 CHARSET and whose position-codes are C1 and C2. Designation and
1731 invocation codes are also produced in advance if necessary. */
1733 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
1735 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
1737 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
1738 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
1740 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
1741 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
1744 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
1746 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
1749 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
1751 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
1755 /* Since CHARSET is not yet invoked to any graphic planes, we \
1756 must invoke it, or, at first, designate it to some graphic \
1757 register. Then repeat the loop to actually produce the \
1759 dst = encode_invocation_designation (charset, coding, dst); \
1762 #define ENCODE_ISO_CHARACTER(c) \
1764 int charset, c1, c2; \
1766 SPLIT_CHAR (c, charset, c1, c2); \
1767 if (CHARSET_DEFINED_P (charset)) \
1769 if (CHARSET_DIMENSION (charset) == 1) \
1771 if (charset == CHARSET_ASCII \
1772 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
1773 charset = charset_latin_jisx0201; \
1774 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
1778 if (charset == charset_jisx0208 \
1779 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
1780 charset = charset_jisx0208_1978; \
1781 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
1793 /* Instead of encoding character C, produce one or two `?'s. */
1795 #define ENCODE_UNSAFE_CHARACTER(c) \
1797 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \
1798 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
1799 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \
1803 /* Produce designation and invocation codes at a place pointed by DST
1804 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
1808 encode_invocation_designation (charset
, coding
, dst
)
1810 struct coding_system
*coding
;
1813 int reg
; /* graphic register number */
1815 /* At first, check designations. */
1816 for (reg
= 0; reg
< 4; reg
++)
1817 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
1822 /* CHARSET is not yet designated to any graphic registers. */
1823 /* At first check the requested designation. */
1824 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
1825 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
1826 /* Since CHARSET requests no special designation, designate it
1827 to graphic register 0. */
1830 ENCODE_DESIGNATION (charset
, reg
, coding
);
1833 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
1834 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
1836 /* Since the graphic register REG is not invoked to any graphic
1837 planes, invoke it to graphic plane 0. */
1840 case 0: /* graphic register 0 */
1844 case 1: /* graphic register 1 */
1848 case 2: /* graphic register 2 */
1849 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1850 ENCODE_SINGLE_SHIFT_2
;
1852 ENCODE_LOCKING_SHIFT_2
;
1855 case 3: /* graphic register 3 */
1856 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
1857 ENCODE_SINGLE_SHIFT_3
;
1859 ENCODE_LOCKING_SHIFT_3
;
1867 /* Produce 2-byte codes for encoded composition rule RULE. */
1869 #define ENCODE_COMPOSITION_RULE(rule) \
1872 COMPOSITION_DECODE_RULE (rule, gref, nref); \
1873 *dst++ = 32 + 81 + gref; \
1874 *dst++ = 32 + nref; \
1877 /* Produce codes for indicating the start of a composition sequence
1878 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
1879 which specify information about the composition. See the comment
1880 in coding.h for the format of DATA. */
1882 #define ENCODE_COMPOSITION_START(coding, data) \
1884 coding->composing = data[3]; \
1885 *dst++ = ISO_CODE_ESC; \
1886 if (coding->composing == COMPOSITION_RELATIVE) \
1890 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
1892 coding->cmp_data_index = coding->cmp_data_start + 4; \
1893 coding->composition_rule_follows = 0; \
1897 /* Produce codes for indicating the end of the current composition. */
1899 #define ENCODE_COMPOSITION_END(coding, data) \
1901 *dst++ = ISO_CODE_ESC; \
1903 coding->cmp_data_start += data[0]; \
1904 coding->composing = COMPOSITION_NO; \
1905 if (coding->cmp_data_start == coding->cmp_data->used \
1906 && coding->cmp_data->next) \
1908 coding->cmp_data = coding->cmp_data->next; \
1909 coding->cmp_data_start = 0; \
1913 /* Produce composition start sequence ESC 0. Here, this sequence
1914 doesn't mean the start of a new composition but means that we have
1915 just produced components (alternate chars and composition rules) of
1916 the composition and the actual text follows in SRC. */
1918 #define ENCODE_COMPOSITION_FAKE_START(coding) \
1920 *dst++ = ISO_CODE_ESC; \
1922 coding->composing = COMPOSITION_RELATIVE; \
1925 /* The following three macros produce codes for indicating direction
1927 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
1929 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
1930 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
1932 *dst++ = ISO_CODE_CSI; \
1935 #define ENCODE_DIRECTION_R2L \
1936 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
1938 #define ENCODE_DIRECTION_L2R \
1939 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
1941 /* Produce codes for designation and invocation to reset the graphic
1942 planes and registers to initial state. */
1943 #define ENCODE_RESET_PLANE_AND_REGISTER \
1946 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
1948 for (reg = 0; reg < 4; reg++) \
1949 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
1950 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
1951 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
1952 ENCODE_DESIGNATION \
1953 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
1956 /* Produce designation sequences of charsets in the line started from
1957 SRC to a place pointed by DST, and return updated DST.
1959 If the current block ends before any end-of-line, we may fail to
1960 find all the necessary designations. */
1962 static unsigned char *
1963 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
1964 struct coding_system
*coding
;
1965 Lisp_Object translation_table
;
1966 unsigned char *src
, *src_end
, *dst
;
1968 int charset
, c
, found
= 0, reg
;
1969 /* Table of charsets to be designated to each graphic register. */
1972 for (reg
= 0; reg
< 4; reg
++)
1981 charset
= CHAR_CHARSET (c
);
1982 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
1983 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
1993 for (reg
= 0; reg
< 4; reg
++)
1995 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
1996 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2002 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2005 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2006 struct coding_system
*coding
;
2007 unsigned char *source
, *destination
;
2008 int src_bytes
, dst_bytes
;
2010 unsigned char *src
= source
;
2011 unsigned char *src_end
= source
+ src_bytes
;
2012 unsigned char *dst
= destination
;
2013 unsigned char *dst_end
= destination
+ dst_bytes
;
2014 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2015 from DST_END to assure overflow checking is necessary only at the
2017 unsigned char *adjusted_dst_end
= dst_end
- 19;
2018 /* SRC_BASE remembers the start position in source in each loop.
2019 The loop will be exited when there's not enough source text to
2020 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2021 there's not enough destination area to produce encoded codes
2022 (within macro EMIT_BYTES). */
2023 unsigned char *src_base
;
2025 Lisp_Object translation_table
;
2026 Lisp_Object safe_chars
;
2028 safe_chars
= coding_safe_chars (coding
);
2030 if (NILP (Venable_character_translation
))
2031 translation_table
= Qnil
;
2034 translation_table
= coding
->translation_table_for_encode
;
2035 if (NILP (translation_table
))
2036 translation_table
= Vstandard_translation_table_for_encode
;
2039 coding
->consumed_char
= 0;
2045 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2047 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2051 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2052 && CODING_SPEC_ISO_BOL (coding
))
2054 /* We have to produce designation sequences if any now. */
2055 dst
= encode_designation_at_bol (coding
, translation_table
,
2057 CODING_SPEC_ISO_BOL (coding
) = 0;
2060 /* Check composition start and end. */
2061 if (coding
->composing
!= COMPOSITION_DISABLED
2062 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2064 struct composition_data
*cmp_data
= coding
->cmp_data
;
2065 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2066 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2068 if (coding
->composing
== COMPOSITION_RELATIVE
)
2070 if (this_pos
== data
[2])
2072 ENCODE_COMPOSITION_END (coding
, data
);
2073 cmp_data
= coding
->cmp_data
;
2074 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2077 else if (COMPOSING_P (coding
))
2079 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2080 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2081 /* We have consumed components of the composition.
2082 What follows in SRC is the compositions's base
2084 ENCODE_COMPOSITION_FAKE_START (coding
);
2087 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2088 if (coding
->composition_rule_follows
)
2090 ENCODE_COMPOSITION_RULE (c
);
2091 coding
->composition_rule_follows
= 0;
2095 if (coding
->flags
& CODING_FLAG_ISO_SAFE
2096 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2097 ENCODE_UNSAFE_CHARACTER (c
);
2099 ENCODE_ISO_CHARACTER (c
);
2100 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2101 coding
->composition_rule_follows
= 1;
2106 if (!COMPOSING_P (coding
))
2108 if (this_pos
== data
[1])
2110 ENCODE_COMPOSITION_START (coding
, data
);
2118 /* Now encode the character C. */
2119 if (c
< 0x20 || c
== 0x7F)
2123 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2125 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2126 ENCODE_RESET_PLANE_AND_REGISTER
;
2130 /* fall down to treat '\r' as '\n' ... */
2135 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2136 ENCODE_RESET_PLANE_AND_REGISTER
;
2137 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2138 bcopy (coding
->spec
.iso2022
.initial_designation
,
2139 coding
->spec
.iso2022
.current_designation
,
2140 sizeof coding
->spec
.iso2022
.initial_designation
);
2141 if (coding
->eol_type
== CODING_EOL_LF
2142 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2143 *dst
++ = ISO_CODE_LF
;
2144 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2145 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2147 *dst
++ = ISO_CODE_CR
;
2148 CODING_SPEC_ISO_BOL (coding
) = 1;
2152 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2153 ENCODE_RESET_PLANE_AND_REGISTER
;
2157 else if (ASCII_BYTE_P (c
))
2158 ENCODE_ISO_CHARACTER (c
);
2159 else if (SINGLE_BYTE_CHAR_P (c
))
2164 else if (coding
->flags
& CODING_FLAG_ISO_SAFE
2165 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2166 ENCODE_UNSAFE_CHARACTER (c
);
2168 ENCODE_ISO_CHARACTER (c
);
2170 coding
->consumed_char
++;
2174 coding
->consumed
= src_base
- source
;
2175 coding
->produced
= coding
->produced_char
= dst
- destination
;
2179 /*** 4. SJIS and BIG5 handlers ***/
2181 /* Although SJIS and BIG5 are not ISO's coding system, they are used
2182 quite widely. So, for the moment, Emacs supports them in the bare
2183 C code. But, in the future, they may be supported only by CCL. */
2185 /* SJIS is a coding system encoding three character sets: ASCII, right
2186 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2187 as is. A character of charset katakana-jisx0201 is encoded by
2188 "position-code + 0x80". A character of charset japanese-jisx0208
2189 is encoded in 2-byte but two position-codes are divided and shifted
2190 so that it fit in the range below.
2192 --- CODE RANGE of SJIS ---
2193 (character set) (range)
2195 KATAKANA-JISX0201 0xA0 .. 0xDF
2196 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2197 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2198 -------------------------------
2202 /* BIG5 is a coding system encoding two character sets: ASCII and
2203 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2204 character set and is encoded in two-byte.
2206 --- CODE RANGE of BIG5 ---
2207 (character set) (range)
2209 Big5 (1st byte) 0xA1 .. 0xFE
2210 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2211 --------------------------
2213 Since the number of characters in Big5 is larger than maximum
2214 characters in Emacs' charset (96x96), it can't be handled as one
2215 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2216 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2217 contains frequently used characters and the latter contains less
2218 frequently used characters. */
2220 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2221 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2222 C1 and C2 are the 1st and 2nd position-codes of of Emacs' internal
2223 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2225 /* Number of Big5 characters which have the same code in 1st byte. */
2226 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2228 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2231 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2233 charset = charset_big5_1; \
2236 charset = charset_big5_2; \
2237 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2239 c1 = temp / (0xFF - 0xA1) + 0x21; \
2240 c2 = temp % (0xFF - 0xA1) + 0x21; \
2243 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2245 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2246 if (charset == charset_big5_2) \
2247 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2248 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2249 b2 = temp % BIG5_SAME_ROW; \
2250 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2253 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2254 Check if a text is encoded in SJIS. If it is, return
2255 CODING_CATEGORY_MASK_SJIS, else return 0. */
2258 detect_coding_sjis (src
, src_end
)
2259 unsigned char *src
, *src_end
;
2262 /* Dummy for ONE_MORE_BYTE. */
2263 struct coding_system dummy_coding
;
2264 struct coding_system
*coding
= &dummy_coding
;
2269 if ((c
>= 0x80 && c
< 0xA0) || c
>= 0xE0)
2277 return CODING_CATEGORY_MASK_SJIS
;
2280 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2281 Check if a text is encoded in BIG5. If it is, return
2282 CODING_CATEGORY_MASK_BIG5, else return 0. */
2285 detect_coding_big5 (src
, src_end
)
2286 unsigned char *src
, *src_end
;
2289 /* Dummy for ONE_MORE_BYTE. */
2290 struct coding_system dummy_coding
;
2291 struct coding_system
*coding
= &dummy_coding
;
2299 if (c
< 0x40 || (c
>= 0x7F && c
<= 0xA0))
2304 return CODING_CATEGORY_MASK_BIG5
;
2307 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2308 Check if a text is encoded in UTF-8. If it is, return
2309 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2311 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2312 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2313 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2314 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2315 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2316 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2317 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2320 detect_coding_utf_8 (src
, src_end
)
2321 unsigned char *src
, *src_end
;
2324 int seq_maybe_bytes
;
2325 /* Dummy for ONE_MORE_BYTE. */
2326 struct coding_system dummy_coding
;
2327 struct coding_system
*coding
= &dummy_coding
;
2332 if (UTF_8_1_OCTET_P (c
))
2334 else if (UTF_8_2_OCTET_LEADING_P (c
))
2335 seq_maybe_bytes
= 1;
2336 else if (UTF_8_3_OCTET_LEADING_P (c
))
2337 seq_maybe_bytes
= 2;
2338 else if (UTF_8_4_OCTET_LEADING_P (c
))
2339 seq_maybe_bytes
= 3;
2340 else if (UTF_8_5_OCTET_LEADING_P (c
))
2341 seq_maybe_bytes
= 4;
2342 else if (UTF_8_6_OCTET_LEADING_P (c
))
2343 seq_maybe_bytes
= 5;
2350 if (!UTF_8_EXTRA_OCTET_P (c
))
2354 while (seq_maybe_bytes
> 0);
2358 return CODING_CATEGORY_MASK_UTF_8
;
2361 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2362 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
2363 Little Endian (otherwise). If it is, return
2364 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
2367 #define UTF_16_INVALID_P(val) \
2368 (((val) == 0xFFFE) \
2369 || ((val) == 0xFFFF))
2371 #define UTF_16_HIGH_SURROGATE_P(val) \
2372 (((val) & 0xD800) == 0xD800)
2374 #define UTF_16_LOW_SURROGATE_P(val) \
2375 (((val) & 0xDC00) == 0xDC00)
2378 detect_coding_utf_16 (src
, src_end
)
2379 unsigned char *src
, *src_end
;
2381 unsigned char c1
, c2
;
2382 /* Dummy for TWO_MORE_BYTES. */
2383 struct coding_system dummy_coding
;
2384 struct coding_system
*coding
= &dummy_coding
;
2386 TWO_MORE_BYTES (c1
, c2
);
2388 if ((c1
== 0xFF) && (c2
== 0xFE))
2389 return CODING_CATEGORY_MASK_UTF_16_LE
;
2390 else if ((c1
== 0xFE) && (c2
== 0xFF))
2391 return CODING_CATEGORY_MASK_UTF_16_BE
;
2397 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
2398 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
2401 decode_coding_sjis_big5 (coding
, source
, destination
,
2402 src_bytes
, dst_bytes
, sjis_p
)
2403 struct coding_system
*coding
;
2404 unsigned char *source
, *destination
;
2405 int src_bytes
, dst_bytes
;
2408 unsigned char *src
= source
;
2409 unsigned char *src_end
= source
+ src_bytes
;
2410 unsigned char *dst
= destination
;
2411 unsigned char *dst_end
= destination
+ dst_bytes
;
2412 /* SRC_BASE remembers the start position in source in each loop.
2413 The loop will be exited when there's not enough source code
2414 (within macro ONE_MORE_BYTE), or when there's not enough
2415 destination area to produce a character (within macro
2417 unsigned char *src_base
;
2418 Lisp_Object translation_table
;
2420 if (NILP (Venable_character_translation
))
2421 translation_table
= Qnil
;
2424 translation_table
= coding
->translation_table_for_decode
;
2425 if (NILP (translation_table
))
2426 translation_table
= Vstandard_translation_table_for_decode
;
2429 coding
->produced_char
= 0;
2432 int c
, charset
, c1
, c2
;
2439 charset
= CHARSET_ASCII
;
2444 if (coding
->eol_type
== CODING_EOL_CRLF
)
2449 else if (coding
->mode
2450 & CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2452 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2453 goto label_end_of_loop
;
2456 /* To process C2 again, SRC is subtracted by 1. */
2459 else if (coding
->eol_type
== CODING_EOL_CR
)
2463 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2464 && (coding
->eol_type
== CODING_EOL_CR
2465 || coding
->eol_type
== CODING_EOL_CRLF
))
2467 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2468 goto label_end_of_loop
;
2477 goto label_invalid_code
;
2478 if (c1
< 0xA0 || c1
>= 0xE0)
2480 /* SJIS -> JISX0208 */
2482 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
2483 goto label_invalid_code
;
2484 DECODE_SJIS (c1
, c2
, c1
, c2
);
2485 charset
= charset_jisx0208
;
2488 /* SJIS -> JISX0201-Kana */
2489 charset
= charset_katakana_jisx0201
;
2494 if (c1
< 0xA1 || c1
> 0xFE)
2495 goto label_invalid_code
;
2497 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
2498 goto label_invalid_code
;
2499 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
2503 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2515 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2516 coding
->produced
= dst
- destination
;
2520 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
2521 This function can encode charsets `ascii', `katakana-jisx0201',
2522 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
2523 are sure that all these charsets are registered as official charset
2524 (i.e. do not have extended leading-codes). Characters of other
2525 charsets are produced without any encoding. If SJIS_P is 1, encode
2526 SJIS text, else encode BIG5 text. */
2529 encode_coding_sjis_big5 (coding
, source
, destination
,
2530 src_bytes
, dst_bytes
, sjis_p
)
2531 struct coding_system
*coding
;
2532 unsigned char *source
, *destination
;
2533 int src_bytes
, dst_bytes
;
2536 unsigned char *src
= source
;
2537 unsigned char *src_end
= source
+ src_bytes
;
2538 unsigned char *dst
= destination
;
2539 unsigned char *dst_end
= destination
+ dst_bytes
;
2540 /* SRC_BASE remembers the start position in source in each loop.
2541 The loop will be exited when there's not enough source text to
2542 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2543 there's not enough destination area to produce encoded codes
2544 (within macro EMIT_BYTES). */
2545 unsigned char *src_base
;
2546 Lisp_Object translation_table
;
2548 if (NILP (Venable_character_translation
))
2549 translation_table
= Qnil
;
2552 translation_table
= coding
->translation_table_for_decode
;
2553 if (NILP (translation_table
))
2554 translation_table
= Vstandard_translation_table_for_decode
;
2559 int c
, charset
, c1
, c2
;
2564 /* Now encode the character C. */
2565 if (SINGLE_BYTE_CHAR_P (c
))
2570 if (!coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
)
2577 if (coding
->eol_type
== CODING_EOL_CRLF
)
2579 EMIT_TWO_BYTES ('\r', c
);
2582 else if (coding
->eol_type
== CODING_EOL_CR
)
2590 SPLIT_CHAR (c
, charset
, c1
, c2
);
2593 if (charset
== charset_jisx0208
2594 || charset
== charset_jisx0208_1978
)
2596 ENCODE_SJIS (c1
, c2
, c1
, c2
);
2597 EMIT_TWO_BYTES (c1
, c2
);
2599 else if (charset
== charset_latin_jisx0201
)
2602 /* There's no way other than producing the internal
2604 EMIT_BYTES (src_base
, src
);
2608 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
2610 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
2611 EMIT_TWO_BYTES (c1
, c2
);
2614 /* There's no way other than producing the internal
2616 EMIT_BYTES (src_base
, src
);
2619 coding
->consumed_char
++;
2623 coding
->consumed
= src_base
- source
;
2624 coding
->produced
= coding
->produced_char
= dst
- destination
;
2628 /*** 5. CCL handlers ***/
2630 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2631 Check if a text is encoded in a coding system of which
2632 encoder/decoder are written in CCL program. If it is, return
2633 CODING_CATEGORY_MASK_CCL, else return 0. */
2636 detect_coding_ccl (src
, src_end
)
2637 unsigned char *src
, *src_end
;
2639 unsigned char *valid
;
2641 /* Dummy for ONE_MORE_BYTE. */
2642 struct coding_system dummy_coding
;
2643 struct coding_system
*coding
= &dummy_coding
;
2645 /* No coding system is assigned to coding-category-ccl. */
2646 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
2649 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
2657 return CODING_CATEGORY_MASK_CCL
;
2661 /*** 6. End-of-line handlers ***/
2663 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
2666 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
2667 struct coding_system
*coding
;
2668 unsigned char *source
, *destination
;
2669 int src_bytes
, dst_bytes
;
2671 unsigned char *src
= source
;
2672 unsigned char *dst
= destination
;
2673 unsigned char *src_end
= src
+ src_bytes
;
2674 unsigned char *dst_end
= dst
+ dst_bytes
;
2675 Lisp_Object translation_table
;
2676 /* SRC_BASE remembers the start position in source in each loop.
2677 The loop will be exited when there's not enough source code
2678 (within macro ONE_MORE_BYTE), or when there's not enough
2679 destination area to produce a character (within macro
2681 unsigned char *src_base
;
2684 translation_table
= Qnil
;
2685 switch (coding
->eol_type
)
2687 case CODING_EOL_CRLF
:
2697 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2699 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2700 goto label_end_of_loop
;
2707 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
2709 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2710 goto label_end_of_loop
;
2723 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
2725 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
2726 goto label_end_of_loop
;
2735 default: /* no need for EOL handling */
2745 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2746 coding
->produced
= dst
- destination
;
2750 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
2751 format of end-of-line according to `coding->eol_type'. It also
2752 convert multibyte form 8-bit characers to unibyte if
2753 CODING->src_multibyte is nonzero. If `coding->mode &
2754 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
2755 also means end-of-line. */
2758 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
2759 struct coding_system
*coding
;
2760 unsigned char *source
, *destination
;
2761 int src_bytes
, dst_bytes
;
2763 unsigned char *src
= source
;
2764 unsigned char *dst
= destination
;
2765 unsigned char *src_end
= src
+ src_bytes
;
2766 unsigned char *dst_end
= dst
+ dst_bytes
;
2767 Lisp_Object translation_table
;
2768 /* SRC_BASE remembers the start position in source in each loop.
2769 The loop will be exited when there's not enough source text to
2770 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2771 there's not enough destination area to produce encoded codes
2772 (within macro EMIT_BYTES). */
2773 unsigned char *src_base
;
2775 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
2777 translation_table
= Qnil
;
2778 if (coding
->src_multibyte
2779 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
2783 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
2786 if (coding
->eol_type
== CODING_EOL_CRLF
)
2788 while (src
< src_end
)
2794 else if (c
== '\n' || (c
== '\r' && selective_display
))
2795 EMIT_TWO_BYTES ('\r', '\n');
2805 if (src_bytes
<= dst_bytes
)
2807 safe_bcopy (src
, dst
, src_bytes
);
2813 if (coding
->src_multibyte
2814 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
2816 safe_bcopy (src
, dst
, dst_bytes
);
2817 src_base
= src
+ dst_bytes
;
2818 dst
= destination
+ dst_bytes
;
2819 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2821 if (coding
->eol_type
== CODING_EOL_CR
)
2823 for (src
= destination
; src
< dst
; src
++)
2824 if (*src
== '\n') *src
= '\r';
2826 else if (selective_display
)
2828 for (src
= destination
; src
< dst
; src
++)
2829 if (*src
== '\r') *src
= '\n';
2832 if (coding
->src_multibyte
)
2833 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
2835 coding
->consumed
= src_base
- source
;
2836 coding
->produced
= dst
- destination
;
2840 /*** 7. C library functions ***/
2842 /* In Emacs Lisp, coding system is represented by a Lisp symbol which
2843 has a property `coding-system'. The value of this property is a
2844 vector of length 5 (called as coding-vector). Among elements of
2845 this vector, the first (element[0]) and the fifth (element[4])
2846 carry important information for decoding/encoding. Before
2847 decoding/encoding, this information should be set in fields of a
2848 structure of type `coding_system'.
2850 A value of property `coding-system' can be a symbol of another
2851 subsidiary coding-system. In that case, Emacs gets coding-vector
2854 `element[0]' contains information to be set in `coding->type'. The
2855 value and its meaning is as follows:
2857 0 -- coding_type_emacs_mule
2858 1 -- coding_type_sjis
2859 2 -- coding_type_iso2022
2860 3 -- coding_type_big5
2861 4 -- coding_type_ccl encoder/decoder written in CCL
2862 nil -- coding_type_no_conversion
2863 t -- coding_type_undecided (automatic conversion on decoding,
2864 no-conversion on encoding)
2866 `element[4]' contains information to be set in `coding->flags' and
2867 `coding->spec'. The meaning varies by `coding->type'.
2869 If `coding->type' is `coding_type_iso2022', element[4] is a vector
2870 of length 32 (of which the first 13 sub-elements are used now).
2871 Meanings of these sub-elements are:
2873 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
2874 If the value is an integer of valid charset, the charset is
2875 assumed to be designated to graphic register N initially.
2877 If the value is minus, it is a minus value of charset which
2878 reserves graphic register N, which means that the charset is
2879 not designated initially but should be designated to graphic
2880 register N just before encoding a character in that charset.
2882 If the value is nil, graphic register N is never used on
2885 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
2886 Each value takes t or nil. See the section ISO2022 of
2887 `coding.h' for more information.
2889 If `coding->type' is `coding_type_big5', element[4] is t to denote
2890 BIG5-ETen or nil to denote BIG5-HKU.
2892 If `coding->type' takes the other value, element[4] is ignored.
2894 Emacs Lisp's coding system also carries information about format of
2895 end-of-line in a value of property `eol-type'. If the value is
2896 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
2897 means CODING_EOL_CR. If it is not integer, it should be a vector
2898 of subsidiary coding systems of which property `eol-type' has one
2903 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
2904 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
2905 is setup so that no conversion is necessary and return -1, else
2909 setup_coding_system (coding_system
, coding
)
2910 Lisp_Object coding_system
;
2911 struct coding_system
*coding
;
2913 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
2917 /* Initialize some fields required for all kinds of coding systems. */
2918 coding
->symbol
= coding_system
;
2919 coding
->common_flags
= 0;
2921 coding
->heading_ascii
= -1;
2922 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
2923 coding
->composing
= COMPOSITION_DISABLED
;
2924 coding
->cmp_data
= NULL
;
2926 if (NILP (coding_system
))
2927 goto label_invalid_coding_system
;
2929 coding_spec
= Fget (coding_system
, Qcoding_system
);
2931 if (!VECTORP (coding_spec
)
2932 || XVECTOR (coding_spec
)->size
!= 5
2933 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
2934 goto label_invalid_coding_system
;
2936 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
2937 if (VECTORP (eol_type
))
2939 coding
->eol_type
= CODING_EOL_UNDECIDED
;
2940 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
2942 else if (XFASTINT (eol_type
) == 1)
2944 coding
->eol_type
= CODING_EOL_CRLF
;
2945 coding
->common_flags
2946 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
2948 else if (XFASTINT (eol_type
) == 2)
2950 coding
->eol_type
= CODING_EOL_CR
;
2951 coding
->common_flags
2952 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
2955 coding
->eol_type
= CODING_EOL_LF
;
2957 coding_type
= XVECTOR (coding_spec
)->contents
[0];
2958 /* Try short cut. */
2959 if (SYMBOLP (coding_type
))
2961 if (EQ (coding_type
, Qt
))
2963 coding
->type
= coding_type_undecided
;
2964 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
2967 coding
->type
= coding_type_no_conversion
;
2971 /* Get values of coding system properties:
2972 `post-read-conversion', `pre-write-conversion',
2973 `translation-table-for-decode', `translation-table-for-encode'. */
2974 plist
= XVECTOR (coding_spec
)->contents
[3];
2975 /* Pre & post conversion functions should be disabled if
2976 inhibit_eol_conversion is nozero. This is the case that a code
2977 conversion function is called while those functions are running. */
2978 if (! inhibit_pre_post_conversion
)
2980 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
2981 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
2983 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
2985 val
= Fget (val
, Qtranslation_table_for_decode
);
2986 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
2987 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
2989 val
= Fget (val
, Qtranslation_table_for_encode
);
2990 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
2991 val
= Fplist_get (plist
, Qcoding_category
);
2994 val
= Fget (val
, Qcoding_category_index
);
2996 coding
->category_idx
= XINT (val
);
2998 goto label_invalid_coding_system
;
3001 goto label_invalid_coding_system
;
3003 /* If the coding system has non-nil `composition' property, enable
3004 composition handling. */
3005 val
= Fplist_get (plist
, Qcomposition
);
3007 coding
->composing
= COMPOSITION_NO
;
3009 switch (XFASTINT (coding_type
))
3012 coding
->type
= coding_type_emacs_mule
;
3013 if (!NILP (coding
->post_read_conversion
))
3014 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3015 if (!NILP (coding
->pre_write_conversion
))
3016 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3020 coding
->type
= coding_type_sjis
;
3021 coding
->common_flags
3022 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3026 coding
->type
= coding_type_iso2022
;
3027 coding
->common_flags
3028 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3030 Lisp_Object val
, temp
;
3032 int i
, charset
, reg_bits
= 0;
3034 val
= XVECTOR (coding_spec
)->contents
[4];
3036 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3037 goto label_invalid_coding_system
;
3039 flags
= XVECTOR (val
)->contents
;
3041 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3042 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3043 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3044 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3045 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3046 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3047 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3048 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3049 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3050 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3051 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3052 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3053 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3056 /* Invoke graphic register 0 to plane 0. */
3057 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3058 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3059 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3060 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3061 /* Not single shifting at first. */
3062 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3063 /* Beginning of buffer should also be regarded as bol. */
3064 CODING_SPEC_ISO_BOL (coding
) = 1;
3066 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3067 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3068 val
= Vcharset_revision_alist
;
3071 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3073 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3074 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3075 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3079 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3080 FLAGS[REG] can be one of below:
3081 integer CHARSET: CHARSET occupies register I,
3082 t: designate nothing to REG initially, but can be used
3084 list of integer, nil, or t: designate the first
3085 element (if integer) to REG initially, the remaining
3086 elements (if integer) is designated to REG on request,
3087 if an element is t, REG can be used by any charsets,
3088 nil: REG is never used. */
3089 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3090 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3091 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3092 for (i
= 0; i
< 4; i
++)
3094 if (INTEGERP (flags
[i
])
3095 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
))
3096 || (charset
= get_charset_id (flags
[i
])) >= 0)
3098 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3099 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3101 else if (EQ (flags
[i
], Qt
))
3103 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3105 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3107 else if (CONSP (flags
[i
]))
3112 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3113 if (INTEGERP (XCAR (tail
))
3114 && (charset
= XINT (XCAR (tail
)),
3115 CHARSET_VALID_P (charset
))
3116 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3118 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3119 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3122 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3124 while (CONSP (tail
))
3126 if (INTEGERP (XCAR (tail
))
3127 && (charset
= XINT (XCAR (tail
)),
3128 CHARSET_VALID_P (charset
))
3129 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3130 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3132 else if (EQ (XCAR (tail
), Qt
))
3138 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3140 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3141 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3144 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3146 /* REG 1 can be used only by locking shift in 7-bit env. */
3147 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3149 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3150 /* Without any shifting, only REG 0 and 1 can be used. */
3155 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3157 if (CHARSET_VALID_P (charset
)
3158 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3159 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3161 /* There exist some default graphic registers to be
3164 /* We had better avoid designating a charset of
3165 CHARS96 to REG 0 as far as possible. */
3166 if (CHARSET_CHARS (charset
) == 96)
3167 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3169 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3171 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3173 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3177 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3178 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3182 coding
->type
= coding_type_big5
;
3183 coding
->common_flags
3184 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3186 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3187 ? CODING_FLAG_BIG5_HKU
3188 : CODING_FLAG_BIG5_ETEN
);
3192 coding
->type
= coding_type_ccl
;
3193 coding
->common_flags
3194 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3196 val
= XVECTOR (coding_spec
)->contents
[4];
3198 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3200 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3202 goto label_invalid_coding_system
;
3204 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3205 val
= Fplist_get (plist
, Qvalid_codes
);
3210 for (; CONSP (val
); val
= XCDR (val
))
3214 && XINT (this) >= 0 && XINT (this) < 256)
3215 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3216 else if (CONSP (this)
3217 && INTEGERP (XCAR (this))
3218 && INTEGERP (XCDR (this)))
3220 int start
= XINT (XCAR (this));
3221 int end
= XINT (XCDR (this));
3223 if (start
>= 0 && start
<= end
&& end
< 256)
3224 while (start
<= end
)
3225 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3230 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3231 coding
->spec
.ccl
.cr_carryover
= 0;
3235 coding
->type
= coding_type_raw_text
;
3239 goto label_invalid_coding_system
;
3243 label_invalid_coding_system
:
3244 coding
->type
= coding_type_no_conversion
;
3245 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3246 coding
->common_flags
= 0;
3247 coding
->eol_type
= CODING_EOL_LF
;
3248 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3252 /* Free memory blocks allocated for storing composition information. */
3255 coding_free_composition_data (coding
)
3256 struct coding_system
*coding
;
3258 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3262 /* Memory blocks are chained. At first, rewind to the first, then,
3263 free blocks one by one. */
3264 while (cmp_data
->prev
)
3265 cmp_data
= cmp_data
->prev
;
3268 next
= cmp_data
->next
;
3272 coding
->cmp_data
= NULL
;
3275 /* Set `char_offset' member of all memory blocks pointed by
3276 coding->cmp_data to POS. */
3279 coding_adjust_composition_offset (coding
, pos
)
3280 struct coding_system
*coding
;
3283 struct composition_data
*cmp_data
;
3285 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3286 cmp_data
->char_offset
= pos
;
3289 /* Setup raw-text or one of its subsidiaries in the structure
3290 coding_system CODING according to the already setup value eol_type
3291 in CODING. CODING should be setup for some coding system in
3295 setup_raw_text_coding_system (coding
)
3296 struct coding_system
*coding
;
3298 if (coding
->type
!= coding_type_raw_text
)
3300 coding
->symbol
= Qraw_text
;
3301 coding
->type
= coding_type_raw_text
;
3302 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3304 Lisp_Object subsidiaries
;
3305 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3307 if (VECTORP (subsidiaries
)
3308 && XVECTOR (subsidiaries
)->size
== 3)
3310 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3312 setup_coding_system (coding
->symbol
, coding
);
3317 /* Emacs has a mechanism to automatically detect a coding system if it
3318 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3319 it's impossible to distinguish some coding systems accurately
3320 because they use the same range of codes. So, at first, coding
3321 systems are categorized into 7, those are:
3323 o coding-category-emacs-mule
3325 The category for a coding system which has the same code range
3326 as Emacs' internal format. Assigned the coding-system (Lisp
3327 symbol) `emacs-mule' by default.
3329 o coding-category-sjis
3331 The category for a coding system which has the same code range
3332 as SJIS. Assigned the coding-system (Lisp
3333 symbol) `japanese-shift-jis' by default.
3335 o coding-category-iso-7
3337 The category for a coding system which has the same code range
3338 as ISO2022 of 7-bit environment. This doesn't use any locking
3339 shift and single shift functions. This can encode/decode all
3340 charsets. Assigned the coding-system (Lisp symbol)
3341 `iso-2022-7bit' by default.
3343 o coding-category-iso-7-tight
3345 Same as coding-category-iso-7 except that this can
3346 encode/decode only the specified charsets.
3348 o coding-category-iso-8-1
3350 The category for a coding system which has the same code range
3351 as ISO2022 of 8-bit environment and graphic plane 1 used only
3352 for DIMENSION1 charset. This doesn't use any locking shift
3353 and single shift functions. Assigned the coding-system (Lisp
3354 symbol) `iso-latin-1' by default.
3356 o coding-category-iso-8-2
3358 The category for a coding system which has the same code range
3359 as ISO2022 of 8-bit environment and graphic plane 1 used only
3360 for DIMENSION2 charset. This doesn't use any locking shift
3361 and single shift functions. Assigned the coding-system (Lisp
3362 symbol) `japanese-iso-8bit' by default.
3364 o coding-category-iso-7-else
3366 The category for a coding system which has the same code range
3367 as ISO2022 of 7-bit environemnt but uses locking shift or
3368 single shift functions. Assigned the coding-system (Lisp
3369 symbol) `iso-2022-7bit-lock' by default.
3371 o coding-category-iso-8-else
3373 The category for a coding system which has the same code range
3374 as ISO2022 of 8-bit environemnt but uses locking shift or
3375 single shift functions. Assigned the coding-system (Lisp
3376 symbol) `iso-2022-8bit-ss2' by default.
3378 o coding-category-big5
3380 The category for a coding system which has the same code range
3381 as BIG5. Assigned the coding-system (Lisp symbol)
3382 `cn-big5' by default.
3384 o coding-category-utf-8
3386 The category for a coding system which has the same code range
3387 as UTF-8 (cf. RFC2279). Assigned the coding-system (Lisp
3388 symbol) `utf-8' by default.
3390 o coding-category-utf-16-be
3392 The category for a coding system in which a text has an
3393 Unicode signature (cf. Unicode Standard) in the order of BIG
3394 endian at the head. Assigned the coding-system (Lisp symbol)
3395 `utf-16-be' by default.
3397 o coding-category-utf-16-le
3399 The category for a coding system in which a text has an
3400 Unicode signature (cf. Unicode Standard) in the order of
3401 LITTLE endian at the head. Assigned the coding-system (Lisp
3402 symbol) `utf-16-le' by default.
3404 o coding-category-ccl
3406 The category for a coding system of which encoder/decoder is
3407 written in CCL programs. The default value is nil, i.e., no
3408 coding system is assigned.
3410 o coding-category-binary
3412 The category for a coding system not categorized in any of the
3413 above. Assigned the coding-system (Lisp symbol)
3414 `no-conversion' by default.
3416 Each of them is a Lisp symbol and the value is an actual
3417 `coding-system's (this is also a Lisp symbol) assigned by a user.
3418 What Emacs does actually is to detect a category of coding system.
3419 Then, it uses a `coding-system' assigned to it. If Emacs can't
3420 decide only one possible category, it selects a category of the
3421 highest priority. Priorities of categories are also specified by a
3422 user in a Lisp variable `coding-category-list'.
3427 int ascii_skip_code
[256];
3429 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
3430 If it detects possible coding systems, return an integer in which
3431 appropriate flag bits are set. Flag bits are defined by macros
3432 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
3433 it should point the table `coding_priorities'. In that case, only
3434 the flag bit for a coding system of the highest priority is set in
3437 How many ASCII characters are at the head is returned as *SKIP. */
3440 detect_coding_mask (source
, src_bytes
, priorities
, skip
)
3441 unsigned char *source
;
3442 int src_bytes
, *priorities
, *skip
;
3444 register unsigned char c
;
3445 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
3446 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
3449 /* At first, skip all ASCII characters and control characters except
3450 for three ISO2022 specific control characters. */
3451 ascii_skip_code
[ISO_CODE_SO
] = 0;
3452 ascii_skip_code
[ISO_CODE_SI
] = 0;
3453 ascii_skip_code
[ISO_CODE_ESC
] = 0;
3455 label_loop_detect_coding
:
3456 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
3457 *skip
= src
- source
;
3460 /* We found nothing other than ASCII. There's nothing to do. */
3464 /* The text seems to be encoded in some multilingual coding system.
3465 Now, try to find in which coding system the text is encoded. */
3468 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
3469 /* C is an ISO2022 specific control code of C0. */
3470 mask
= detect_coding_iso2022 (src
, src_end
);
3473 /* No valid ISO2022 code follows C. Try again. */
3475 if (c
== ISO_CODE_ESC
)
3476 ascii_skip_code
[ISO_CODE_ESC
] = 1;
3478 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
3479 goto label_loop_detect_coding
;
3483 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
3485 if (mask
& priorities
[i
])
3486 return priorities
[i
];
3488 return CODING_CATEGORY_MASK_RAW_TEXT
;
3497 /* C is the first byte of SJIS character code,
3498 or a leading-code of Emacs' internal format (emacs-mule),
3499 or the first byte of UTF-16. */
3500 try = (CODING_CATEGORY_MASK_SJIS
3501 | CODING_CATEGORY_MASK_EMACS_MULE
3502 | CODING_CATEGORY_MASK_UTF_16_BE
3503 | CODING_CATEGORY_MASK_UTF_16_LE
);
3505 /* Or, if C is a special latin extra code,
3506 or is an ISO2022 specific control code of C1 (SS2 or SS3),
3507 or is an ISO2022 control-sequence-introducer (CSI),
3508 we should also consider the possibility of ISO2022 codings. */
3509 if ((VECTORP (Vlatin_extra_code_table
)
3510 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
3511 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
3512 || (c
== ISO_CODE_CSI
3515 || ((*src
== '0' || *src
== '1' || *src
== '2')
3516 && src
+ 1 < src_end
3517 && src
[1] == ']')))))
3518 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
3519 | CODING_CATEGORY_MASK_ISO_8BIT
);
3522 /* C is a character of ISO2022 in graphic plane right,
3523 or a SJIS's 1-byte character code (i.e. JISX0201),
3524 or the first byte of BIG5's 2-byte code,
3525 or the first byte of UTF-8/16. */
3526 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
3527 | CODING_CATEGORY_MASK_ISO_8BIT
3528 | CODING_CATEGORY_MASK_SJIS
3529 | CODING_CATEGORY_MASK_BIG5
3530 | CODING_CATEGORY_MASK_UTF_8
3531 | CODING_CATEGORY_MASK_UTF_16_BE
3532 | CODING_CATEGORY_MASK_UTF_16_LE
);
3534 /* Or, we may have to consider the possibility of CCL. */
3535 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
3536 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
3537 ->spec
.ccl
.valid_codes
)[c
])
3538 try |= CODING_CATEGORY_MASK_CCL
;
3541 utf16_examined_p
= iso2022_examined_p
= 0;
3544 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
3546 if (!iso2022_examined_p
3547 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
3549 mask
|= detect_coding_iso2022 (src
, src_end
);
3550 iso2022_examined_p
= 1;
3552 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
3553 mask
|= detect_coding_sjis (src
, src_end
);
3554 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
3555 mask
|= detect_coding_utf_8 (src
, src_end
);
3556 else if (!utf16_examined_p
3557 && (priorities
[i
] & try &
3558 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
3560 mask
|= detect_coding_utf_16 (src
, src_end
);
3561 utf16_examined_p
= 1;
3563 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
3564 mask
|= detect_coding_big5 (src
, src_end
);
3565 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
3566 mask
|= detect_coding_emacs_mule (src
, src_end
);
3567 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
3568 mask
|= detect_coding_ccl (src
, src_end
);
3569 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
3570 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
3571 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
3572 mask
|= CODING_CATEGORY_MASK_BINARY
;
3573 if (mask
& priorities
[i
])
3574 return priorities
[i
];
3576 return CODING_CATEGORY_MASK_RAW_TEXT
;
3578 if (try & CODING_CATEGORY_MASK_ISO
)
3579 mask
|= detect_coding_iso2022 (src
, src_end
);
3580 if (try & CODING_CATEGORY_MASK_SJIS
)
3581 mask
|= detect_coding_sjis (src
, src_end
);
3582 if (try & CODING_CATEGORY_MASK_BIG5
)
3583 mask
|= detect_coding_big5 (src
, src_end
);
3584 if (try & CODING_CATEGORY_MASK_UTF_8
)
3585 mask
|= detect_coding_utf_8 (src
, src_end
);
3586 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
3587 mask
|= detect_coding_utf_16 (src
, src_end
);
3588 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
3589 mask
|= detect_coding_emacs_mule (src
, src_end
);
3590 if (try & CODING_CATEGORY_MASK_CCL
)
3591 mask
|= detect_coding_ccl (src
, src_end
);
3593 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
3596 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
3597 The information of the detected coding system is set in CODING. */
3600 detect_coding (coding
, src
, src_bytes
)
3601 struct coding_system
*coding
;
3609 val
= Vcoding_category_list
;
3610 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
);
3611 coding
->heading_ascii
= skip
;
3615 /* We found a single coding system of the highest priority in MASK. */
3617 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
3619 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
3621 val
= XSYMBOL (XVECTOR (Vcoding_category_table
)->contents
[idx
])->value
;
3623 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3627 tmp
= Fget (val
, Qeol_type
);
3629 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
3632 /* Setup this new coding system while preserving some slots. */
3634 int src_multibyte
= coding
->src_multibyte
;
3635 int dst_multibyte
= coding
->dst_multibyte
;
3637 setup_coding_system (val
, coding
);
3638 coding
->src_multibyte
= src_multibyte
;
3639 coding
->dst_multibyte
= dst_multibyte
;
3640 coding
->heading_ascii
= skip
;
3644 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
3645 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
3646 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
3648 How many non-eol characters are at the head is returned as *SKIP. */
3650 #define MAX_EOL_CHECK_COUNT 3
3653 detect_eol_type (source
, src_bytes
, skip
)
3654 unsigned char *source
;
3655 int src_bytes
, *skip
;
3657 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
3659 int total
= 0; /* How many end-of-lines are found so far. */
3660 int eol_type
= CODING_EOL_UNDECIDED
;
3665 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
3668 if (c
== '\n' || c
== '\r')
3671 *skip
= src
- 1 - source
;
3674 this_eol_type
= CODING_EOL_LF
;
3675 else if (src
>= src_end
|| *src
!= '\n')
3676 this_eol_type
= CODING_EOL_CR
;
3678 this_eol_type
= CODING_EOL_CRLF
, src
++;
3680 if (eol_type
== CODING_EOL_UNDECIDED
)
3681 /* This is the first end-of-line. */
3682 eol_type
= this_eol_type
;
3683 else if (eol_type
!= this_eol_type
)
3685 /* The found type is different from what found before. */
3686 eol_type
= CODING_EOL_INCONSISTENT
;
3693 *skip
= src_end
- source
;
3697 /* Like detect_eol_type, but detect EOL type in 2-octet
3698 big-endian/little-endian format for coding systems utf-16-be and
3702 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
3703 unsigned char *source
;
3704 int src_bytes
, *skip
;
3706 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
3707 unsigned int c1
, c2
;
3708 int total
= 0; /* How many end-of-lines are found so far. */
3709 int eol_type
= CODING_EOL_UNDECIDED
;
3720 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
3722 c1
= (src
[msb
] << 8) | (src
[lsb
]);
3725 if (c1
== '\n' || c1
== '\r')
3728 *skip
= src
- 2 - source
;
3732 this_eol_type
= CODING_EOL_LF
;
3736 if ((src
+ 1) >= src_end
)
3738 this_eol_type
= CODING_EOL_CR
;
3742 c2
= (src
[msb
] << 8) | (src
[lsb
]);
3744 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
3746 this_eol_type
= CODING_EOL_CR
;
3750 if (eol_type
== CODING_EOL_UNDECIDED
)
3751 /* This is the first end-of-line. */
3752 eol_type
= this_eol_type
;
3753 else if (eol_type
!= this_eol_type
)
3755 /* The found type is different from what found before. */
3756 eol_type
= CODING_EOL_INCONSISTENT
;
3763 *skip
= src_end
- source
;
3767 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
3768 is encoded. If it detects an appropriate format of end-of-line, it
3769 sets the information in *CODING. */
3772 detect_eol (coding
, src
, src_bytes
)
3773 struct coding_system
*coding
;
3781 switch (coding
->category_idx
)
3783 case CODING_CATEGORY_IDX_UTF_16_BE
:
3784 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
3786 case CODING_CATEGORY_IDX_UTF_16_LE
:
3787 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
3790 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
3794 if (coding
->heading_ascii
> skip
)
3795 coding
->heading_ascii
= skip
;
3797 skip
= coding
->heading_ascii
;
3799 if (eol_type
== CODING_EOL_UNDECIDED
)
3801 if (eol_type
== CODING_EOL_INCONSISTENT
)
3804 /* This code is suppressed until we find a better way to
3805 distinguish raw text file and binary file. */
3807 /* If we have already detected that the coding is raw-text, the
3808 coding should actually be no-conversion. */
3809 if (coding
->type
== coding_type_raw_text
)
3811 setup_coding_system (Qno_conversion
, coding
);
3814 /* Else, let's decode only text code anyway. */
3816 eol_type
= CODING_EOL_LF
;
3819 val
= Fget (coding
->symbol
, Qeol_type
);
3820 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
3822 int src_multibyte
= coding
->src_multibyte
;
3823 int dst_multibyte
= coding
->dst_multibyte
;
3825 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
3826 coding
->src_multibyte
= src_multibyte
;
3827 coding
->dst_multibyte
= dst_multibyte
;
3828 coding
->heading_ascii
= skip
;
3832 #define CONVERSION_BUFFER_EXTRA_ROOM 256
3834 #define DECODING_BUFFER_MAG(coding) \
3835 (coding->type == coding_type_iso2022 \
3837 : (coding->type == coding_type_ccl \
3838 ? coding->spec.ccl.decoder.buf_magnification \
3841 /* Return maximum size (bytes) of a buffer enough for decoding
3842 SRC_BYTES of text encoded in CODING. */
3845 decoding_buffer_size (coding
, src_bytes
)
3846 struct coding_system
*coding
;
3849 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
3850 + CONVERSION_BUFFER_EXTRA_ROOM
);
3853 /* Return maximum size (bytes) of a buffer enough for encoding
3854 SRC_BYTES of text to CODING. */
3857 encoding_buffer_size (coding
, src_bytes
)
3858 struct coding_system
*coding
;
3863 if (coding
->type
== coding_type_ccl
)
3864 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
3865 else if (CODING_REQUIRE_ENCODING (coding
))
3870 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
3873 #ifndef MINIMUM_CONVERSION_BUFFER_SIZE
3874 #define MINIMUM_CONVERSION_BUFFER_SIZE 1024
3877 char *conversion_buffer
;
3878 int conversion_buffer_size
;
3880 /* Return a pointer to a SIZE bytes of buffer to be used for encoding
3881 or decoding. Sufficient memory is allocated automatically. If we
3882 run out of memory, return NULL. */
3885 get_conversion_buffer (size
)
3888 if (size
> conversion_buffer_size
)
3891 int real_size
= conversion_buffer_size
* 2;
3893 while (real_size
< size
) real_size
*= 2;
3894 buf
= (char *) xmalloc (real_size
);
3895 xfree (conversion_buffer
);
3896 conversion_buffer
= buf
;
3897 conversion_buffer_size
= real_size
;
3899 return conversion_buffer
;
3903 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
3904 struct coding_system
*coding
;
3905 unsigned char *source
, *destination
;
3906 int src_bytes
, dst_bytes
, encodep
;
3908 struct ccl_program
*ccl
3909 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
3912 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
3914 ccl
->eol_type
= coding
->eol_type
;
3915 coding
->produced
= ccl_driver (ccl
, source
, destination
,
3916 src_bytes
, dst_bytes
, &(coding
->consumed
));
3918 coding
->produced_char
= coding
->produced
;
3922 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
3923 coding
->produced
= str_as_multibyte (destination
, bytes
,
3925 &(coding
->produced_char
));
3928 switch (ccl
->status
)
3930 case CCL_STAT_SUSPEND_BY_SRC
:
3931 result
= CODING_FINISH_INSUFFICIENT_SRC
;
3933 case CCL_STAT_SUSPEND_BY_DST
:
3934 result
= CODING_FINISH_INSUFFICIENT_DST
;
3937 case CCL_STAT_INVALID_CMD
:
3938 result
= CODING_FINISH_INTERRUPT
;
3941 result
= CODING_FINISH_NORMAL
;
3947 /* Decode EOL format of the text at PTR of BYTES length destructively
3948 according to CODING->eol_type. This is called after the CCL
3949 program produced a decoded text at PTR. If we do CRLF->LF
3950 conversion, update CODING->produced and CODING->produced_char. */
3953 decode_eol_post_ccl (coding
, ptr
, bytes
)
3954 struct coding_system
*coding
;
3958 Lisp_Object val
, saved_coding_symbol
;
3959 unsigned char *pend
= ptr
+ bytes
;
3962 /* Remember the current coding system symbol. We set it back when
3963 an inconsistent EOL is found so that `last-coding-system-used' is
3964 set to the coding system that doesn't specify EOL conversion. */
3965 saved_coding_symbol
= coding
->symbol
;
3967 coding
->spec
.ccl
.cr_carryover
= 0;
3968 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
3970 /* Here, to avoid the call of setup_coding_system, we directly
3971 call detect_eol_type. */
3972 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
3973 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
3974 coding
->eol_type
= CODING_EOL_LF
;
3975 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3977 val
= Fget (coding
->symbol
, Qeol_type
);
3978 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
3979 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
3981 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
3984 if (coding
->eol_type
== CODING_EOL_LF
3985 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
3987 /* We have nothing to do. */
3990 else if (coding
->eol_type
== CODING_EOL_CRLF
)
3992 unsigned char *pstart
= ptr
, *p
= ptr
;
3994 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
3995 && *(pend
- 1) == '\r')
3997 /* If the last character is CR, we can't handle it here
3998 because LF will be in the not-yet-decoded source text.
3999 Recorded that the CR is not yet processed. */
4000 coding
->spec
.ccl
.cr_carryover
= 1;
4002 coding
->produced_char
--;
4009 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4016 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4017 goto undo_eol_conversion
;
4021 else if (*ptr
== '\n'
4022 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4023 goto undo_eol_conversion
;
4028 undo_eol_conversion
:
4029 /* We have faced with inconsistent EOL format at PTR.
4030 Convert all LFs before PTR back to CRLFs. */
4031 for (p
--, ptr
--; p
>= pstart
; p
--)
4034 *ptr
-- = '\n', *ptr
-- = '\r';
4038 /* If carryover is recorded, cancel it because we don't
4039 convert CRLF anymore. */
4040 if (coding
->spec
.ccl
.cr_carryover
)
4042 coding
->spec
.ccl
.cr_carryover
= 0;
4044 coding
->produced_char
++;
4048 coding
->eol_type
= CODING_EOL_LF
;
4049 coding
->symbol
= saved_coding_symbol
;
4053 /* As each two-byte sequence CRLF was converted to LF, (PEND
4054 - P) is the number of deleted characters. */
4055 coding
->produced
-= pend
- p
;
4056 coding
->produced_char
-= pend
- p
;
4059 else /* i.e. coding->eol_type == CODING_EOL_CR */
4061 unsigned char *p
= ptr
;
4063 for (; ptr
< pend
; ptr
++)
4067 else if (*ptr
== '\n'
4068 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4070 for (; p
< ptr
; p
++)
4076 coding
->eol_type
= CODING_EOL_LF
;
4077 coding
->symbol
= saved_coding_symbol
;
4083 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4084 decoding, it may detect coding system and format of end-of-line if
4085 those are not yet decided. The source should be unibyte, the
4086 result is multibyte if CODING->dst_multibyte is nonzero, else
4090 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4091 struct coding_system
*coding
;
4092 unsigned char *source
, *destination
;
4093 int src_bytes
, dst_bytes
;
4095 if (coding
->type
== coding_type_undecided
)
4096 detect_coding (coding
, source
, src_bytes
);
4098 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4099 && coding
->type
!= coding_type_ccl
)
4100 detect_eol (coding
, source
, src_bytes
);
4102 coding
->produced
= coding
->produced_char
= 0;
4103 coding
->consumed
= coding
->consumed_char
= 0;
4105 coding
->result
= CODING_FINISH_NORMAL
;
4107 switch (coding
->type
)
4109 case coding_type_sjis
:
4110 decode_coding_sjis_big5 (coding
, source
, destination
,
4111 src_bytes
, dst_bytes
, 1);
4114 case coding_type_iso2022
:
4115 decode_coding_iso2022 (coding
, source
, destination
,
4116 src_bytes
, dst_bytes
);
4119 case coding_type_big5
:
4120 decode_coding_sjis_big5 (coding
, source
, destination
,
4121 src_bytes
, dst_bytes
, 0);
4124 case coding_type_emacs_mule
:
4125 decode_coding_emacs_mule (coding
, source
, destination
,
4126 src_bytes
, dst_bytes
);
4129 case coding_type_ccl
:
4130 if (coding
->spec
.ccl
.cr_carryover
)
4132 /* Set the CR which is not processed by the previous call of
4133 decode_eol_post_ccl in DESTINATION. */
4134 *destination
= '\r';
4136 coding
->produced_char
++;
4139 ccl_coding_driver (coding
, source
,
4140 destination
+ coding
->spec
.ccl
.cr_carryover
,
4141 src_bytes
, dst_bytes
, 0);
4142 if (coding
->eol_type
!= CODING_EOL_LF
)
4143 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4147 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4150 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4151 && coding
->consumed
== src_bytes
)
4152 coding
->result
= CODING_FINISH_NORMAL
;
4154 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4155 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4157 unsigned char *src
= source
+ coding
->consumed
;
4158 unsigned char *dst
= destination
+ coding
->produced
;
4160 src_bytes
-= coding
->consumed
;
4162 if (COMPOSING_P (coding
))
4163 DECODE_COMPOSITION_END ('1');
4167 dst
+= CHAR_STRING (c
, dst
);
4168 coding
->produced_char
++;
4170 coding
->consumed
= coding
->consumed_char
= src
- source
;
4171 coding
->produced
= dst
- destination
;
4174 if (!coding
->dst_multibyte
)
4176 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4177 coding
->produced_char
= coding
->produced
;
4180 return coding
->result
;
4183 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4184 multibyteness of the source is CODING->src_multibyte, the
4185 multibyteness of the result is always unibyte. */
4188 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4189 struct coding_system
*coding
;
4190 unsigned char *source
, *destination
;
4191 int src_bytes
, dst_bytes
;
4193 coding
->produced
= coding
->produced_char
= 0;
4194 coding
->consumed
= coding
->consumed_char
= 0;
4196 coding
->result
= CODING_FINISH_NORMAL
;
4198 switch (coding
->type
)
4200 case coding_type_sjis
:
4201 encode_coding_sjis_big5 (coding
, source
, destination
,
4202 src_bytes
, dst_bytes
, 1);
4205 case coding_type_iso2022
:
4206 encode_coding_iso2022 (coding
, source
, destination
,
4207 src_bytes
, dst_bytes
);
4210 case coding_type_big5
:
4211 encode_coding_sjis_big5 (coding
, source
, destination
,
4212 src_bytes
, dst_bytes
, 0);
4215 case coding_type_emacs_mule
:
4216 encode_coding_emacs_mule (coding
, source
, destination
,
4217 src_bytes
, dst_bytes
);
4220 case coding_type_ccl
:
4221 ccl_coding_driver (coding
, source
, destination
,
4222 src_bytes
, dst_bytes
, 1);
4226 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4229 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4230 && coding
->consumed
== src_bytes
)
4231 coding
->result
= CODING_FINISH_NORMAL
;
4233 if (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4235 unsigned char *src
= source
+ coding
->consumed
;
4236 unsigned char *src_end
= src
+ src_bytes
;
4237 unsigned char *dst
= destination
+ coding
->produced
;
4239 if (coding
->type
== coding_type_iso2022
)
4240 ENCODE_RESET_PLANE_AND_REGISTER
;
4241 if (COMPOSING_P (coding
))
4242 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
4243 if (coding
->consumed
< src_bytes
)
4245 int len
= src_bytes
- coding
->consumed
;
4247 BCOPY_SHORT (source
+ coding
->consumed
, dst
, len
);
4248 if (coding
->src_multibyte
)
4249 len
= str_as_unibyte (dst
, len
);
4251 coding
->consumed
= src_bytes
;
4253 coding
->produced
= coding
->produced_char
= dst
- destination
;
4256 return coding
->result
;
4259 /* Scan text in the region between *BEG and *END (byte positions),
4260 skip characters which we don't have to decode by coding system
4261 CODING at the head and tail, then set *BEG and *END to the region
4262 of the text we actually have to convert. The caller should move
4263 the gap out of the region in advance if the region is from a
4266 If STR is not NULL, *BEG and *END are indices into STR. */
4269 shrink_decoding_region (beg
, end
, coding
, str
)
4271 struct coding_system
*coding
;
4274 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
4276 Lisp_Object translation_table
;
4278 if (coding
->type
== coding_type_ccl
4279 || coding
->type
== coding_type_undecided
4280 || coding
->eol_type
!= CODING_EOL_LF
4281 || !NILP (coding
->post_read_conversion
)
4282 || coding
->composing
!= COMPOSITION_DISABLED
)
4284 /* We can't skip any data. */
4287 if (coding
->type
== coding_type_no_conversion
4288 || coding
->type
== coding_type_raw_text
4289 || coding
->type
== coding_type_emacs_mule
)
4291 /* We need no conversion, but don't have to skip any data here.
4292 Decoding routine handles them effectively anyway. */
4296 translation_table
= coding
->translation_table_for_decode
;
4297 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
4298 translation_table
= Vstandard_translation_table_for_decode
;
4299 if (CHAR_TABLE_P (translation_table
))
4302 for (i
= 0; i
< 128; i
++)
4303 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
4306 /* Some ASCII character should be translated. We give up
4311 if (coding
->heading_ascii
>= 0)
4312 /* Detection routine has already found how much we can skip at the
4314 *beg
+= coding
->heading_ascii
;
4318 begp_orig
= begp
= str
+ *beg
;
4319 endp_orig
= endp
= str
+ *end
;
4323 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
4324 endp_orig
= endp
= begp
+ *end
- *beg
;
4327 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
4328 || coding
->eol_type
== CODING_EOL_CRLF
);
4330 switch (coding
->type
)
4332 case coding_type_sjis
:
4333 case coding_type_big5
:
4334 /* We can skip all ASCII characters at the head. */
4335 if (coding
->heading_ascii
< 0)
4338 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
4340 while (begp
< endp
&& *begp
< 0x80) begp
++;
4342 /* We can skip all ASCII characters at the tail except for the
4343 second byte of SJIS or BIG5 code. */
4345 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
4347 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
4348 /* Do not consider LF as ascii if preceded by CR, since that
4349 confuses eol decoding. */
4350 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
4352 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
4356 case coding_type_iso2022
:
4357 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
4358 /* We can't skip any data. */
4360 if (coding
->heading_ascii
< 0)
4362 /* We can skip all ASCII characters at the head except for a
4363 few control codes. */
4364 while (begp
< endp
&& (c
= *begp
) < 0x80
4365 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
4366 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
4367 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
4370 switch (coding
->category_idx
)
4372 case CODING_CATEGORY_IDX_ISO_8_1
:
4373 case CODING_CATEGORY_IDX_ISO_8_2
:
4374 /* We can skip all ASCII characters at the tail. */
4376 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
4378 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
4379 /* Do not consider LF as ascii if preceded by CR, since that
4380 confuses eol decoding. */
4381 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
4385 case CODING_CATEGORY_IDX_ISO_7
:
4386 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
4388 /* We can skip all charactes at the tail except for 8-bit
4389 codes and ESC and the following 2-byte at the tail. */
4390 unsigned char *eight_bit
= NULL
;
4394 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
4396 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
4401 && (c
= endp
[-1]) != ISO_CODE_ESC
)
4403 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
4406 /* Do not consider LF as ascii if preceded by CR, since that
4407 confuses eol decoding. */
4408 if (begp
< endp
&& endp
< endp_orig
4409 && endp
[-1] == '\r' && endp
[0] == '\n')
4411 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
4413 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
4414 /* This is an ASCII designation sequence. We can
4415 surely skip the tail. But, if we have
4416 encountered an 8-bit code, skip only the codes
4418 endp
= eight_bit
? eight_bit
: endp
+ 2;
4420 /* Hmmm, we can't skip the tail. */
4432 *beg
+= begp
- begp_orig
;
4433 *end
+= endp
- endp_orig
;
4437 /* Like shrink_decoding_region but for encoding. */
4440 shrink_encoding_region (beg
, end
, coding
, str
)
4442 struct coding_system
*coding
;
4445 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
4447 Lisp_Object translation_table
;
4449 if (coding
->type
== coding_type_ccl
4450 || coding
->eol_type
== CODING_EOL_CRLF
4451 || coding
->eol_type
== CODING_EOL_CR
4452 || coding
->cmp_data
&& coding
->cmp_data
->used
> 0)
4454 /* We can't skip any data. */
4457 if (coding
->type
== coding_type_no_conversion
4458 || coding
->type
== coding_type_raw_text
4459 || coding
->type
== coding_type_emacs_mule
4460 || coding
->type
== coding_type_undecided
)
4462 /* We need no conversion, but don't have to skip any data here.
4463 Encoding routine handles them effectively anyway. */
4467 translation_table
= coding
->translation_table_for_encode
;
4468 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
4469 translation_table
= Vstandard_translation_table_for_encode
;
4470 if (CHAR_TABLE_P (translation_table
))
4473 for (i
= 0; i
< 128; i
++)
4474 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
4477 /* Some ASCII character should be tranlsated. We give up
4484 begp_orig
= begp
= str
+ *beg
;
4485 endp_orig
= endp
= str
+ *end
;
4489 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
4490 endp_orig
= endp
= begp
+ *end
- *beg
;
4493 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
4494 || coding
->eol_type
== CODING_EOL_CRLF
);
4496 /* Here, we don't have to check coding->pre_write_conversion because
4497 the caller is expected to have handled it already. */
4498 switch (coding
->type
)
4500 case coding_type_iso2022
:
4501 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
4502 /* We can't skip any data. */
4504 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
4506 unsigned char *bol
= begp
;
4507 while (begp
< endp
&& *begp
< 0x80)
4510 if (begp
[-1] == '\n')
4514 goto label_skip_tail
;
4518 case coding_type_sjis
:
4519 case coding_type_big5
:
4520 /* We can skip all ASCII characters at the head and tail. */
4522 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
4524 while (begp
< endp
&& *begp
< 0x80) begp
++;
4527 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
4529 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
4536 *beg
+= begp
- begp_orig
;
4537 *end
+= endp
- endp_orig
;
4541 /* As shrinking conversion region requires some overhead, we don't try
4542 shrinking if the length of conversion region is less than this
4544 static int shrink_conversion_region_threshhold
= 1024;
4546 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
4548 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
4550 if (encodep) shrink_encoding_region (beg, end, coding, str); \
4551 else shrink_decoding_region (beg, end, coding, str); \
4556 code_convert_region_unwind (dummy
)
4559 inhibit_pre_post_conversion
= 0;
4563 /* Store information about all compositions in the range FROM and TO
4564 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
4565 buffer or a string, defaults to the current buffer. */
4568 coding_save_composition (coding
, from
, to
, obj
)
4569 struct coding_system
*coding
;
4576 if (coding
->composing
== COMPOSITION_DISABLED
)
4578 if (!coding
->cmp_data
)
4579 coding_allocate_composition_data (coding
, from
);
4580 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
4584 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
4587 coding
->composing
= COMPOSITION_NO
;
4590 if (COMPOSITION_VALID_P (start
, end
, prop
))
4592 enum composition_method method
= COMPOSITION_METHOD (prop
);
4593 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
4594 >= COMPOSITION_DATA_SIZE
)
4595 coding_allocate_composition_data (coding
, from
);
4596 /* For relative composition, we remember start and end
4597 positions, for the other compositions, we also remember
4599 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
4600 if (method
!= COMPOSITION_RELATIVE
)
4602 /* We must store a*/
4603 Lisp_Object val
, ch
;
4605 val
= COMPOSITION_COMPONENTS (prop
);
4609 ch
= XCAR (val
), val
= XCDR (val
);
4610 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
4612 else if (VECTORP (val
) || STRINGP (val
))
4614 int len
= (VECTORP (val
)
4615 ? XVECTOR (val
)->size
: XSTRING (val
)->size
);
4617 for (i
= 0; i
< len
; i
++)
4620 ? Faref (val
, make_number (i
))
4621 : XVECTOR (val
)->contents
[i
]);
4622 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
4625 else /* INTEGERP (val) */
4626 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
4628 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
4633 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
4636 /* Make coding->cmp_data point to the first memory block. */
4637 while (coding
->cmp_data
->prev
)
4638 coding
->cmp_data
= coding
->cmp_data
->prev
;
4639 coding
->cmp_data_start
= 0;
4642 /* Reflect the saved information about compositions to OBJ.
4643 CODING->cmp_data points to a memory block for the informaiton. OBJ
4644 is a buffer or a string, defaults to the current buffer. */
4647 coding_restore_composition (coding
, obj
)
4648 struct coding_system
*coding
;
4651 struct composition_data
*cmp_data
= coding
->cmp_data
;
4656 while (cmp_data
->prev
)
4657 cmp_data
= cmp_data
->prev
;
4663 for (i
= 0; i
< cmp_data
->used
; i
+= cmp_data
->data
[i
])
4665 int *data
= cmp_data
->data
+ i
;
4666 enum composition_method method
= (enum composition_method
) data
[3];
4667 Lisp_Object components
;
4669 if (method
== COMPOSITION_RELATIVE
)
4673 int len
= data
[0] - 4, j
;
4674 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
4676 for (j
= 0; j
< len
; j
++)
4677 args
[j
] = make_number (data
[4 + j
]);
4678 components
= (method
== COMPOSITION_WITH_ALTCHARS
4679 ? Fstring (len
, args
) : Fvector (len
, args
));
4681 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
4683 cmp_data
= cmp_data
->next
;
4687 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
4688 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
4689 coding system CODING, and return the status code of code conversion
4690 (currently, this value has no meaning).
4692 How many characters (and bytes) are converted to how many
4693 characters (and bytes) are recorded in members of the structure
4696 If REPLACE is nonzero, we do various things as if the original text
4697 is deleted and a new text is inserted. See the comments in
4698 replace_range (insdel.c) to know what we are doing.
4700 If REPLACE is zero, it is assumed that the source text is unibyte.
4701 Otherwize, it is assumed that the source text is multibyte. */
4704 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
4705 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
4706 struct coding_system
*coding
;
4708 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
4709 int require
, inserted
, inserted_byte
;
4710 int head_skip
, tail_skip
, total_skip
= 0;
4711 Lisp_Object saved_coding_symbol
;
4713 unsigned char *src
, *dst
;
4714 Lisp_Object deletion
;
4715 int orig_point
= PT
, orig_len
= len
;
4717 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
4719 coding
->src_multibyte
= replace
&& multibyte_p
;
4720 coding
->dst_multibyte
= multibyte_p
;
4723 saved_coding_symbol
= Qnil
;
4725 if (from
< PT
&& PT
< to
)
4727 TEMP_SET_PT_BOTH (from
, from_byte
);
4733 int saved_from
= from
;
4734 int saved_inhibit_modification_hooks
;
4736 prepare_to_modify_buffer (from
, to
, &from
);
4737 if (saved_from
!= from
)
4740 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
4741 len_byte
= to_byte
- from_byte
;
4744 /* The code conversion routine can not preserve text properties
4745 for now. So, we must remove all text properties in the
4746 region. Here, we must suppress all modification hooks. */
4747 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
4748 inhibit_modification_hooks
= 1;
4749 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
4750 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
4753 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
4755 /* We must detect encoding of text and eol format. */
4757 if (from
< GPT
&& to
> GPT
)
4758 move_gap_both (from
, from_byte
);
4759 if (coding
->type
== coding_type_undecided
)
4761 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
4762 if (coding
->type
== coding_type_undecided
)
4763 /* It seems that the text contains only ASCII, but we
4764 should not left it undecided because the deeper
4765 decoding routine (decode_coding) tries to detect the
4766 encodings again in vain. */
4767 coding
->type
= coding_type_emacs_mule
;
4769 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4770 && coding
->type
!= coding_type_ccl
)
4772 saved_coding_symbol
= coding
->symbol
;
4773 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
4774 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4775 coding
->eol_type
= CODING_EOL_LF
;
4776 /* We had better recover the original eol format if we
4777 encounter an inconsitent eol format while decoding. */
4778 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4782 /* Now we convert the text. */
4784 /* For encoding, we must process pre-write-conversion in advance. */
4785 if (! inhibit_pre_post_conversion
4787 && SYMBOLP (coding
->pre_write_conversion
)
4788 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
4790 /* The function in pre-write-conversion may put a new text in a
4792 struct buffer
*prev
= current_buffer
;
4794 int count
= specpdl_ptr
- specpdl
;
4796 record_unwind_protect (code_convert_region_unwind
, Qnil
);
4797 /* We should not call any more pre-write/post-read-conversion
4798 functions while this pre-write-conversion is running. */
4799 inhibit_pre_post_conversion
= 1;
4800 call2 (coding
->pre_write_conversion
,
4801 make_number (from
), make_number (to
));
4802 inhibit_pre_post_conversion
= 0;
4803 /* Discard the unwind protect. */
4806 if (current_buffer
!= prev
)
4809 new = Fcurrent_buffer ();
4810 set_buffer_internal_1 (prev
);
4811 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
4812 TEMP_SET_PT_BOTH (from
, from_byte
);
4813 insert_from_buffer (XBUFFER (new), 1, len
, 0);
4815 if (orig_point
>= to
)
4816 orig_point
+= len
- orig_len
;
4817 else if (orig_point
> from
)
4821 from_byte
= CHAR_TO_BYTE (from
);
4822 to_byte
= CHAR_TO_BYTE (to
);
4823 len_byte
= to_byte
- from_byte
;
4824 TEMP_SET_PT_BOTH (from
, from_byte
);
4829 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
4831 if (coding
->composing
!= COMPOSITION_DISABLED
)
4834 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
4836 coding_allocate_composition_data (coding
, from
);
4839 /* Try to skip the heading and tailing ASCIIs. */
4840 if (coding
->type
!= coding_type_ccl
)
4842 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
4844 if (from
< GPT
&& GPT
< to
)
4845 move_gap_both (from
, from_byte
);
4846 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
4847 if (from_byte
== to_byte
4848 && (encodep
|| NILP (coding
->post_read_conversion
))
4849 && ! CODING_REQUIRE_FLUSHING (coding
))
4851 coding
->produced
= len_byte
;
4852 coding
->produced_char
= len
;
4854 /* We must record and adjust for this new text now. */
4855 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
4859 head_skip
= from_byte
- from_byte_orig
;
4860 tail_skip
= to_byte_orig
- to_byte
;
4861 total_skip
= head_skip
+ tail_skip
;
4864 len
-= total_skip
; len_byte
-= total_skip
;
4867 /* For converion, we must put the gap before the text in addition to
4868 making the gap larger for efficient decoding. The required gap
4869 size starts from 2000 which is the magic number used in make_gap.
4870 But, after one batch of conversion, it will be incremented if we
4871 find that it is not enough . */
4874 if (GAP_SIZE
< require
)
4875 make_gap (require
- GAP_SIZE
);
4876 move_gap_both (from
, from_byte
);
4878 inserted
= inserted_byte
= 0;
4880 GAP_SIZE
+= len_byte
;
4883 ZV_BYTE
-= len_byte
;
4886 if (GPT
- BEG
< BEG_UNCHANGED
)
4887 BEG_UNCHANGED
= GPT
- BEG
;
4888 if (Z
- GPT
< END_UNCHANGED
)
4889 END_UNCHANGED
= Z
- GPT
;
4891 if (!encodep
&& coding
->src_multibyte
)
4893 /* Decoding routines expects that the source text is unibyte.
4894 We must convert 8-bit characters of multibyte form to
4896 int len_byte_orig
= len_byte
;
4897 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
4898 if (len_byte
< len_byte_orig
)
4899 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
4901 coding
->src_multibyte
= 0;
4908 /* The buffer memory is now:
4909 +--------+converted-text+---------+-------original-text-------+---+
4910 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
4911 |<---------------------- GAP ----------------------->| */
4912 src
= GAP_END_ADDR
- len_byte
;
4913 dst
= GPT_ADDR
+ inserted_byte
;
4916 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
4918 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
4920 /* The buffer memory is now:
4921 +--------+-------converted-text----+--+------original-text----+---+
4922 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
4923 |<---------------------- GAP ----------------------->| */
4925 inserted
+= coding
->produced_char
;
4926 inserted_byte
+= coding
->produced
;
4927 len_byte
-= coding
->consumed
;
4929 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
4931 coding_allocate_composition_data (coding
, from
+ inserted
);
4935 src
+= coding
->consumed
;
4936 dst
+= coding
->produced
;
4938 if (result
== CODING_FINISH_NORMAL
)
4943 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
4945 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
4946 Lisp_Object eol_type
;
4948 /* Encode LFs back to the original eol format (CR or CRLF). */
4949 if (coding
->eol_type
== CODING_EOL_CR
)
4951 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
4957 while (p
< pend
) if (*p
++ == '\n') count
++;
4958 if (src
- dst
< count
)
4960 /* We don't have sufficient room for encoding LFs
4961 back to CRLF. We must record converted and
4962 not-yet-converted text back to the buffer
4963 content, enlarge the gap, then record them out of
4964 the buffer contents again. */
4965 int add
= len_byte
+ inserted_byte
;
4968 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
4969 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
4970 make_gap (count
- GAP_SIZE
);
4972 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
4973 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
4974 /* Don't forget to update SRC, DST, and PEND. */
4975 src
= GAP_END_ADDR
- len_byte
;
4976 dst
= GPT_ADDR
+ inserted_byte
;
4980 inserted_byte
+= count
;
4981 coding
->produced
+= count
;
4982 p
= dst
= pend
+ count
;
4986 if (*p
== '\n') count
--, *--p
= '\r';
4990 /* Suppress eol-format conversion in the further conversion. */
4991 coding
->eol_type
= CODING_EOL_LF
;
4993 /* Set the coding system symbol to that for Unix-like EOL. */
4994 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
4995 if (VECTORP (eol_type
)
4996 && XVECTOR (eol_type
)->size
== 3
4997 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
4998 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5000 coding
->symbol
= saved_coding_symbol
;
5006 if (coding
->type
!= coding_type_ccl
5007 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5009 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5012 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5014 /* The source text ends in invalid codes. Let's just
5015 make them valid buffer contents, and finish conversion. */
5016 inserted
+= len_byte
;
5017 inserted_byte
+= len_byte
;
5022 if (result
== CODING_FINISH_INTERRUPT
)
5024 /* The conversion procedure was interrupted by a user. */
5027 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5028 if (coding
->consumed
< 1)
5030 /* It's quite strange to require more memory without
5031 consuming any bytes. Perhaps CCL program bug. */
5036 /* We have just done the first batch of conversion which was
5037 stoped because of insufficient gap. Let's reconsider the
5038 required gap size (i.e. SRT - DST) now.
5040 We have converted ORIG bytes (== coding->consumed) into
5041 NEW bytes (coding->produced). To convert the remaining
5042 LEN bytes, we may need REQUIRE bytes of gap, where:
5043 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5044 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5045 Here, we are sure that NEW >= ORIG. */
5046 float ratio
= coding
->produced
- coding
->consumed
;
5047 ratio
/= coding
->consumed
;
5048 require
= len_byte
* ratio
;
5051 if ((src
- dst
) < (require
+ 2000))
5053 /* See the comment above the previous call of make_gap. */
5054 int add
= len_byte
+ inserted_byte
;
5057 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5058 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5059 make_gap (require
+ 2000);
5061 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5062 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5065 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5067 if (encodep
&& coding
->dst_multibyte
)
5069 /* The output is unibyte. We must convert 8-bit characters to
5071 if (inserted_byte
* 2 > GAP_SIZE
)
5073 GAP_SIZE
-= inserted_byte
;
5074 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5075 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5076 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5077 make_gap (inserted_byte
- GAP_SIZE
);
5078 GAP_SIZE
+= inserted_byte
;
5079 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5080 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5081 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5083 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5086 /* If we have shrinked the conversion area, adjust it now. */
5090 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5091 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5092 GAP_SIZE
+= total_skip
;
5093 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5094 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5095 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5096 from
-= head_skip
; from_byte
-= head_skip
;
5097 to
+= tail_skip
; to_byte
+= tail_skip
;
5101 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5102 inserted
= Z
- prev_Z
;
5104 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5105 coding_restore_composition (coding
, Fcurrent_buffer ());
5106 coding_free_composition_data (coding
);
5108 if (! inhibit_pre_post_conversion
5109 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5112 int count
= specpdl_ptr
- specpdl
;
5115 TEMP_SET_PT_BOTH (from
, from_byte
);
5117 record_unwind_protect (code_convert_region_unwind
, Qnil
);
5118 /* We should not call any more pre-write/post-read-conversion
5119 functions while this post-read-conversion is running. */
5120 inhibit_pre_post_conversion
= 1;
5121 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
5122 inhibit_pre_post_conversion
= 0;
5123 /* Discard the unwind protect. */
5125 CHECK_NUMBER (val
, 0);
5126 inserted
+= Z
- prev_Z
;
5129 if (orig_point
>= from
)
5131 if (orig_point
>= from
+ orig_len
)
5132 orig_point
+= inserted
- orig_len
;
5135 TEMP_SET_PT (orig_point
);
5140 signal_after_change (from
, to
- from
, inserted
);
5141 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
5145 coding
->consumed
= to_byte
- from_byte
;
5146 coding
->consumed_char
= to
- from
;
5147 coding
->produced
= inserted_byte
;
5148 coding
->produced_char
= inserted
;
5155 run_pre_post_conversion_on_str (str
, coding
, encodep
)
5157 struct coding_system
*coding
;
5160 int count
= specpdl_ptr
- specpdl
;
5161 struct gcpro gcpro1
;
5162 struct buffer
*prev
= current_buffer
;
5163 int multibyte
= STRING_MULTIBYTE (str
);
5165 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
5166 record_unwind_protect (code_convert_region_unwind
, Qnil
);
5168 temp_output_buffer_setup (" *code-converting-work*");
5169 set_buffer_internal (XBUFFER (Vstandard_output
));
5170 /* We must insert the contents of STR as is without
5171 unibyte<->multibyte conversion. For that, we adjust the
5172 multibyteness of the working buffer to that of STR. */
5174 current_buffer
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
5175 insert_from_string (str
, 0, 0,
5176 XSTRING (str
)->size
, STRING_BYTES (XSTRING (str
)), 0);
5178 inhibit_pre_post_conversion
= 1;
5180 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
5183 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
5184 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
5186 inhibit_pre_post_conversion
= 0;
5187 str
= make_buffer_string (BEG
, Z
, 0);
5188 return unbind_to (count
, str
);
5192 decode_coding_string (str
, coding
, nocopy
)
5194 struct coding_system
*coding
;
5199 int from
, to
, to_byte
;
5200 struct gcpro gcpro1
;
5201 Lisp_Object saved_coding_symbol
;
5205 to
= XSTRING (str
)->size
;
5206 to_byte
= STRING_BYTES (XSTRING (str
));
5208 saved_coding_symbol
= Qnil
;
5209 if (CODING_REQUIRE_DETECTION (coding
))
5211 /* See the comments in code_convert_region. */
5212 if (coding
->type
== coding_type_undecided
)
5214 detect_coding (coding
, XSTRING (str
)->data
, to_byte
);
5215 if (coding
->type
== coding_type_undecided
)
5216 coding
->type
= coding_type_emacs_mule
;
5218 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5219 && coding
->type
!= coding_type_ccl
)
5221 saved_coding_symbol
= coding
->symbol
;
5222 detect_eol (coding
, XSTRING (str
)->data
, to_byte
);
5223 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5224 coding
->eol_type
= CODING_EOL_LF
;
5225 /* We had better recover the original eol format if we
5226 encounter an inconsitent eol format while decoding. */
5227 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5231 if (! CODING_REQUIRE_DECODING (coding
))
5233 if (!STRING_MULTIBYTE (str
))
5235 str
= Fstring_as_multibyte (str
);
5238 return (nocopy
? str
: Fcopy_sequence (str
));
5241 if (STRING_MULTIBYTE (str
))
5243 /* Decoding routines expect the source text to be unibyte. */
5244 str
= Fstring_as_unibyte (str
);
5245 to_byte
= STRING_BYTES (XSTRING (str
));
5247 coding
->src_multibyte
= 0;
5249 coding
->dst_multibyte
= 1;
5251 if (coding
->composing
!= COMPOSITION_DISABLED
)
5252 coding_allocate_composition_data (coding
, from
);
5254 /* Try to skip the heading and tailing ASCIIs. */
5255 if (coding
->type
!= coding_type_ccl
)
5257 int from_orig
= from
;
5259 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, XSTRING (str
)->data
,
5261 if (from
== to_byte
)
5262 return (nocopy
? str
: Fcopy_sequence (str
));
5265 len
= decoding_buffer_size (coding
, to_byte
- from
);
5266 len
+= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5268 buf
= get_conversion_buffer (len
);
5272 bcopy (XSTRING (str
)->data
, buf
, from
);
5273 result
= decode_coding (coding
, XSTRING (str
)->data
+ from
,
5274 buf
+ from
, to_byte
- from
, len
);
5275 if (result
== CODING_FINISH_INCONSISTENT_EOL
)
5277 /* We simply try to decode the whole string again but without
5278 eol-conversion this time. */
5279 coding
->eol_type
= CODING_EOL_LF
;
5280 coding
->symbol
= saved_coding_symbol
;
5281 coding_free_composition_data (coding
);
5282 return decode_coding_string (str
, coding
, nocopy
);
5285 bcopy (XSTRING (str
)->data
+ to_byte
, buf
+ from
+ coding
->produced
,
5286 STRING_BYTES (XSTRING (str
)) - to_byte
);
5288 len
= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5289 str
= make_multibyte_string (buf
, len
+ coding
->produced_char
,
5290 len
+ coding
->produced
);
5292 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
5293 coding_restore_composition (coding
, str
);
5294 coding_free_composition_data (coding
);
5296 if (SYMBOLP (coding
->post_read_conversion
)
5297 && !NILP (Ffboundp (coding
->post_read_conversion
)))
5298 str
= run_pre_post_conversion_on_str (str
, coding
, 0);
5304 encode_coding_string (str
, coding
, nocopy
)
5306 struct coding_system
*coding
;
5311 int from
, to
, to_byte
;
5312 struct gcpro gcpro1
;
5313 Lisp_Object saved_coding_symbol
;
5316 if (SYMBOLP (coding
->pre_write_conversion
)
5317 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
5318 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
5321 to
= XSTRING (str
)->size
;
5322 to_byte
= STRING_BYTES (XSTRING (str
));
5324 saved_coding_symbol
= Qnil
;
5325 if (! CODING_REQUIRE_ENCODING (coding
))
5327 if (STRING_MULTIBYTE (str
))
5329 str
= Fstring_as_unibyte (str
);
5332 return (nocopy
? str
: Fcopy_sequence (str
));
5335 /* Encoding routines determine the multibyteness of the source text
5336 by coding->src_multibyte. */
5337 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
5338 coding
->dst_multibyte
= 0;
5340 if (coding
->composing
!= COMPOSITION_DISABLED
)
5341 coding_save_composition (coding
, from
, to
, str
);
5343 /* Try to skip the heading and tailing ASCIIs. */
5344 if (coding
->type
!= coding_type_ccl
)
5346 int from_orig
= from
;
5348 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, XSTRING (str
)->data
,
5350 if (from
== to_byte
)
5351 return (nocopy
? str
: Fcopy_sequence (str
));
5354 len
= encoding_buffer_size (coding
, to_byte
- from
);
5355 len
+= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5357 buf
= get_conversion_buffer (len
);
5361 bcopy (XSTRING (str
)->data
, buf
, from
);
5362 result
= encode_coding (coding
, XSTRING (str
)->data
+ from
,
5363 buf
+ from
, to_byte
- from
, len
);
5364 bcopy (XSTRING (str
)->data
+ to_byte
, buf
+ from
+ coding
->produced
,
5365 STRING_BYTES (XSTRING (str
)) - to_byte
);
5367 len
= from
+ STRING_BYTES (XSTRING (str
)) - to_byte
;
5368 str
= make_unibyte_string (buf
, len
+ coding
->produced
);
5369 coding_free_composition_data (coding
);
5376 /*** 8. Emacs Lisp library functions ***/
5378 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
5379 "Return t if OBJECT is nil or a coding-system.\n\
5380 See the documentation of `make-coding-system' for information\n\
5381 about coding-system objects.")
5389 /* Get coding-spec vector for OBJ. */
5390 obj
= Fget (obj
, Qcoding_system
);
5391 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
5395 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
5396 Sread_non_nil_coding_system
, 1, 1, 0,
5397 "Read a coding system from the minibuffer, prompting with string PROMPT.")
5404 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
5405 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
5407 while (XSTRING (val
)->size
== 0);
5408 return (Fintern (val
, Qnil
));
5411 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
5412 "Read a coding system from the minibuffer, prompting with string PROMPT.\n\
5413 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM.")
5414 (prompt
, default_coding_system
)
5415 Lisp_Object prompt
, default_coding_system
;
5418 if (SYMBOLP (default_coding_system
))
5419 XSETSTRING (default_coding_system
, XSYMBOL (default_coding_system
)->name
);
5420 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
5421 Qt
, Qnil
, Qcoding_system_history
,
5422 default_coding_system
, Qnil
);
5423 return (XSTRING (val
)->size
== 0 ? Qnil
: Fintern (val
, Qnil
));
5426 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
5428 "Check validity of CODING-SYSTEM.\n\
5429 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.\n\
5430 It is valid if it is a symbol with a non-nil `coding-system' property.\n\
5431 The value of property should be a vector of length 5.")
5433 Lisp_Object coding_system
;
5435 CHECK_SYMBOL (coding_system
, 0);
5436 if (!NILP (Fcoding_system_p (coding_system
)))
5437 return coding_system
;
5439 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
5443 detect_coding_system (src
, src_bytes
, highest
)
5445 int src_bytes
, highest
;
5447 int coding_mask
, eol_type
;
5448 Lisp_Object val
, tmp
;
5451 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
);
5452 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
5453 if (eol_type
== CODING_EOL_INCONSISTENT
)
5454 eol_type
= CODING_EOL_UNDECIDED
;
5459 if (eol_type
!= CODING_EOL_UNDECIDED
)
5462 val2
= Fget (Qundecided
, Qeol_type
);
5464 val
= XVECTOR (val2
)->contents
[eol_type
];
5466 return (highest
? val
: Fcons (val
, Qnil
));
5469 /* At first, gather possible coding systems in VAL. */
5471 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
5473 Lisp_Object category_val
, category_index
;
5475 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
5476 category_val
= Fsymbol_value (XCAR (tmp
));
5477 if (!NILP (category_val
)
5478 && NATNUMP (category_index
)
5479 && (coding_mask
& (1 << XFASTINT (category_index
))))
5481 val
= Fcons (category_val
, val
);
5487 val
= Fnreverse (val
);
5489 /* Then, replace the elements with subsidiary coding systems. */
5490 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
5492 if (eol_type
!= CODING_EOL_UNDECIDED
5493 && eol_type
!= CODING_EOL_INCONSISTENT
)
5496 eol
= Fget (XCAR (tmp
), Qeol_type
);
5498 XCAR (tmp
) = XVECTOR (eol
)->contents
[eol_type
];
5501 return (highest
? XCAR (val
) : val
);
5504 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
5506 "Detect coding system of the text in the region between START and END.\n\
5507 Return a list of possible coding systems ordered by priority.\n\
5509 If only ASCII characters are found, it returns a list of single element\n\
5510 `undecided' or its subsidiary coding system according to a detected\n\
5511 end-of-line format.\n\
5513 If optional argument HIGHEST is non-nil, return the coding system of\n\
5515 (start
, end
, highest
)
5516 Lisp_Object start
, end
, highest
;
5519 int from_byte
, to_byte
;
5521 CHECK_NUMBER_COERCE_MARKER (start
, 0);
5522 CHECK_NUMBER_COERCE_MARKER (end
, 1);
5524 validate_region (&start
, &end
);
5525 from
= XINT (start
), to
= XINT (end
);
5526 from_byte
= CHAR_TO_BYTE (from
);
5527 to_byte
= CHAR_TO_BYTE (to
);
5529 if (from
< GPT
&& to
>= GPT
)
5530 move_gap_both (to
, to_byte
);
5532 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
5533 to_byte
- from_byte
,
5537 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
5539 "Detect coding system of the text in STRING.\n\
5540 Return a list of possible coding systems ordered by priority.\n\
5542 If only ASCII characters are found, it returns a list of single element\n\
5543 `undecided' or its subsidiary coding system according to a detected\n\
5544 end-of-line format.\n\
5546 If optional argument HIGHEST is non-nil, return the coding system of\n\
5549 Lisp_Object string
, highest
;
5551 CHECK_STRING (string
, 0);
5553 return detect_coding_system (XSTRING (string
)->data
,
5554 STRING_BYTES (XSTRING (string
)),
5558 /* Return an intersection of lists L1 and L2. */
5561 intersection (l1
, l2
)
5566 for (val
= Qnil
; CONSP (l1
); l1
= XCDR (l1
))
5568 if (!NILP (Fmemq (XCAR (l1
), l2
)))
5569 val
= Fcons (XCAR (l1
), val
);
5575 /* Subroutine for Fsafe_coding_systems_region_internal.
5577 Return a list of coding systems that safely encode the multibyte
5578 text between P and PEND. SAFE_CODINGS, if non-nil, is a list of
5579 possible coding systems. If it is nil, it means that we have not
5580 yet found any coding systems.
5582 WORK_TABLE is a copy of the char-table Vchar_coding_system_table. An
5583 element of WORK_TABLE is set to t once the element is looked up.
5585 If a non-ASCII single byte char is found, set
5586 *single_byte_char_found to 1. */
5589 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
5590 unsigned char *p
, *pend
;
5591 Lisp_Object safe_codings
, work_table
;
5592 int *single_byte_char_found
;
5599 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
5601 if (ASCII_BYTE_P (c
))
5602 /* We can ignore ASCII characters here. */
5604 if (SINGLE_BYTE_CHAR_P (c
))
5605 *single_byte_char_found
= 1;
5606 if (NILP (safe_codings
))
5608 /* Check the safe coding systems for C. */
5609 val
= char_table_ref_and_index (work_table
, c
, &idx
);
5611 /* This element was already checked. Ignore it. */
5613 /* Remember that we checked this element. */
5614 CHAR_TABLE_SET (work_table
, idx
, Qt
);
5616 /* If there are some safe coding systems for C and we have
5617 already found the other set of coding systems for the
5618 different characters, get the intersection of them. */
5619 if (!EQ (safe_codings
, Qt
) && !NILP (val
))
5620 val
= intersection (safe_codings
, val
);
5623 return safe_codings
;
5627 /* Return a list of coding systems that safely encode the text between
5628 START and END. If the text contains only ASCII or is unibyte,
5631 DEFUN ("find-coding-systems-region-internal",
5632 Ffind_coding_systems_region_internal
,
5633 Sfind_coding_systems_region_internal
, 2, 2, 0,
5634 "Internal use only.")
5636 Lisp_Object start
, end
;
5638 Lisp_Object work_table
, safe_codings
;
5639 int non_ascii_p
= 0;
5640 int single_byte_char_found
= 0;
5641 unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
5642 Lisp_Object args
[2];
5644 if (STRINGP (start
))
5646 if (!STRING_MULTIBYTE (start
))
5648 p1
= XSTRING (start
)->data
, p1end
= p1
+ STRING_BYTES (XSTRING (start
));
5650 if (XSTRING (start
)->size
!= STRING_BYTES (XSTRING (start
)))
5657 CHECK_NUMBER_COERCE_MARKER (start
, 0);
5658 CHECK_NUMBER_COERCE_MARKER (end
, 1);
5659 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
5660 args_out_of_range (start
, end
);
5661 if (NILP (current_buffer
->enable_multibyte_characters
))
5663 from
= CHAR_TO_BYTE (XINT (start
));
5664 to
= CHAR_TO_BYTE (XINT (end
));
5665 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
5666 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
5670 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
5671 if (XINT (end
) - XINT (start
) != to
- from
)
5677 /* We are sure that the text contains no multibyte character.
5678 Check if it contains eight-bit-graphic. */
5680 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
5683 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
5689 /* The text contains non-ASCII characters. */
5690 work_table
= Fcopy_sequence (Vchar_coding_system_table
);
5691 safe_codings
= find_safe_codings (p1
, p1end
, Qt
, work_table
,
5692 &single_byte_char_found
);
5694 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
5695 &single_byte_char_found
);
5697 if (!single_byte_char_found
)
5699 /* Append generic coding systems. */
5700 Lisp_Object args
[2];
5701 args
[0] = safe_codings
;
5702 args
[1] = Fchar_table_extra_slot (Vchar_coding_system_table
,
5704 safe_codings
= Fappend (make_number (2), args
);
5707 safe_codings
= Fcons (Qraw_text
, Fcons (Qemacs_mule
, safe_codings
));
5708 return safe_codings
;
5713 code_convert_region1 (start
, end
, coding_system
, encodep
)
5714 Lisp_Object start
, end
, coding_system
;
5717 struct coding_system coding
;
5720 CHECK_NUMBER_COERCE_MARKER (start
, 0);
5721 CHECK_NUMBER_COERCE_MARKER (end
, 1);
5722 CHECK_SYMBOL (coding_system
, 2);
5724 validate_region (&start
, &end
);
5725 from
= XFASTINT (start
);
5726 to
= XFASTINT (end
);
5728 if (NILP (coding_system
))
5729 return make_number (to
- from
);
5731 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
5732 error ("Invalid coding system: %s", XSYMBOL (coding_system
)->name
->data
);
5734 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
5735 coding
.src_multibyte
= coding
.dst_multibyte
5736 = !NILP (current_buffer
->enable_multibyte_characters
);
5737 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
5738 &coding
, encodep
, 1);
5739 Vlast_coding_system_used
= coding
.symbol
;
5740 return make_number (coding
.produced_char
);
5743 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
5744 3, 3, "r\nzCoding system: ",
5745 "Decode the current region by specified coding system.\n\
5746 When called from a program, takes three arguments:\n\
5747 START, END, and CODING-SYSTEM. START and END are buffer positions.\n\
5748 This function sets `last-coding-system-used' to the precise coding system\n\
5749 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5750 not fully specified.)\n\
5751 It returns the length of the decoded text.")
5752 (start
, end
, coding_system
)
5753 Lisp_Object start
, end
, coding_system
;
5755 return code_convert_region1 (start
, end
, coding_system
, 0);
5758 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
5759 3, 3, "r\nzCoding system: ",
5760 "Encode the current region by specified coding system.\n\
5761 When called from a program, takes three arguments:\n\
5762 START, END, and CODING-SYSTEM. START and END are buffer positions.\n\
5763 This function sets `last-coding-system-used' to the precise coding system\n\
5764 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5765 not fully specified.)\n\
5766 It returns the length of the encoded text.")
5767 (start
, end
, coding_system
)
5768 Lisp_Object start
, end
, coding_system
;
5770 return code_convert_region1 (start
, end
, coding_system
, 1);
5774 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
5775 Lisp_Object string
, coding_system
, nocopy
;
5778 struct coding_system coding
;
5780 CHECK_STRING (string
, 0);
5781 CHECK_SYMBOL (coding_system
, 1);
5783 if (NILP (coding_system
))
5784 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
5786 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
5787 error ("Invalid coding system: %s", XSYMBOL (coding_system
)->name
->data
);
5789 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
5791 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
5792 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
5793 Vlast_coding_system_used
= coding
.symbol
;
5798 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
5800 "Decode STRING which is encoded in CODING-SYSTEM, and return the result.\n\
5801 Optional arg NOCOPY non-nil means it is ok to return STRING itself\n\
5802 if the decoding operation is trivial.\n\
5803 This function sets `last-coding-system-used' to the precise coding system\n\
5804 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5805 not fully specified.)")
5806 (string
, coding_system
, nocopy
)
5807 Lisp_Object string
, coding_system
, nocopy
;
5809 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
5812 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
5814 "Encode STRING to CODING-SYSTEM, and return the result.\n\
5815 Optional arg NOCOPY non-nil means it is ok to return STRING itself\n\
5816 if the encoding operation is trivial.\n\
5817 This function sets `last-coding-system-used' to the precise coding system\n\
5818 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is\n\
5819 not fully specified.)")
5820 (string
, coding_system
, nocopy
)
5821 Lisp_Object string
, coding_system
, nocopy
;
5823 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
5826 /* Encode or decode STRING according to CODING_SYSTEM.
5827 Do not set Vlast_coding_system_used.
5829 This function is called only from macros DECODE_FILE and
5830 ENCODE_FILE, thus we ignore character composition. */
5833 code_convert_string_norecord (string
, coding_system
, encodep
)
5834 Lisp_Object string
, coding_system
;
5837 struct coding_system coding
;
5839 CHECK_STRING (string
, 0);
5840 CHECK_SYMBOL (coding_system
, 1);
5842 if (NILP (coding_system
))
5845 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
5846 error ("Invalid coding system: %s", XSYMBOL (coding_system
)->name
->data
);
5848 coding
.composing
= COMPOSITION_DISABLED
;
5849 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
5851 ? encode_coding_string (string
, &coding
, 1)
5852 : decode_coding_string (string
, &coding
, 1));
5855 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
5856 "Decode a Japanese character which has CODE in shift_jis encoding.\n\
5857 Return the corresponding character.")
5861 unsigned char c1
, c2
, s1
, s2
;
5864 CHECK_NUMBER (code
, 0);
5865 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
5869 XSETFASTINT (val
, s2
);
5870 else if (s2
>= 0xA0 || s2
<= 0xDF)
5871 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
5873 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
5877 if ((s1
< 0x80 || s1
> 0x9F && s1
< 0xE0 || s1
> 0xEF)
5878 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
5879 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
5880 DECODE_SJIS (s1
, s2
, c1
, c2
);
5881 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
5886 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
5887 "Encode a Japanese character CHAR to shift_jis encoding.\n\
5888 Return the corresponding code in SJIS.")
5892 int charset
, c1
, c2
, s1
, s2
;
5895 CHECK_NUMBER (ch
, 0);
5896 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
5897 if (charset
== CHARSET_ASCII
)
5901 else if (charset
== charset_jisx0208
5902 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
5904 ENCODE_SJIS (c1
, c2
, s1
, s2
);
5905 XSETFASTINT (val
, (s1
<< 8) | s2
);
5907 else if (charset
== charset_katakana_jisx0201
5908 && c1
> 0x20 && c2
< 0xE0)
5910 XSETFASTINT (val
, c1
| 0x80);
5913 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
5917 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
5918 "Decode a Big5 character which has CODE in BIG5 coding system.\n\
5919 Return the corresponding character.")
5924 unsigned char b1
, b2
, c1
, c2
;
5927 CHECK_NUMBER (code
, 0);
5928 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
5932 error ("Invalid BIG5 code: %x", XFASTINT (code
));
5937 if ((b1
< 0xA1 || b1
> 0xFE)
5938 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
5939 error ("Invalid BIG5 code: %x", XFASTINT (code
));
5940 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
5941 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
5946 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
5947 "Encode the Big5 character CHAR to BIG5 coding system.\n\
5948 Return the corresponding character code in Big5.")
5952 int charset
, c1
, c2
, b1
, b2
;
5955 CHECK_NUMBER (ch
, 0);
5956 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
5957 if (charset
== CHARSET_ASCII
)
5961 else if ((charset
== charset_big5_1
5962 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
5963 || (charset
== charset_big5_2
5964 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
5966 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
5967 XSETFASTINT (val
, (b1
<< 8) | b2
);
5970 error ("Can't encode to Big5: %d", XFASTINT (ch
));
5974 DEFUN ("set-terminal-coding-system-internal",
5975 Fset_terminal_coding_system_internal
,
5976 Sset_terminal_coding_system_internal
, 1, 1, 0, "")
5978 Lisp_Object coding_system
;
5980 CHECK_SYMBOL (coding_system
, 0);
5981 setup_coding_system (Fcheck_coding_system (coding_system
), &terminal_coding
);
5982 /* We had better not send unsafe characters to terminal. */
5983 terminal_coding
.flags
|= CODING_FLAG_ISO_SAFE
;
5984 /* Characer composition should be disabled. */
5985 terminal_coding
.composing
= COMPOSITION_DISABLED
;
5986 terminal_coding
.src_multibyte
= 1;
5987 terminal_coding
.dst_multibyte
= 0;
5991 DEFUN ("set-safe-terminal-coding-system-internal",
5992 Fset_safe_terminal_coding_system_internal
,
5993 Sset_safe_terminal_coding_system_internal
, 1, 1, 0, "")
5995 Lisp_Object coding_system
;
5997 CHECK_SYMBOL (coding_system
, 0);
5998 setup_coding_system (Fcheck_coding_system (coding_system
),
5999 &safe_terminal_coding
);
6000 /* Characer composition should be disabled. */
6001 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
6002 safe_terminal_coding
.src_multibyte
= 1;
6003 safe_terminal_coding
.dst_multibyte
= 0;
6007 DEFUN ("terminal-coding-system",
6008 Fterminal_coding_system
, Sterminal_coding_system
, 0, 0, 0,
6009 "Return coding system specified for terminal output.")
6012 return terminal_coding
.symbol
;
6015 DEFUN ("set-keyboard-coding-system-internal",
6016 Fset_keyboard_coding_system_internal
,
6017 Sset_keyboard_coding_system_internal
, 1, 1, 0, "")
6019 Lisp_Object coding_system
;
6021 CHECK_SYMBOL (coding_system
, 0);
6022 setup_coding_system (Fcheck_coding_system (coding_system
), &keyboard_coding
);
6023 /* Characer composition should be disabled. */
6024 keyboard_coding
.composing
= COMPOSITION_DISABLED
;
6028 DEFUN ("keyboard-coding-system",
6029 Fkeyboard_coding_system
, Skeyboard_coding_system
, 0, 0, 0,
6030 "Return coding system specified for decoding keyboard input.")
6033 return keyboard_coding
.symbol
;
6037 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
6038 Sfind_operation_coding_system
, 1, MANY
, 0,
6039 "Choose a coding system for an operation based on the target name.\n\
6040 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).\n\
6041 DECODING-SYSTEM is the coding system to use for decoding\n\
6042 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system\n\
6043 for encoding (in case OPERATION does encoding).\n\
6045 The first argument OPERATION specifies an I/O primitive:\n\
6046 For file I/O, `insert-file-contents' or `write-region'.\n\
6047 For process I/O, `call-process', `call-process-region', or `start-process'.\n\
6048 For network I/O, `open-network-stream'.\n\
6050 The remaining arguments should be the same arguments that were passed\n\
6051 to the primitive. Depending on which primitive, one of those arguments\n\
6052 is selected as the TARGET. For example, if OPERATION does file I/O,\n\
6053 whichever argument specifies the file name is TARGET.\n\
6055 TARGET has a meaning which depends on OPERATION:\n\
6056 For file I/O, TARGET is a file name.\n\
6057 For process I/O, TARGET is a process name.\n\
6058 For network I/O, TARGET is a service name or a port number\n\
6060 This function looks up what specified for TARGET in,\n\
6061 `file-coding-system-alist', `process-coding-system-alist',\n\
6062 or `network-coding-system-alist' depending on OPERATION.\n\
6063 They may specify a coding system, a cons of coding systems,\n\
6064 or a function symbol to call.\n\
6065 In the last case, we call the function with one argument,\n\
6066 which is a list of all the arguments given to this function.")
6071 Lisp_Object operation
, target_idx
, target
, val
;
6072 register Lisp_Object chain
;
6075 error ("Too few arguments");
6076 operation
= args
[0];
6077 if (!SYMBOLP (operation
)
6078 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
6079 error ("Invalid first arguement");
6080 if (nargs
< 1 + XINT (target_idx
))
6081 error ("Too few arguments for operation: %s",
6082 XSYMBOL (operation
)->name
->data
);
6083 target
= args
[XINT (target_idx
) + 1];
6084 if (!(STRINGP (target
)
6085 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
6086 error ("Invalid %dth argument", XINT (target_idx
) + 1);
6088 chain
= ((EQ (operation
, Qinsert_file_contents
)
6089 || EQ (operation
, Qwrite_region
))
6090 ? Vfile_coding_system_alist
6091 : (EQ (operation
, Qopen_network_stream
)
6092 ? Vnetwork_coding_system_alist
6093 : Vprocess_coding_system_alist
));
6097 for (; CONSP (chain
); chain
= XCDR (chain
))
6103 && ((STRINGP (target
)
6104 && STRINGP (XCAR (elt
))
6105 && fast_string_match (XCAR (elt
), target
) >= 0)
6106 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
6109 /* Here, if VAL is both a valid coding system and a valid
6110 function symbol, we return VAL as a coding system. */
6113 if (! SYMBOLP (val
))
6115 if (! NILP (Fcoding_system_p (val
)))
6116 return Fcons (val
, val
);
6117 if (! NILP (Ffboundp (val
)))
6119 val
= call1 (val
, Flist (nargs
, args
));
6122 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
6123 return Fcons (val
, val
);
6131 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
6132 Supdate_coding_systems_internal
, 0, 0, 0,
6133 "Update internal database for ISO2022 and CCL based coding systems.\n\
6134 When values of any coding categories are changed, you must\n\
6135 call this function")
6140 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
6144 val
= XSYMBOL (XVECTOR (Vcoding_category_table
)->contents
[i
])->value
;
6147 if (! coding_system_table
[i
])
6148 coding_system_table
[i
] = ((struct coding_system
*)
6149 xmalloc (sizeof (struct coding_system
)));
6150 setup_coding_system (val
, coding_system_table
[i
]);
6152 else if (coding_system_table
[i
])
6154 xfree (coding_system_table
[i
]);
6155 coding_system_table
[i
] = NULL
;
6162 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
6163 Sset_coding_priority_internal
, 0, 0, 0,
6164 "Update internal database for the current value of `coding-category-list'.\n\
6165 This function is internal use only.")
6171 val
= Vcoding_category_list
;
6173 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
6175 if (! SYMBOLP (XCAR (val
)))
6177 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
6178 if (idx
>= CODING_CATEGORY_IDX_MAX
)
6180 coding_priorities
[i
++] = (1 << idx
);
6183 /* If coding-category-list is valid and contains all coding
6184 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
6185 the following code saves Emacs from crashing. */
6186 while (i
< CODING_CATEGORY_IDX_MAX
)
6187 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
6195 /*** 9. Post-amble ***/
6200 conversion_buffer
= (char *) xmalloc (MINIMUM_CONVERSION_BUFFER_SIZE
);
6208 /* Emacs' internal format specific initialize routine. */
6209 for (i
= 0; i
<= 0x20; i
++)
6210 emacs_code_class
[i
] = EMACS_control_code
;
6211 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
6212 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
6213 for (i
= 0x21 ; i
< 0x7F; i
++)
6214 emacs_code_class
[i
] = EMACS_ascii_code
;
6215 emacs_code_class
[0x7F] = EMACS_control_code
;
6216 for (i
= 0x80; i
< 0xFF; i
++)
6217 emacs_code_class
[i
] = EMACS_invalid_code
;
6218 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
6219 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
6220 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
6221 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
6223 /* ISO2022 specific initialize routine. */
6224 for (i
= 0; i
< 0x20; i
++)
6225 iso_code_class
[i
] = ISO_control_0
;
6226 for (i
= 0x21; i
< 0x7F; i
++)
6227 iso_code_class
[i
] = ISO_graphic_plane_0
;
6228 for (i
= 0x80; i
< 0xA0; i
++)
6229 iso_code_class
[i
] = ISO_control_1
;
6230 for (i
= 0xA1; i
< 0xFF; i
++)
6231 iso_code_class
[i
] = ISO_graphic_plane_1
;
6232 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
6233 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
6234 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
6235 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
6236 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
6237 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
6238 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
6239 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
6240 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
6241 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
6243 conversion_buffer_size
= MINIMUM_CONVERSION_BUFFER_SIZE
;
6245 setup_coding_system (Qnil
, &keyboard_coding
);
6246 setup_coding_system (Qnil
, &terminal_coding
);
6247 setup_coding_system (Qnil
, &safe_terminal_coding
);
6248 setup_coding_system (Qnil
, &default_buffer_file_coding
);
6250 bzero (coding_system_table
, sizeof coding_system_table
);
6252 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
6253 for (i
= 0; i
< 128; i
++)
6254 ascii_skip_code
[i
] = 1;
6256 #if defined (MSDOS) || defined (WINDOWSNT)
6257 system_eol_type
= CODING_EOL_CRLF
;
6259 system_eol_type
= CODING_EOL_LF
;
6262 inhibit_pre_post_conversion
= 0;
6270 Qtarget_idx
= intern ("target-idx");
6271 staticpro (&Qtarget_idx
);
6273 Qcoding_system_history
= intern ("coding-system-history");
6274 staticpro (&Qcoding_system_history
);
6275 Fset (Qcoding_system_history
, Qnil
);
6277 /* Target FILENAME is the first argument. */
6278 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
6279 /* Target FILENAME is the third argument. */
6280 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
6282 Qcall_process
= intern ("call-process");
6283 staticpro (&Qcall_process
);
6284 /* Target PROGRAM is the first argument. */
6285 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
6287 Qcall_process_region
= intern ("call-process-region");
6288 staticpro (&Qcall_process_region
);
6289 /* Target PROGRAM is the third argument. */
6290 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
6292 Qstart_process
= intern ("start-process");
6293 staticpro (&Qstart_process
);
6294 /* Target PROGRAM is the third argument. */
6295 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
6297 Qopen_network_stream
= intern ("open-network-stream");
6298 staticpro (&Qopen_network_stream
);
6299 /* Target SERVICE is the fourth argument. */
6300 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
6302 Qcoding_system
= intern ("coding-system");
6303 staticpro (&Qcoding_system
);
6305 Qeol_type
= intern ("eol-type");
6306 staticpro (&Qeol_type
);
6308 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
6309 staticpro (&Qbuffer_file_coding_system
);
6311 Qpost_read_conversion
= intern ("post-read-conversion");
6312 staticpro (&Qpost_read_conversion
);
6314 Qpre_write_conversion
= intern ("pre-write-conversion");
6315 staticpro (&Qpre_write_conversion
);
6317 Qno_conversion
= intern ("no-conversion");
6318 staticpro (&Qno_conversion
);
6320 Qundecided
= intern ("undecided");
6321 staticpro (&Qundecided
);
6323 Qcoding_system_p
= intern ("coding-system-p");
6324 staticpro (&Qcoding_system_p
);
6326 Qcoding_system_error
= intern ("coding-system-error");
6327 staticpro (&Qcoding_system_error
);
6329 Fput (Qcoding_system_error
, Qerror_conditions
,
6330 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
6331 Fput (Qcoding_system_error
, Qerror_message
,
6332 build_string ("Invalid coding system"));
6334 Qcoding_category
= intern ("coding-category");
6335 staticpro (&Qcoding_category
);
6336 Qcoding_category_index
= intern ("coding-category-index");
6337 staticpro (&Qcoding_category_index
);
6339 Vcoding_category_table
6340 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
6341 staticpro (&Vcoding_category_table
);
6344 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
6346 XVECTOR (Vcoding_category_table
)->contents
[i
]
6347 = intern (coding_category_name
[i
]);
6348 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
6349 Qcoding_category_index
, make_number (i
));
6353 Qtranslation_table
= intern ("translation-table");
6354 staticpro (&Qtranslation_table
);
6355 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (1));
6357 Qtranslation_table_id
= intern ("translation-table-id");
6358 staticpro (&Qtranslation_table_id
);
6360 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
6361 staticpro (&Qtranslation_table_for_decode
);
6363 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
6364 staticpro (&Qtranslation_table_for_encode
);
6366 Qsafe_chars
= intern ("safe-chars");
6367 staticpro (&Qsafe_chars
);
6369 Qchar_coding_system
= intern ("char-coding-system");
6370 staticpro (&Qchar_coding_system
);
6372 /* Intern this now in case it isn't already done.
6373 Setting this variable twice is harmless.
6374 But don't staticpro it here--that is done in alloc.c. */
6375 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
6376 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
6377 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (1));
6379 Qvalid_codes
= intern ("valid-codes");
6380 staticpro (&Qvalid_codes
);
6382 Qemacs_mule
= intern ("emacs-mule");
6383 staticpro (&Qemacs_mule
);
6385 Qraw_text
= intern ("raw-text");
6386 staticpro (&Qraw_text
);
6388 defsubr (&Scoding_system_p
);
6389 defsubr (&Sread_coding_system
);
6390 defsubr (&Sread_non_nil_coding_system
);
6391 defsubr (&Scheck_coding_system
);
6392 defsubr (&Sdetect_coding_region
);
6393 defsubr (&Sdetect_coding_string
);
6394 defsubr (&Sfind_coding_systems_region_internal
);
6395 defsubr (&Sdecode_coding_region
);
6396 defsubr (&Sencode_coding_region
);
6397 defsubr (&Sdecode_coding_string
);
6398 defsubr (&Sencode_coding_string
);
6399 defsubr (&Sdecode_sjis_char
);
6400 defsubr (&Sencode_sjis_char
);
6401 defsubr (&Sdecode_big5_char
);
6402 defsubr (&Sencode_big5_char
);
6403 defsubr (&Sset_terminal_coding_system_internal
);
6404 defsubr (&Sset_safe_terminal_coding_system_internal
);
6405 defsubr (&Sterminal_coding_system
);
6406 defsubr (&Sset_keyboard_coding_system_internal
);
6407 defsubr (&Skeyboard_coding_system
);
6408 defsubr (&Sfind_operation_coding_system
);
6409 defsubr (&Supdate_coding_systems_internal
);
6410 defsubr (&Sset_coding_priority_internal
);
6412 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
6413 "List of coding systems.\n\
6415 Do not alter the value of this variable manually. This variable should be\n\
6416 updated by the functions `make-coding-system' and\n\
6417 `define-coding-system-alias'.");
6418 Vcoding_system_list
= Qnil
;
6420 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
6421 "Alist of coding system names.\n\
6422 Each element is one element list of coding system name.\n\
6423 This variable is given to `completing-read' as TABLE argument.\n\
6425 Do not alter the value of this variable manually. This variable should be\n\
6426 updated by the functions `make-coding-system' and\n\
6427 `define-coding-system-alias'.");
6428 Vcoding_system_alist
= Qnil
;
6430 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
6431 "List of coding-categories (symbols) ordered by priority.");
6435 Vcoding_category_list
= Qnil
;
6436 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
6437 Vcoding_category_list
6438 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
6439 Vcoding_category_list
);
6442 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
6443 "Specify the coding system for read operations.\n\
6444 It is useful to bind this variable with `let', but do not set it globally.\n\
6445 If the value is a coding system, it is used for decoding on read operation.\n\
6446 If not, an appropriate element is used from one of the coding system alists:\n\
6447 There are three such tables, `file-coding-system-alist',\n\
6448 `process-coding-system-alist', and `network-coding-system-alist'.");
6449 Vcoding_system_for_read
= Qnil
;
6451 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
6452 "Specify the coding system for write operations.\n\
6453 Programs bind this variable with `let', but you should not set it globally.\n\
6454 If the value is a coding system, it is used for encoding of output,\n\
6455 when writing it to a file and when sending it to a file or subprocess.\n\
6457 If this does not specify a coding system, an appropriate element\n\
6458 is used from one of the coding system alists:\n\
6459 There are three such tables, `file-coding-system-alist',\n\
6460 `process-coding-system-alist', and `network-coding-system-alist'.\n\
6461 For output to files, if the above procedure does not specify a coding system,\n\
6462 the value of `buffer-file-coding-system' is used.");
6463 Vcoding_system_for_write
= Qnil
;
6465 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
6466 "Coding system used in the latest file or process I/O.");
6467 Vlast_coding_system_used
= Qnil
;
6469 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
6470 "*Non-nil means always inhibit code conversion of end-of-line format.\n\
6471 See info node `Coding Systems' and info node `Text and Binary' concerning\n\
6473 inhibit_eol_conversion
= 0;
6475 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
6476 "Non-nil means process buffer inherits coding system of process output.\n\
6477 Bind it to t if the process output is to be treated as if it were a file\n\
6478 read from some filesystem.");
6479 inherit_process_coding_system
= 0;
6481 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
6482 "Alist to decide a coding system to use for a file I/O operation.\n\
6483 The format is ((PATTERN . VAL) ...),\n\
6484 where PATTERN is a regular expression matching a file name,\n\
6485 VAL is a coding system, a cons of coding systems, or a function symbol.\n\
6486 If VAL is a coding system, it is used for both decoding and encoding\n\
6487 the file contents.\n\
6488 If VAL is a cons of coding systems, the car part is used for decoding,\n\
6489 and the cdr part is used for encoding.\n\
6490 If VAL is a function symbol, the function must return a coding system\n\
6491 or a cons of coding systems which are used as above.\n\
6493 See also the function `find-operation-coding-system'\n\
6494 and the variable `auto-coding-alist'.");
6495 Vfile_coding_system_alist
= Qnil
;
6497 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
6498 "Alist to decide a coding system to use for a process I/O operation.\n\
6499 The format is ((PATTERN . VAL) ...),\n\
6500 where PATTERN is a regular expression matching a program name,\n\
6501 VAL is a coding system, a cons of coding systems, or a function symbol.\n\
6502 If VAL is a coding system, it is used for both decoding what received\n\
6503 from the program and encoding what sent to the program.\n\
6504 If VAL is a cons of coding systems, the car part is used for decoding,\n\
6505 and the cdr part is used for encoding.\n\
6506 If VAL is a function symbol, the function must return a coding system\n\
6507 or a cons of coding systems which are used as above.\n\
6509 See also the function `find-operation-coding-system'.");
6510 Vprocess_coding_system_alist
= Qnil
;
6512 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
6513 "Alist to decide a coding system to use for a network I/O operation.\n\
6514 The format is ((PATTERN . VAL) ...),\n\
6515 where PATTERN is a regular expression matching a network service name\n\
6516 or is a port number to connect to,\n\
6517 VAL is a coding system, a cons of coding systems, or a function symbol.\n\
6518 If VAL is a coding system, it is used for both decoding what received\n\
6519 from the network stream and encoding what sent to the network stream.\n\
6520 If VAL is a cons of coding systems, the car part is used for decoding,\n\
6521 and the cdr part is used for encoding.\n\
6522 If VAL is a function symbol, the function must return a coding system\n\
6523 or a cons of coding systems which are used as above.\n\
6525 See also the function `find-operation-coding-system'.");
6526 Vnetwork_coding_system_alist
= Qnil
;
6528 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
6529 "Coding system to use with system messages.");
6530 Vlocale_coding_system
= Qnil
;
6532 /* The eol mnemonics are reset in startup.el system-dependently. */
6533 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
6534 "*String displayed in mode line for UNIX-like (LF) end-of-line format.");
6535 eol_mnemonic_unix
= build_string (":");
6537 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
6538 "*String displayed in mode line for DOS-like (CRLF) end-of-line format.");
6539 eol_mnemonic_dos
= build_string ("\\");
6541 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
6542 "*String displayed in mode line for MAC-like (CR) end-of-line format.");
6543 eol_mnemonic_mac
= build_string ("/");
6545 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
6546 "*String displayed in mode line when end-of-line format is not yet determined.");
6547 eol_mnemonic_undecided
= build_string (":");
6549 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
6550 "*Non-nil enables character translation while encoding and decoding.");
6551 Venable_character_translation
= Qt
;
6553 DEFVAR_LISP ("standard-translation-table-for-decode",
6554 &Vstandard_translation_table_for_decode
,
6555 "Table for translating characters while decoding.");
6556 Vstandard_translation_table_for_decode
= Qnil
;
6558 DEFVAR_LISP ("standard-translation-table-for-encode",
6559 &Vstandard_translation_table_for_encode
,
6560 "Table for translationg characters while encoding.");
6561 Vstandard_translation_table_for_encode
= Qnil
;
6563 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
6564 "Alist of charsets vs revision numbers.\n\
6565 While encoding, if a charset (car part of an element) is found,\n\
6566 designate it with the escape sequence identifing revision (cdr part of the element).");
6567 Vcharset_revision_alist
= Qnil
;
6569 DEFVAR_LISP ("default-process-coding-system",
6570 &Vdefault_process_coding_system
,
6571 "Cons of coding systems used for process I/O by default.\n\
6572 The car part is used for decoding a process output,\n\
6573 the cdr part is used for encoding a text to be sent to a process.");
6574 Vdefault_process_coding_system
= Qnil
;
6576 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
6577 "Table of extra Latin codes in the range 128..159 (inclusive).\n\
6578 This is a vector of length 256.\n\
6579 If Nth element is non-nil, the existence of code N in a file\n\
6580 \(or output of subprocess) doesn't prevent it to be detected as\n\
6581 a coding system of ISO 2022 variant which has a flag\n\
6582 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file\n\
6583 or reading output of a subprocess.\n\
6584 Only 128th through 159th elements has a meaning.");
6585 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
6587 DEFVAR_LISP ("select-safe-coding-system-function",
6588 &Vselect_safe_coding_system_function
,
6589 "Function to call to select safe coding system for encoding a text.\n\
6591 If set, this function is called to force a user to select a proper\n\
6592 coding system which can encode the text in the case that a default\n\
6593 coding system used in each operation can't encode the text.\n\
6595 The default value is `select-safe-coding-system' (which see).");
6596 Vselect_safe_coding_system_function
= Qnil
;
6598 DEFVAR_LISP ("char-coding-system-table", &Vchar_coding_system_table
,
6599 "Char-table containing safe coding systems of each characters.\n\
6600 Each element doesn't include such generic coding systems that can\n\
6601 encode any characters. They are in the first extra slot.");
6602 Vchar_coding_system_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
6604 DEFVAR_BOOL ("inhibit-iso-escape-detection",
6605 &inhibit_iso_escape_detection
,
6606 "If non-nil, Emacs ignores ISO2022's escape sequence on code detection.\n\
6608 By default, on reading a file, Emacs tries to detect how the text is\n\
6609 encoded. This code detection is sensitive to escape sequences. If\n\
6610 the sequence is valid as ISO2022, the code is determined as one of\n\
6611 the ISO2022 encodings, and the file is decoded by the corresponding\n\
6612 coding system (e.g. `iso-2022-7bit').\n\
6614 However, there may be a case that you want to read escape sequences in\n\
6615 a file as is. In such a case, you can set this variable to non-nil.\n\
6616 Then, as the code detection ignores any escape sequences, no file is\n\
6617 detected as encoded in some ISO2022 encoding. The result is that all\n\
6618 escape sequences become visible in a buffer.\n\
6620 The default value is nil, and it is strongly recommended not to change\n\
6621 it. That is because many Emacs Lisp source files that contain\n\
6622 non-ASCII characters are encoded by the coding system `iso-2022-7bit'\n\
6623 in Emacs's distribution, and they won't be decoded correctly on\n\
6624 reading if you suppress escape sequence detection.\n\
6626 The other way to read escape sequences in a file without decoding is\n\
6627 to explicitly specify some coding system that doesn't use ISO2022's\n\
6628 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument].");
6629 inhibit_iso_escape_detection
= 0;
6633 emacs_strerror (error_number
)
6638 synchronize_system_messages_locale ();
6639 str
= strerror (error_number
);
6641 if (! NILP (Vlocale_coding_system
))
6643 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
6644 Vlocale_coding_system
,
6646 str
= (char *) XSTRING (dec
)->data
;