@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
@c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2007, 2009,
-@c 2010, 2011 Free Software Foundation, Inc.
+@c 2010, 2011, 2013 Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@node Input and Output
* Port Types:: Types of port and how to make them.
* R6RS I/O Ports:: The R6RS port API.
* I/O Extensions:: Using and extending ports in C.
+* BOM Handling:: Handling of Unicode byte order marks.
@end menu
@end defvr
@deffn {Scheme Procedure} port-encoding port
-@deffnx {C Function} scm_port_encoding
+@deffnx {C Function} scm_port_encoding (port)
Returns, as a string, the character encoding that @var{port} uses to interpret
its input and output. The value @code{#f} is equivalent to @code{"ISO-8859-1"}.
@end deffn
created.
@end deffn
+@deffn {Scheme Variable} %default-port-conversion-strategy
+The fluid that defines the conversion strategy for newly created ports,
+and for other conversion routines such as @code{scm_to_stringn},
+@code{scm_from_stringn}, @code{string->pointer}, and
+@code{pointer->string}.
+
+Its value must be one of the symbols described above, with the same
+semantics: @code{'error}, @code{'substitute}, or @code{'escape}.
+
+When Guile starts, its value is @code{'substitute}.
+
+Note that @code{(set-port-conversion-strategy! #f @var{sym})} is
+equivalent to @code{(fluid-set! %default-port-conversion-strategy
+@var{sym})}.
+@end deffn
@node Reading
@deffn {Scheme Procedure} unread-char cobj [port]
@deffnx {C Function} scm_unread_char (cobj, port)
-Place @var{char} in @var{port} so that it will be read by the
+Place character @var{cobj} in @var{port} so that it will be read by the
next read operation. If called multiple times, the unread characters
will be read again in last-in first-out order. If @var{port} is
not supplied, the current input port is used.
@var{message} can contain @code{~A} (was @code{%s}) and
@code{~S} (was @code{%S}) escapes. When printed,
the escapes are replaced with corresponding members of
-@var{ARGS}:
+@var{args}:
@code{~A} formats using @code{display} and @code{~S} formats
using @code{write}.
If @var{destination} is @code{#t}, then use the current output
@deffn {Scheme Procedure} seek fd_port offset whence
@deffnx {C Function} scm_seek (fd_port, offset, whence)
-Sets the current position of @var{fd/port} to the integer
+Sets the current position of @var{fd_port} to the integer
@var{offset}, which is interpreted according to the value of
@var{whence}.
@defvar SEEK_END
Seek from the end of the file.
@end defvar
-If @var{fd/port} is a file descriptor, the underlying system
+If @var{fd_port} is a file descriptor, the underlying system
call is @code{lseek}. @var{port} may be a string port.
The value returned is the new position in the file. This means
@deffn {Scheme Procedure} ftell fd_port
@deffnx {C Function} scm_ftell (fd_port)
Return an integer representing the current position of
-@var{fd/port}, measured from the beginning. Equivalent to:
+@var{fd_port}, measured from the beginning. Equivalent to:
@lisp
(seek port 0 SEEK_CUR)
@end lisp
@end deffn
+In the past, Guile did not have a procedure that would just read out all
+of the characters from a port. As a workaround, many people just called
+@code{read-delimited} with no delimiters, knowing that would produce the
+behavior they wanted. This prompted Guile developers to add some
+routines that would read all characters from a port. So it is that
+@code{(ice-9 rdelim)} is also the home for procedures that can reading
+undelimited text:
+
+@deffn {Scheme Procedure} read-string [port] [count]
+Read all of the characters out of @var{port} and return them as a
+string. If the @var{count} is present, treat it as a limit to the
+number of characters to read.
+
+By default, read from the current input port, with no size limit on the
+result. This procedure always returns a string, even if no characters
+were read.
+@end deffn
+
+@deffn {Scheme Procedure} read-string! buf [port] [start] [end]
+Fill @var{buf} with characters read from @var{port}, defaulting to the
+current input port. Return the number of characters read.
+
+If @var{start} or @var{end} are specified, store data only into the
+substring of @var{str} bounded by @var{start} and @var{end} (which
+default to the beginning and end of the string, respectively).
+@end deffn
+
Some of the aforementioned I/O functions rely on the following C
primitives. These will mainly be of interest to people hacking Guile
internals.
Add line-buffering to the port. The port output buffer will be
automatically flushed whenever a newline character is written.
@item b
-Use binary mode. On DOS systems the default text mode converts CR+LF
-in the file to newline for the program, whereas binary mode reads and
-writes all bytes unchanged. On Unix-like systems there is no such
-distinction, text files already contain just newlines and no
-conversion is ever made. The @code{b} flag is accepted on all
-systems, but has no effect on Unix-like systems.
-
-(For reference, Guile leaves text versus binary up to the C library,
-@code{b} here just adds @code{O_BINARY} to the underlying @code{open}
-call, when that flag is available.)
-
-Also, open the file using the 8-bit character encoding "ISO-8859-1",
-ignoring any coding declaration or port encoding.
-
-Note that, when reading or writing binary data with ports, the
-bytevector ports in the @code{(rnrs io ports)} module are preferred,
-as they return vectors, and not strings (@pxref{R6RS I/O Ports}).
+Use binary mode, ensuring that each byte in the file will be read as one
+Scheme character.
+
+To provide this property, the file will be opened with the 8-bit
+character encoding "ISO-8859-1", ignoring the default port encoding.
+@xref{Ports}, for more information on port encodings.
+
+Note that while it is possible to read and write binary data as
+characters or strings, it is usually better to treat bytes as octets,
+and byte sequences as bytevectors. @xref{R6RS Binary Input}, and
+@ref{R6RS Binary Output}, for more.
+
+This option had another historical meaning, for DOS compatibility: in
+the default (textual) mode, DOS reads a CR-LF sequence as one LF byte.
+The @code{b} flag prevents this from happening, adding @code{O_BINARY}
+to the underlying @code{open} call. Still, the flag is generally useful
+because of its port encoding ramifications.
@end table
If a file cannot be opened with the access
requested, @code{open-file} throws an exception.
-When the file is opened, this procedure will scan for a coding
-declaration (@pxref{Character Encoding of Source Files}). If present
-will use that encoding for interpreting the file. Otherwise, the
-port's encoding will be used. To suppress this behavior, open
-the file in binary mode and then set the port encoding explicitly
-using @code{set-port-encoding!}.
+When the file is opened, its encoding is set to the current
+@code{%default-port-encoding}, unless the @code{b} flag was supplied.
+Sometimes it is desirable to honor Emacs-style coding declarations in
+files@footnote{Guile 2.0.0 to 2.0.7 would do this by default. This
+behavior was deemed inappropriate and disabled starting from Guile
+2.0.8.}. When that is the case, the @code{file-encoding} procedure can
+be used as follows (@pxref{Character Encoding of Source Files,
+@code{file-encoding}}):
+
+@example
+(let* ((port (open-input-file file))
+ (encoding (file-encoding port)))
+ (set-port-encoding! port (or encoding (port-encoding port))))
+@end example
In theory we could create read/write ports which were buffered
in one direction only. However this isn't included in the
The current port setting is managed with @code{dynamic-wind}, so the
previous value is restored no matter how @var{thunk} exits (eg.@: an
exception), and if @var{thunk} is re-entered (via a captured
-continuation) then it's set again to the @var{FILENAME} port.
+continuation) then it's set again to the @var{filename} port.
The port is closed when @var{thunk} returns normally, but not when
exited via an exception or new continuation. This ensures it's still
Some of the procedures described in this chapter accept a file name as an
argument. Valid values for such a file name include strings that name a file
-using the native notation of filesystem paths on an implementation's
+using the native notation of file system paths on an implementation's
underlying operating system, and may include implementation-dependent
values as well.
An exception with this type is raised when one of the operations for
textual output to a port encounters a character that cannot be
translated into bytes by the output direction of the port's transcoder.
-@var{Char} is the character that could not be encoded.
+@var{char} is the character that could not be encoded.
@end deffn
@deffn {Scheme Syntax} error-handling-mode @var{error-handling-mode-symbol}
raised.
@quotation Note
- Only the name of @var{error-handling-style-symbol} is significant.
+ Only the name of @var{error-handling-mode-symbol} is significant.
@end quotation
The error-handling mode of a transcoder specifies the behavior
symbol.
@var{eol-style} may be omitted, in which case it defaults to the native
-end-of-line style of the underlying platform. @var{Handling-mode} may
+end-of-line style of the underlying platform. @var{handling-mode} may
be omitted, in which case it defaults to @code{replace}. The result is
a transcoder with the behavior specified by its arguments.
@end deffn
@end deffn
@deffn {Scheme Procedure} textual-port? port
-Always return @var{#t}, as all ports can be used for textual I/O in
+Always return @code{#t}, as all ports can be used for textual I/O in
Guile.
@end deffn
-@deffn {Scheme Procedure} transcoded-port obj
+@deffn {Scheme Procedure} transcoded-port binary-port transcoder
The @code{transcoded-port} procedure
returns a new textual port with the specified @var{transcoder}.
Otherwise the new textual port's state is largely the same as
@node R6RS Input Ports
@subsubsection Input Ports
-@deffn {Scheme Procedure} input-port? obj@
+@deffn {Scheme Procedure} input-port? obj
Returns @code{#t} if the argument is an input port (or a combined input
and output port), and returns @code{#f} otherwise.
@end deffn
-@deffn {Scheme Procedure} port-eof? port
+@deffn {Scheme Procedure} port-eof? input-port
Returns @code{#t}
if the @code{lookahead-u8} procedure (if @var{input-port} is a binary port)
or the @code{lookahead-char} procedure (if @var{input-port} is a textual port)
@deffnx {Scheme Procedure} open-file-input-port filename file-options
@deffnx {Scheme Procedure} open-file-input-port filename file-options buffer-mode
@deffnx {Scheme Procedure} open-file-input-port filename file-options buffer-mode maybe-transcoder
-@var{Maybe-transcoder} must be either a transcoder or @code{#f}.
+@var{maybe-transcoder} must be either a transcoder or @code{#f}.
The @code{open-file-input-port} procedure returns an
input port for the named file. The @var{file-options} and
end-of-file.
Optionally, if @var{get-position} is not @code{#f}, it must be a thunk
-that will be called when @var{port-position} is invoked on the custom
+that will be called when @code{port-position} is invoked on the custom
binary port and should return an integer indicating the position within
the underlying data stream; if @var{get-position} was not supplied, the
-returned port does not support @var{port-position}.
+returned port does not support @code{port-position}.
Likewise, if @var{set-position!} is not @code{#f}, it should be a
-one-argument procedure. When @var{set-port-position!} is invoked on the
+one-argument procedure. When @code{set-port-position!} is invoked on the
custom binary input port, @var{set-position!} is passed an integer
indicating the position of the next byte is to read.
@deffn {Scheme Procedure} get-bytevector-some port
@deffnx {C Function} scm_get_bytevector_some (port)
-Read from @var{port}, blocking as necessary, until data are available or
-and end-of-file is reached. Return either a new bytevector containing
-the data read or the end-of-file object.
+Read from @var{port}, blocking as necessary, until bytes are available
+or an end-of-file is reached. Return either the end-of-file object or a
+new bytevector containing some of the available bytes (at least one),
+and update the port position to point just past these bytes.
@end deffn
@deffn {Scheme Procedure} get-bytevector-all port
@node R6RS Textual Input
@subsubsection Textual Input
-@deffn {Scheme Procedure} get-char port
+@deffn {Scheme Procedure} get-char textual-input-port
Reads from @var{textual-input-port}, blocking as necessary, until a
complete character is available from @var{textual-input-port},
or until an end of file is reached.
character is read, @code{get-char} returns the end-of-file object.
@end deffn
-@deffn {Scheme Procedure} lookahead-char port
+@deffn {Scheme Procedure} lookahead-char textual-input-port
The @code{lookahead-char} procedure is like @code{get-char}, but it does
not update @var{textual-input-port} to point past the character.
@end deffn
-@deffn {Scheme Procedure} get-string-n port count
+@deffn {Scheme Procedure} get-string-n textual-input-port count
-@var{Count} must be an exact, non-negative integer object, representing
+@var{count} must be an exact, non-negative integer object, representing
the number of characters to be read.
The @code{get-string-n} procedure reads from @var{textual-input-port},
before an end of file, the end-of-file object is returned.
@end deffn
-@deffn {Scheme Procedure} get-string-n! port string start count
+@deffn {Scheme Procedure} get-string-n! textual-input-port string start count
-@var{Start} and @var{count} must be exact, non-negative integer objects,
+@var{start} and @var{count} must be exact, non-negative integer objects,
with @var{count} representing the number of characters to be read.
-@var{String} must be a string with at least $@var{start} + @var{count}$
+@var{string} must be a string with at least $@var{start} + @var{count}$
characters.
The @code{get-string-n!} procedure reads from @var{textual-input-port}
file, the end-of-file object is returned.
@end deffn
-@deffn {Scheme Procedure} get-string-all port count
+@deffn {Scheme Procedure} get-string-all textual-input-port count
Reads from @var{textual-input-port} until an end of file, decoding
characters in the same manner as @code{get-string-n} and
@code{get-string-n!}.
precedes the end of file, the end-of-file object is returned.
@end deffn
-@deffn {Scheme Procedure} get-line port
+@deffn {Scheme Procedure} get-line textual-input-port
Reads from @var{textual-input-port} up to and including the linefeed
character or end of file, decoding characters in the same manner as
@code{get-string-n} and @code{get-string-n!}.
@end quotation
@end deffn
-@deffn {Scheme Procedure} get-datum port count
+@deffn {Scheme Procedure} get-datum textual-input-port count
Reads an external representation from @var{textual-input-port} and returns the
datum it represents. The @code{get-datum} procedure returns the next
datum that can be parsed from the given @var{textual-input-port}, updating
@code{put-string} procedure returns an unspecified value.
@end deffn
-@deffn {Scheme Procedure} put-datum port datum
+@deffn {Scheme Procedure} put-datum textual-output-port datum
@var{datum} should be a datum value. The @code{put-datum} procedure
writes an external representation of @var{datum} to
@var{textual-output-port}. The specific external representation is
@end table
+@node BOM Handling
+@subsection Handling of Unicode byte order marks.
+@cindex BOM
+@cindex byte order mark
+
+This section documents the finer points of Guile's handling of Unicode
+byte order marks (BOMs). A byte order mark (U+FEFF) is typically found
+at the start of a UTF-16 or UTF-32 stream, to allow readers to reliably
+determine the byte order. Occasionally, a BOM is found at the start of
+a UTF-8 stream, but this is much less common and not generally
+recommended.
+
+Guile attempts to handle BOMs automatically, and in accordance with the
+recommendations of the Unicode Standard, when the port encoding is set
+to @code{UTF-8}, @code{UTF-16}, or @code{UTF-32}. In brief, Guile
+automatically writes a BOM at the start of a UTF-16 or UTF-32 stream,
+and automatically consumes one from the start of a UTF-8, UTF-16, or
+UTF-32 stream.
+
+As specified in the Unicode Standard, a BOM is only handled specially at
+the start of a stream, and only if the port encoding is set to
+@code{UTF-8}, @code{UTF-16} or @code{UTF-32}. If the port encoding is
+set to @code{UTF-16BE}, @code{UTF-16LE}, @code{UTF-32BE}, or
+@code{UTF-32LE}, then BOMs are @emph{not} handled specially, and none of
+the special handling described in this section applies.
+
+@itemize @bullet
+@item
+To ensure that Guile will properly detect the byte order of a UTF-16 or
+UTF-32 stream, you must perform a textual read before any writes, seeks,
+or binary I/O. Guile will not attempt to read a BOM unless a read is
+explicitly requested at the start of the stream.
+
+@item
+If a textual write is performed before the first read, then an arbitrary
+byte order will be chosen. Currently, big endian is the default on all
+platforms, but that may change in the future. If you wish to explicitly
+control the byte order of an output stream, set the port encoding to
+@code{UTF-16BE}, @code{UTF-16LE}, @code{UTF-32BE}, or @code{UTF-32LE},
+and explicitly write a BOM (@code{#\xFEFF}) if desired.
+
+@item
+If @code{set-port-encoding!} is called in the middle of a stream, Guile
+treats this as a new logical ``start of stream'' for purposes of BOM
+handling, and will forget about any BOMs that had previously been seen.
+Therefore, it may choose a different byte order than had been used
+previously. This is intended to support multiple logical text streams
+embedded within a larger binary stream.
+
+@item
+Binary I/O operations are not guaranteed to update Guile's notion of
+whether the port is at the ``start of the stream'', nor are they
+guaranteed to produce or consume BOMs.
+
+@item
+For ports that support seeking (e.g. normal files), the input and output
+streams are considered linked: if the user reads first, then a BOM will
+be consumed (if appropriate), but later writes will @emph{not} produce a
+BOM. Similarly, if the user writes first, then later reads will
+@emph{not} consume a BOM.
+
+@item
+For ports that do not support seeking (e.g. pipes, sockets, and
+terminals), the input and output streams are considered
+@emph{independent} for purposes of BOM handling: the first read will
+consume a BOM (if appropriate), and the first write will @emph{also}
+produce a BOM (if appropriate). However, the input and output streams
+will always use the same byte order.
+
+@item
+Seeks to the beginning of a file will set the ``start of stream'' flags.
+Therefore, a subsequent textual read or write will consume or produce a
+BOM. However, unlike @code{set-port-encoding!}, if a byte order had
+already been chosen for the port, it will remain in effect after a seek,
+and cannot be changed by the presence of a BOM. Seeks anywhere other
+than the beginning of a file clear the ``start of stream'' flags.
+@end itemize
+
@c Local Variables:
@c TeX-master: "guile.texi"
@c End:
HCoop / bpt / guile.git / blobdiff