@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
-@c Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+@c Copyright (C) 2010, 2011, 2012, 2013 Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@node Web
* URIs:: Universal Resource Identifiers.
* HTTP:: The Hyper-Text Transfer Protocol.
* HTTP Headers:: How Guile represents specific header values.
+* Transfer Codings:: HTTP Transfer Codings.
* Requests:: HTTP requests.
* Responses:: HTTP responses.
+* Web Client:: Accessing web resources over HTTP.
* Web Server:: Serving HTTP to the internet.
* Web Examples:: How to use this thing.
@end menu
For example, in the URI, @indicateurl{http://www.gnu.org/help/}, the
scheme is @code{http}, the host is @code{www.gnu.org}, the path is
-@code{/help/}, and there is no userinfo, port, query, or path. All URIs
-have a scheme and a path (though the path might be empty). Some URIs
-have a host, and some of those have ports and userinfo. Any URI might
-have a query part or a fragment.
+@code{/help/}, and there is no userinfo, port, query, or fragment. All
+URIs have a scheme and a path (though the path might be empty). Some
+URIs have a host, and some of those have ports and userinfo. Any URI
+might have a query part or a fragment.
Userinfo is something of an abstraction, as some legacy URI schemes
allowed userinfo of the form @code{@var{username}:@var{passwd}}. But
@deffn {Scheme Procedure} build-uri scheme [#:userinfo=@code{#f}] [#:host=@code{#f}] @
[#:port=@code{#f}] [#:path=@code{""}] [#:query=@code{#f}] @
[#:fragment=@code{#f}] [#:validate?=@code{#t}]
-Construct a URI object. @var{scheme} should be a symbol, and the rest
-of the fields are either strings or @code{#f}. If @var{validate?} is
-true, also run some consistency checks to make sure that the constructed
-URI is valid.
+Construct a URI object. @var{scheme} should be a symbol, @var{port}
+either a positive, exact integer or @code{#f}, and the rest of the
+fields are either strings or @code{#f}. If @var{validate?} is true,
+also run some consistency checks to make sure that the constructed URI
+is valid.
@end deffn
@deffn {Scheme Procedure} uri? x
@deffnx {Scheme Procedure} uri-query uri
@deffnx {Scheme Procedure} uri-fragment uri
A predicate and field accessors for the URI record type. The URI scheme
-will be a symbol, and the rest either strings or @code{#f} if not
-present.
+will be a symbol, the port either a positive, exact integer or @code{#f},
+and the rest either strings or @code{#f} if not present.
@end deffn
@deffn {Scheme Procedure} string->uri string
elements of the HTTP protocol.
If you are want to skip the low-level details for now and move on to web
-pages, @pxref{Web Server}. Otherwise, load the HTTP module, and read
-on.
+pages, @pxref{Web Client}, and @pxref{Web Server}. Otherwise, load the
+HTTP module, and read on.
@example
(use-modules (web http))
(display (inet-ntoa ip) port)))
@end example
+@deffn {Scheme Procedure} declare-opaque-header! name
+A specialised version of @code{declare-header!} for the case in which
+you want a header's value to be returned/written ``as-is''.
+@end deffn
+
@deffn {Scheme Procedure} valid-header? sym val
Return a true value iff @var{val} is a valid Scheme value for the header
with name @var{sym}.
@end deffn
@deffn {Scheme Procedure} read-headers port
-Read the headers of an HTTP message from @var{port}, returning the
-headers as an ordered alist.
+Read the headers of an HTTP message from @var{port}, returning them
+as an ordered alist.
@end deffn
@deffn {Scheme Procedure} write-headers headers port
A list of content codings, as symbols.
@example
(parse-header 'content-encoding "gzip")
-@result{} (GET HEAD)
+@result{} (gzip)
@end example
@end deftypevr
Indicates that a response should proceed if and only if the resource has
been modified since the given date.
@example
-(parse-header if-modified-since "Tue, 15 Nov 1994 08:12:31 GMT")
+(parse-header 'if-modified-since "Tue, 15 Nov 1994 08:12:31 GMT")
@result{} #<date ...>
@end example
@end deftypevr
@end example
@end deftypevr
+@node Transfer Codings
+@subsection Transfer Codings
+
+HTTP 1.1 allows for various transfer codings to be applied to message
+bodies. These include various types of compression, and HTTP chunked
+encoding. Currently, only chunked encoding is supported by guile.
+
+Chunked coding is an optional coding that may be applied to message
+bodies, to allow messages whose length is not known beforehand to be
+returned. Such messages can be split into chunks, terminated by a final
+zero length chunk.
+
+In order to make dealing with encodings more simple, guile provides
+procedures to create ports that ``wrap'' existing ports, applying
+transformations transparently under the hood.
+
+These procedures are in the @code{(web http)} module.
+
+@example
+(use-modules (web http))
+@end example
+
+@deffn {Scheme Procedure} make-chunked-input-port port [#:keep-alive?=#f]
+Returns a new port, that transparently reads and decodes chunk-encoded
+data from @var{port}. If no more chunk-encoded data is available, it
+returns the end-of-file object. When the port is closed, @var{port} will
+also be closed, unless @var{keep-alive?} is true.
+@end deffn
+
+@example
+(use-modules (ice-9 rdelim))
+
+(define s "5\r\nFirst\r\nA\r\n line\n Sec\r\n8\r\nond line\r\n0\r\n")
+(define p (make-chunked-input-port (open-input-string s)))
+(read-line s)
+@result{} "First line"
+(read-line s)
+@result{} "Second line"
+@end example
+
+@deffn {Scheme Procedure} make-chunked-output-port port [#:keep-alive?=#f]
+Returns a new port, which transparently encodes data as chunk-encoded
+before writing it to @var{port}. Whenever a write occurs on this port,
+it buffers it, until the port is flushed, at which point it writes a
+chunk containing all the data written so far. When the port is closed,
+the data remaining is written to @var{port}, as is the terminating zero
+chunk. It also causes @var{port} to be closed, unless @var{keep-alive?}
+is true.
+
+Note. Forcing a chunked output port when there is no data is buffered
+does not write a zero chunk, as this would cause the data to be
+interpreted incorrectly by the client.
+@end deffn
+
+@example
+(call-with-output-string
+ (lambda (out)
+ (define out* (make-chunked-output-port out #:keep-alive? #t))
+ (display "first chunk" out*)
+ (force-output out*)
+ (force-output out*) ; note this does not write a zero chunk
+ (display "second chunk" out*)
+ (close-port out*)))
+@result{} "b\r\nfirst chunk\r\nc\r\nsecond chunk\r\n0\r\n"
+@end example
@node Requests
@subsection HTTP Requests
@end deffn
@deffn {Scheme Procedure} write-request-body r bv
-Write @var{body}, a bytevector, to the port corresponding to the HTTP
+Write @var{bv}, a bytevector, to the port corresponding to the HTTP
request @var{r}.
@end deffn
discussion of character sets in @ref{Responses}, for more information.
@end deffn
-@deffn {Scheme Procedure} build-response [#:version='(1 . 1)] [#:code=200] [#:reason-phrase=#f] [#:headers='()] [#:port=#f] [#:validate-headers=#t]
+@deffn {Scheme Procedure} build-response [#:version='(1 . 1)] [#:code=200] [#:reason-phrase=#f] [#:headers='()] [#:port=#f] [#:validate-headers?=#t]
Construct an HTTP response object. If @var{validate-headers?} is true,
the headers are each run through their respective validators.
@end deffn
on @var{port}, perhaps using some transfer encoding.
@end deffn
+@deffn {Scheme Procedure} response-must-not-include-body? r
+Some responses, like those with status code 304, are specified as never
+having bodies. This predicate returns @code{#t} for those responses.
+
+Note also, though, that responses to @code{HEAD} requests must also not
+have a body.
+@end deffn
+
+@deffn {Scheme Procedure} response-body-port r [#:decode?=#t] [#:keep-alive?=#t]
+Return an input port from which the body of @var{r} can be read. The encoding
+of the returned port is set according to @var{r}'s @code{content-type} header,
+when it's textual, except if @var{decode?} is @code{#f}. Return @code{#f}
+when no body is available.
+
+When @var{keep-alive?} is @code{#f}, closing the returned port also closes
+@var{r}'s response port.
+@end deffn
+
@deffn {Scheme Procedure} read-response-body r
Read the response body from @var{r}, as a bytevector. Returns @code{#f}
if there was no response body.
@end deffn
@deffn {Scheme Procedure} write-response-body r bv
-Write @var{body}, a bytevector, to the port corresponding to the HTTP
+Write @var{bv}, a bytevector, to the port corresponding to the HTTP
response @var{r}.
@end deffn
Return the given response header, or @var{default} if none was present.
@end deffn
+@deffn {Scheme Procedure} text-content-type? @var{type}
+Return @code{#t} if @var{type}, a symbol as returned by
+@code{response-content-type}, represents a textual type such as
+@code{text/plain}.
+@end deffn
+
+
+@node Web Client
+@subsection Web Client
+
+@code{(web client)} provides a simple, synchronous HTTP client, built on
+the lower-level HTTP, request, and response modules.
+
+@deffn {Scheme Procedure} open-socket-for-uri uri
+Return an open input/output port for a connection to URI.
+@end deffn
+
+@deffn {Scheme Procedure} http-get uri arg...
+@deffnx {Scheme Procedure} http-head uri arg...
+@deffnx {Scheme Procedure} http-post uri arg...
+@deffnx {Scheme Procedure} http-put uri arg...
+@deffnx {Scheme Procedure} http-delete uri arg...
+@deffnx {Scheme Procedure} http-trace uri arg...
+@deffnx {Scheme Procedure} http-options uri arg...
+
+Connect to the server corresponding to @var{uri} and make a request over
+HTTP, using the appropriate method (@code{GET}, @code{HEAD}, etc.).
+
+All of these procedures have the same prototype: a URI followed by an
+optional sequence of keyword arguments. These keyword arguments allow
+you to modify the requests in various ways, for example attaching a body
+to the request, or setting specific headers. The following table lists
+the keyword arguments and their default values.
+
+@table @code
+@item #:body #f
+@item #:port (open-socket-for-uri @var{uri})]
+@item #:version '(1 . 1)
+@item #:keep-alive? #f
+@item #:headers '()
+@item #:decode-body? #t
+@item #:streaming? #f
+@end table
+
+If you already have a port open, pass it as @var{port}. Otherwise, a
+connection will be opened to the server corresponding to @var{uri}. Any
+extra headers in the alist @var{headers} will be added to the request.
+
+If @var{body} is not #f, a message body will also be sent with the HTTP
+request. If @var{body} is a string, it is encoded according to the
+content-type in @var{headers}, defaulting to UTF-8. Otherwise
+@var{body} should be a bytevector, or @code{#f} for no body. Although a
+message body may be sent with any request, usually only @code{POST} and
+@code{PUT} requests have bodies.
+
+If @var{decode-body?} is true, as is the default, the body of the
+response will be decoded to string, if it is a textual content-type.
+Otherwise it will be returned as a bytevector.
+
+However, if @var{streaming?} is true, instead of eagerly reading the
+response body from the server, this function only reads off the headers.
+The response body will be returned as a port on which the data may be
+read.
+
+Unless @var{keep-alive?} is true, the port will be closed after the full
+response body has been read.
+
+Returns two values: the response read from the server, and the response
+body as a string, bytevector, #f value, or as a port (if
+@var{streaming?} is true).
+@end deffn
+
+@code{http-get} is useful for making one-off requests to web sites. If
+you are writing a web spider or some other client that needs to handle a
+number of requests in parallel, it's better to build an event-driven URL
+fetcher, similar in structure to the web server (@pxref{Web Server}).
+
+Another option, good but not as performant, would be to use threads,
+possibly via par-map or futures.
+
@node Web Server
@subsection Web Server
socket, request, and body.
@item
-A user-provided handler procedure is called, with the request
-and body as its arguments. The handler should return two
-values: the response, as a @code{<response>} record from @code{(web
-response)}, and the response body as a string, bytevector, or
-@code{#f} if not present. We also allow the response to be simply an
-alist of headers, in which case a default response object is
-constructed with those headers.
+A user-provided handler procedure is called, with the request and body
+as its arguments. The handler should return two values: the response,
+as a @code{<response>} record from @code{(web response)}, and the
+response body as bytevector, or @code{#f} if not present.
+
+The respose and response body are run through @code{sanitize-response},
+documented below. This allows the handler writer to take some
+convenient shortcuts: for example, instead of a @code{<response>}, the
+handler can simply return an alist of headers, in which case a default
+response object is constructed with those headers. Instead of a
+bytevector for the body, the handler can return a string, which will be
+serialized into an appropriate encoding; or it can return a procedure,
+which will be called on a port to write out the data. See the
+@code{sanitize-response} documentation, for more.
@item
The @code{write} hook is called with three arguments: the client
A user may define a server implementation with the following form:
-@deffn {Scheme Procedure} define-server-impl name open read write close
+@deffn {Scheme Syntax} define-server-impl name open read write close
Make a @code{<server-impl>} object with the hooks @var{open},
@var{read}, @var{write}, and @var{close}, and bind it to the symbol
@var{name} in the current module.
@end deffn
@deffn {Scheme Procedure} lookup-server-impl impl
-Look up a server implementation. If @var{impl} is a server
-implementation already, it is returned directly. If it is a symbol, the
+Look up a server implementation. If @var{impl} is a server
+implementation already, it is returned directly. If it is a symbol, the
binding named @var{impl} in the @code{(web server @var{impl})} module is
-looked up. Otherwise an error is signaled.
+looked up. Otherwise an error is signaled.
Currently a server implementation is a somewhat opaque type, useful only
for passing to other procedures in this module, like @code{read-client}.
@deffn {Scheme Procedure} open-server impl open-params
Open a server for the given implementation. Return one value, the new
-server object. The implementation's @code{open} procedure is applied to
+server object. The implementation's @code{open} procedure is applied to
@var{open-params}, which should be a list.
@end deffn
Read a new client from @var{server}, by applying the implementation's
@code{read} procedure to the server. If successful, return three
values: an object corresponding to the client, a request object, and the
-request body. If any exception occurs, return @code{#f} for all three
+request body. If any exception occurs, return @code{#f} for all three
values.
@end deffn
@var{handler} with @var{request} and @var{body} as arguments.
The elements of @var{state} are also passed to @var{handler} as
-arguments, and may be returned as additional values. The new
+arguments, and may be returned as additional values. The new
@var{state}, collected from the @var{handler}'s return values, is then
-returned as a list. The idea is that a server loop receives a handler
+returned as a list. The idea is that a server loop receives a handler
from the user, along with whatever state values the user is interested
in, allowing the user's handler to explicitly manage its state.
@end deffn
As a convenience to web handler authors, @var{response} may be given as
an alist of headers, in which case it is used to construct a default
-response. Ensures that the response version corresponds to the request
-version. If @var{body} is a string, encodes the string to a bytevector,
-in an encoding appropriate for @var{response}. Adds a
+response. Ensures that the response version corresponds to the request
+version. If @var{body} is a string, encodes the string to a bytevector,
+in an encoding appropriate for @var{response}. Adds a
@code{content-length} and @code{content-type} header, as necessary.
If @var{body} is a procedure, it is called with a port as an argument,
-and the output collected as a bytevector. In the future we might try to
+and the output collected as a bytevector. In the future we might try to
instead use a compressing, chunk-encoded port, and call this procedure
-later, in the write-client procedure. Authors are advised not to rely on
+later, in the write-client procedure. Authors are advised not to rely on
the procedure being called at any particular time.
@end deffn
@deffn {Scheme Procedure} write-client impl server client response body
-Write an HTTP response and body to @var{client}. If the server and
+Write an HTTP response and body to @var{client}. If the server and
client support persistent connections, it is the implementation's
responsibility to keep track of the client thereafter, presumably by
attaching it to the @var{server} argument somehow.
Additional arguments to @var{handler} are taken from @var{state}.
Additional return values are accumulated into a new @var{state}, which
-will be used for subsequent requests. In this way a handler can
+will be used for subsequent requests. In this way a handler can
explicitly manage its state.
@end deffn
HCoop / bpt / guile.git / blobdiff