@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
-@c Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+@c Copyright (C) 2010, 2011, 2012, 2013, 2015 Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@node Web
@cindex HTTP
It has always been possible to connect computers together and share
-information between them, but the rise of the World-Wide Web over the
+information between them, but the rise of the World Wide Web over the
last couple of decades has made it much easier to do so. The result is
a richly connected network of computation, in which Guile forms a part.
* URIs:: Universal Resource Identifiers.
* HTTP:: The Hyper-Text Transfer Protocol.
* HTTP Headers:: How Guile represents specific header values.
+* Transfer Codings:: HTTP Transfer Codings.
* Requests:: HTTP requests.
* Responses:: HTTP responses.
+* Web Client:: Accessing web resources over HTTP.
* Web Server:: Serving HTTP to the internet.
* Web Examples:: How to use this thing.
@end menu
dates, occurs on the boundary in which we produce a SRFI 19 date record
from other types, like strings.
-With regards to the web, data types are help in the two broad phases of
-HTTP messages: parsing and generation.
+With regards to the web, data types are helpful in the two broad phases
+of HTTP messages: parsing and generation.
Consider a server, which has to parse a request, and produce a response.
Guile will parse the request into an HTTP request object
element, represented as a tagged list. So @samp{foo} becomes
@samp{"foo"}, and @samp{<b>foo</b>} becomes @samp{(b "foo")}.
Attributes, if present, go in a tagged list headed by @samp{@@}, like
-@samp{(img (@@ (src "http://example.com/foo.png")))}. @xref{sxml
-simple}, for more information.
+@samp{(img (@@ (src "http://example.com/foo.png")))}. @xref{SXML}, for
+more information.
The good thing about SXML is that HTML elements cannot be confused with
text. Let's make a new definition of @code{para}:
For example, in the URI, @indicateurl{http://www.gnu.org/help/}, the
scheme is @code{http}, the host is @code{www.gnu.org}, the path is
-@code{/help/}, and there is no userinfo, port, query, or path. All URIs
-have a scheme and a path (though the path might be empty). Some URIs
-have a host, and some of those have ports and userinfo. Any URI might
-have a query part or a fragment.
+@code{/help/}, and there is no userinfo, port, query, or fragment. All
+URIs have a scheme and a path (though the path might be empty). Some
+URIs have a host, and some of those have ports and userinfo. Any URI
+might have a query part or a fragment.
+
+There is also a ``URI-reference'' data type, which is the same as a URI
+but where the scheme is optional. In this case, the scheme is taken to
+be relative to some other related URI. A common use of URI references
+is when you want to be vague regarding the choice of HTTP or HTTPS --
+serving a web page referring to @code{/foo.css} will use HTTPS if loaded
+over HTTPS, or HTTP otherwise.
Userinfo is something of an abstraction, as some legacy URI schemes
allowed userinfo of the form @code{@var{username}:@var{passwd}}. But
module. Load it into your Guile, using a form like the above, to have
access to them.
-@defun build-uri scheme [#:userinfo=@code{#f}] [#:host=@code{#f}] @
- [#:port=@code{#f}] [#:path=@code{""}] [#:query=@code{#f}] @
- [#:fragment=@code{#f}] [#:validate?=@code{#t}]
-Construct a URI object. @var{scheme} should be a symbol, and the rest
-of the fields are either strings or @code{#f}. If @var{validate?} is
-true, also run some consistency checks to make sure that the constructed
-URI is valid.
-@end defun
-
-@defun uri? x
-@defunx uri-scheme uri
-@defunx uri-userinfo uri
-@defunx uri-host uri
-@defunx uri-port uri
-@defunx uri-path uri
-@defunx uri-query uri
-@defunx uri-fragment uri
+@deffn {Scheme Procedure} build-uri scheme @
+ [#:userinfo=@code{#f}] [#:host=@code{#f}] [#:port=@code{#f}] @
+ [#:path=@code{""}] [#:query=@code{#f}] [#:fragment=@code{#f}] @
+ [#:validate?=@code{#t}]
+Construct a URI object. @var{scheme} should be a symbol, @var{port}
+either a positive, exact integer or @code{#f}, and the rest of the
+fields are either strings or @code{#f}. If @var{validate?} is true,
+also run some consistency checks to make sure that the constructed URI
+is valid.
+@end deffn
+
+@deffn {Scheme Procedure} build-uri-reference [#:scheme=@code{#f}]@
+ [#:userinfo=@code{#f}] [#:host=@code{#f}] [#:port=@code{#f}] @
+ [#:path=@code{""}] [#:query=@code{#f}] [#:fragment=@code{#f}] @
+ [#:validate?=@code{#t}]
+Like @code{build-uri}, but with an optional scheme.
+@end deffn
+
+In Guile, both URI and URI reference data types are represented in the
+same way, as URI objects.
+
+@deffn {Scheme Procedure} uri? obj
+@deffnx {Scheme Procedure} uri-scheme uri
+@deffnx {Scheme Procedure} uri-userinfo uri
+@deffnx {Scheme Procedure} uri-host uri
+@deffnx {Scheme Procedure} uri-port uri
+@deffnx {Scheme Procedure} uri-path uri
+@deffnx {Scheme Procedure} uri-query uri
+@deffnx {Scheme Procedure} uri-fragment uri
A predicate and field accessors for the URI record type. The URI scheme
-will be a symbol, and the rest either strings or @code{#f} if not
+will be a symbol, or @code{#f} if the object is a URI reference but not
+a URI. The port will be either a positive, exact integer or @code{#f},
+and the rest of the fields will be either strings or @code{#f} if not
present.
-@end defun
+@end deffn
-@defun string->uri string
+@deffn {Scheme Procedure} string->uri string
Parse @var{string} into a URI object. Return @code{#f} if the string
could not be parsed.
-@end defun
+@end deffn
-@defun uri->string uri
+@deffn {Scheme Procedure} string->uri-reference string
+Parse @var{string} into a URI object, while not requiring a scheme.
+Return @code{#f} if the string could not be parsed.
+@end deffn
+
+@deffn {Scheme Procedure} uri->string uri
Serialize @var{uri} to a string. If the URI has a port that is the
default port for its scheme, the port is not included in the
serialization.
-@end defun
+@end deffn
-@defun declare-default-port! scheme port
+@deffn {Scheme Procedure} declare-default-port! scheme port
Declare a default port for the given URI scheme.
-@end defun
+@end deffn
-@defun uri-decode str [#:encoding=@code{"utf-8"}]
+@deffn {Scheme Procedure} uri-decode str [#:encoding=@code{"utf-8"}]
Percent-decode the given @var{str}, according to @var{encoding}, which
should be the name of a character encoding.
Note that this function should not generally be applied to a full URI
-string. For paths, use split-and-decode-uri-path instead. For query
-strings, split the query on @code{&} and @code{=} boundaries, and decode
-the components separately.
+string. For paths, use @code{split-and-decode-uri-path} instead. For
+query strings, split the query on @code{&} and @code{=} boundaries, and
+decode the components separately.
Note also that percent-encoded strings encode @emph{bytes}, not
characters. There is no guarantee that a given byte sequence is a valid
Returns a string of the decoded characters, or a bytevector if
@var{encoding} was @code{#f}.
-@end defun
-
-Fixme: clarify return type. indicate default values. type of
-unescaped-chars.
+@end deffn
-@defun uri-encode str [#:encoding=@code{"utf-8"}] [#:unescaped-chars]
+@deffn {Scheme Procedure} uri-encode str [#:encoding=@code{"utf-8"}] [#:unescaped-chars]
Percent-encode any character not in the character set,
@var{unescaped-chars}.
a bytevector within the given @var{encoding}, then encoding each byte as
@code{%@var{HH}}, where @var{HH} is the hexadecimal representation of
the byte.
-@end defun
+@end deffn
-@defun split-and-decode-uri-path path
+@deffn {Scheme Procedure} split-and-decode-uri-path path
Split @var{path} into its components, and decode each component,
removing empty components.
For example, @code{"/foo/bar%20baz/"} decodes to the two-element list,
@code{("foo" "bar baz")}.
-@end defun
+@end deffn
-@defun encode-and-join-uri-path parts
+@deffn {Scheme Procedure} encode-and-join-uri-path parts
URI-encode each element of @var{parts}, which should be a list of
strings, and join the parts together with @code{/} as a delimiter.
For example, the list @code{("scrambled eggs" "biscuits&gravy")} encodes
as @code{"scrambled%20eggs/biscuits%26gravy"}.
-@end defun
+@end deffn
@node HTTP
@subsection The Hyper-Text Transfer Protocol
elements of the HTTP protocol.
If you are want to skip the low-level details for now and move on to web
-pages, @pxref{Web Server}. Otherwise, load the HTTP module, and read
-on.
+pages, @pxref{Web Client}, and @pxref{Web Server}. Otherwise, load the
+HTTP module, and read on.
@example
(use-modules (web http))
Header names are represented as lower-case symbols.
-@defun string->header name
+@deffn {Scheme Procedure} string->header name
Parse @var{name} to a symbolic header name.
-@end defun
+@end deffn
-@defun header->string sym
+@deffn {Scheme Procedure} header->string sym
Return the string form for the header named @var{sym}.
-@end defun
+@end deffn
For example:
(string->header "FOO")
@result{} foo
-(header->string 'foo
+(header->string 'foo)
@result{} "Foo"
@end example
parsing and serialization procedures. If a header is unknown, its
string name is simply its symbol name in title-case.
-@defun known-header? sym
-Return @code{#t} iff @var{sym} is a known header, with associated
-parsers and serialization procedures.
-@end defun
+@deffn {Scheme Procedure} known-header? sym
+Return @code{#t} if @var{sym} is a known header, with associated
+parsers and serialization procedures, or @code{#f} otherwise.
+@end deffn
-@defun header-parser sym
+@deffn {Scheme Procedure} header-parser sym
Return the value parser for headers named @var{sym}. The result is a
procedure that takes one argument, a string, and returns the parsed
value. If the header isn't known to Guile, a default parser is returned
that passes through the string unchanged.
-@end defun
+@end deffn
-@defun header-validator sym
+@deffn {Scheme Procedure} header-validator sym
Return a predicate which returns @code{#t} if the given value is valid
for headers named @var{sym}. The default validator for unknown headers
is @code{string?}.
-@end defun
+@end deffn
-@defun header-writer sym
+@deffn {Scheme Procedure} header-writer sym
Return a procedure that writes values for headers named @var{sym} to a
port. The resulting procedure takes two arguments: a value and a port.
The default writer is @code{display}.
-@end defun
+@end deffn
For more on the set of headers that Guile knows about out of the box,
@pxref{HTTP Headers}. To add your own, use the @code{declare-header!}
procedure:
-@defun declare-header! name parser validator writer [#:multiple?=@code{#f}]
+@deffn {Scheme Procedure} declare-header! name parser validator writer @
+ [#:multiple?=@code{#f}]
Declare a parser, validator, and writer for a given header.
-@end defun
+@end deffn
For example, let's say you are running a web server behind some sort of
proxy, and your proxy adds an @code{X-Client-Address} header, indicating
HTTP stack like this:
@example
-(define (parse-ip str)
- (inet-aton str)
-(define (validate-ip ip)
-(define (write-ip ip port)
- (display (inet-ntoa ip) port))
-
(declare-header! "X-Client-Address"
(lambda (str)
(inet-aton str))
(display (inet-ntoa ip) port)))
@end example
-@defun valid-header? sym val
-Return a true value iff @var{val} is a valid Scheme value for the header
-with name @var{sym}.
-@end defun
+@deffn {Scheme Procedure} declare-opaque-header! name
+A specialised version of @code{declare-header!} for the case in which
+you want a header's value to be returned/written ``as-is''.
+@end deffn
+
+@deffn {Scheme Procedure} valid-header? sym val
+Return a true value if @var{val} is a valid Scheme value for the header
+with name @var{sym}, or @code{#f} otherwise.
+@end deffn
Now that we have a generic interface for reading and writing headers, we
do just that.
-@defun read-header port
+@deffn {Scheme Procedure} read-header port
Read one HTTP header from @var{port}. Return two values: the header
name and the parsed Scheme value. May raise an exception if the header
was known but the value was invalid.
Returns the end-of-file object for both values if the end of the message
body was reached (i.e., a blank line).
-@end defun
+@end deffn
-@defun parse-header name val
+@deffn {Scheme Procedure} parse-header name val
Parse @var{val}, a string, with the parser for the header named
@var{name}. Returns the parsed value.
-@end defun
+@end deffn
-@defun write-header name val port
+@deffn {Scheme Procedure} write-header name val port
Write the given header name and value to @var{port}, using the writer
from @code{header-writer}.
-@end defun
+@end deffn
-@defun read-headers port
-Read the headers of an HTTP message from @var{port}, returning the
-headers as an ordered alist.
-@end defun
+@deffn {Scheme Procedure} read-headers port
+Read the headers of an HTTP message from @var{port}, returning them
+as an ordered alist.
+@end deffn
-@defun write-headers headers port
+@deffn {Scheme Procedure} write-headers headers port
Write the given header alist to @var{port}. Doesn't write the final
@samp{\r\n}, as the user might want to add another header.
-@end defun
+@end deffn
The @code{(web http)} module also has some utility procedures to read
and write request and response lines.
-@defun parse-http-method str [start] [end]
+@deffn {Scheme Procedure} parse-http-method str [start] [end]
Parse an HTTP method from @var{str}. The result is an upper-case symbol,
like @code{GET}.
-@end defun
+@end deffn
-@defun parse-http-version str [start] [end]
-Parse an HTTP version from @var{str}, returning it as a major-minor
+@deffn {Scheme Procedure} parse-http-version str [start] [end]
+Parse an HTTP version from @var{str}, returning it as a major--minor
pair. For example, @code{HTTP/1.1} parses as the pair of integers,
@code{(1 . 1)}.
-@end defun
+@end deffn
-@defun parse-request-uri str [start] [end]
+@deffn {Scheme Procedure} parse-request-uri str [start] [end]
Parse a URI from an HTTP request line. Note that URIs in requests do not
have to have a scheme or host name. The result is a URI object.
-@end defun
+@end deffn
-@defun read-request-line port
+@deffn {Scheme Procedure} read-request-line port
Read the first line of an HTTP request from @var{port}, returning three
values: the method, the URI, and the version.
-@end defun
+@end deffn
-@defun write-request-line method uri version port
+@deffn {Scheme Procedure} write-request-line method uri version port
Write the first line of an HTTP request to @var{port}.
-@end defun
+@end deffn
-@defun read-response-line port
+@deffn {Scheme Procedure} read-response-line port
Read the first line of an HTTP response from @var{port}, returning three
-values: the HTTP version, the response code, and the "reason phrase".
-@end defun
+values: the HTTP version, the response code, and the ``reason phrase''.
+@end deffn
-@defun write-response-line version code reason-phrase port
+@deffn {Scheme Procedure} write-response-line version code reason-phrase port
Write the first line of an HTTP response to @var{port}.
-@end defun
+@end deffn
@node HTTP Headers
transfer coding defined by the specification is @code{chunked}.
@example
(parse-header 'transfer-encoding "chunked")
-@result{} (chunked)
+@result{} ((chunked))
@end example
@end deftypevr
A list of content codings, as symbols.
@example
(parse-header 'content-encoding "gzip")
-@result{} (GET HEAD)
+@result{} (gzip)
@end example
@end deftypevr
(parse-header 'accept "text/html,text/plain;charset=utf-8")
@result{} ((text/html) (text/plain (charset . "utf-8")))
@end example
-Preference is expressed with qualitiy values:
+Preference is expressed with quality values:
@example
(parse-header 'accept "text/html;q=0.8,text/plain;q=0.6")
@result{} ((text/html (q . 800)) (text/plain (q . 600)))
Indicates that a response should proceed if and only if the resource has
been modified since the given date.
@example
-(parse-header if-modified-since "Tue, 15 Nov 1994 08:12:31 GMT")
+(parse-header 'if-modified-since "Tue, 15 Nov 1994 08:12:31 GMT")
@result{} #<date ...>
@end example
@end deftypevr
@end example
@end deftypevr
-@deftypevr {HTTP Header} URI location
-A URI on which a request may be completed. Used in combination with a
-redirecting status code to perform client-side redirection.
+@deftypevr {HTTP Header} URI-reference location
+A URI reference on which a request may be completed. Used in
+combination with a redirecting status code to perform client-side
+redirection.
@example
(parse-header 'location "http://example.com/other")
@result{} #<uri ...>
@deftypevr {HTTP Header} *|List vary
A set of request headers that were used in computing this response.
-Used to indicate that server-side content negotation was performed, for
+Used to indicate that server-side content negotiation was performed, for
example in response to the @code{accept-language} header. Can also be
the symbol @code{*}, indicating that all headers were considered.
@example
@end example
@end deftypevr
+@node Transfer Codings
+@subsection Transfer Codings
+
+HTTP 1.1 allows for various transfer codings to be applied to message
+bodies. These include various types of compression, and HTTP chunked
+encoding. Currently, only chunked encoding is supported by guile.
+
+Chunked coding is an optional coding that may be applied to message
+bodies, to allow messages whose length is not known beforehand to be
+returned. Such messages can be split into chunks, terminated by a final
+zero length chunk.
+
+In order to make dealing with encodings more simple, guile provides
+procedures to create ports that ``wrap'' existing ports, applying
+transformations transparently under the hood.
+
+These procedures are in the @code{(web http)} module.
+
+@example
+(use-modules (web http))
+@end example
+
+@deffn {Scheme Procedure} make-chunked-input-port port [#:keep-alive?=#f]
+Returns a new port, that transparently reads and decodes chunk-encoded
+data from @var{port}. If no more chunk-encoded data is available, it
+returns the end-of-file object. When the port is closed, @var{port} will
+also be closed, unless @var{keep-alive?} is true.
+@end deffn
+
+@example
+(use-modules (ice-9 rdelim))
+
+(define s "5\r\nFirst\r\nA\r\n line\n Sec\r\n8\r\nond line\r\n0\r\n")
+(define p (make-chunked-input-port (open-input-string s)))
+(read-line s)
+@result{} "First line"
+(read-line s)
+@result{} "Second line"
+@end example
+
+@deffn {Scheme Procedure} make-chunked-output-port port [#:keep-alive?=#f]
+Returns a new port, which transparently encodes data as chunk-encoded
+before writing it to @var{port}. Whenever a write occurs on this port,
+it buffers it, until the port is flushed, at which point it writes a
+chunk containing all the data written so far. When the port is closed,
+the data remaining is written to @var{port}, as is the terminating zero
+chunk. It also causes @var{port} to be closed, unless @var{keep-alive?}
+is true.
+
+Note. Forcing a chunked output port when there is no data is buffered
+does not write a zero chunk, as this would cause the data to be
+interpreted incorrectly by the client.
+@end deffn
+
+@example
+(call-with-output-string
+ (lambda (out)
+ (define out* (make-chunked-output-port out #:keep-alive? #t))
+ (display "first chunk" out*)
+ (force-output out*)
+ (force-output out*) ; note this does not write a zero chunk
+ (display "second chunk" out*)
+ (close-port out*)))
+@result{} "b\r\nfirst chunk\r\nc\r\nsecond chunk\r\n0\r\n"
+@end example
@node Requests
@subsection HTTP Requests
@subsubsection Request API
-@defun request?
-@defunx request-method
-@defunx request-uri
-@defunx request-version
-@defunx request-headers
-@defunx request-meta
-@defunx request-port
+@deffn {Scheme Procedure} request? obj
+@deffnx {Scheme Procedure} request-method request
+@deffnx {Scheme Procedure} request-uri request
+@deffnx {Scheme Procedure} request-version request
+@deffnx {Scheme Procedure} request-headers request
+@deffnx {Scheme Procedure} request-meta request
+@deffnx {Scheme Procedure} request-port request
A predicate and field accessors for the request type. The fields are as
follows:
@table @code
@item port
The port on which to read or write a request body, if any.
@end table
-@end defun
+@end deffn
-@defun read-request port [meta='()]
+@deffn {Scheme Procedure} read-request port [meta='()]
Read an HTTP request from @var{port}, optionally attaching the given
metadata, @var{meta}.
Note that the body is not part of the request. Once you have read a
request, you may read the body separately, and likewise for writing
requests.
-@end defun
+@end deffn
-@defun build-request uri [#:method='GET] [#:version='(1 . 1)] [#:headers='()] [#:port=#f] [#:meta='()] [#:validate-headers?=#t]
+@deffn {Scheme Procedure} build-request uri [#:method='GET] @
+ [#:version='(1 . 1)] [#:headers='()] [#:port=#f] [#:meta='()] @
+ [#:validate-headers?=#t]
Construct an HTTP request object. If @var{validate-headers?} is true,
the headers are each run through their respective validators.
-@end defun
+@end deffn
-@defun write-request r port
+@deffn {Scheme Procedure} write-request r port
Write the given HTTP request to @var{port}.
Return a new request, whose @code{request-port} will continue writing
on @var{port}, perhaps using some transfer encoding.
-@end defun
+@end deffn
-@defun read-request-body r
+@deffn {Scheme Procedure} read-request-body r
Reads the request body from @var{r}, as a bytevector. Return @code{#f}
if there was no request body.
-@end defun
+@end deffn
-@defun write-request-body r bv
-Write @var{body}, a bytevector, to the port corresponding to the HTTP
+@deffn {Scheme Procedure} write-request-body r bv
+Write @var{bv}, a bytevector, to the port corresponding to the HTTP
request @var{r}.
-@end defun
+@end deffn
The various headers that are typically associated with HTTP requests may
be accessed with these dedicated accessors. @xref{HTTP Headers}, for
more information on the format of parsed headers.
-@defun request-accept request [default='()]
-@defunx request-accept-charset request [default='()]
-@defunx request-accept-encoding request [default='()]
-@defunx request-accept-language request [default='()]
-@defunx request-allow request [default='()]
-@defunx request-authorization request [default=#f]
-@defunx request-cache-control request [default='()]
-@defunx request-connection request [default='()]
-@defunx request-content-encoding request [default='()]
-@defunx request-content-language request [default='()]
-@defunx request-content-length request [default=#f]
-@defunx request-content-location request [default=#f]
-@defunx request-content-md5 request [default=#f]
-@defunx request-content-range request [default=#f]
-@defunx request-content-type request [default=#f]
-@defunx request-date request [default=#f]
-@defunx request-expect request [default='()]
-@defunx request-expires request [default=#f]
-@defunx request-from request [default=#f]
-@defunx request-host request [default=#f]
-@defunx request-if-match request [default=#f]
-@defunx request-if-modified-since request [default=#f]
-@defunx request-if-none-match request [default=#f]
-@defunx request-if-range request [default=#f]
-@defunx request-if-unmodified-since request [default=#f]
-@defunx request-last-modified request [default=#f]
-@defunx request-max-forwards request [default=#f]
-@defunx request-pragma request [default='()]
-@defunx request-proxy-authorization request [default=#f]
-@defunx request-range request [default=#f]
-@defunx request-referer request [default=#f]
-@defunx request-te request [default=#f]
-@defunx request-trailer request [default='()]
-@defunx request-transfer-encoding request [default='()]
-@defunx request-upgrade request [default='()]
-@defunx request-user-agent request [default=#f]
-@defunx request-via request [default='()]
-@defunx request-warning request [default='()]
+@deffn {Scheme Procedure} request-accept request [default='()]
+@deffnx {Scheme Procedure} request-accept-charset request [default='()]
+@deffnx {Scheme Procedure} request-accept-encoding request [default='()]
+@deffnx {Scheme Procedure} request-accept-language request [default='()]
+@deffnx {Scheme Procedure} request-allow request [default='()]
+@deffnx {Scheme Procedure} request-authorization request [default=#f]
+@deffnx {Scheme Procedure} request-cache-control request [default='()]
+@deffnx {Scheme Procedure} request-connection request [default='()]
+@deffnx {Scheme Procedure} request-content-encoding request [default='()]
+@deffnx {Scheme Procedure} request-content-language request [default='()]
+@deffnx {Scheme Procedure} request-content-length request [default=#f]
+@deffnx {Scheme Procedure} request-content-location request [default=#f]
+@deffnx {Scheme Procedure} request-content-md5 request [default=#f]
+@deffnx {Scheme Procedure} request-content-range request [default=#f]
+@deffnx {Scheme Procedure} request-content-type request [default=#f]
+@deffnx {Scheme Procedure} request-date request [default=#f]
+@deffnx {Scheme Procedure} request-expect request [default='()]
+@deffnx {Scheme Procedure} request-expires request [default=#f]
+@deffnx {Scheme Procedure} request-from request [default=#f]
+@deffnx {Scheme Procedure} request-host request [default=#f]
+@deffnx {Scheme Procedure} request-if-match request [default=#f]
+@deffnx {Scheme Procedure} request-if-modified-since request [default=#f]
+@deffnx {Scheme Procedure} request-if-none-match request [default=#f]
+@deffnx {Scheme Procedure} request-if-range request [default=#f]
+@deffnx {Scheme Procedure} request-if-unmodified-since request [default=#f]
+@deffnx {Scheme Procedure} request-last-modified request [default=#f]
+@deffnx {Scheme Procedure} request-max-forwards request [default=#f]
+@deffnx {Scheme Procedure} request-pragma request [default='()]
+@deffnx {Scheme Procedure} request-proxy-authorization request [default=#f]
+@deffnx {Scheme Procedure} request-range request [default=#f]
+@deffnx {Scheme Procedure} request-referer request [default=#f]
+@deffnx {Scheme Procedure} request-te request [default=#f]
+@deffnx {Scheme Procedure} request-trailer request [default='()]
+@deffnx {Scheme Procedure} request-transfer-encoding request [default='()]
+@deffnx {Scheme Procedure} request-upgrade request [default='()]
+@deffnx {Scheme Procedure} request-user-agent request [default=#f]
+@deffnx {Scheme Procedure} request-via request [default='()]
+@deffnx {Scheme Procedure} request-warning request [default='()]
Return the given request header, or @var{default} if none was present.
-@end defun
+@end deffn
-@defun request-absolute-uri r [default-host=#f] [default-port=#f]
+@deffn {Scheme Procedure} request-absolute-uri r [default-host=#f] @
+ [default-port=#f] [default-scheme=#f]
A helper routine to determine the absolute URI of a request, using the
-@code{host} header and the default host and port.
-@end defun
-
+@code{host} header and the default scheme, host and port. If there is
+no default scheme and the URI is not itself absolute, an error is
+signalled.
+@end deffn
@node Responses
@subsection HTTP Responses
As with requests (@pxref{Requests}), Guile offers a data type for HTTP
responses. Again, the body is represented separately from the request.
-@defun response?
-@defunx response-version
-@defunx response-code
-@defunx response-reason-phrase response
-@defunx response-headers
-@defunx response-port
+@deffn {Scheme Procedure} response? obj
+@deffnx {Scheme Procedure} response-version response
+@deffnx {Scheme Procedure} response-code response
+@deffnx {Scheme Procedure} response-reason-phrase response
+@deffnx {Scheme Procedure} response-headers response
+@deffnx {Scheme Procedure} response-port response
A predicate and field accessors for the response type. The fields are as
follows:
@table @code
@item port
The port on which to read or write a response body, if any.
@end table
-@end defun
+@end deffn
-@defun read-response port
+@deffn {Scheme Procedure} read-response port
Read an HTTP response from @var{port}.
As a side effect, sets the encoding on @var{port} to ISO-8859-1
(latin-1), so that reading one character reads one byte. See the
discussion of character sets in @ref{Responses}, for more information.
-@end defun
+@end deffn
-@defun build-response [#:version='(1 . 1)] [#:code=200] [#:reason-phrase=#f] [#:headers='()] [#:port=#f] [#:validate-headers=#t]
+@deffn {Scheme Procedure} build-response [#:version='(1 . 1)] [#:code=200] [#:reason-phrase=#f] [#:headers='()] [#:port=#f] [#:validate-headers?=#t]
Construct an HTTP response object. If @var{validate-headers?} is true,
the headers are each run through their respective validators.
-@end defun
+@end deffn
-@defun adapt-response-version response version
+@deffn {Scheme Procedure} adapt-response-version response version
Adapt the given response to a different HTTP version. Return a new HTTP
response.
programmatic transformations to respond to older HTTP versions (0.9 and
1.0). But currently this function is a bit heavy-handed, just updating
the version field.
-@end defun
+@end deffn
-@defun write-response r port
+@deffn {Scheme Procedure} write-response r port
Write the given HTTP response to @var{port}.
Return a new response, whose @code{response-port} will continue writing
on @var{port}, perhaps using some transfer encoding.
-@end defun
+@end deffn
+
+@deffn {Scheme Procedure} response-must-not-include-body? r
+Some responses, like those with status code 304, are specified as never
+having bodies. This predicate returns @code{#t} for those responses.
+
+Note also, though, that responses to @code{HEAD} requests must also not
+have a body.
+@end deffn
-@defun read-response-body r
+@deffn {Scheme Procedure} response-body-port r [#:decode?=#t] [#:keep-alive?=#t]
+Return an input port from which the body of @var{r} can be read. The encoding
+of the returned port is set according to @var{r}'s @code{content-type} header,
+when it's textual, except if @var{decode?} is @code{#f}. Return @code{#f}
+when no body is available.
+
+When @var{keep-alive?} is @code{#f}, closing the returned port also closes
+@var{r}'s response port.
+@end deffn
+
+@deffn {Scheme Procedure} read-response-body r
Read the response body from @var{r}, as a bytevector. Returns @code{#f}
if there was no response body.
-@end defun
+@end deffn
-@defun write-response-body r bv
-Write @var{body}, a bytevector, to the port corresponding to the HTTP
+@deffn {Scheme Procedure} write-response-body r bv
+Write @var{bv}, a bytevector, to the port corresponding to the HTTP
response @var{r}.
-@end defun
+@end deffn
As with requests, the various headers that are typically associated with
HTTP responses may be accessed with these dedicated accessors.
@xref{HTTP Headers}, for more information on the format of parsed
headers.
-@defun response-accept-ranges response [default=#f]
-@defunx response-age response [default='()]
-@defunx response-allow response [default='()]
-@defunx response-cache-control response [default='()]
-@defunx response-connection response [default='()]
-@defunx response-content-encoding response [default='()]
-@defunx response-content-language response [default='()]
-@defunx response-content-length response [default=#f]
-@defunx response-content-location response [default=#f]
-@defunx response-content-md5 response [default=#f]
-@defunx response-content-range response [default=#f]
-@defunx response-content-type response [default=#f]
-@defunx response-date response [default=#f]
-@defunx response-etag response [default=#f]
-@defunx response-expires response [default=#f]
-@defunx response-last-modified response [default=#f]
-@defunx response-location response [default=#f]
-@defunx response-pragma response [default='()]
-@defunx response-proxy-authenticate response [default=#f]
-@defunx response-retry-after response [default=#f]
-@defunx response-server response [default=#f]
-@defunx response-trailer response [default='()]
-@defunx response-transfer-encoding response [default='()]
-@defunx response-upgrade response [default='()]
-@defunx response-vary response [default='()]
-@defunx response-via response [default='()]
-@defunx response-warning response [default='()]
-@defunx response-www-authenticate response [default=#f]
+@deffn {Scheme Procedure} response-accept-ranges response [default=#f]
+@deffnx {Scheme Procedure} response-age response [default='()]
+@deffnx {Scheme Procedure} response-allow response [default='()]
+@deffnx {Scheme Procedure} response-cache-control response [default='()]
+@deffnx {Scheme Procedure} response-connection response [default='()]
+@deffnx {Scheme Procedure} response-content-encoding response [default='()]
+@deffnx {Scheme Procedure} response-content-language response [default='()]
+@deffnx {Scheme Procedure} response-content-length response [default=#f]
+@deffnx {Scheme Procedure} response-content-location response [default=#f]
+@deffnx {Scheme Procedure} response-content-md5 response [default=#f]
+@deffnx {Scheme Procedure} response-content-range response [default=#f]
+@deffnx {Scheme Procedure} response-content-type response [default=#f]
+@deffnx {Scheme Procedure} response-date response [default=#f]
+@deffnx {Scheme Procedure} response-etag response [default=#f]
+@deffnx {Scheme Procedure} response-expires response [default=#f]
+@deffnx {Scheme Procedure} response-last-modified response [default=#f]
+@deffnx {Scheme Procedure} response-location response [default=#f]
+@deffnx {Scheme Procedure} response-pragma response [default='()]
+@deffnx {Scheme Procedure} response-proxy-authenticate response [default=#f]
+@deffnx {Scheme Procedure} response-retry-after response [default=#f]
+@deffnx {Scheme Procedure} response-server response [default=#f]
+@deffnx {Scheme Procedure} response-trailer response [default='()]
+@deffnx {Scheme Procedure} response-transfer-encoding response [default='()]
+@deffnx {Scheme Procedure} response-upgrade response [default='()]
+@deffnx {Scheme Procedure} response-vary response [default='()]
+@deffnx {Scheme Procedure} response-via response [default='()]
+@deffnx {Scheme Procedure} response-warning response [default='()]
+@deffnx {Scheme Procedure} response-www-authenticate response [default=#f]
Return the given response header, or @var{default} if none was present.
-@end defun
+@end deffn
+
+@deffn {Scheme Procedure} text-content-type? @var{type}
+Return @code{#t} if @var{type}, a symbol as returned by
+@code{response-content-type}, represents a textual type such as
+@code{text/plain}.
+@end deffn
+
+
+@node Web Client
+@subsection Web Client
+
+@code{(web client)} provides a simple, synchronous HTTP client, built on
+the lower-level HTTP, request, and response modules.
+
+@example
+(use-modules (web client))
+@end example
+
+@deffn {Scheme Procedure} open-socket-for-uri uri
+Return an open input/output port for a connection to URI.
+@end deffn
+
+@deffn {Scheme Procedure} http-get uri arg...
+@deffnx {Scheme Procedure} http-head uri arg...
+@deffnx {Scheme Procedure} http-post uri arg...
+@deffnx {Scheme Procedure} http-put uri arg...
+@deffnx {Scheme Procedure} http-delete uri arg...
+@deffnx {Scheme Procedure} http-trace uri arg...
+@deffnx {Scheme Procedure} http-options uri arg...
+
+Connect to the server corresponding to @var{uri} and make a request over
+HTTP, using the appropriate method (@code{GET}, @code{HEAD}, etc.).
+
+All of these procedures have the same prototype: a URI followed by an
+optional sequence of keyword arguments. These keyword arguments allow
+you to modify the requests in various ways, for example attaching a body
+to the request, or setting specific headers. The following table lists
+the keyword arguments and their default values.
+
+@table @code
+@item #:body #f
+@item #:port (open-socket-for-uri @var{uri})]
+@item #:version '(1 . 1)
+@item #:keep-alive? #f
+@item #:headers '()
+@item #:decode-body? #t
+@item #:streaming? #f
+@end table
+
+If you already have a port open, pass it as @var{port}. Otherwise, a
+connection will be opened to the server corresponding to @var{uri}. Any
+extra headers in the alist @var{headers} will be added to the request.
+
+If @var{body} is not @code{#f}, a message body will also be sent with
+the HTTP request. If @var{body} is a string, it is encoded according to
+the content-type in @var{headers}, defaulting to UTF-8. Otherwise
+@var{body} should be a bytevector, or @code{#f} for no body. Although a
+message body may be sent with any request, usually only @code{POST} and
+@code{PUT} requests have bodies.
+
+If @var{decode-body?} is true, as is the default, the body of the
+response will be decoded to string, if it is a textual content-type.
+Otherwise it will be returned as a bytevector.
+
+However, if @var{streaming?} is true, instead of eagerly reading the
+response body from the server, this function only reads off the headers.
+The response body will be returned as a port on which the data may be
+read.
+
+Unless @var{keep-alive?} is true, the port will be closed after the full
+response body has been read.
+
+Returns two values: the response read from the server, and the response
+body as a string, bytevector, #f value, or as a port (if
+@var{streaming?} is true).
+@end deffn
+
+@code{http-get} is useful for making one-off requests to web sites. If
+you are writing a web spider or some other client that needs to handle a
+number of requests in parallel, it's better to build an event-driven URL
+fetcher, similar in structure to the web server (@pxref{Web Server}).
+
+Another option, good but not as performant, would be to use threads,
+possibly via par-map or futures.
+
+@deffn {Scheme Parameter} current-http-proxy
+Either @code{#f} or a non-empty string containing the URL of the HTTP
+proxy server to be used by the procedures in the @code{(web client)}
+module, including @code{open-socket-for-uri}. Its initial value is
+based on the @env{http_proxy} environment variable.
+
+@example
+(current-http-proxy) @result{} "http://localhost:8123/"
+(parameterize ((current-http-proxy #f))
+ (http-get "http://example.com/")) ; temporarily bypass proxy
+(current-http-proxy) @result{} "http://localhost:8123/"
+@end example
+@end deffn
@node Web Server
@enumerate
@item
-The @code{open} hook is called, to open the server. @code{open} takes 0 or
-more arguments, depending on the backend, and returns an opaque
+The @code{open} hook is called, to open the server. @code{open} takes
+zero or more arguments, depending on the backend, and returns an opaque
server socket object, or signals an error.
@item
socket, request, and body.
@item
-A user-provided handler procedure is called, with the request
-and body as its arguments. The handler should return two
-values: the response, as a @code{<response>} record from @code{(web
-response)}, and the response body as a string, bytevector, or
-@code{#f} if not present. We also allow the reponse to be simply an
-alist of headers, in which case a default response object is
-constructed with those headers.
+A user-provided handler procedure is called, with the request and body
+as its arguments. The handler should return two values: the response,
+as a @code{<response>} record from @code{(web response)}, and the
+response body as bytevector, or @code{#f} if not present.
+
+The respose and response body are run through @code{sanitize-response},
+documented below. This allows the handler writer to take some
+convenient shortcuts: for example, instead of a @code{<response>}, the
+handler can simply return an alist of headers, in which case a default
+response object is constructed with those headers. Instead of a
+bytevector for the body, the handler can return a string, which will be
+serialized into an appropriate encoding; or it can return a procedure,
+which will be called on a port to write out the data. See the
+@code{sanitize-response} documentation, for more.
@item
The @code{write} hook is called with three arguments: the client
A user may define a server implementation with the following form:
-@defun define-server-impl name open read write close
+@deffn {Scheme Syntax} define-server-impl name open read write close
Make a @code{<server-impl>} object with the hooks @var{open},
@var{read}, @var{write}, and @var{close}, and bind it to the symbol
@var{name} in the current module.
-@end defun
+@end deffn
-@defun lookup-server-impl impl
-Look up a server implementation. If @var{impl} is a server
-implementation already, it is returned directly. If it is a symbol, the
+@deffn {Scheme Procedure} lookup-server-impl impl
+Look up a server implementation. If @var{impl} is a server
+implementation already, it is returned directly. If it is a symbol, the
binding named @var{impl} in the @code{(web server @var{impl})} module is
-looked up. Otherwise an error is signaled.
+looked up. Otherwise an error is signaled.
Currently a server implementation is a somewhat opaque type, useful only
for passing to other procedures in this module, like @code{read-client}.
-@end defun
+@end deffn
The @code{(web server)} module defines a number of routines that use
@code{<server-impl>} objects to implement parts of a web server. Given
@code{<server-impl>}, indeed these routines are the only procedures with
any access to the impl objects.
-@defun open-server impl open-params
-Open a server for the given implementation. Returns one value, the new
-server object. The implementation's @code{open} procedure is applied to
+@deffn {Scheme Procedure} open-server impl open-params
+Open a server for the given implementation. Return one value, the new
+server object. The implementation's @code{open} procedure is applied to
@var{open-params}, which should be a list.
-@end defun
+@end deffn
-@defun read-client impl server
+@deffn {Scheme Procedure} read-client impl server
Read a new client from @var{server}, by applying the implementation's
-@code{read} procedure to the server. If successful, returns three
+@code{read} procedure to the server. If successful, return three
values: an object corresponding to the client, a request object, and the
-request body. If any exception occurs, returns @code{#f} for all three
+request body. If any exception occurs, return @code{#f} for all three
values.
-@end defun
+@end deffn
-@defun handle-request handler request body state
+@deffn {Scheme Procedure} handle-request handler request body state
Handle a given request, returning the response and body.
The response and response body are produced by calling the given
@var{handler} with @var{request} and @var{body} as arguments.
The elements of @var{state} are also passed to @var{handler} as
-arguments, and may be returned as additional values. The new
+arguments, and may be returned as additional values. The new
@var{state}, collected from the @var{handler}'s return values, is then
-returned as a list. The idea is that a server loop receives a handler
+returned as a list. The idea is that a server loop receives a handler
from the user, along with whatever state values the user is interested
in, allowing the user's handler to explicitly manage its state.
-@end defun
+@end deffn
-@defun sanitize-response request response body
-"Sanitize" the given response and body, making them appropriate for the
-given request.
+@deffn {Scheme Procedure} sanitize-response request response body
+``Sanitize'' the given response and body, making them appropriate for
+the given request.
As a convenience to web handler authors, @var{response} may be given as
an alist of headers, in which case it is used to construct a default
-response. Ensures that the response version corresponds to the request
-version. If @var{body} is a string, encodes the string to a bytevector,
-in an encoding appropriate for @var{response}. Adds a
+response. Ensures that the response version corresponds to the request
+version. If @var{body} is a string, encodes the string to a bytevector,
+in an encoding appropriate for @var{response}. Adds a
@code{content-length} and @code{content-type} header, as necessary.
If @var{body} is a procedure, it is called with a port as an argument,
-and the output collected as a bytevector. In the future we might try to
+and the output collected as a bytevector. In the future we might try to
instead use a compressing, chunk-encoded port, and call this procedure
-later, in the write-client procedure. Authors are advised not to rely on
+later, in the write-client procedure. Authors are advised not to rely on
the procedure being called at any particular time.
-@end defun
+@end deffn
-@defun write-client impl server client response body
-Write an HTTP response and body to @var{client}. If the server and
+@deffn {Scheme Procedure} write-client impl server client response body
+Write an HTTP response and body to @var{client}. If the server and
client support persistent connections, it is the implementation's
responsibility to keep track of the client thereafter, presumably by
attaching it to the @var{server} argument somehow.
-@end defun
+@end deffn
-@defun close-server impl server
+@deffn {Scheme Procedure} close-server impl server
Release resources allocated by a previous invocation of
@code{open-server}.
-@end defun
+@end deffn
Given the procedures above, it is a small matter to make a web server:
-@defun serve-one-client handler impl server state
+@deffn {Scheme Procedure} serve-one-client handler impl server state
Read one request from @var{server}, call @var{handler} on the request
-and body, and write the response to the client. Returns the new state
+and body, and write the response to the client. Return the new state
produced by the handler procedure.
-@end defun
+@end deffn
-@defun run-server handler [impl] [open-params] . state
+@deffn {Scheme Procedure} run-server handler @
+ [impl='http] [open-params='()] @
+ arg @dots{}
Run Guile's built-in web server.
@var{handler} should be a procedure that takes two or more arguments,
the HTTP request and request body, and returns two or more values, the
response and response body.
-For example, here is a simple "Hello, World!" server:
-
-@example
- (define (handler request body)
- (values '((content-type . ("text/plain")))
- "Hello, World!"))
- (run-server handler)
-@end example
+For examples, skip ahead to the next section, @ref{Web Examples}.
The response and body will be run through @code{sanitize-response}
before sending back to the client.
-Additional arguments to @var{handler} are taken from @var{state}.
-Additional return values are accumulated into a new @var{state}, which
-will be used for subsequent requests. In this way a handler can
-explicitly manage its state.
-
-The default server implementation is @code{http}, which accepts
-@var{open-params} like @code{(#:port 8081)}, among others. See "Web
-Server" in the manual, for more information.
-@end defun
-
+Additional arguments to @var{handler} are taken from @var{arg}
+@enddots{}. These arguments comprise a @dfn{state}. Additional return
+values are accumulated into a new state, which will be used for
+subsequent requests. In this way a handler can explicitly manage its
+state.
+@end deffn
+
+The default web server implementation is @code{http}, which binds to a
+socket, listening for request on that port.
+
+@deffn {HTTP Implementation} http [#:host=#f] @
+ [#:family=AF_INET] @
+ [#:addr=INADDR_LOOPBACK] @
+ [#:port 8080] [#:socket]
+The default HTTP implementation. We document it as a function with
+keyword arguments, because that is precisely the way that it is -- all
+of the @var{open-params} to @code{run-server} get passed to the
+implementation's open function.
+
+@example
+;; The defaults: localhost:8080
+(run-server handler)
+;; Same thing
+(run-server handler 'http '())
+;; On a different port
+(run-server handler 'http '(#:port 8081))
+;; IPv6
+(run-server handler 'http '(#:family AF_INET6 #:port 8081))
+;; Custom socket
+(run-server handler 'http `(#:socket ,(sudo-make-me-a-socket)))
+@end example
+@end deffn
@node Web Examples
@subsection Web Examples
@example
(define (hello-world-handler request request-body)
- (values '((content-type . ("text/plain")))
+ (values '((content-type . (text/plain)))
"Hello World!"))
@end example
(define (hello-hacker-handler request body)
(if (equal? (request-path-components request)
'("hacker"))
- (values '((content-type . ("text/plain")))
+ (values '((content-type . (text/plain)))
"Hello hacker!")
(not-found request)))
(define (not-found request)
(values (build-response #:code 404)
(string-append "Resource not found: "
- (unparse-uri (request-uri request)))))
+ (uri->string (request-uri request)))))
;; Now paste this to let the web server keep going:
,continue
The web handler interface is a common baseline that all kinds of Guile
web applications can use. You will usually want to build something on
top of it, however, especially when producing HTML. Here is a simple
-example that builds up HTML output using SXML (@pxref{sxml simple}).
+example that builds up HTML output using SXML (@pxref{SXML}).
First, load up the modules:
(status 200)
(title "Hello hello!")
(doctype "<!DOCTYPE html>\n")
- (content-type-params '(("charset" . "utf-8")))
- (content-type "text/html")
+ (content-type-params '((charset . "utf-8")))
+ (content-type 'text/html)
(extra-headers '())
(sxml (and body (templatize title body))))
(values (build-response
the time the arguments are fully parsed, the @code{sxml} local variable
will hold the templated SXML, ready for sending out to the client.
-Instead of returning the body as a string, here we give a procedure,
-which will be called by the web server to write out the response to the
-client.
+Also, instead of returning the body as a string, @code{respond} gives a
+procedure, which will be called by the web server to write out the
+response to the client.
Now, a simple example using this responder, which lays out the incoming
headers in an HTML table.
HCoop / bpt / guile.git / blobdiff