2 @c This is part of the GNU Guile Reference Manual.
3 @c Copyright (C) 2010 Free Software Foundation, Inc.
4 @c See the file guile.texi for copying conditions.
7 @section @acronym{HTTP}, the Web, and All That
12 When Guile started back in the mid-nineties, the GNU system was still
13 focused on producing a good POSIX implementation. This is why Guile's
14 POSIX support is good, and has been so for a while.
16 But times change, and in a way these days the web is the new POSIX: a
17 standard and a motley set of implementations on which much computing is
18 done. So today's Guile also supports the web at the programming
19 language level, by defining common data types and operations for the
20 technologies underpinning the web: URIs, HTTP, and XML.
22 It is particularly important to define native web data types. Though
23 the web is text in motion, programming the web in text is like
24 programming with @code{goto}: muddy, and error-prone. Most current
25 security problems on the web are due to treating the web as text instead
26 of as instances of the proper data types.
28 In addition, common web data types help programmers to share code.
30 Well. That's all very nice and opinionated and such, but how do I use
34 * URIs:: Universal Resource Identifiers.
35 * HTTP:: The Hyper-Text Transfer Protocol.
36 * Requests:: HTTP requests.
37 * Responses:: HTTP responses.
38 * Web Handlers:: A simple web application interface.
39 * Web Server:: Serving HTTP to the internet.
43 @subsection Universal Resource Identifiers
46 (use-modules (web uri))
50 A data type for Universal Resource Identifiers, as defined in RFC
78 @defun build-uri scheme [#:userinfo] [#:host] [#:port] [#:path] [#:query] [#:fragment] [#:validate?]
79 Construct a URI object. If @var{validate?} is true, also run some
80 consistency checks to make sure that the constructed URI is valid.
83 @defun declare-default-port! scheme port
84 Declare a default port for the given URI scheme.
86 Default ports are for printing URI objects: a default port is not
90 @defun parse-uri string
91 Parse @var{string} into a URI object. Returns @code{#f} if the string
95 @defun unparse-uri uri
96 Serialize @var{uri} to a string.
99 @defun uri-decode str [#:charset]
100 Percent-decode the given @var{str}, according to @var{charset}.
102 Note that this function should not generally be applied to a full URI
103 string. For paths, use split-and-decode-uri-path instead. For query
104 strings, split the query on @code{&} and @code{=} boundaries, and decode
105 the components separately.
107 Note that percent-encoded strings encode @emph{bytes}, not characters.
108 There is no guarantee that a given byte sequence is a valid string
109 encoding. Therefore this routine may signal an error if the decoded
110 bytes are not valid for the given encoding. Pass @code{#f} for
111 @var{charset} if you want decoded bytes as a bytevector directly.
114 @defun uri-encode str [#:charset] [#:unescaped-chars]
115 Percent-encode any character not in @var{unescaped-chars}.
117 Percent-encoding first writes out the given character to a bytevector
118 within the given @var{charset}, then encodes each byte as
119 @code{%@var{HH}}, where @var{HH} is the hexadecimal representation of
123 @defun split-and-decode-uri-path path
124 Split @var{path} into its components, and decode each component,
125 removing empty components.
127 For example, @code{"/foo/bar/"} decodes to the two-element list,
128 @code{("foo" "bar")}.
131 @defun encode-and-join-uri-path parts
132 URI-encode each element of @var{parts}, which should be a list of
133 strings, and join the parts together with @code{/} as a delimiter.
137 @subsection The Hyper-Text Transfer Protocol
140 (use-modules (web http))
143 This module has a number of routines to parse textual
144 representations of HTTP data into native Scheme data structures.
146 It tries to follow RFCs fairly strictly---the road to perdition
147 being paved with compatibility hacks---though some allowances are
148 made for not-too-divergent texts (like a quality of .2 which should
151 @defspec header-decl?
154 @defspec make-header-decl
157 @defspec header-decl-sym
160 @defspec header-decl-name
163 @defspec header-decl-multiple?
166 @defspec header-decl-parser
169 @defspec header-decl-validator
172 @defspec header-decl-writer
175 @defun lookup-header-decl name
176 Return the @var{header-decl} object registered for the given @var{name}.
178 @var{name} may be a symbol or a string. Strings are mapped to headers in
179 a case-insensitive fashion.
182 @defun declare-header! sym name [#:multiple?] [#:parser] [#:validator] [#:writer]
183 Define a parser, validator, and writer for the HTTP header, @var{name}.
185 @var{parser} should be a procedure that takes a string and returns a
186 Scheme value. @var{validator} is a predicate for whether the given
187 Scheme value is valid for this header. @var{writer} takes a value and a
188 port, and writes the value to the port.
191 @defun read-header port
192 Reads one HTTP header from @var{port}. Returns two values: the header
193 name and the parsed Scheme value. May raise an exception if the header
194 was known but the value was invalid.
196 Returns @var{#f} for both values if the end of the message body was
197 reached (i.e., a blank line).
200 @defun parse-header name val
201 Parse @var{val}, a string, with the parser for the header named
204 Returns two values, the header name and parsed value. If a parser was
205 found, the header name will be returned as a symbol. If a parser was not
206 found, both the header name and the value are returned as strings.
209 @defun valid-header? sym val
210 Returns a true value iff @var{val} is a valid Scheme value for the
211 header with name @var{sym}.
214 @defun write-header name val port
215 Writes the given header name and value to @var{port}. If @var{name} is a
216 symbol, looks up a declared header and uses that writer. Otherwise the
217 value is written using @var{display}.
220 @defun read-headers port
221 Read an HTTP message from @var{port}, returning the headers as an
225 @defun write-headers headers port
226 Write the given header alist to @var{port}. Doesn't write the final
227 \r\n, as the user might want to add another header.
230 @defun parse-http-method str [start] [end]
231 Parse an HTTP method from @var{str}. The result is an upper-case symbol,
235 @defun parse-http-version str [start] [end]
236 Parse an HTTP version from @var{str}, returning it as a major-minor
237 pair. For example, @code{HTTP/1.1} parses as the pair of integers,
241 @defun parse-request-uri str [start] [end]
242 Parse a URI from an HTTP request line. Note that URIs in requests do not
243 have to have a scheme or host name. The result is a URI object.
246 @defun read-request-line port
247 Read the first line of an HTTP request from @var{port}, returning three
248 values: the method, the URI, and the version.
251 @defun write-request-line method uri version port
252 Write the first line of an HTTP request to @var{port}.
255 @defun read-response-line port
256 Read the first line of an HTTP response from @var{port}, returning three
257 values: the HTTP version, the response code, and the "reason phrase".
260 @defun write-response-line version code reason-phrase port
261 Write the first line of an HTTP response to @var{port}.
266 @subsection HTTP Requests
269 (use-modules (web request))
275 @defspec request-method
281 @defspec request-version
284 @defspec request-headers
287 @defspec request-meta
290 @defspec request-port
293 @defun read-request port [meta]
294 Read an HTTP request from @var{port}, optionally attaching the given
295 metadata, @var{meta}.
297 As a side effect, sets the encoding on @var{port} to ISO-8859-1
298 (latin-1), so that reading one character reads one byte. See the
299 discussion of character sets in "HTTP Requests" in the manual, for more
303 @defun build-request [#:method] [#:uri] [#:version] [#:headers] [#:port] [#:meta] [#:validate-headers?]
304 Construct an HTTP request object. If @var{validate-headers?} is true,
305 the headers are each run through their respective validators.
308 @defun write-request r port
309 Write the given HTTP request to @var{port}.
311 Returns a new request, whose @code{request-port} will continue writing
312 on @var{port}, perhaps using some transfer encoding.
315 @defun read-request-body/latin-1 r
316 Reads the request body from @var{r}, as a string.
318 Assumes that the request port has ISO-8859-1 encoding, so that the
319 number of characters to read is the same as the
320 @code{request-content-length}. Returns @code{#f} if there was no request
324 @defun write-request-body/latin-1 r body
325 Write @var{body}, a string encodable in ISO-8859-1, to the port
326 corresponding to the HTTP request @var{r}.
329 @defun read-request-body/bytevector r
330 Reads the request body from @var{r}, as a bytevector. Returns @code{#f}
331 if there was no request body.
334 @defun write-request-body/bytevector r bv
335 Write @var{body}, a bytevector, to the port corresponding to the HTTP
339 @defun request-accept request [default='()]
340 @defunx request-accept-charset request [default='()]
341 @defunx request-accept-encoding request [default='()]
342 @defunx request-accept-language request [default='()]
343 @defunx request-allow request [default='()]
344 @defunx request-authorization request [default=#f]
345 @defunx request-cache-control request [default='()]
346 @defunx request-connection request [default='()]
347 @defunx request-content-encoding request [default='()]
348 @defunx request-content-language request [default='()]
349 @defunx request-content-length request [default=#f]
350 @defunx request-content-location request [default=#f]
351 @defunx request-content-md5 request [default=#f]
352 @defunx request-content-range request [default=#f]
353 @defunx request-content-type request [default=#f]
354 @defunx request-date request [default=#f]
355 @defunx request-expect request [default='()]
356 @defunx request-expires request [default=#f]
357 @defunx request-from request [default=#f]
358 @defunx request-host request [default=#f]
359 @defunx request-if-match request [default=#f]
360 @defunx request-if-modified-since request [default=#f]
361 @defunx request-if-none-match request [default=#f]
362 @defunx request-if-range request [default=#f]
363 @defunx request-if-unmodified-since request [default=#f]
364 @defunx request-last-modified request [default=#f]
365 @defunx request-max-forwards request [default=#f]
366 @defunx request-pragma request [default='()]
367 @defunx request-proxy-authorization request [default=#f]
368 @defunx request-range request [default=#f]
369 @defunx request-referer request [default=#f]
370 @defunx request-te request [default=#f]
371 @defunx request-trailer request [default='()]
372 @defunx request-transfer-encoding request [default='()]
373 @defunx request-upgrade request [default='()]
374 @defunx request-user-agent request [default=#f]
375 @defunx request-via request [default='()]
376 @defunx request-warning request [default='()]
379 @defun request-absolute-uri r [default-host] [default-port]
385 @subsection HTTP Responses
388 (use-modules (web response))
395 @defspec response-version
398 @defspec response-code
401 @defun response-reason-phrase response
402 Return the reason phrase given in @var{response}, or the standard reason
403 phrase for the response's code.
406 @defspec response-headers
409 @defspec response-port
412 @defun read-response port
413 Read an HTTP response from @var{port}, optionally attaching the given
414 metadata, @var{meta}.
416 As a side effect, sets the encoding on @var{port} to ISO-8859-1
417 (latin-1), so that reading one character reads one byte. See the
418 discussion of character sets in "HTTP Responses" in the manual, for more
422 @defun build-response [#:version] [#:code] [#:reason-phrase] [#:headers] [#:port]
423 Construct an HTTP response object. If @var{validate-headers?} is true,
424 the headers are each run through their respective validators.
427 @defun extend-response r k v . additional
428 Extend an HTTP response by setting additional HTTP headers @var{k},
429 @var{v}. Returns a new HTTP response.
432 @defun adapt-response-version response version
433 Adapt the given response to a different HTTP version. Returns a new HTTP
436 The idea is that many applications might just build a response for the
437 default HTTP version, and this method could handle a number of
438 programmatic transformations to respond to older HTTP versions (0.9 and
439 1.0). But currently this function is a bit heavy-handed, just updating
443 @defun write-response r port
444 Write the given HTTP response to @var{port}.
446 Returns a new response, whose @code{response-port} will continue writing
447 on @var{port}, perhaps using some transfer encoding.
450 @defun read-response-body/latin-1 r
451 Reads the response body from @var{r}, as a string.
453 Assumes that the response port has ISO-8859-1 encoding, so that the
454 number of characters to read is the same as the
455 @code{response-content-length}. Returns @code{#f} if there was no
459 @defun write-response-body/latin-1 r body
460 Write @var{body}, a string encodable in ISO-8859-1, to the port
461 corresponding to the HTTP response @var{r}.
464 @defun read-response-body/bytevector r
465 Reads the response body from @var{r}, as a bytevector. Returns @code{#f}
466 if there was no response body.
469 @defun write-response-body/bytevector r bv
470 Write @var{body}, a bytevector, to the port corresponding to the HTTP
474 @defun response-accept-ranges response [default=#f]
475 @defunx response-age response [default='()]
476 @defunx response-allow response [default='()]
477 @defunx response-cache-control response [default='()]
478 @defunx response-connection response [default='()]
479 @defunx response-content-encoding response [default='()]
480 @defunx response-content-language response [default='()]
481 @defunx response-content-length response [default=#f]
482 @defunx response-content-location response [default=#f]
483 @defunx response-content-md5 response [default=#f]
484 @defunx response-content-range response [default=#f]
485 @defunx response-content-type response [default=#f]
486 @defunx response-date response [default=#f]
487 @defunx response-etag response [default=#f]
488 @defunx response-expires response [default=#f]
489 @defunx response-last-modified response [default=#f]
490 @defunx response-location response [default=#f]
491 @defunx response-pragma response [default='()]
492 @defunx response-proxy-authenticate response [default=#f]
493 @defunx response-retry-after response [default=#f]
494 @defunx response-server response [default=#f]
495 @defunx response-trailer response [default='()]
496 @defunx response-transfer-encoding response [default='()]
497 @defunx response-upgrade response [default='()]
498 @defunx response-vary response [default='()]
499 @defunx response-via response [default='()]
500 @defunx response-warning response [default='()]
501 @defunx response-www-authenticate response [default=#f]
506 @subsection Web Handlers
508 from request to response
511 @subsection Web Server
513 @code{(web server)} is a generic web server interface, along with a main
514 loop implementation for web servers controlled by Guile.
516 The lowest layer is the <server-impl> object, which defines a set of
517 hooks to open a server, read a request from a client, write a
518 response to a client, and close a server. These hooks -- open,
519 read, write, and close, respectively -- are bound together in a
520 <server-impl> object. Procedures in this module take a
521 <server-impl> object, if needed.
523 A <server-impl> may also be looked up by name. If you pass the
524 @code{http} symbol to @code{run-server}, Guile looks for a variable named
525 @code{http} in the @code{(web server http)} module, which should be bound to a
526 <server-impl> object. Such a binding is made by instantiation of
527 the @code{define-server-impl} syntax. In this way the run-server loop can
528 automatically load other backends if available.
530 The life cycle of a server goes as follows:
534 The @code{open} hook is called, to open the server. @code{open} takes 0 or
535 more arguments, depending on the backend, and returns an opaque
536 server socket object, or signals an error.
539 The @code{read} hook is called, to read a request from a new client.
540 The @code{read} hook takes one arguments, the server socket. It
541 should return three values: an opaque client socket, the
542 request, and the request body. The request should be a
543 @code{<request>} object, from @code{(web request)}. The body should be a
544 string or a bytevector, or @code{#f} if there is no body.
546 If the read failed, the @code{read} hook may return #f for the client
547 socket, request, and body.
550 A user-provided handler procedure is called, with the request
551 and body as its arguments. The handler should return two
552 values: the response, as a @code{<response>} record from @code{(web
553 response)}, and the response body as a string, bytevector, or
554 @code{#f} if not present. We also allow the reponse to be simply an
555 alist of headers, in which case a default response object is
556 constructed with those headers.
559 The @code{write} hook is called with three arguments: the client
560 socket, the response, and the body. The @code{write} hook returns no
564 At this point the request handling is complete. For a loop, we
565 loop back and try to read a new request.
568 If the user interrupts the loop, the @code{close} hook is called on
572 @defspec define-server-impl name open read write close
575 @defun lookup-server-impl impl
576 Look up a server implementation. If @var{impl} is a server
577 implementation already, it is returned directly. If it is a symbol, the
578 binding named @var{impl} in the @code{(web server @var{impl})} module is
579 looked up. Otherwise an error is signaled.
581 Currently a server implementation is a somewhat opaque type, useful only
582 for passing to other procedures in this module, like @code{read-client}.
585 @defun open-server impl open-params
586 Open a server for the given implementation. Returns one value, the new
587 server object. The implementation's @code{open} procedure is applied to
588 @var{open-params}, which should be a list.
591 @defun read-client impl server
592 Read a new client from @var{server}, by applying the implementation's
593 @code{read} procedure to the server. If successful, returns three
594 values: an object corresponding to the client, a request object, and the
595 request body. If any exception occurs, returns @code{#f} for all three
599 @defun handle-request handler request body state
600 Handle a given request, returning the response and body.
602 The response and response body are produced by calling the given
603 @var{handler} with @var{request} and @var{body} as arguments.
605 The elements of @var{state} are also passed to @var{handler} as
606 arguments, and may be returned as additional values. The new
607 @var{state}, collected from the @var{handler}'s return values, is then
608 returned as a list. The idea is that a server loop receives a handler
609 from the user, along with whatever state values the user is interested
610 in, allowing the user's handler to explicitly manage its state.
613 @defun sanitize-response request response body
614 "Sanitize" the given response and body, making them appropriate for the
617 As a convenience to web handler authors, @var{response} may be given as
618 an alist of headers, in which case it is used to construct a default
619 response. Ensures that the response version corresponds to the request
620 version. If @var{body} is a string, encodes the string to a bytevector,
621 in an encoding appropriate for @var{response}. Adds a
622 @code{content-length} and @code{content-type} header, as necessary.
624 If @var{body} is a procedure, it is called with a port as an argument,
625 and the output collected as a bytevector. In the future we might try to
626 instead use a compressing, chunk-encoded port, and call this procedure
627 later, in the write-client procedure. Authors are advised not to rely on
628 the procedure being called at any particular time.
631 @defun write-client impl server client response body
632 Write an HTTP response and body to @var{client}. If the server and
633 client support persistent connections, it is the implementation's
634 responsibility to keep track of the client thereafter, presumably by
635 attaching it to the @var{server} argument somehow.
638 @defun close-server impl server
639 Release resources allocated by a previous invocation of
643 @defun serve-one-client handler impl server state
644 Read one request from @var{server}, call @var{handler} on the request
645 and body, and write the response to the client. Returns the new state
646 produced by the handler procedure.
649 @defun run-server handler [impl] [open-params] . state
650 Run Guile's built-in web server.
652 @var{handler} should be a procedure that takes two or more arguments,
653 the HTTP request and request body, and returns two or more values, the
654 response and response body.
656 For example, here is a simple "Hello, World!" server:
659 (define (handler request body)
660 (values '((content-type . ("text/plain")))
665 The response and body will be run through @code{sanitize-response}
666 before sending back to the client.
668 Additional arguments to @var{handler} are taken from @var{state}.
669 Additional return values are accumulated into a new @var{state}, which
670 will be used for subsequent requests. In this way a handler can
671 explicitly manage its state.
673 The default server implementation is @code{http}, which accepts
674 @var{open-params} like @code{(#:port 8081)}, among others. See "Web
675 Server" in the manual, for more information.
679 (use-modules (web server))
684 @c TeX-master: "guile.texi"