Commit | Line | Data |
---|---|---|
feec80c3 C |
1 | |
2 | ||
3 | README: library "Sexplib" | |
4 | ************************* | |
5 | Copyright (C) 2012 Jane Street Holding, LLC (1) | |
6 | ===================================================== | |
7 | Author: Markus Mottl | |
8 | ====================== | |
9 | New York, 2012-03-20 | |
10 | ==================== | |
11 | ||
12 | ||
13 | 1 Directory contents | |
14 | *=*=*=*=*=*=*=*=*=*=* | |
15 | ||
16 | ||
17 | ||
18 | ------------------------------------------------------------------------ | |
19 | | CHANGES | History of code changes | | |
20 | ------------------------------------------------------------------------ | |
21 | | COPYRIGHT | Notes on copyright | | |
22 | ------------------------------------------------------------------------ | |
23 | | INSTALL | Short notes on compiling and | | |
24 | | | installing the library | | |
25 | ------------------------------------------------------------------------ | |
26 | | LICENSE | "GNU LESSER GENERAL PUBLIC LICENSE" | | |
27 | ------------------------------------------------------------------------ | |
28 | | LICENSE.Tywith | License of Tywith, from which Sexplib is derived| | |
29 | ------------------------------------------------------------------------ | |
30 | | Makefile | Top Makefile | | |
31 | ------------------------------------------------------------------------ | |
32 | | OCamlMakefile | Generic Makefile for OCaml-projects | | |
33 | ------------------------------------------------------------------------ | |
34 | | OMakefile | Ignore this file | | |
35 | ------------------------------------------------------------------------ | |
36 | | README.txt | This file | | |
37 | ------------------------------------------------------------------------ | |
38 | | lib/ | OCaml-library for S-expression conversions | | |
39 | ------------------------------------------------------------------------ | |
40 | | lib_test/ | Test applications for the Sexplib-library | | |
41 | ------------------------------------------------------------------------ | |
42 | ||
43 | ||
44 | ||
45 | ||
46 | 2 What is "Sexplib"? | |
47 | *=*=*=*=*=*=*=*=*=*=* | |
48 | ||
49 | ||
50 | This library contains functionality for parsing and pretty-printing | |
51 | S-expressions. In addition to that it contains a preprocessing module for | |
52 | Camlp4, which can be used to automatically generate code from type definitions | |
53 | for efficiently converting OCaml-values to S-expressions and vice versa. In | |
54 | combination with the parsing and pretty-printing functionality this frees | |
55 | users from having to write their own I/O-routines for datastructures they | |
56 | define. Possible errors during automatic conversions from S-expressions to | |
57 | OCaml-values are reported in human-readable ways with exact location | |
58 | information. Another module in this library allows you to extract and replace | |
59 | sub-expressions in S-expressions. | |
60 | ||
61 | ||
62 | 3 How can you use it? | |
63 | *=*=*=*=*=*=*=*=*=*=*= | |
64 | ||
65 | ||
66 | Make sure you have installed the 'type_conv' package on your system, too. It | |
67 | should be obtainable at the same site as 'sexplib'. | |
68 | ||
69 | The API (.mli-files) in the 'sexplib' library directory is fully documented. | |
70 | Module 'Sexp' contains all I/O-functions for S-expressions, module 'Conv' | |
71 | helper functions for converting OCaml-values of standard types to | |
72 | S-expressions. Module 'Path' supports sub-expression extraction and | |
73 | substitution. | |
74 | ||
75 | Module 'pa_sexp_conv.ml' contains the extensions for the | |
76 | Camlp4-preprocessor. It adds the new construct 'with sexp' (and 'with sexp_of' | |
77 | and 'with of_sexp', which are implied by the first). When using this construct | |
78 | right after a type definition, function definitions will be generated | |
79 | automatically, which perform S-expression conversions. | |
80 | ||
81 | E.g. given the following type definition: | |
82 | << type t = A | B | |
83 | with sexp | |
84 | >> | |
85 | ||
86 | The above will generate the functions 'sexp_of_t' and 't_of_sexp'. The | |
87 | preprocessor also supports automatic addition of conversion functions to | |
88 | signatures. Just add 'with sexp' to the type in a signature, and the | |
89 | appropriate function signatures will be generated. | |
90 | ||
91 | Converters for standard types (int, list, Hashtbl.t, etc.) become visible to | |
92 | the macro-generated code by opening the standard module before their first use | |
93 | in a type definition. Users will therefore usually want to place the following | |
94 | at the top of their files: | |
95 | << open Sexplib.Std | |
96 | >> | |
97 | ||
98 | See the file 'lib_test/conv_test.ml' for example usage. It also demonstrates | |
99 | how to extract and substitute sub-expressions. | |
100 | ||
101 | ||
102 | 3.1 Compiling and linking | |
103 | ========================== | |
104 | ||
105 | To compile a file you will have to add a preprocessing flag to the compiler | |
106 | invocation, e.g. for foo.ml: | |
107 | << ocamlc -pp "camlp4o -I {path to type_conv} \ | |
108 | -I {path to sexplib} pa_type_conv.cmo pa_sexp_conv.cmo" \ | |
109 | -I {path to sexplib} foo.ml | |
110 | >> | |
111 | ||
112 | If you are using OCamlMakefile, just put the following line at the top of | |
113 | the file, assuming you have installed both 'type_conv' and 'sexplib' with | |
114 | ocamlfind. The comment must start at the beginning of the line, and you must | |
115 | not break lines (here broken for readability only): | |
116 | << (*pp camlp4o -I `ocamlfind query type_conv` \ | |
117 | -I `ocamlfind query sexplib` \ | |
118 | pa_type_conv.cmo pa_sexp_conv.cmo *) | |
119 | >> | |
120 | ||
121 | Don't forget to place the macro 'TYPE_CONV_PATH', which takes a string | |
122 | argument, at the top of the file to be preprocessed (see example | |
123 | 'conv_test.ml'). It is supposed to set the module path to the given module | |
124 | file. This is necessary, because modules may get packed at a later stage, and | |
125 | error messages generated by Sexplib will refer to this location to help | |
126 | pinpointing the associated type. | |
127 | ||
128 | In the linking stage you will only have to link with 'sexplib'. E.g. when | |
129 | using OCamlMakefile just add it to the 'PACKS'-variable. | |
130 | ||
131 | ||
132 | 4 Syntax Specification of S-expressions | |
133 | *=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*= | |
134 | ||
135 | ||
136 | ||
137 | ||
138 | 4.1 Lexical conventions of S-expression | |
139 | ======================================== | |
140 | ||
141 | Whitespace, which consists of space, newline, carriage return, horizontal | |
142 | tab and form feed, is ignored unless within an OCaml-string, where it is | |
143 | treated according to OCaml-conventions. The semicolon introduces comments. | |
144 | Comments are ignored, and range up to the next newline character. The left | |
145 | parenthesis opens a new list, the right parenthesis closes it again. Lists can | |
146 | be empty. The double quote denotes the beginning and end of a string following | |
147 | the lexical conventions of OCaml (see OCaml-manual for details). All | |
148 | characters other than double quotes, left- and right parentheses, and | |
149 | whitespace are considered part of a contiguous string. | |
150 | ||
151 | ||
152 | 4.2 Grammar of S-expressions | |
153 | ============================= | |
154 | ||
155 | S-expressions are either strings (= atoms) or lists. The lists can | |
156 | recursively contain further S-expressions or be empty, and must be balanced, | |
157 | i.e. parentheses must match. | |
158 | ||
159 | ||
160 | 4.3 Examples | |
161 | ============= | |
162 | ||
163 | << this_is_an_atom_123'&^%! ; this is a comment | |
164 | "another atom in an OCaml-string \"string in a string\" \123" | |
165 | ||
166 | ; empty list follows below | |
167 | () | |
168 | ||
169 | ; a more complex example | |
170 | ( | |
171 | ( | |
172 | list in a list ; comment within a list | |
173 | (list in a list in a list) | |
174 | 42 is the answer to all questions | |
175 | ) | |
176 | ) | |
177 | >> | |
178 | ||
179 | ||
180 | ||
181 | 4.4 Conversion of basic OCaml-values | |
182 | ===================================== | |
183 | ||
184 | Basic OCaml-values like the unit-value, integers (in all representations), | |
185 | floats, strings, and booleans are represented in S-exp syntax in the same way | |
186 | as in OCaml. Strings may also appear without quotes if this does not clash | |
187 | with the lexical conventions for S-expressions. | |
188 | ||
189 | ||
190 | 4.5 Conversion of OCaml-tuples | |
191 | =============================== | |
192 | ||
193 | OCaml-tuples are simple lists of values in the same order as in the tuple. | |
194 | E.g.: | |
195 | << (3.14, "foo", "bar bla", 27) <===> (3.14 foo "bar bla" 27) | |
196 | >> | |
197 | ||
198 | ||
199 | ||
200 | 4.6 Conversion of OCaml-records | |
201 | ================================ | |
202 | ||
203 | OCaml-records are represented as lists of pairs in S-expression syntax. Each | |
204 | pair consists of the name of the record field (first element), and its value | |
205 | (second element). E.g.: | |
206 | << { | |
207 | foo = 3; | |
208 | bar = "some string"; | |
209 | } | |
210 | ||
211 | <===> | |
212 | ||
213 | ( | |
214 | (foo 3) | |
215 | (bar "some string") | |
216 | ) | |
217 | >> | |
218 | ||
219 | Type specifications of records allow the use of a special type 'sexp_option' | |
220 | which indicates that a record field should be optional. E.g.: | |
221 | << type t = | |
222 | { | |
223 | x : int option; | |
224 | y : int sexp_option; | |
225 | } | |
226 | >> | |
227 | ||
228 | The type 'sexp_option' is equivalent to ordinary options, but is treated | |
229 | specially by the code generator. The above would lead to the following | |
230 | equivalences of values and S-expressions: | |
231 | << { | |
232 | x = Some 1; | |
233 | y = Some 2; | |
234 | } | |
235 | ||
236 | <===> | |
237 | ||
238 | ( | |
239 | (x (some 1)) | |
240 | (y 2) | |
241 | ) | |
242 | >> | |
243 | ||
244 | And: | |
245 | << { | |
246 | x = None; | |
247 | y = None; | |
248 | } | |
249 | ||
250 | <===> | |
251 | ||
252 | ( | |
253 | (x none) | |
254 | ) | |
255 | >> | |
256 | ||
257 | Note how 'sexp_option' allows you to leave away record fields that should | |
258 | default to 'None'. It is also unnecessary (and actually wrong) now to write | |
259 | down such a value as an option, i.e. the 'some'-tag must be dropped if the | |
260 | field should be defined. | |
261 | ||
262 | The types 'sexp_list', 'sexp_array', and 'sexp_bool' can be used in ways | |
263 | similar to the type 'sexp_option'. They assume the empty list, empty array, | |
264 | and false value respectively as default values. | |
265 | ||
266 | ||
267 | 4.7 Conversion of sum types | |
268 | ============================ | |
269 | ||
270 | Constant constructors in sum types are represented as strings. Constructors | |
271 | with arguments are represented as lists, the first element being the | |
272 | constructor name, the rest being its arguments. Constructors may also be | |
273 | started in lowercase in S-expressions, but will always be converted to | |
274 | uppercase when converting from OCaml-values. | |
275 | ||
276 | For example: | |
277 | << type t = A | B of int * float * t with sexp | |
278 | ||
279 | B (42, 3.14, B (-1, 2.72, A)) <===> (B 42 3.14 (B -1 2.72 A)) | |
280 | >> | |
281 | ||
282 | The above example also demonstrates recursion in datastructures. | |
283 | ||
284 | ||
285 | 4.8 Conversion of variant types | |
286 | ================================ | |
287 | ||
288 | The conversion of polymorphic variants is almost the same as with sum types. | |
289 | The notable difference is that variant constructors must always start with an | |
290 | either lower- or uppercase character, matching the way it was specified in the | |
291 | type definition. This is because OCaml also distinguishes between upper- and | |
292 | lowercase variant constructors. Note that type specifications containing | |
293 | unions of variant types are also supported by the S-expression converter. | |
294 | ||
295 | ||
296 | 4.9 Conversion of OCaml-lists and arrays | |
297 | ========================================= | |
298 | ||
299 | OCaml-lists and arrays are straightforwardly represented as S-expression | |
300 | lists. | |
301 | ||
302 | ||
303 | 4.10 Conversion of option types | |
304 | ================================ | |
305 | ||
306 | The option type is converted like ordinary polymorphic sum types, i.e.: | |
307 | << None <===> none | |
308 | Some value <===> (some value) | |
309 | >> | |
310 | ||
311 | There is a deprecated version of the syntax in which values of option type | |
312 | are represented as lists in S-expressions: | |
313 | << None <===> () | |
314 | Some value <===> (value) | |
315 | >> | |
316 | ||
317 | Reading of the old-style S-expression syntax for option values is only | |
318 | supported if the reference 'Conv.read_old_option_format' is set to 'true' | |
319 | (currently the default, which may change soon). A conversion exception is | |
320 | raised otherwise. The old format will be written only if | |
321 | 'Conv.write_old_option_format' is true (also currently the default). Reading | |
322 | of the new format is always supported. | |
323 | ||
324 | ||
325 | 4.11 Conversion of polymorphic values | |
326 | ====================================== | |
327 | ||
328 | There is nothing special about polymorphic values as long as there are | |
329 | conversion functions for the type parameters. E.g.: | |
330 | << type 'a t = A | B of 'a with sexp | |
331 | type foo = int t with sexp | |
332 | >> | |
333 | ||
334 | In the above case the conversion functions will behave as if 'foo' had been | |
335 | defined as a monomorphic version of 't' with ''a' replaced by 'int' on the | |
336 | right hand side. | |
337 | ||
338 | If a datastructure is indeed polymorphic, and you want to convert it, you | |
339 | will have to supply the conversion functions for the type parameters at | |
340 | runtime. E.g. in the above example, if you wanted to convert a value of type | |
341 | ''a t', you would have to write something like this: | |
342 | << sexp_of_t sexp_of_a v | |
343 | >> | |
344 | ||
345 | where 'sexp_of_a', which may also be named differently in this particular | |
346 | case, is a function that converts values of type ''a' to an S-expression. | |
347 | Types with more than one parameter require passing conversion functions for | |
348 | those parameters in the order of their appearance on the left hand side of the | |
349 | type definition. | |
350 | ||
351 | ||
352 | 4.12 Conversion of abstract datatypes | |
353 | ====================================== | |
354 | ||
355 | Of course, if you want to convert an abstract datatype to an S-expression, | |
356 | you will have to roll your own conversion function, which should produce | |
357 | values of type 'Sexp.t' directly. If, however, you want to make use of your | |
358 | abstract type within definitions of other types, make sure that you call your | |
359 | conversion function appropriately: it should be in the same scope as the | |
360 | typename, and must be named 'sexp_of_{typename}'. | |
361 | ||
362 | ||
363 | 4.13 Conversion of hashtables | |
364 | ============================== | |
365 | ||
366 | Hashtables, which are abstract values in OCaml, are represented as | |
367 | association lists, i.e. lists of key-value pairs, e.g.: | |
368 | << ((foo 42) (bar 3)) | |
369 | >> | |
370 | ||
371 | Reading in the above S-expression as hashtable mapping strings to integers | |
372 | ('(string, int) Hashtbl.t') will map '"foo"' to 42 and '"bar"' to 3. | |
373 | ||
374 | Note that the order of elements in the list may matter, because duplicates | |
375 | are kept: bindings will be inserted into the hashtable in order of appearence. | |
376 | Therefore, the last binding of a key will be the "visible" one, the others are | |
377 | "hidden". See the OCaml-documentation on hashtables for details. | |
378 | ||
379 | Note, too, that polymorphic equality may not hold between conversions. You | |
380 | will have to use a function implementing logical equality for that purpose. | |
381 | ||
382 | ||
383 | 4.14 Conversion of opaque values | |
384 | ================================= | |
385 | ||
386 | Opaque values are ones for which we do not want to perform conversions. This | |
387 | may be, because we do not have S-expression converters for them, or because we | |
388 | do not want to apply them in a particular type context, e.g. if the resulting | |
389 | S-expression should be printed out but without superfluous information. To | |
390 | prevent the preprocessor from generating calls to converters, simply apply the | |
391 | qualifier 'sexp_opaque' as if it were a type constructor, e.g.: | |
392 | << type foo = int * stuff sexp_opaque with sexp | |
393 | >> | |
394 | ||
395 | Thus, there is no need to specify converters for type 'stuff', and if there | |
396 | are any, they will not be used in this particular context. Needless to say, it | |
397 | is not possible to convert such an S-expression back to the original value. | |
398 | Here is an example conversion: | |
399 | << (42, some_stuff) ===> (42, <opaque>) | |
400 | >> | |
401 | ||
402 | ||
403 | ||
404 | 4.15 Conversion of exceptions | |
405 | ============================== | |
406 | ||
407 | S-expression converters for exceptions can be automatically registered using | |
408 | the 'with sexp' macro, e.g.: | |
409 | << module M = struct | |
410 | exception Foo of int with sexp | |
411 | end | |
412 | >> | |
413 | ||
414 | Such exceptions will be translated in a similar way as sum types, but their | |
415 | constructor will be prefixed with the fully qualified module path (here: | |
416 | 'M.Foo') so as to be able to discriminate between them without problems. | |
417 | The user can then easily convert an exception matching the above one to an | |
418 | S-expression using 'Sexplib.Conv.sexp_of_exn'. User-defined conversion | |
419 | functions can be registered, too, by calling 'Sexplib.Conv.add_exn_converter'. | |
420 | This should make it very convenient for users to catch arbitrary exceptions | |
421 | escaping their program and pretty-printing them, including all arguments, as | |
422 | S-expressions. The library already contains mappings for all known exceptions | |
423 | that can escape functions in the OCaml standard library. | |
424 | ||
425 | ||
426 | 5 I/O and type conversions | |
427 | *=*=*=*=*=*=*=*=*=*=*=*=*=* | |
428 | ||
429 | ||
430 | There are multiple ways of performing I/O with S-expressions. If exact error | |
431 | locations are required when type conversions fail, S-expressions need to be | |
432 | parsed with location annotations. In most cases users may want to use | |
433 | functions like e.g. 'load_sexp_conv' or 'load_sexp_conv_exn', which load | |
434 | S-expressions from files and convert them. Only when conversions fail, the | |
435 | file will be reparsed with annotations, which is slower, and type errors will | |
436 | be reported accurately with file, line number, column, and file position. | |
437 | ||
438 | ||
439 | 6 Contact information | |
440 | *=*=*=*=*=*=*=*=*=*=*= | |
441 | ||
442 | ||
443 | In the case of bugs, feature requests and similar, you can contact us here: | |
444 | opensource@janestreet.com | |
445 | Up-to-date information concerning this library should be available here: | |
446 | http://www.janestreet.com/ocaml | |
447 | Enjoy!! | |
448 | ||
449 | ----------------------------------------------------------------------------- | |
450 | ||
451 | This document was translated from LaTeX by HeVeA (2). | |
452 | -------------------------------------- | |
453 | ||
454 | ||
455 | (1) http://www.janestreet.com | |
456 | ||
457 | (2) http://hevea.inria.fr/index.html |