doc/guide/src/MLLex.adoc

   1 MLLex
   2 =====
   3
   4 <:MLLex:> is a lexical analyzer generator for <:StandardML:Standard ML>
   5 modeled after the Lex lexical analyzer generator.
   6
   7 A version of MLLex, ported from the <:SMLNJ:SML/NJ> sources, is
   8 distributed with MLton.
   9
  10 == Description ==
  11
  12 MLLex takes as input the lex language as defined in the ML-Lex manual,
  13 and outputs a lexical analyzer in SML.
  14
  15 == Implementation ==
  16
  17 * <!ViewGitFile(mlton,master,mllex/lexgen.sml)>
  18 * <!ViewGitFile(mlton,master,mllex/main.sml)>
  19 * <!ViewGitFile(mlton,master,mllex/call-main.sml)>
  20
  21 == Details and Notes ==
  22
  23 There are 3 main passes in the MLLex tool:
  24
  25 * Source parsing. In this pass, lex source program are parsed into internal representations. The core part of this pass is a hand-written lexer and an LL(1) parser. The output of this pass is a record of user code, rules (along with start states) and actions. (MLLex definitions are wiped off.)
  26 * DFA construction. In this pass, a DFA is constructed by the algorithm of H. Yamada et. al.
  27 * Output. In this pass, the generated DFA is written out as a transition table, along with a table-driven algorithm, to an SML file.
  28
  29 == Also see ==
  30
  31 * <!Attachment(Documentation,mllex.pdf)>
  32 * <:MLYacc:>
  33 * <!Cite(AppelEtAl94)>
  34 * <!Cite(Price09)>