doc/guide/localhost/MLLex

   1 <!DOCTYPE html>
   2 <html lang="en">
   3 <head>
   4 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
   5 <meta name="generator" content="AsciiDoc 8.6.9">
   6 <title>MLLex</title>
   7 <link rel="stylesheet" href="./asciidoc.css" type="text/css">
   8 <link rel="stylesheet" href="./pygments.css" type="text/css">
   9
  10
  11 <script type="text/javascript" src="./asciidoc.js"></script>
  12 <script type="text/javascript">
  13 /*<![CDATA[*/
  14 asciidoc.install();
  15 /*]]>*/
  16 </script>
  17 <link rel="stylesheet" href="./mlton.css" type="text/css">
  18 </head>
  19 <body class="article">
  20 <div id="banner">
  21 <div id="banner-home">
  22 <a href="./Home">MLton 20180207</a>
  23 </div>
  24 </div>
  25 <div id="header">
  26 <h1>MLLex</h1>
  27 </div>
  28 <div id="content">
  29 <div id="preamble">
  30 <div class="sectionbody">
  31 <div class="paragraph"><p><a href="MLLex">MLLex</a> is a lexical analyzer generator for <a href="StandardML">Standard ML</a>
  32 modeled after the Lex lexical analyzer generator.</p></div>
  33 <div class="paragraph"><p>A version of MLLex, ported from the <a href="SMLNJ">SML/NJ</a> sources, is
  34 distributed with MLton.</p></div>
  35 </div>
  36 </div>
  37 <div class="sect1">
  38 <h2 id="_description">Description</h2>
  39 <div class="sectionbody">
  40 <div class="paragraph"><p>MLLex takes as input the lex language as defined in the ML-Lex manual,
  41 and outputs a lexical analyzer in SML.</p></div>
  42 </div>
  43 </div>
  44 <div class="sect1">
  45 <h2 id="_implementation">Implementation</h2>
  46 <div class="sectionbody">
  47 <div class="ulist"><ul>
  48 <li>
  49 <p>
  50 <a href="https://github.com/MLton/mlton/blob/master/mllex/lexgen.sml"><span class="monospaced">lexgen.sml</span></a>
  51 </p>
  52 </li>
  53 <li>
  54 <p>
  55 <a href="https://github.com/MLton/mlton/blob/master/mllex/main.sml"><span class="monospaced">main.sml</span></a>
  56 </p>
  57 </li>
  58 <li>
  59 <p>
  60 <a href="https://github.com/MLton/mlton/blob/master/mllex/call-main.sml"><span class="monospaced">call-main.sml</span></a>
  61 </p>
  62 </li>
  63 </ul></div>
  64 </div>
  65 </div>
  66 <div class="sect1">
  67 <h2 id="_details_and_notes">Details and Notes</h2>
  68 <div class="sectionbody">
  69 <div class="paragraph"><p>There are 3 main passes in the MLLex tool:</p></div>
  70 <div class="ulist"><ul>
  71 <li>
  72 <p>
  73 Source parsing. In this pass, lex source program are parsed into internal representations. The core part of this pass is a hand-written lexer and an LL(1) parser. The output of this pass is a record of user code, rules (along with start states) and actions. (MLLex definitions are wiped off.)
  74 </p>
  75 </li>
  76 <li>
  77 <p>
  78 DFA construction. In this pass, a DFA is constructed by the algorithm of H. Yamada et. al.
  79 </p>
  80 </li>
  81 <li>
  82 <p>
  83 Output. In this pass, the generated DFA is written out as a transition table, along with a table-driven algorithm, to an SML file.
  84 </p>
  85 </li>
  86 </ul></div>
  87 </div>
  88 </div>
  89 <div class="sect1">
  90 <h2 id="_also_see">Also see</h2>
  91 <div class="sectionbody">
  92 <div class="ulist"><ul>
  93 <li>
  94 <p>
  95 <a href="Documentation.attachments/mllex.pdf"><span class="monospaced">mllex.pdf</span></a>
  96 </p>
  97 </li>
  98 <li>
  99 <p>
 100 <a href="MLYacc">MLYacc</a>
 101 </p>
 102 </li>
 103 <li>
 104 <p>
 105 <a href="References#AppelEtAl94">AppelEtAl94</a>
 106 </p>
 107 </li>
 108 <li>
 109 <p>
 110 <a href="References#Price09">Price09</a>
 111 </p>
 112 </li>
 113 </ul></div>
 114 </div>
 115 </div>
 116 </div>
 117 <div id="footnotes"><hr></div>
 118 <div id="footer">
 119 <div id="footer-text">
 120 </div>
 121 <div id="footer-badges">
 122 </div>
 123 </div>
 124 </body>
 125 </html>