doc/guide/localhost/MLNLFFIImplementation

   1 <!DOCTYPE html>
   2 <html lang="en">
   3 <head>
   4 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
   5 <meta name="generator" content="AsciiDoc 8.6.9">
   6 <title>MLNLFFIImplementation</title>
   7 <link rel="stylesheet" href="./asciidoc.css" type="text/css">
   8 <link rel="stylesheet" href="./pygments.css" type="text/css">
   9
  10
  11 <script type="text/javascript" src="./asciidoc.js"></script>
  12 <script type="text/javascript">
  13 /*<![CDATA[*/
  14 asciidoc.install();
  15 /*]]>*/
  16 </script>
  17 <link rel="stylesheet" href="./mlton.css" type="text/css">
  18 </head>
  19 <body class="article">
  20 <div id="banner">
  21 <div id="banner-home">
  22 <a href="./Home">MLton 20180207</a>
  23 </div>
  24 </div>
  25 <div id="header">
  26 <h1>MLNLFFIImplementation</h1>
  27 </div>
  28 <div id="content">
  29 <div id="preamble">
  30 <div class="sectionbody">
  31 <div class="paragraph"><p>MLton&#8217;s implementation(s) of the <a href="MLNLFFI">MLNLFFI</a> library differs from the
  32 SML/NJ implementation in two important ways:</p></div>
  33 <div class="ulist"><ul>
  34 <li>
  35 <p>
  36 MLton cannot utilize the <span class="monospaced">Unsafe.cast</span> "cheat" described in Section
  37 3.7 of <a href="References#Blume01">Blume01</a>.  (MLton&#8217;s representation of
  38 <a href="Closure">closures</a> and
  39 <a href="PackedRepresentation">aggressive representation</a> optimizations make
  40 an <span class="monospaced">Unsafe.cast</span> even more "unsafe" than in other implementations.)
  41 </p>
  42 <div class="openblock">
  43 <div class="content">
  44 <div class="paragraph"><p>We have considered two solutions:</p></div>
  45 <div class="ulist"><ul>
  46 <li>
  47 <p>
  48 One solution is to utilize an additional type parameter (as
  49 described in Section 3.7 of <a href="References#Blume01">Blume01</a>):
  50 </p>
  51 <div class="openblock">
  52 <div class="content">
  53 <div class="quoteblock">
  54 <div class="content">
  55 <div class="listingblock">
  56 <div class="content"><div class="highlight"><pre><span class="k">signature</span><span class="w"> </span><span class="n">C</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="k">sig</span><span class="w"></span>
  57 <span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;t</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;c</span><span class="p">)</span><span class="w"> </span><span class="n">obj</span><span class="w"></span>
  58 <span class="w">    </span><span class="k">eqtype</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;t</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;c</span><span class="p">)</span><span class="w"> </span><span class="n">obj&#39;</span><span class="w"></span>
  59 <span class="w">    </span><span class="p">...</span><span class="w"></span>
  60 <span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;o</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">)</span><span class="w"> </span><span class="n">ptr</span><span class="w"></span>
  61 <span class="w">    </span><span class="k">eqtype</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;o</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">)</span><span class="w"> </span><span class="n">ptr&#39;</span><span class="w"></span>
  62 <span class="w">    </span><span class="p">...</span><span class="w"></span>
  63 <span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="n">&#39;f</span><span class="w"> </span><span class="n">fptr</span><span class="w"></span>
  64 <span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="n">&#39;f</span><span class="w"> </span><span class="n">ptr&#39;</span><span class="w"></span>
  65 <span class="w">    </span><span class="p">...</span><span class="w"></span>
  66 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">T</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="k">sig</span><span class="w"></span>
  67 <span class="w">        </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;t</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">)</span><span class="w"> </span><span class="n">typ</span><span class="w"></span>
  68 <span class="w">        </span><span class="p">...</span><span class="w"></span>
  69 <span class="w">    </span><span class="k">end</span><span class="w"></span>
  70 <span class="k">end</span><span class="w"></span>
  71 </pre></div></div></div>
  72 <div class="paragraph"><p>The rule for <span class="monospaced">('t, 'f, 'c) obj</span>,<span class="monospaced">('t, 'f, 'c) ptr</span>, and also <span class="monospaced">('t, 'f)
  73 T.typ</span> is that whenever <span class="monospaced">F fptr</span> occurs within the instantiation of
  74 <span class="monospaced">'t</span>, then <span class="monospaced">'f</span> must be instantiated to <span class="monospaced">F</span>.  In all other cases, <span class="monospaced">'f</span>
  75 will be instantiated to <span class="monospaced">unit</span>.</p></div>
  76 </div>
  77 <div class="attribution">
  78 </div></div>
  79 <div class="paragraph"><p>(In the actual MLton implementation, an abstract type <span class="monospaced">naf</span>
  80 (not-a-function) is used instead of <span class="monospaced">unit</span>.)</p></div>
  81 <div class="paragraph"><p>While this means that type-annotated programs may not type-check under
  82 both the SML/NJ implementation and the MLton implementation, this
  83 should not be a problem in practice.  Tools, like <span class="monospaced">ml-nlffigen</span>, which
  84 are necessarily implementation dependent (in order to make
  85 <a href="CallingFromSMLToCFunctionPointer">calls through a C function
  86 pointer</a>), may be easily extended to emit the additional type
  87 parameter.  Client code which uses such generated glue-code (e.g.,
  88 Section 1 of <a href="References#Blume01">Blume01</a>) need rarely write type-annotations,
  89 thanks to the magic of type inference.</p></div>
  90 </div></div>
  91 </li>
  92 <li>
  93 <p>
  94 The above implementation suffers from two disadvantages.
  95 </p>
  96 <div class="openblock">
  97 <div class="content">
  98 <div class="paragraph"><p>First, it changes the MLNLFFI Library interface, meaning that the same
  99 program may not type-check under both the SML/NJ implementation and
 100 the MLton implementation (though, in light of type inference and the
 101 richer <span class="monospaced">MLRep</span> structure provided by MLton, this point is mostly
 102 moot).</p></div>
 103 <div class="paragraph"><p>Second, it appears to unnecessarily duplicate type information.  For
 104 example, an external C variable of type <span class="monospaced">int (* f[3])(int)</span> (that is,
 105 an array of three function pointers), would be represented by the SML
 106 type <span class="monospaced">(((sint -&gt; sint) fptr, dec dg3) arr, sint -&gt; sint, rw) obj</span>.
 107 One might well ask why the <span class="monospaced">'f</span> instantiation (<span class="monospaced">sint -&gt; sint</span> in this
 108 case) cannot be <em>extracted</em> from the <span class="monospaced">'t</span> instantiation
 109 (<span class="monospaced">((sint -&gt; sint) fptr, dec dg3) arr</span> in this case), obviating the
 110 need for a separate <em>function-type</em> type argument.  There are a number
 111 of components to an complete answer to this question.  Foremost is the
 112 fact that <a href="StandardML"> Standard ML</a> supports neither (general)
 113 type-level functions nor intensional polymorphism.</p></div>
 114 <div class="paragraph"><p>A more direct answer for MLNLFFI is that in the SML/NJ implemention,
 115 the definition of the types <span class="monospaced">('t, 'c) obj</span> and <span class="monospaced">('t, 'c) ptr</span> are made
 116 in such a way that the type variables <span class="monospaced">'t</span> and <span class="monospaced">'c</span> are <a href="PhantomType">
 117 phantom</a> (not contributing to the run-time representation of an
 118 <span class="monospaced">('t, 'c) obj</span> or <span class="monospaced">('t, 'c) ptr</span> value), despite the fact that the
 119 types <span class="monospaced">((sint -&gt; sint) fptr, rw) ptr</span> and
 120 <span class="monospaced">((double -&gt; double) fptr, rw) ptr</span> necessarily carry distinct (and
 121 type incompatible) run-time (C-)type information (RTTI), corresponding
 122 to the different calling conventions of the two C functions.  The
 123 <span class="monospaced">Unsafe.cast</span> "cheat" overcomes the type incompatibility without
 124 introducing a new type variable (as in the first solution above).</p></div>
 125 <div class="paragraph"><p>Hence, the reason that <em>function-type</em> type cannot be extracted from
 126 the <span class="monospaced">'t</span> type variable instantiation is that the type of the
 127 representation of RTTI doesn&#8217;t even <em>see</em> the (phantom) <span class="monospaced">'t</span> type
 128 variable.  The solution which presents itself is to give up on the
 129 phantomness of the <span class="monospaced">'t</span> type variable, making it available to the
 130 representation of RTTI.</p></div>
 131 <div class="paragraph"><p>This is not without some small drawbacks.  Because many of the types
 132 used to instantiate <span class="monospaced">'t</span> carry more structure than is strictly
 133 necessary for <span class="monospaced">'t</span>'s RTTI, it is sometimes necessary to wrap and
 134 unwrap RTTI to accommodate the additional structure.  (In the other
 135 implementations, the corresponding operations can pass along the RTTI
 136 unchanged.)  However, these coercions contribute minuscule overhead;
 137 in fact, in a majority of cases, MLton&#8217;s optimizations will completely
 138 eliminate the RTTI from the final program.</p></div>
 139 </div></div>
 140 </li>
 141 </ul></div>
 142 <div class="paragraph"><p>The implementation distributed with MLton uses the second solution.</p></div>
 143 <div class="paragraph"><p>Bonus question: Why can&#8217;t one use a <a href="UniversalType"> universal type</a>
 144 to eliminate the use of <span class="monospaced">Unsafe.cast</span>?</p></div>
 145 <div class="ulist"><ul>
 146 <li>
 147 <p>
 148 Answer: ???
 149 </p>
 150 </li>
 151 </ul></div>
 152 </div></div>
 153 </li>
 154 <li>
 155 <p>
 156 MLton (in both of the above implementations) provides a richer
 157 <span class="monospaced">MLRep</span> structure, utilizing <span class="monospaced">Int<em>&lt;N&gt;</em></span> and <span class="monospaced">Word<em>&lt;N&gt;</em></span>
 158 structures.
 159 </p>
 160 <div class="openblock">
 161 <div class="content">
 162 <div class="listingblock">
 163 <div class="content"><div class="highlight"><pre><span class="k">structure</span><span class="w"> </span><span class="n">MLRep</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="k">struct</span><span class="w"></span>
 164 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Char</span><span class="w"> </span><span class="p">=</span><span class="w"></span>
 165 <span class="w">       </span><span class="k">struct</span><span class="w"></span>
 166 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int8</span><span class="w"></span>
 167 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word8</span><span class="w"></span>
 168 <span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>
 169 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>
 170 <span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>
 171 <span class="w">       </span><span class="k">end</span><span class="w"></span>
 172 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Short</span><span class="w"> </span><span class="p">=</span><span class="w"></span>
 173 <span class="w">       </span><span class="k">struct</span><span class="w"></span>
 174 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int16</span><span class="w"></span>
 175 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word16</span><span class="w"></span>
 176 <span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>
 177 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>
 178 <span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>
 179 <span class="w">       </span><span class="k">end</span><span class="w"></span>
 180 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Int</span><span class="w"> </span><span class="p">=</span><span class="w"></span>
 181 <span class="w">       </span><span class="k">struct</span><span class="w"></span>
 182 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int32</span><span class="w"></span>
 183 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word32</span><span class="w"></span>
 184 <span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>
 185 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>
 186 <span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>
 187 <span class="w">       </span><span class="k">end</span><span class="w"></span>
 188 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Long</span><span class="w"> </span><span class="p">=</span><span class="w"></span>
 189 <span class="w">       </span><span class="k">struct</span><span class="w"></span>
 190 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int32</span><span class="w"></span>
 191 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word32</span><span class="w"></span>
 192 <span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>
 193 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>
 194 <span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>
 195 <span class="w">       </span><span class="k">end</span><span class="w"></span>
 196 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:LongLong</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"></span>
 197 <span class="w">       </span><span class="k">struct</span><span class="w"></span>
 198 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int64</span><span class="w"></span>
 199 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word64</span><span class="w"></span>
 200 <span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>
 201 <span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>
 202 <span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>
 203 <span class="w">       </span><span class="k">end</span><span class="w"></span>
 204 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Float</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Real32</span><span class="w"></span>
 205 <span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Real64</span><span class="w"></span>
 206 <span class="k">end</span><span class="w"></span>
 207 </pre></div></div></div>
 208 <div class="paragraph"><p>This would appear to be a better interface, even when an
 209 implementation must choose <span class="monospaced">Int32</span> and <span class="monospaced">Word32</span> as the representation
 210 for smaller C-types.</p></div>
 211 </div></div>
 212 </li>
 213 </ul></div>
 214 </div>
 215 </div>
 216 </div>
 217 <div id="footnotes"><hr></div>
 218 <div id="footer">
 219 <div id="footer-text">
 220 </div>
 221 <div id="footer-badges">
 222 </div>
 223 </div>
 224 </body>
 225 </html>