[hcoop/debian/mlton.git] / doc / guide / localhost / MLNLFFIImplementation

<!DOCTYPE html>\r
<html lang="en">\r
<head>\r
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">\r
<meta name="generator" content="AsciiDoc 8.6.9">\r
<title>MLNLFFIImplementation</title>\r
<link rel="stylesheet" href="./asciidoc.css" type="text/css">\r
<link rel="stylesheet" href="./pygments.css" type="text/css">\r
\r
\r
<script type="text/javascript" src="./asciidoc.js"></script>\r
<script type="text/javascript">\r
/*<![CDATA[*/\r
asciidoc.install();\r
/*]]>*/\r
</script>\r
<link rel="stylesheet" href="./mlton.css" type="text/css">\r
</head>\r
<body class="article">\r
<div id="banner">\r
<div id="banner-home">\r
<a href="./Home">MLton 20180207</a>\r
</div>\r
</div>\r
<div id="header">\r
<h1>MLNLFFIImplementation</h1>\r
</div>\r
<div id="content">\r
<div id="preamble">\r
<div class="sectionbody">\r
<div class="paragraph"><p>MLton&#8217;s implementation(s) of the <a href="MLNLFFI">MLNLFFI</a> library differs from the\r
SML/NJ implementation in two important ways:</p></div>\r
<div class="ulist"><ul>\r
<li>\r
<p>\r
MLton cannot utilize the <span class="monospaced">Unsafe.cast</span> "cheat" described in Section\r
3.7 of <a href="References#Blume01">Blume01</a>.  (MLton&#8217;s representation of\r
<a href="Closure">closures</a> and\r
<a href="PackedRepresentation">aggressive representation</a> optimizations make\r
an <span class="monospaced">Unsafe.cast</span> even more "unsafe" than in other implementations.)\r
</p>\r
<div class="openblock">\r
<div class="content">\r
<div class="paragraph"><p>We have considered two solutions:</p></div>\r
<div class="ulist"><ul>\r
<li>\r
<p>\r
One solution is to utilize an additional type parameter (as\r
described in Section 3.7 of <a href="References#Blume01">Blume01</a>):\r
</p>\r
<div class="openblock">\r
<div class="content">\r
<div class="quoteblock">\r
<div class="content">\r
<div class="listingblock">\r
<div class="content"><div class="highlight"><pre><span class="k">signature</span><span class="w"> </span><span class="n">C</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="k">sig</span><span class="w"></span>\r
<span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;t</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;c</span><span class="p">)</span><span class="w"> </span><span class="n">obj</span><span class="w"></span>\r
<span class="w">    </span><span class="k">eqtype</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;t</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;c</span><span class="p">)</span><span class="w"> </span><span class="n">obj&#39;</span><span class="w"></span>\r
<span class="w">    </span><span class="p">...</span><span class="w"></span>\r
<span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;o</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">)</span><span class="w"> </span><span class="n">ptr</span><span class="w"></span>\r
<span class="w">    </span><span class="k">eqtype</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;o</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">)</span><span class="w"> </span><span class="n">ptr&#39;</span><span class="w"></span>\r
<span class="w">    </span><span class="p">...</span><span class="w"></span>\r
<span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="n">&#39;f</span><span class="w"> </span><span class="n">fptr</span><span class="w"></span>\r
<span class="w">    </span><span class="k">type</span><span class="w"> </span><span class="n">&#39;f</span><span class="w"> </span><span class="n">ptr&#39;</span><span class="w"></span>\r
<span class="w">    </span><span class="p">...</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">T</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="k">sig</span><span class="w"></span>\r
<span class="w">        </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">&#39;t</span><span class="p">,</span><span class="w"> </span><span class="n">&#39;f</span><span class="p">)</span><span class="w"> </span><span class="n">typ</span><span class="w"></span>\r
<span class="w">        </span><span class="p">...</span><span class="w"></span>\r
<span class="w">    </span><span class="k">end</span><span class="w"></span>\r
<span class="k">end</span><span class="w"></span>\r
</pre></div></div></div>\r
<div class="paragraph"><p>The rule for <span class="monospaced">('t, 'f, 'c) obj</span>,<span class="monospaced">('t, 'f, 'c) ptr</span>, and also <span class="monospaced">('t, 'f)\r
T.typ</span> is that whenever <span class="monospaced">F fptr</span> occurs within the instantiation of\r
<span class="monospaced">'t</span>, then <span class="monospaced">'f</span> must be instantiated to <span class="monospaced">F</span>.  In all other cases, <span class="monospaced">'f</span>\r
will be instantiated to <span class="monospaced">unit</span>.</p></div>\r
</div>\r
<div class="attribution">\r
</div></div>\r
<div class="paragraph"><p>(In the actual MLton implementation, an abstract type <span class="monospaced">naf</span>\r
(not-a-function) is used instead of <span class="monospaced">unit</span>.)</p></div>\r
<div class="paragraph"><p>While this means that type-annotated programs may not type-check under\r
both the SML/NJ implementation and the MLton implementation, this\r
should not be a problem in practice.  Tools, like <span class="monospaced">ml-nlffigen</span>, which\r
are necessarily implementation dependent (in order to make\r
<a href="CallingFromSMLToCFunctionPointer">calls through a C function\r
pointer</a>), may be easily extended to emit the additional type\r
parameter.  Client code which uses such generated glue-code (e.g.,\r
Section 1 of <a href="References#Blume01">Blume01</a>) need rarely write type-annotations,\r
thanks to the magic of type inference.</p></div>\r
</div></div>\r
</li>\r
<li>\r
<p>\r
The above implementation suffers from two disadvantages.\r
</p>\r
<div class="openblock">\r
<div class="content">\r
<div class="paragraph"><p>First, it changes the MLNLFFI Library interface, meaning that the same\r
program may not type-check under both the SML/NJ implementation and\r
the MLton implementation (though, in light of type inference and the\r
richer <span class="monospaced">MLRep</span> structure provided by MLton, this point is mostly\r
moot).</p></div>\r
<div class="paragraph"><p>Second, it appears to unnecessarily duplicate type information.  For\r
example, an external C variable of type <span class="monospaced">int (* f[3])(int)</span> (that is,\r
an array of three function pointers), would be represented by the SML\r
type <span class="monospaced">(((sint -&gt; sint) fptr, dec dg3) arr, sint -&gt; sint, rw) obj</span>.\r
One might well ask why the <span class="monospaced">'f</span> instantiation (<span class="monospaced">sint -&gt; sint</span> in this\r
case) cannot be <em>extracted</em> from the <span class="monospaced">'t</span> instantiation\r
(<span class="monospaced">((sint -&gt; sint) fptr, dec dg3) arr</span> in this case), obviating the\r
need for a separate <em>function-type</em> type argument.  There are a number\r
of components to an complete answer to this question.  Foremost is the\r
fact that <a href="StandardML"> Standard ML</a> supports neither (general)\r
type-level functions nor intensional polymorphism.</p></div>\r
<div class="paragraph"><p>A more direct answer for MLNLFFI is that in the SML/NJ implemention,\r
the definition of the types <span class="monospaced">('t, 'c) obj</span> and <span class="monospaced">('t, 'c) ptr</span> are made\r
in such a way that the type variables <span class="monospaced">'t</span> and <span class="monospaced">'c</span> are <a href="PhantomType">\r
phantom</a> (not contributing to the run-time representation of an\r
<span class="monospaced">('t, 'c) obj</span> or <span class="monospaced">('t, 'c) ptr</span> value), despite the fact that the\r
types <span class="monospaced">((sint -&gt; sint) fptr, rw) ptr</span> and\r
<span class="monospaced">((double -&gt; double) fptr, rw) ptr</span> necessarily carry distinct (and\r
type incompatible) run-time (C-)type information (RTTI), corresponding\r
to the different calling conventions of the two C functions.  The\r
<span class="monospaced">Unsafe.cast</span> "cheat" overcomes the type incompatibility without\r
introducing a new type variable (as in the first solution above).</p></div>\r
<div class="paragraph"><p>Hence, the reason that <em>function-type</em> type cannot be extracted from\r
the <span class="monospaced">'t</span> type variable instantiation is that the type of the\r
representation of RTTI doesn&#8217;t even <em>see</em> the (phantom) <span class="monospaced">'t</span> type\r
variable.  The solution which presents itself is to give up on the\r
phantomness of the <span class="monospaced">'t</span> type variable, making it available to the\r
representation of RTTI.</p></div>\r
<div class="paragraph"><p>This is not without some small drawbacks.  Because many of the types\r
used to instantiate <span class="monospaced">'t</span> carry more structure than is strictly\r
necessary for <span class="monospaced">'t</span>'s RTTI, it is sometimes necessary to wrap and\r
unwrap RTTI to accommodate the additional structure.  (In the other\r
implementations, the corresponding operations can pass along the RTTI\r
unchanged.)  However, these coercions contribute minuscule overhead;\r
in fact, in a majority of cases, MLton&#8217;s optimizations will completely\r
eliminate the RTTI from the final program.</p></div>\r
</div></div>\r
</li>\r
</ul></div>\r
<div class="paragraph"><p>The implementation distributed with MLton uses the second solution.</p></div>\r
<div class="paragraph"><p>Bonus question: Why can&#8217;t one use a <a href="UniversalType"> universal type</a>\r
to eliminate the use of <span class="monospaced">Unsafe.cast</span>?</p></div>\r
<div class="ulist"><ul>\r
<li>\r
<p>\r
Answer: ???\r
</p>\r
</li>\r
</ul></div>\r
</div></div>\r
</li>\r
<li>\r
<p>\r
MLton (in both of the above implementations) provides a richer\r
<span class="monospaced">MLRep</span> structure, utilizing <span class="monospaced">Int<em>&lt;N&gt;</em></span> and <span class="monospaced">Word<em>&lt;N&gt;</em></span>\r
structures.\r
</p>\r
<div class="openblock">\r
<div class="content">\r
<div class="listingblock">\r
<div class="content"><div class="highlight"><pre><span class="k">structure</span><span class="w"> </span><span class="n">MLRep</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="k">struct</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Char</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r
<span class="w">       </span><span class="k">struct</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int8</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word8</span><span class="w"></span>\r
<span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r
<span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r
<span class="w">       </span><span class="k">end</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Short</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r
<span class="w">       </span><span class="k">struct</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int16</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word16</span><span class="w"></span>\r
<span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r
<span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r
<span class="w">       </span><span class="k">end</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Int</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r
<span class="w">       </span><span class="k">struct</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int32</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word32</span><span class="w"></span>\r
<span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r
<span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r
<span class="w">       </span><span class="k">end</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Long</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r
<span class="w">       </span><span class="k">struct</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int32</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word32</span><span class="w"></span>\r
<span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r
<span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r
<span class="w">       </span><span class="k">end</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:LongLong</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r
<span class="w">       </span><span class="k">struct</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int64</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word64</span><span class="w"></span>\r
<span class="w">          </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r
<span class="w">          </span><span class="k">structure</span><span class="w"> </span><span class="n">&lt;:SignedBitops</span><span class="p">:&gt;</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r
<span class="w">                                             </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r
<span class="w">       </span><span class="k">end</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Float</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Real32</span><span class="w"></span>\r
<span class="w">    </span><span class="k">structure</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Real64</span><span class="w"></span>\r
<span class="k">end</span><span class="w"></span>\r
</pre></div></div></div>\r
<div class="paragraph"><p>This would appear to be a better interface, even when an\r
implementation must choose <span class="monospaced">Int32</span> and <span class="monospaced">Word32</span> as the representation\r
for smaller C-types.</p></div>\r
</div></div>\r
</li>\r
</ul></div>\r
</div>\r
</div>\r
</div>\r
<div id="footnotes"><hr></div>\r
<div id="footer">\r
<div id="footer-text">\r
</div>\r
<div id="footer-badges">\r
</div>\r
</div>\r
</body>\r
</html>\r