| 1 | <!DOCTYPE html>\r |
| 2 | <html lang="en">\r |
| 3 | <head>\r |
| 4 | <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">\r |
| 5 | <meta name="generator" content="AsciiDoc 8.6.9">\r |
| 6 | <title>MLNLFFIImplementation</title>\r |
| 7 | <link rel="stylesheet" href="./asciidoc.css" type="text/css">\r |
| 8 | <link rel="stylesheet" href="./pygments.css" type="text/css">\r |
| 9 | \r |
| 10 | \r |
| 11 | <script type="text/javascript" src="./asciidoc.js"></script>\r |
| 12 | <script type="text/javascript">\r |
| 13 | /*<![CDATA[*/\r |
| 14 | asciidoc.install();\r |
| 15 | /*]]>*/\r |
| 16 | </script>\r |
| 17 | <link rel="stylesheet" href="./mlton.css" type="text/css">\r |
| 18 | </head>\r |
| 19 | <body class="article">\r |
| 20 | <div id="banner">\r |
| 21 | <div id="banner-home">\r |
| 22 | <a href="./Home">MLton 20180207</a>\r |
| 23 | </div>\r |
| 24 | </div>\r |
| 25 | <div id="header">\r |
| 26 | <h1>MLNLFFIImplementation</h1>\r |
| 27 | </div>\r |
| 28 | <div id="content">\r |
| 29 | <div id="preamble">\r |
| 30 | <div class="sectionbody">\r |
| 31 | <div class="paragraph"><p>MLton’s implementation(s) of the <a href="MLNLFFI">MLNLFFI</a> library differs from the\r |
| 32 | SML/NJ implementation in two important ways:</p></div>\r |
| 33 | <div class="ulist"><ul>\r |
| 34 | <li>\r |
| 35 | <p>\r |
| 36 | MLton cannot utilize the <span class="monospaced">Unsafe.cast</span> "cheat" described in Section\r |
| 37 | 3.7 of <a href="References#Blume01">Blume01</a>. (MLton’s representation of\r |
| 38 | <a href="Closure">closures</a> and\r |
| 39 | <a href="PackedRepresentation">aggressive representation</a> optimizations make\r |
| 40 | an <span class="monospaced">Unsafe.cast</span> even more "unsafe" than in other implementations.)\r |
| 41 | </p>\r |
| 42 | <div class="openblock">\r |
| 43 | <div class="content">\r |
| 44 | <div class="paragraph"><p>We have considered two solutions:</p></div>\r |
| 45 | <div class="ulist"><ul>\r |
| 46 | <li>\r |
| 47 | <p>\r |
| 48 | One solution is to utilize an additional type parameter (as\r |
| 49 | described in Section 3.7 of <a href="References#Blume01">Blume01</a>):\r |
| 50 | </p>\r |
| 51 | <div class="openblock">\r |
| 52 | <div class="content">\r |
| 53 | <div class="quoteblock">\r |
| 54 | <div class="content">\r |
| 55 | <div class="listingblock">\r |
| 56 | <div class="content"><div class="highlight"><pre><span class="k">signature</span><span class="w"> </span><span class="n">C</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="k">sig</span><span class="w"></span>\r |
| 57 | <span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">'t</span><span class="p">,</span><span class="w"> </span><span class="n">'f</span><span class="p">,</span><span class="w"> </span><span class="n">'c</span><span class="p">)</span><span class="w"> </span><span class="n">obj</span><span class="w"></span>\r |
| 58 | <span class="w"> </span><span class="k">eqtype</span><span class="w"> </span><span class="p">(</span><span class="n">'t</span><span class="p">,</span><span class="w"> </span><span class="n">'f</span><span class="p">,</span><span class="w"> </span><span class="n">'c</span><span class="p">)</span><span class="w"> </span><span class="n">obj'</span><span class="w"></span>\r |
| 59 | <span class="w"> </span><span class="p">...</span><span class="w"></span>\r |
| 60 | <span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">'o</span><span class="p">,</span><span class="w"> </span><span class="n">'f</span><span class="p">)</span><span class="w"> </span><span class="n">ptr</span><span class="w"></span>\r |
| 61 | <span class="w"> </span><span class="k">eqtype</span><span class="w"> </span><span class="p">(</span><span class="n">'o</span><span class="p">,</span><span class="w"> </span><span class="n">'f</span><span class="p">)</span><span class="w"> </span><span class="n">ptr'</span><span class="w"></span>\r |
| 62 | <span class="w"> </span><span class="p">...</span><span class="w"></span>\r |
| 63 | <span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="n">'f</span><span class="w"> </span><span class="n">fptr</span><span class="w"></span>\r |
| 64 | <span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="n">'f</span><span class="w"> </span><span class="n">ptr'</span><span class="w"></span>\r |
| 65 | <span class="w"> </span><span class="p">...</span><span class="w"></span>\r |
| 66 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">T</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="k">sig</span><span class="w"></span>\r |
| 67 | <span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="p">(</span><span class="n">'t</span><span class="p">,</span><span class="w"> </span><span class="n">'f</span><span class="p">)</span><span class="w"> </span><span class="n">typ</span><span class="w"></span>\r |
| 68 | <span class="w"> </span><span class="p">...</span><span class="w"></span>\r |
| 69 | <span class="w"> </span><span class="k">end</span><span class="w"></span>\r |
| 70 | <span class="k">end</span><span class="w"></span>\r |
| 71 | </pre></div></div></div>\r |
| 72 | <div class="paragraph"><p>The rule for <span class="monospaced">('t, 'f, 'c) obj</span>,<span class="monospaced">('t, 'f, 'c) ptr</span>, and also <span class="monospaced">('t, 'f)\r |
| 73 | T.typ</span> is that whenever <span class="monospaced">F fptr</span> occurs within the instantiation of\r |
| 74 | <span class="monospaced">'t</span>, then <span class="monospaced">'f</span> must be instantiated to <span class="monospaced">F</span>. In all other cases, <span class="monospaced">'f</span>\r |
| 75 | will be instantiated to <span class="monospaced">unit</span>.</p></div>\r |
| 76 | </div>\r |
| 77 | <div class="attribution">\r |
| 78 | </div></div>\r |
| 79 | <div class="paragraph"><p>(In the actual MLton implementation, an abstract type <span class="monospaced">naf</span>\r |
| 80 | (not-a-function) is used instead of <span class="monospaced">unit</span>.)</p></div>\r |
| 81 | <div class="paragraph"><p>While this means that type-annotated programs may not type-check under\r |
| 82 | both the SML/NJ implementation and the MLton implementation, this\r |
| 83 | should not be a problem in practice. Tools, like <span class="monospaced">ml-nlffigen</span>, which\r |
| 84 | are necessarily implementation dependent (in order to make\r |
| 85 | <a href="CallingFromSMLToCFunctionPointer">calls through a C function\r |
| 86 | pointer</a>), may be easily extended to emit the additional type\r |
| 87 | parameter. Client code which uses such generated glue-code (e.g.,\r |
| 88 | Section 1 of <a href="References#Blume01">Blume01</a>) need rarely write type-annotations,\r |
| 89 | thanks to the magic of type inference.</p></div>\r |
| 90 | </div></div>\r |
| 91 | </li>\r |
| 92 | <li>\r |
| 93 | <p>\r |
| 94 | The above implementation suffers from two disadvantages.\r |
| 95 | </p>\r |
| 96 | <div class="openblock">\r |
| 97 | <div class="content">\r |
| 98 | <div class="paragraph"><p>First, it changes the MLNLFFI Library interface, meaning that the same\r |
| 99 | program may not type-check under both the SML/NJ implementation and\r |
| 100 | the MLton implementation (though, in light of type inference and the\r |
| 101 | richer <span class="monospaced">MLRep</span> structure provided by MLton, this point is mostly\r |
| 102 | moot).</p></div>\r |
| 103 | <div class="paragraph"><p>Second, it appears to unnecessarily duplicate type information. For\r |
| 104 | example, an external C variable of type <span class="monospaced">int (* f[3])(int)</span> (that is,\r |
| 105 | an array of three function pointers), would be represented by the SML\r |
| 106 | type <span class="monospaced">(((sint -> sint) fptr, dec dg3) arr, sint -> sint, rw) obj</span>.\r |
| 107 | One might well ask why the <span class="monospaced">'f</span> instantiation (<span class="monospaced">sint -> sint</span> in this\r |
| 108 | case) cannot be <em>extracted</em> from the <span class="monospaced">'t</span> instantiation\r |
| 109 | (<span class="monospaced">((sint -> sint) fptr, dec dg3) arr</span> in this case), obviating the\r |
| 110 | need for a separate <em>function-type</em> type argument. There are a number\r |
| 111 | of components to an complete answer to this question. Foremost is the\r |
| 112 | fact that <a href="StandardML"> Standard ML</a> supports neither (general)\r |
| 113 | type-level functions nor intensional polymorphism.</p></div>\r |
| 114 | <div class="paragraph"><p>A more direct answer for MLNLFFI is that in the SML/NJ implemention,\r |
| 115 | the definition of the types <span class="monospaced">('t, 'c) obj</span> and <span class="monospaced">('t, 'c) ptr</span> are made\r |
| 116 | in such a way that the type variables <span class="monospaced">'t</span> and <span class="monospaced">'c</span> are <a href="PhantomType">\r |
| 117 | phantom</a> (not contributing to the run-time representation of an\r |
| 118 | <span class="monospaced">('t, 'c) obj</span> or <span class="monospaced">('t, 'c) ptr</span> value), despite the fact that the\r |
| 119 | types <span class="monospaced">((sint -> sint) fptr, rw) ptr</span> and\r |
| 120 | <span class="monospaced">((double -> double) fptr, rw) ptr</span> necessarily carry distinct (and\r |
| 121 | type incompatible) run-time (C-)type information (RTTI), corresponding\r |
| 122 | to the different calling conventions of the two C functions. The\r |
| 123 | <span class="monospaced">Unsafe.cast</span> "cheat" overcomes the type incompatibility without\r |
| 124 | introducing a new type variable (as in the first solution above).</p></div>\r |
| 125 | <div class="paragraph"><p>Hence, the reason that <em>function-type</em> type cannot be extracted from\r |
| 126 | the <span class="monospaced">'t</span> type variable instantiation is that the type of the\r |
| 127 | representation of RTTI doesn’t even <em>see</em> the (phantom) <span class="monospaced">'t</span> type\r |
| 128 | variable. The solution which presents itself is to give up on the\r |
| 129 | phantomness of the <span class="monospaced">'t</span> type variable, making it available to the\r |
| 130 | representation of RTTI.</p></div>\r |
| 131 | <div class="paragraph"><p>This is not without some small drawbacks. Because many of the types\r |
| 132 | used to instantiate <span class="monospaced">'t</span> carry more structure than is strictly\r |
| 133 | necessary for <span class="monospaced">'t</span>'s RTTI, it is sometimes necessary to wrap and\r |
| 134 | unwrap RTTI to accommodate the additional structure. (In the other\r |
| 135 | implementations, the corresponding operations can pass along the RTTI\r |
| 136 | unchanged.) However, these coercions contribute minuscule overhead;\r |
| 137 | in fact, in a majority of cases, MLton’s optimizations will completely\r |
| 138 | eliminate the RTTI from the final program.</p></div>\r |
| 139 | </div></div>\r |
| 140 | </li>\r |
| 141 | </ul></div>\r |
| 142 | <div class="paragraph"><p>The implementation distributed with MLton uses the second solution.</p></div>\r |
| 143 | <div class="paragraph"><p>Bonus question: Why can’t one use a <a href="UniversalType"> universal type</a>\r |
| 144 | to eliminate the use of <span class="monospaced">Unsafe.cast</span>?</p></div>\r |
| 145 | <div class="ulist"><ul>\r |
| 146 | <li>\r |
| 147 | <p>\r |
| 148 | Answer: ???\r |
| 149 | </p>\r |
| 150 | </li>\r |
| 151 | </ul></div>\r |
| 152 | </div></div>\r |
| 153 | </li>\r |
| 154 | <li>\r |
| 155 | <p>\r |
| 156 | MLton (in both of the above implementations) provides a richer\r |
| 157 | <span class="monospaced">MLRep</span> structure, utilizing <span class="monospaced">Int<em><N></em></span> and <span class="monospaced">Word<em><N></em></span>\r |
| 158 | structures.\r |
| 159 | </p>\r |
| 160 | <div class="openblock">\r |
| 161 | <div class="content">\r |
| 162 | <div class="listingblock">\r |
| 163 | <div class="content"><div class="highlight"><pre><span class="k">structure</span><span class="w"> </span><span class="n">MLRep</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="k">struct</span><span class="w"></span>\r |
| 164 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Char</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r |
| 165 | <span class="w"> </span><span class="k">struct</span><span class="w"></span>\r |
| 166 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int8</span><span class="w"></span>\r |
| 167 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word8</span><span class="w"></span>\r |
| 168 | <span class="w"> </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r |
| 169 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n"><:SignedBitops</span><span class="p">:></span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r |
| 170 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r |
| 171 | <span class="w"> </span><span class="k">end</span><span class="w"></span>\r |
| 172 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Short</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r |
| 173 | <span class="w"> </span><span class="k">struct</span><span class="w"></span>\r |
| 174 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int16</span><span class="w"></span>\r |
| 175 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word16</span><span class="w"></span>\r |
| 176 | <span class="w"> </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r |
| 177 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n"><:SignedBitops</span><span class="p">:></span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r |
| 178 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r |
| 179 | <span class="w"> </span><span class="k">end</span><span class="w"></span>\r |
| 180 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Int</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r |
| 181 | <span class="w"> </span><span class="k">struct</span><span class="w"></span>\r |
| 182 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int32</span><span class="w"></span>\r |
| 183 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word32</span><span class="w"></span>\r |
| 184 | <span class="w"> </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r |
| 185 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n"><:SignedBitops</span><span class="p">:></span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r |
| 186 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r |
| 187 | <span class="w"> </span><span class="k">end</span><span class="w"></span>\r |
| 188 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Long</span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r |
| 189 | <span class="w"> </span><span class="k">struct</span><span class="w"></span>\r |
| 190 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int32</span><span class="w"></span>\r |
| 191 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word32</span><span class="w"></span>\r |
| 192 | <span class="w"> </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r |
| 193 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n"><:SignedBitops</span><span class="p">:></span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r |
| 194 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r |
| 195 | <span class="w"> </span><span class="k">end</span><span class="w"></span>\r |
| 196 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n"><:LongLong</span><span class="p">:></span><span class="w"> </span><span class="p">=</span><span class="w"></span>\r |
| 197 | <span class="w"> </span><span class="k">struct</span><span class="w"></span>\r |
| 198 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Signed</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Int64</span><span class="w"></span>\r |
| 199 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Unsigned</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Word64</span><span class="w"></span>\r |
| 200 | <span class="w"> </span><span class="cm">(* word-style bit-operations on integers... *)</span><span class="w"></span>\r |
| 201 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n"><:SignedBitops</span><span class="p">:></span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">IntBitOps</span><span class="p">(</span><span class="k">structure</span><span class="w"> </span><span class="n">I</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Signed</span><span class="w"></span>\r |
| 202 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">W</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Unsigned</span><span class="p">)</span><span class="w"></span>\r |
| 203 | <span class="w"> </span><span class="k">end</span><span class="w"></span>\r |
| 204 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Float</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Real32</span><span class="w"></span>\r |
| 205 | <span class="w"> </span><span class="k">structure</span><span class="w"> </span><span class="n">Double</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="n">Real64</span><span class="w"></span>\r |
| 206 | <span class="k">end</span><span class="w"></span>\r |
| 207 | </pre></div></div></div>\r |
| 208 | <div class="paragraph"><p>This would appear to be a better interface, even when an\r |
| 209 | implementation must choose <span class="monospaced">Int32</span> and <span class="monospaced">Word32</span> as the representation\r |
| 210 | for smaller C-types.</p></div>\r |
| 211 | </div></div>\r |
| 212 | </li>\r |
| 213 | </ul></div>\r |
| 214 | </div>\r |
| 215 | </div>\r |
| 216 | </div>\r |
| 217 | <div id="footnotes"><hr></div>\r |
| 218 | <div id="footer">\r |
| 219 | <div id="footer-text">\r |
| 220 | </div>\r |
| 221 | <div id="footer-badges">\r |
| 222 | </div>\r |
| 223 | </div>\r |
| 224 | </body>\r |
| 225 | </html>\r |