Backport from sid to buster

[hcoop/debian/mlton.git] / doc / guide / src / MLNLFFIImplementation.adoc

MLNLFFIImplementation
=====================

MLton's implementation(s) of the <:MLNLFFI:> library differs from the
SML/NJ implementation in two important ways:

* MLton cannot utilize the `Unsafe.cast` "cheat" described in Section
3.7 of <!Cite(Blume01)>.  (MLton's representation of
<:Closure:closures> and
<:PackedRepresentation:aggressive representation> optimizations make
an `Unsafe.cast` even more "unsafe" than in other implementations.)
+
--
We have considered two solutions:

** One solution is to utilize an additional type parameter (as
described in Section 3.7 of <!Cite(Blume01)>):
+
--
__________
[source,sml]
----
signature C = sig
    type ('t, 'f, 'c) obj
    eqtype ('t, 'f, 'c) obj'
    ...
    type ('o, 'f) ptr
    eqtype ('o, 'f) ptr'
    ...
    type 'f fptr
    type 'f ptr'
    ...
    structure T : sig
        type ('t, 'f) typ
        ...
    end
end
----

The rule for `('t, 'f, 'c) obj`,`('t, 'f, 'c) ptr`, and also `('t, 'f)
T.typ` is that whenever `F fptr` occurs within the instantiation of
`'t`, then `'f` must be instantiated to `F`.  In all other cases, `'f`
will be instantiated to `unit`.
__________

(In the actual MLton implementation, an abstract type `naf`
(not-a-function) is used instead of `unit`.)

While this means that type-annotated programs may not type-check under
both the SML/NJ implementation and the MLton implementation, this
should not be a problem in practice.  Tools, like `ml-nlffigen`, which
are necessarily implementation dependent (in order to make
<:CallingFromSMLToCFunctionPointer:calls through a C function
pointer>), may be easily extended to emit the additional type
parameter.  Client code which uses such generated glue-code (e.g.,
Section 1 of <!Cite(Blume01)>) need rarely write type-annotations,
thanks to the magic of type inference.
--

** The above implementation suffers from two disadvantages.
+
--
First, it changes the MLNLFFI Library interface, meaning that the same
program may not type-check under both the SML/NJ implementation and
the MLton implementation (though, in light of type inference and the
richer `MLRep` structure provided by MLton, this point is mostly
moot).

Second, it appears to unnecessarily duplicate type information.  For
example, an external C variable of type `int (* f[3])(int)` (that is,
an array of three function pointers), would be represented by the SML
type `(((sint -> sint) fptr, dec dg3) arr, sint -> sint, rw) obj`.
One might well ask why the `'f` instantiation (`sint -> sint` in this
case) cannot be _extracted_ from the `'t` instantiation
(`((sint -> sint) fptr, dec dg3) arr` in this case), obviating the
need for a separate _function-type_ type argument.  There are a number
of components to an complete answer to this question.  Foremost is the
fact that <:StandardML: Standard ML> supports neither (general)
type-level functions nor intensional polymorphism.

A more direct answer for MLNLFFI is that in the SML/NJ implemention,
the definition of the types `('t, 'c) obj` and `('t, 'c) ptr` are made
in such a way that the type variables `'t` and `'c` are <:PhantomType:
phantom> (not contributing to the run-time representation of an
`('t, 'c) obj` or `('t, 'c) ptr` value), despite the fact that the
types `((sint -> sint) fptr, rw) ptr` and
`((double -> double) fptr, rw) ptr` necessarily carry distinct (and
type incompatible) run-time (C-)type information (RTTI), corresponding
to the different calling conventions of the two C functions.  The
`Unsafe.cast` "cheat" overcomes the type incompatibility without
introducing a new type variable (as in the first solution above).

Hence, the reason that _function-type_ type cannot be extracted from
the `'t` type variable instantiation is that the type of the
representation of RTTI doesn't even _see_ the (phantom) `'t` type
variable.  The solution which presents itself is to give up on the
phantomness of the `'t` type variable, making it available to the
representation of RTTI.

This is not without some small drawbacks.  Because many of the types
used to instantiate `'t` carry more structure than is strictly
necessary for `'t`'s RTTI, it is sometimes necessary to wrap and
unwrap RTTI to accommodate the additional structure.  (In the other
implementations, the corresponding operations can pass along the RTTI
unchanged.)  However, these coercions contribute minuscule overhead;
in fact, in a majority of cases, MLton's optimizations will completely
eliminate the RTTI from the final program.
--

The implementation distributed with MLton uses the second solution.

Bonus question: Why can't one use a <:UniversalType: universal type>
to eliminate the use of `Unsafe.cast`?

** Answer: ???
--

* MLton (in both of the above implementations) provides a richer
`MLRep` structure, utilizing ++Int__<N>__++ and ++Word__<N>__++
structures.
+
--
[source,sml]
-----
structure MLRep = struct
    structure Char =
       struct
          structure Signed = Int8
          structure Unsigned = Word8
          (* word-style bit-operations on integers... *)
          structure <:SignedBitops:> = IntBitOps(structure I = Signed
                                             structure W = Unsigned)
       end
    structure Short =
       struct
          structure Signed = Int16
          structure Unsigned = Word16
          (* word-style bit-operations on integers... *)
          structure <:SignedBitops:> = IntBitOps(structure I = Signed
                                             structure W = Unsigned)
       end
    structure Int =
       struct
          structure Signed = Int32
          structure Unsigned = Word32
          (* word-style bit-operations on integers... *)
          structure <:SignedBitops:> = IntBitOps(structure I = Signed
                                             structure W = Unsigned)
       end
    structure Long =
       struct
          structure Signed = Int32
          structure Unsigned = Word32
          (* word-style bit-operations on integers... *)
          structure <:SignedBitops:> = IntBitOps(structure I = Signed
                                             structure W = Unsigned)
       end
    structure <:LongLong:> =
       struct
          structure Signed = Int64
          structure Unsigned = Word64
          (* word-style bit-operations on integers... *)
          structure <:SignedBitops:> = IntBitOps(structure I = Signed
                                             structure W = Unsigned)
       end
    structure Float = Real32
    structure Double = Real64
end
----

This would appear to be a better interface, even when an
implementation must choose `Int32` and `Word32` as the representation
for smaller C-types.
--