[hcoop/debian/mlton.git] / doc / guide / src / PortingMLton.adoc

PortingMLton
============

Porting MLton to a new target platform (architecture or OS) involves
the following steps.

1. Make the necessary changes to the scripts, runtime system,
<:BasisLibrary: Basis Library> implementation, and compiler.

2. Get the regressions working using a cross compiler.

3. <:CrossCompiling: Cross compile> MLton and bootstrap on the target.

MLton has a native code generator only for AMD64 and X86, so, if you
are porting to another architecture, you must use the C code
generator.  These notes do not cover building a new native code
generator.

Some of the following steps will not be necessary if MLton already
supports the architecture or operating system you are porting to.


== What code to change ==

* Scripts.
+
--
* In `bin/platform`, add new cases to define `$HOST_OS` and `$HOST_ARCH`.
--

* Runtime system.
+
--
The goal of this step is to be able to successfully run `make` in the
`runtime` directory on the target machine.

* In `platform.h`, add a new case to include `platform/<arch>.h` and `platform/<os>.h`.

* In `platform/<arch>.h`:
** define `MLton_Platform_Arch_host`.

* In `platform/<os>.h`:
** include platform-specific includes.
** define `MLton_Platform_OS_host`.
** define all of the `HAS_*` macros.

* In `platform/<os>.c` implement any platform-dependent functions that the runtime needs.

* Add rounding mode control to `basis/Real/IEEEReal.c` for the new arch (if not `HAS_FEROUND`)

* Compile and install the <:GnuMP:>.  This varies from platform to platform.  In `platform/<os>.h`, you need to include the appropriate `gmp.h`.
--

* Basis Library implementation (`basis-library/*`)
+
--
* In `primitive/prim-mlton.sml`:
** Add a new variant to the `MLton.Platform.Arch.t` datatype.
** modify the constants that define `MLton.Platform.Arch.host` to match with `MLton_Platform_Arch_host`, as set in `runtime/platform/<arch>.h`.
** Add a new variant to the `MLton.Platform.OS.t` datatype.
** modify the constants that define `MLton.Platform.OS.host` to match with `MLton_Platform_OS_host`, as set in `runtime/platform/<os>.h`.

* In `mlton/platform.{sig,sml}` add a new variant.

* In `sml-nj/sml-nj.sml`, modify `getOSKind`.

* Look at all the uses of `MLton.Platform` in the Basis Library implementation and see if you need to do anything special.  You might use the following command to see where to look.
+
----
find basis-library -type f | xargs grep 'MLton\.Platform'
----
+
If in doubt, leave the code alone and wait to see what happens when you run the regression tests.
--

* Compiler.
+
--
* In `lib/stubs/mlton-stubs/platform.sig` add any new variants, as was done in the Basis Library.

* In `lib/stubs/mlton-stubs/mlton.sml` add any new variants in `MLton.Platform`, as was done in the Basis Library.
--

The string used to identify a particular architecture or operating
system must be the same (except for possibly case of letters) in the
scripts, runtime, Basis Library implementation, and compiler (stubs).
In `mlton/main/main.fun`, MLton itself uses the conversions to and
from strings:
----
MLton.Platform.{Arch,OS}.{from,to}String
----

If the there is a mismatch, you may see the error message
`strange arch` or `strange os`.


== Running the regressions with a cross compiler ==

When porting to a new platform, it is always best to get all (or as
many as possible) of the regressions working before moving to a self
compile.  It is easiest to do this by modifying and rebuilding the
compiler on a working machine and then running the regressions with a
cross compiler.  It is not easy to build a gcc cross compiler, so we
recommend generating the C and assembly on a working machine (using
MLton's `-target` and `-stop g` flags, copying the generated files to
the target machine, then compiling and linking there.

1. Remake the compiler on a working machine.

2. Use `bin/add-cross` to add support for the new target.  In particular, this should create `build/lib/mlton/targets/<target>/` with the platform-specific necessary cross-compilation information.

3. Run the regression tests with the cross-compiler.  To cross-compile all the tests, do
+
----
bin/regression -cross <target>
----
+
This will create all the executables.  Then, copy `bin/regression` and
the `regression` directory to the target machine, and do
+
----
bin/regression -run-only <target>
----
+
This should run all the tests.

Repeat this step, interleaved with appropriate compiler modifications,
until all the regressions pass.


== Bootstrap ==

Once you've got all the regressions working, you can build MLton for
the new target.  As with the regressions, the idea for bootstrapping
is to generate the C and assembly on a working machine, copy it to the
target machine, and then compile and link there.  Here's the sequence
of steps.

1. On a working machine, with the newly rebuilt compiler, in the `mlton` directory, do:
+
----
mlton -stop g -target <target> mlton.mlb
----

2. Copy to the target machine.

3. On the target machine, move the libraries to the right place. That is, in `build/lib/mlton/targets`, do:
+
----
rm -rf self
mv <target> self
----
+
Also make sure you have all the header files in build/lib/mlton/include. You can copy them from a host machine that has run `make runtime`.

4. On the target machine, compile and link MLton.  That is, in the  mlton directory, do something like:
+
----
gcc -c -Ibuild/lib/mlton/include -Ibuild/lib/mlton/targets/self/include -O1 -w mlton/mlton.*.[cs]
gcc -o build/lib/mlton/mlton-compile \
        -Lbuild/lib/mlton/targets/self \
        -L/usr/local/lib \
        mlton.*.o \
        -lmlton -lgmp -lgdtoa -lm
----

5. At this point, MLton should be working and you can finish the rest of a usual make on the target machine.
+
----
make basis-no-check script mlbpathmap constants libraries tools
----

6. Making the last tool, mlyacc, will fail, because mlyacc cannot bootstrap its own yacc.grm.* files. On the host machine, run `make -C mlyacc src/yacc.grm.sml`. Then copy both files to the target machine, and compile mlyacc, making sure to supply the path to your newly compile mllex: `make -C mlyacc MLLEX=mllex/mllex`. 

There are other details to get right, like making sure that the tools
directories were clean so that the tools are rebuilt on the new
platform, but hopefully this structure works.  Once you've got a
compiler on the target machine, you should test it by running all the
regressions normally (i.e. without the `-cross` flag) and by running a
couple rounds of self compiles.


== Also see ==

The above description is based on the following emails sent to the
MLton list.

* http://www.mlton.org/pipermail/mlton/2002-October/013110.html
* http://www.mlton.org/pipermail/mlton/2004-July/016029.html
Commit	Line	Data
7f918cf1 CE	1	PortingMLton
	2	============
	3
	4	Porting MLton to a new target platform (architecture or OS) involves
	5	the following steps.
	6
	7	1. Make the necessary changes to the scripts, runtime system,
	8	<:BasisLibrary: Basis Library> implementation, and compiler.
	9
	10	2. Get the regressions working using a cross compiler.
	11
	12	3. <:CrossCompiling: Cross compile> MLton and bootstrap on the target.
	13
	14	MLton has a native code generator only for AMD64 and X86, so, if you
	15	are porting to another architecture, you must use the C code
	16	generator. These notes do not cover building a new native code
	17	generator.
	18
	19	Some of the following steps will not be necessary if MLton already
	20	supports the architecture or operating system you are porting to.
	21
	22
	23	== What code to change ==
	24
	25	* Scripts.
	26	+
	27	--
	28	* In `bin/platform`, add new cases to define `$HOST_OS` and `$HOST_ARCH`.
	29	--
	30
	31	* Runtime system.
	32	+
	33	--
	34	The goal of this step is to be able to successfully run `make` in the
	35	`runtime` directory on the target machine.
	36
	37	* In `platform.h`, add a new case to include `platform/<arch>.h` and `platform/<os>.h`.
	38
	39	* In `platform/<arch>.h`:
	40	** define `MLton_Platform_Arch_host`.
	41
	42	* In `platform/<os>.h`:
	43	** include platform-specific includes.
	44	** define `MLton_Platform_OS_host`.
	45	** define all of the `HAS_*` macros.
	46
	47	* In `platform/<os>.c` implement any platform-dependent functions that the runtime needs.
	48
	49	* Add rounding mode control to `basis/Real/IEEEReal.c` for the new arch (if not `HAS_FEROUND`)
	50
	51	* Compile and install the <:GnuMP:>. This varies from platform to platform. In `platform/<os>.h`, you need to include the appropriate `gmp.h`.
	52	--
	53
	54	* Basis Library implementation (`basis-library/*`)
	55	+
	56	--
	57	* In `primitive/prim-mlton.sml`:
	58	** Add a new variant to the `MLton.Platform.Arch.t` datatype.
	59	** modify the constants that define `MLton.Platform.Arch.host` to match with `MLton_Platform_Arch_host`, as set in `runtime/platform/<arch>.h`.
	60	** Add a new variant to the `MLton.Platform.OS.t` datatype.
	61	** modify the constants that define `MLton.Platform.OS.host` to match with `MLton_Platform_OS_host`, as set in `runtime/platform/<os>.h`.
	62
	63	* In `mlton/platform.{sig,sml}` add a new variant.
	64
65	* In `sml-nj/sml-nj.sml`, modify `getOSKind`.
66
67	* Look at all the uses of `MLton.Platform` in the Basis Library implementation and see if you need to do anything special. You might use the following command to see where to look.
68	+
69	----
70	find basis-library -type f \| xargs grep 'MLton\.Platform'
71	----
72	+
73	If in doubt, leave the code alone and wait to see what happens when you run the regression tests.
74	--
75
76	* Compiler.
77	+
78	--
79	* In `lib/stubs/mlton-stubs/platform.sig` add any new variants, as was done in the Basis Library.
80
81	* In `lib/stubs/mlton-stubs/mlton.sml` add any new variants in `MLton.Platform`, as was done in the Basis Library.
82	--
83
84	The string used to identify a particular architecture or operating
85	system must be the same (except for possibly case of letters) in the
86	scripts, runtime, Basis Library implementation, and compiler (stubs).
87	In `mlton/main/main.fun`, MLton itself uses the conversions to and
88	from strings:
89	----
90	MLton.Platform.{Arch,OS}.{from,to}String
91	----
92
93	If the there is a mismatch, you may see the error message
94	`strange arch` or `strange os`.
95
96
97	== Running the regressions with a cross compiler ==
98
99	When porting to a new platform, it is always best to get all (or as
100	many as possible) of the regressions working before moving to a self
101	compile. It is easiest to do this by modifying and rebuilding the
102	compiler on a working machine and then running the regressions with a
103	cross compiler. It is not easy to build a gcc cross compiler, so we
104	recommend generating the C and assembly on a working machine (using
105	MLton's `-target` and `-stop g` flags, copying the generated files to
106	the target machine, then compiling and linking there.
107
108	1. Remake the compiler on a working machine.
109
110	2. Use `bin/add-cross` to add support for the new target. In particular, this should create `build/lib/mlton/targets/<target>/` with the platform-specific necessary cross-compilation information.
111
112	3. Run the regression tests with the cross-compiler. To cross-compile all the tests, do
113	+
114	----
115	bin/regression -cross <target>
116	----
117	+
118	This will create all the executables. Then, copy `bin/regression` and
119	the `regression` directory to the target machine, and do
120	+
121	----
122	bin/regression -run-only <target>
123	----
124	+
125	This should run all the tests.
126
127	Repeat this step, interleaved with appropriate compiler modifications,
128	until all the regressions pass.
129
130
131	== Bootstrap ==
132
133	Once you've got all the regressions working, you can build MLton for
134	the new target. As with the regressions, the idea for bootstrapping
135	is to generate the C and assembly on a working machine, copy it to the
136	target machine, and then compile and link there. Here's the sequence
137	of steps.
138
139	1. On a working machine, with the newly rebuilt compiler, in the `mlton` directory, do:
140	+
141	----
142	mlton -stop g -target <target> mlton.mlb
143	----
144
145	2. Copy to the target machine.
146
147	3. On the target machine, move the libraries to the right place. That is, in `build/lib/mlton/targets`, do:
148	+
149	----
150	rm -rf self
151	mv <target> self
152	----
153	+
154	Also make sure you have all the header files in build/lib/mlton/include. You can copy them from a host machine that has run `make runtime`.
155
156	4. On the target machine, compile and link MLton. That is, in the mlton directory, do something like:
157	+
158	----
159	gcc -c -Ibuild/lib/mlton/include -Ibuild/lib/mlton/targets/self/include -O1 -w mlton/mlton.*.[cs]
160	gcc -o build/lib/mlton/mlton-compile \
161	-Lbuild/lib/mlton/targets/self \
162	-L/usr/local/lib \
163	mlton.*.o \
164	-lmlton -lgmp -lgdtoa -lm
165	----
166
167	5. At this point, MLton should be working and you can finish the rest of a usual make on the target machine.
168	+
169	----
170	make basis-no-check script mlbpathmap constants libraries tools
171	----
172
173	6. Making the last tool, mlyacc, will fail, because mlyacc cannot bootstrap its own yacc.grm.* files. On the host machine, run `make -C mlyacc src/yacc.grm.sml`. Then copy both files to the target machine, and compile mlyacc, making sure to supply the path to your newly compile mllex: `make -C mlyacc MLLEX=mllex/mllex`.
174
175	There are other details to get right, like making sure that the tools
176	directories were clean so that the tools are rebuilt on the new
177	platform, but hopefully this structure works. Once you've got a
178	compiler on the target machine, you should test it by running all the
179	regressions normally (i.e. without the `-cross` flag) and by running a
180	couple rounds of self compiles.
181
182
183	== Also see ==
184
185	The above description is based on the following emails sent to the
186	MLton list.
187
188	* http://www.mlton.org/pipermail/mlton/2002-October/013110.html
189	* http://www.mlton.org/pipermail/mlton/2004-July/016029.html