[hcoop/debian/mlton.git] / doc / hacker-guide / notes.tex


\chap{Notes}{notes}

This chapter contains random notes (usually old emails) on various
subtle issues.

\section{IntInf and Flattener}

\begin{verbatim}
From: "Stephen T. Weeks" <sweeks@intertrust.com>
Date: Tue, 27 Jun 2000 18:52:19 -0700 (PDT)
To: MLton@research.nj.nec.com
Subject: safe for space ... and IntInf
\end{verbatim}

Your mail also came at a fortunate time, as I was trying to track down
a seg fault I was getting in the smith-normal-form regression test.
For stress testing, I turned off all the cps simplify passes (except
for poly equal) and ran the regressions.  smith-normal-form failed
with a seg fault when compiled normally, and failed with an assertion
failure in \verb+IntInf_do_neg+ when compiled -g.  The assertion
failure was right at the beginning, checking that the frontier is in
the expected place.
\begin{verbatim}
         assert(frontier == (pointer)&bp->limbs[bp->card - 1]);
\end{verbatim}
I'd been tracking this bug for a couple hours when I received your
mail about the flattener.  Do you see the connection? :-)  As a
reminder, here is the code for \verb+bigNegate+
\begin{verbatim}
         fun bigNegate (arg: bigInt): bigInt =
                if Prim.isSmall arg
                      then let val argw = Prim.toWord arg
                           in if argw = badw
                                 then negBad
                                 else Prim.fromWord (Word.- (0w2, argw))
                           end
                      else Prim.~ (arg, allocate (1 + bigSize arg))
\end{verbatim}
The problem is, when the flattener is turned off, there is an
allocation in between the call to allocate and the \verb+Prim.~+ call.  The
argument tuple allocation screws everything up.  So, we are relying on 
the flattener for correctness of the IntInf implementation.  Any ideas 
on how to improve the implementation to remove this reliance, or at
least put an assert somewhere to avoid falling prey to this bug again?
Commit	Line	Data
7f918cf1 CE	1
	2	\chap{Notes}{notes}
	3
	4	This chapter contains random notes (usually old emails) on various
	5	subtle issues.
	6
	7	\section{IntInf and Flattener}
	8
	9	\begin{verbatim}
	10	From: "Stephen T. Weeks" <sweeks@intertrust.com>
	11	Date: Tue, 27 Jun 2000 18:52:19 -0700 (PDT)
	12	To: MLton@research.nj.nec.com
	13	Subject: safe for space ... and IntInf
	14	\end{verbatim}
	15
	16	Your mail also came at a fortunate time, as I was trying to track down
	17	a seg fault I was getting in the smith-normal-form regression test.
	18	For stress testing, I turned off all the cps simplify passes (except
	19	for poly equal) and ran the regressions. smith-normal-form failed
	20	with a seg fault when compiled normally, and failed with an assertion
	21	failure in \verb+IntInf_do_neg+ when compiled -g. The assertion
	22	failure was right at the beginning, checking that the frontier is in
	23	the expected place.
	24	\begin{verbatim}
	25	assert(frontier == (pointer)&bp->limbs[bp->card - 1]);
	26	\end{verbatim}
	27	I'd been tracking this bug for a couple hours when I received your
	28	mail about the flattener. Do you see the connection? :-) As a
	29	reminder, here is the code for \verb+bigNegate+
	30	\begin{verbatim}
	31	fun bigNegate (arg: bigInt): bigInt =
	32	if Prim.isSmall arg
	33	then let val argw = Prim.toWord arg
	34	in if argw = badw
	35	then negBad
	36	else Prim.fromWord (Word.- (0w2, argw))
	37	end
	38	else Prim.~ (arg, allocate (1 + bigSize arg))
	39	\end{verbatim}
	40	The problem is, when the flattener is turned off, there is an
	41	allocation in between the call to allocate and the \verb+Prim.~+ call. The
	42	argument tuple allocation screws everything up. So, we are relying on
	43	the flattener for correctness of the IntInf implementation. Any ideas
	44	on how to improve the implementation to remove this reliance, or at
	45	least put an assert somewhere to avoid falling prey to this bug again?
	46