Removed a few more deprecated function calls; documented closures.

[bpt/guile.git] / doc / guile-vm.texi

\input texinfo  @c -*-texinfo-*-
@c %**start of header
@setfilename guile-vm.info
@settitle Guile VM Specification
@footnotestyle end
@setchapternewpage odd
@c %**end of header

@set EDITION 0.6
@set VERSION 0.6
@set UPDATED 2005-04-26

@c Macro for instruction definitions.
@macro insn{}
Instruction
@end macro

@ifinfo
@dircategory Scheme Programming
@direntry
* Guile VM: (guile-vm).         Guile's Virtual Machine.
@end direntry

This file documents Guile VM.

Copyright @copyright{} 2000 Keisuke Nishida
Copyright @copyright{} 2005 Ludovic Court`es

Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
preserved on all copies.

@ignore
Permission is granted to process this file through TeX and print the
results, provided the printed document carries a copying permission
notice identical to this one except for the removal of this paragraph
(this paragraph not being relevant to the printed manual).

@end ignore
Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the
entire resulting derived work is distributed under the terms of a
permission notice identical to this one.

Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions,
except that this permission notice may be stated in a translation
approved by the Free Software Foundation.
@end ifinfo

@titlepage
@title Guile VM Specification
@subtitle for Guile VM @value{VERSION}
@author Keisuke Nishida

@page
@vskip 0pt plus 1filll
Edition @value{EDITION} @*
Updated for Guile VM @value{VERSION} @*
@value{UPDATED} @*

Copyright @copyright{} 2000 Keisuke Nishida
Copyright @copyright{} 2005 Ludovic Court`es

Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
preserved on all copies.

Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the
entire resulting derived work is distributed under the terms of a
permission notice identical to this one.

Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions,
except that this permission notice may be stated in a translation
approved by the Free Software Foundation.
@end titlepage

@contents

@c *********************************************************************
@node Top, Introduction, (dir), (dir)
@top Guile VM Specification

This document would like to correspond to Guile VM @value{VERSION}.
However, be warned that important parts still correspond to version
0.0 and are not valid anymore.

@menu
* Introduction::                
* Variable Management::         
* Program Execution::           
* Instruction Set::             

@detailmenu
 --- The Detailed Node Listing ---

Instruction Set

* Environment Control Instructions::  
* Branch Instructions::         
* Subprogram Control Instructions::  
* Data Control Instructions::   

@end detailmenu
@end menu

@c *********************************************************************
@node Introduction, Variable Management, Top, Top
@chapter What is Guile VM?

A Guile VM has a set of registers and its own stack memory.  Guile may
have more than one VM's.  Each VM may execute at most one program at a
time.  Guile VM is a CISC system so designed as to execute Scheme and
other languages efficiently.

@unnumberedsubsec Registers

@itemize
@item pc - Program counter    ;; ip (instruction poiner) is better?
@item sp - Stack pointer
@item bp - Base pointer
@item ac - Accumulator
@end itemize

@unnumberedsubsec Engine

A VM may have one of three engines: reckless, regular, or debugging.
Reckless engine is fastest but dangerous.  Regular engine is normally
fail-safe and reasonably fast.  Debugging engine is safest and
functional but very slow.

@unnumberedsubsec Memory

Stack is the only memory that each VM owns.  The other memory is shared
memory that is shared among every VM and other part of Guile.

@unnumberedsubsec Program

A VM program consists of a bytecode that is executed and an environment
in which execution is done.  Each program is allocated in the shared
memory and may be executed by any VM.  A program may call other programs
within a VM.

@unnumberedsubsec Instruction

Guile VM has dozens of system instructions and (possibly) hundreds of
functional instructions.  Some Scheme procedures such as cons and car
are implemented as VM's builtin functions, which are very efficient.
Other procedures defined outside of the VM are also considered as VM's
functional features, since they do not change the state of VM.
Procedures defined within the VM are called subprograms.

Most instructions deal with the accumulator (ac).  The VM stores all
results from functions in ac, instead of pushing them into the stack.
I'm not sure whether this is a good thing or not.

@node Variable Management, Program Execution, Introduction, Top
@chapter Variable Management

A program may have access to local variables, external variables, and
top-level variables.

@section Local/external variables

A stack is logically divided into several blocks during execution.  A
"block" is such a unit that maintains local variables and dynamic chain.
A "frame" is an upper level unit that maintains subprogram calls.

@example
             Stack
  dynamic |          |  |        |
    chain +==========+  -        =
        | |local vars|  |        |
        `-|block data|  | block  |
         /|frame data|  |        |
        | +----------+  -        |
        | |local vars|  |        | frame
        `-|block data|  |        |
         /+----------+  -        |
        | |local vars|  |        |
        `-|block data|  |        |
         /+==========+  -        =
        | |local vars|  |        |
        `-|block data|  |        |
         /|frame data|  |        |
        | +----------+  -        |
        | |          |  |        |
@end example

The first block of each frame may look like this:

@example
       Address  Data
       -------  ----
       xxx0028  Local variable 2
       xxx0024  Local variable 1
  bp ->xxx0020  Local variable 0
       xxx001c  Local link       (block data)
       xxx0018  External link    (block data)
       xxx0014  Stack pointer    (block data)
       xxx0010  Return address   (frame data)
       xxx000c  Parent program   (frame data)
@end example

The base pointer (bp) always points to the lowest address of local
variables of the recent block.  Local variables are referred as "bp[n]".
The local link field has a pointer to the dynamic parent of the block.
The parent's variables are referred as "bp[-1][n]", and grandparent's
are "bp[-1][-1][n]".  Thus, any local variable is represented by its
depth and offset from the current bp.

A variable may be "external", which is allocated in the shared memory.
The external link field of a block has a pointer to such a variable set,
which I call "fragment" (what should I call?).  A fragment has a set of
variables and its own chain.

@example
    local                    external
    chain|     |              chain
       | +-----+     .--------, |
       `-|block|--+->|external|-'
        /+-----+  |  `--------'\,
       `-|block|--'             |
        /+-----+     .--------, |
       `-|block|---->|external|-'
         +-----+     `--------'
         |     |
@end example

An external variable is referred as "bp[-2]->variables[n]" or
"bp[-2]->link->...->variables[n]".  This is also represented by a pair
of depth and offset.  At any point of execution, the value of bp
determines the current local link and external link, and thus the
current environment of a program.

Other data fields are described later.

@section Top-level variables

Guile VM uses the same top-level variables as the regular Guile.  A
program may have direct access to vcells.  Currently this is done by
calling scm_intern0, but a program is possible to have any top-level
environment defined by the current module.

@section Scheme and VM variable

Let's think about the following Scheme code as an example:

@example
  (define (foo a)
    (lambda (b) (list foo a b)))
@end example

In the lambda expression, "foo" is a top-level variable, "a" is an
external variable, and "b" is a local variable.

When a VM executes foo, it allocates a block for "a".  Since "a" may be
externally referred from the closure, the VM creates a fragment with a
copy of "a" in it.  When the VM evaluates the lambda expression, it
creates a subprogram (closure), associating the fragment with the
subprogram as its external environment.  When the closure is executed,
its environment will look like this:

@example
      block          Top-level: foo
  +-------------+
  |local var: b |       fragment
  +-------------+     .-----------,
  |external link|---->|variable: a|
  +-------------+     `-----------'
@end example

The fragment remains as long as the closure exists.

@section Addressing mode

Guile VM has five addressing modes:

@itemize
@item Real address
@item Local position
@item External position
@item Top-level location
@item Constant object
@end itemize

Real address points to the address in the real program and is only used
with the program counter (pc).

Local position and external position are represented as a pair of depth
and offset from bp, as described above.  These are base relative
addresses, and the real address may vary during execution.

Top-level location is represented as a Guile's vcell.  This location is
determined at loading time, so the use of this address is efficient.

Constant object is not an address but gives an instruction an Scheme
object directly.

[ We'll also need dynamic scope addressing to support Emacs Lisp? ]

@unnumberedsubsec At a Glance

Guile VM has a set of instructions for each instruction family.  `%load'
is, for example, a family to load an object from memory and set the
accumulator (ac).  There are four basic `%load' instructions:

@example
  %loadl - Local addressing
  %loade - External addressing
  %loadt - Top-level addressing
  %loadi - Immediate addressing
@end example

A possible program code may look like this:

@example
  %loadl (0 . 1)                ; ac = local[0][1]
  %loade (2 . 3)                ; ac = external[2][3]
  %loadt (foo . #<undefined>)   ; ac = #<undefined>
  %loadi "hello"                ; ac = "hello"
@end example

One instruction that uses real addressing is `%jump', which changes the
value of the program counter:

@example
  %jump  0x80234ab8             ; pc = 0x80234ab8
@end example


@node Program Execution, Instruction Set, Variable Management, Top
@chapter Program Execution

Overall procedure:

@enumerate
@item A source program is compiled into a bytecode.
@item A bytecode is given an environment and becomes a program.
@item A VM starts execution, creating a frame for it.
@item Whenever a program calls a subprogram, a new frame is created for it.
@item When a program finishes execution, it returns a value, and the VM
      continues execution of the parent program.
@item When all programs terminated, the VM returns the final value and stops.
@end enumerate

@section Environment

Local variable:

@example
 (let ((a 1) (b 2) (c 3)) (+ a b c)) ->

   %pushi 1       ; a
   %pushi 2       ; b
   %pushi 3       ; c
   %bind  3       ; create local bindings
   %pushl (0 . 0) ; local variable a
   %pushl (0 . 1) ; local variable b
   %pushl (0 . 2) ; local variable c
   add    3       ; ac = a + b + c
   %unbind        ; remove local bindings
@end example

External variable:

@example
 (define foo (let ((n 0)) (lambda () n)))

   %pushi 0                      ; n
   %bind  1                      ; create local bindings
   %export [0]                   ; make it an external variable
   %make-program #<bytecode xxx> ; create a program in this environment
   %unbind                       ; remove local bindings
   %savet (foo . #<undefined>)   ; save the program in foo

 (foo) ->

   %loadt (foo . #<program xxx>) ; program has an external link
   %call  0                      ; change the current external link
   %loade (0 . 0)                ; external variable n
   %return                       ; recover the external link
@end example

Top-level variable:

@example
 foo ->

   %loadt (foo . #<program xxx>) ; top-level variable foo
@end example

@section Flow control

@example
 (if #t 1 0) ->

      %loadi      #t
      %br-if-not  L1
      %loadi      1
      %jump       L2
  L1: %loadi      0
  L2: 
@end example

@section Function call

Builtin function:

@example
 (1+ 2) ->

   %loadi 2  ; ac = 2
   1+        ; one argument

 (+ 1 2) ->

   %pushi 1  ; 1 -> stack
   %loadi 2  ; ac = 2
   add2      ; two argument

 (+ 1 2 3) ->

   %pushi 1  ; 1 -> stack
   %pushi 2  ; 2 -> stack
   %pushi 3  ; 3 -> stack
   add    3  ; many argument
@end example

External function:

@example
 (version) ->

   %func0 (version . #<primitive-procedure version>) ; no argument

 (display "hello") ->

   %loadi "hello"
   %func1 (display . #<primitive-procedure display>) ; one argument

 (open-file "file" "w") ->

   %pushi "file"
   %loadi "w"
   %func2 (open-file . #<primitive-procedure open-file>) ; two arguments

 (equal 1 2 3)

   %pushi 1
   %pushi 2
   %pushi 3
   %loadi 3                                      ; the number of arguments
   %func  (equal . #<primitive-procedure equal>) ; many arguments
@end example

@section Subprogram call

@example
 (define (plus a b) (+ a b))
 (plus 1 2) ->

   %pushi 1                       ; argument 1
   %pushi 2                       ; argument 2
   %loadt (plus . #<program xxx>) ; load the program
   %call  2                       ; call it with two arguments
   %pushl (0 . 0)                 ; argument 1
   %loadl (0 . 1)                 ; argument 2
   add2                           ; ac = 1 + 2
   %return                        ; result is 3
@end example

@node Instruction Set,  , Program Execution, Top
@chapter Instruction Set

The Guile VM instruction set is roughly divided two groups: system
instructions and functional instructions.  System instructions control
the execution of programs, while functional instructions provide many
useful calculations.

@menu
* Environment Control Instructions::  
* Branch Instructions::         
* Subprogram Control Instructions::  
* Data Control Instructions::   
@end menu

@node Environment Control Instructions, Branch Instructions, Instruction Set, Instruction Set
@section Environment Control Instructions

@deffn @insn{} link binding-name
Look up @var{binding-name} (a string) in the current environment and
push the corresponding variable object onto the stack.  If
@var{binding-name} is not bound yet, then create a new binding and
push its variable object.
@end deffn

@deffn @insn{} variable-ref
Dereference the variable object which is on top of the stack and
replace it by the value of the variable it represents.
@end deffn

@deffn @insn{} variable-set
Set the value of the variable on top of the stack (at @code{sp[0]}) to
the object located immediately before (at @code{sp[-1]}).
@end deffn

As an example, let us look at what a simple function call looks like:

@example
(+ 2 3)
@end example

This call yields the following sequence of instructions:

@example
(link "+")      ;; lookup binding "+"
(variable-ref)  ;; dereference it
(make-int8 2)   ;; push immediate value `2'
(make-int8 3)   ;; push immediate value `3'
(tail-call 2)   ;; call the proc at sp[-3] with two args
@end example

@deffn @insn{} local-ref offset
Push onto the stack the value of the local variable located at
@var{offset} within the current stack frame.
@end deffn

@deffn @insn{} local-set offset
Pop the Scheme object located on top of the stack and make it the new
value of the local variable located at @var{offset} within the current
stack frame.
@end deffn

@deffn @insn{} external-ref offset
Push the value of the closure variable located at position
@var{offset} within the program's list of external variables.
@end deffn

@deffn @insn{} external-set offset
Pop the Scheme object located on top of the stack and make it the new
value of the closure variable located at @var{offset} within the
program's list of external variables.
@end deffn

Let's look at a more complete example:

@example
(let ((x 2))
  (lambda ()
    (let ((x++ (+ 1 x)))
      (set! x x++)
      x++)))
@end example

The resulting program has one external (closure) variable, i.e. its
@var{nexts} is set to 1 (@pxref{Subprogram Control Instructions}).
This yields the following code:

@example
   ;; the traditional program prologue
   0    (make-int8 2)
   2    (external-set 0)
   4    (make-int8 4)
   6    (link "+")    ;; lookup `+'
   9    (vector 1)    ;; create the external variable vector for
                      ;; later use by `object-ref' and `object-set'
        ...
  40    (load-program ##34#)
  59    (return)
@end example

The program loaded here by @var{load-program} contains the following
sequence of instructions:

@example
   0    (object-ref 0)     ;; push the variable for `+'
   2    (variable-ref)     ;; dereference `+'
   3    (make-int8:1)      ;; push 1
   4    (external-ref 0)   ;; push the value of `x'
   6    (call 2)           ;; call `+' and push the result
   8    (local-set 0)      ;; make it the new value of `x++'
  10    (local-ref 0)      ;; push the value of `x++'
  12    (external-set 0)   ;; make it the new value of `x'
  14    (local-ref 0)      ;; push the value of `x++'
  16    (return)           ;; return it
@end example

At this point, you know pretty much everything about the three types
of variables a program may need to access.


@node Branch Instructions, Subprogram Control Instructions, Environment Control Instructions, Instruction Set
@section Branch Instructions

All the conditional branch instructions described below work in the
same way:

@itemize
@item They take the Scheme object located on the stack and use it as
the branch condition;
@item If the condition if false, then program execution continues with
the next instruction;
@item If the condition is true, then the instruction pointer is
increased by the offset passed as an argument to the branch
instruction;
@item Finally, when the instruction finished, the condition object is
removed from the stack.
@end itemize

Note that the offset passed to the instruction is encoded on two 8-bit
integers which are then combined by the VM as one 16-bit integer.

@deffn @insn{} br offset
Jump to @var{offset}.
@end deffn

@deffn @insn{} br-if offset
Jump to @var{offset} if the condition on the stack is not false.
@end deffn

@deffn @insn{} br-if-not offset
Jump to @var{offset} if the condition on the stack is false.
@end deffn

@deffn @insn{} br-if-eq offset
Jump to @var{offset} if the two objects located on the stack are
equal in the sense of @var{eq?}.  Note that, for this instruction, the
stack pointer is decremented by two Scheme objects instead of only
one.
@end deffn

@deffn @insn{} br-if-not-eq offset
Same as @var{br-if-eq} for non-equal objects.
@end deffn

@deffn @insn{} br-if-null offset
Jump to @var{offset} if the object on the stack is @code{'()}.
@end deffn

@deffn @insn{} br-if-not-null offset
Jump to @var{offset} if the object on the stack is not @code{'()}.
@end deffn


@node Subprogram Control Instructions, Data Control Instructions, Branch Instructions, Instruction Set
@section Subprogram Control Instructions

Programs (read: ``compiled procedure'') may refer to external
bindings, like variables or functions defined outside the program
itself, in the environment in which it will evaluate at run-time.  In
a sense, a program's environment and its bindings are an implicit
parameter of every program.

@cindex Object table
In order to handle such bindings, each program has an @dfn{object
table} associated to it.  This table (actually a vector) contains all
the variable objects corresponding to the external bindings referenced
by the program.  The object table of a program is initialized right
before a program is loaded and run with @var{load-program}.

Therefore, external bindings only need to be looked up once before the
program is loaded.  References to the corresponding external variables
from within the program are then performed via the @var{object-ref}
instruction and are almost as fast as local variable references.

Let us consider the following program (procedure) which references
external bindings @code{frob} and @var{%magic}:

@example
(lambda (x)
  (frob x %magic))
@end example

This yields the following assembly code:

@example
(make-int8 64)   ;; number of args, vars, etc. (see below)
(link "frob")
(link "%magic")
(vector 2)       ;; object table (external bindings)
...
(load-program #u8(20 0 23 21 0 20 1 23 36 2))
(return)
@end example

All the instructions occurring before @var{load-program} (some were
omitted for simplicity) form a @dfn{prologue} which, among other
things, pushed an object table (a vector) that contains the variable
objects for the variables bound to @var{frob} and @var{%magic}.  This
vector and other data pushed onto the stack are then popped by the
@var{load-program} instruction.

Besides, the @var{load-program} instruction takes one explicit
argument which is the bytecode of the program itself.  Disassembled,
this bytecode looks like:

@example
z(object-ref 0)  ;; push the variable object of `frob'
(variable-ref)  ;; dereference it
(local-ref 0)   ;; push the value of `x'
(object-ref 1)  ;; push the variable object of `%magic'
(variable-ref)  ;; dereference it
(tail-call 2)   ;; call `frob' with two parameters
@end example

This clearly shows that there is little difference between references
to local variables and references to externally bound variables since
lookup of externally bound variables if performed only once before the
program is run.

@deffn @insn{} load-program bytecode
Load the program whose bytecode is @var{bytecode} (a u8vector), pop
its meta-information from the stack, and push a corresponding program
object onto the stack.  The program's meta-information may consist of
(in the order in which it should be pushed onto the stack):

@itemize
@item optionally, a pair representing meta-data (see the
@var{program-meta} procedure); [FIXME: explain their meaning]
@item optionally, a vector which is the program's object table (a
program that does not reference external bindings does not need an
object table);
@item either one immediate integer or four immediate integers
representing respectively the number of arguments taken by the
function (@var{nargs}), the number of @dfn{rest arguments}
(@var{nrest}, 0 or 1), the number of local variables (@var{nlocs}) and
the number of external variables (@var{nexts}) (@pxref{Environment
Control Instructions}).
@end itemize

@end deffn

@deffn @insn{} object-ref offset
Push the variable object for the external variable located at
@var{offset} within the program's object table.
@end deffn

@deffn @insn{} return
Free the program's frame.
@end deffn

@deffn @insn{} call nargs
Call the procedure, continuation or program located at
@code{sp[-nargs]} with the @var{nargs} arguments located from
@code{sp[0]} to @code{sp[-nargs + 1]}.  The
procedure/continuation/program and its arguments are dropped from the
stack and the result is pushed.  When calling a program, the
@code{call} instruction reserves room for its local variables on the
stack, and initializes its list of closure variables and its vector of
externally bound variables.
@end deffn

@deffn @insn{} tail-call nargs
Same as @code{call} except that, for tail-recursive calls to a
program, the current stack frame is re-used, as required by RnRS.
This instruction is otherwise similar to @code{call}.
@end deffn


@node Data Control Instructions,  , Subprogram Control Instructions, Instruction Set
@section Data Control Instructions

@deffn @insn{} make-int8 value
Push @var{value}, an 8-bit integer, onto the stack.
@end deffn

@deffn @insn{} make-int8:0
Push the immediate value @code{0} onto the stack.
@end deffn

@deffn @insn{} make-int8:1
Push the immediate value @code{1} onto the stack.
@end deffn

@deffn @insn{} make-false
Push @code{#f} onto the stack.
@end deffn

@deffn @insn{} make-true
Push @code{#t} onto the stack.
@end deffn

@itemize
@item %push
@item %pushi
@item %pushl, %pushl:0:0, %pushl:0:1, %pushl:0:2, %pushl:0:3
@item %pushe, %pushe:0:0, %pushe:0:1, %pushe:0:2, %pushe:0:3
@item %pusht
@end itemize

@itemize
@item %loadi
@item %loadl, %loadl:0:0, %loadl:0:1, %loadl:0:2, %loadl:0:3
@item %loade, %loade:0:0, %loade:0:1, %loade:0:2, %loade:0:3
@item %loadt
@end itemize

@itemize
@item %savei
@item %savel, %savel:0:0, %savel:0:1, %savel:0:2, %savel:0:3
@item %savee, %savee:0:0, %savee:0:1, %savee:0:2, %savee:0:3
@item %savet
@end itemize

@section Flow control instructions

@itemize
@item %br-if
@item %br-if-not
@item %jump
@end itemize

@section Function call instructions

@itemize
@item %func, %func0, %func1, %func2
@end itemize

@section Scheme built-in functions

@itemize
@item cons
@item car
@item cdr
@end itemize

@section Mathematical buitin functions

@itemize
@item 1+
@item 1-
@item add, add2
@item sub, sub2, minus
@item mul2
@item div2
@item lt2
@item gt2
@item le2
@item ge2
@item num-eq2
@end itemize

@c *********************************************************************
@c @node Concept Index, Command Index, Related Information, Top
@c @unnumbered Concept Index
@c @printindex cp

@c @node Command Index, Variable Index, Concept Index, Top
@c @unnumbered Command Index
@c @printindex fn

@c @node Variable Index,  , Command Index, Top
@c @unnumbered Variable Index
@c @printindex vr

@bye
\f
@c Local Variables:
@c mode:outline-minor
@c outline-regexp:"@\\(ch\\|sec\\|subs\\)"
@c End:

@c  LocalWords:  bytecode