An Overview of Guile Programming
********************************

Guile is designed as an extension language interpreter that is straightforward to integrate with applications written in C (and C++). The big win here for the application developer is that Guile integration, as the Guile web page says, ``lowers your project's hacktivation energy.''  Lowering the hacktivation energy means that you, as the application developer, and your users, reap the benefits that flow from being able to extend the application in a high level extension language rather than in plain old C.

Underlying this argument is the assumption that programming in a high level language, specifically Guile's implementation of Scheme, is necessarily better in some way than programming in C.  What do we mean by this claim, and how can we be so sure?

One class of advantages applies not only to Scheme, but more generally to any interpretable, high level, scripting language, such as Emacs Lisp, Python, Ruby, or TeX's macro language.  Common features of all such languages, when compared to C, are that:

  * They lend themselves to rapid and experimental development cycles, owing usually to a combination of their interpretability and the integrated development environment in which they are used.

  * They free developers from some of the low level bookkeeping tasks associated with C programming, notably memory management.

  * They provide high level features such as container objects and exception handling that make common programming tasks easier.

In the case of Scheme, further features that make programming easier --- and more fun! --- are its powerful mechanisms for abstracting parts of programs (closures --- see About Closure) and for iteration (see while do).

The evidence in support of this argument is empirical: the huge amount of code that has been written in extension languages for applications that support this mechanism.  Most notable are extensions written in Emacs Lisp for GNU Emacs, in TeX's macro language for TeX, and in Script-Fu for the Gimp, but there is increasingly now a significant code eco-system for Guile-based applications as well, such as Lilypond and GnuCash.  It is close to inconceivable that similar amounts of functionality could have been added to these applications just by writing new code in their base implementation languages.

Example: Using Guile for an Application Testbed
===============================================

As an example of what this means in practice, imagine writing a testbed for an application that is tested by submitting various requests (via a C interface) and validating the output received.  Suppose further that the application keeps an idea of its current state, and that the ``correct'' output for a given request may depend on the current application state.  A complete ``white box'' (A white box test plan is one that incorporates knowledge of the internal design of the application under test.) test plan for this application would aim to submit all possible requests in each distinguishable state, and validate the output for all request/state combinations.

To write all this test code in C would be very tedious.  Suppose instead that the testbed code adds a single new C function, to submit an arbitrary request and return the response, and then uses Guile to export this function as a Scheme procedure.  The rest of the testbed can then be written in Scheme, and so benefits from all the advantages of programming in Scheme that were described in the previous section.

(In this particular example, there is an additional benefit of writing most of the testbed in Scheme.  A common problem for white box testing is that mistakes and mistaken assumptions in the application under test can easily be reproduced in the testbed code.  It is more difficult to copy mistakes like this when the testbed is written in a different language from the application.)

A Choice of Programming Options
===============================

The preceding arguments and example point to a model of Guile programming that is applicable in many cases.  According to this model, Guile programming involves a balance between C and Scheme programming, with the aim being to extract the greatest possible Scheme level benefit from the least amount of C level work.

The C level work required in this model usually consists of packaging and exporting functions and application objects such that they can be seen and manipulated on the Scheme level.  To help with this, Guile's C language interface includes utility features that aim to make this kind of integration very easy for the application developer.  These features are documented later in this part of the manual: see REFFIXME.

This model, though, is really just one of a range of possible programming options.  If all of the functionality that you need is available from Scheme, you could choose instead to write your whole application in Scheme (or one of the other high level languages that Guile supports through translation), and simply use Guile as an interpreter for Scheme.  (In the future, we hope that Guile will also be able to compile Scheme code, so lessening the performance gap between C and Scheme code.)  Or, at the other end of the C--Scheme scale, you could write the majority of your application in C, and only call out to Guile occasionally for specific actions such as reading a configuration file or executing a user-specified extension.  The choices boil down to two basic questions:

  * Which parts of the application do you write in C, and which in Scheme (or another high level translated language)?

  * How do you design the interface between the C and Scheme parts of your application?

These are of course design questions, and the right design for any given application will always depend upon the particular requirements that you are trying to meet.  In the context of Guile, however, there are some generally applicable considerations that can help you when designing your answers.

What Functionality is Already Available?
----------------------------------------

Suppose, for the sake of argument, that you would prefer to write your whole application in Scheme.  Then the API available to you consists of:

  * standard Scheme

  * plus the extensions to standard Scheme provided by Guile in its core distribution

  * plus any additional functionality that you or others have packaged so that it can be loaded as a Guile Scheme module.

A module in the last category can either be a pure Scheme module --- in other words a collection of utility procedures coded in Scheme --- or a module that provides a Scheme interface to an extension library coded in C --- in other words a nice package where someone else has done the work of wrapping up some useful C code for you.  The set of available modules is growing quickly and already includes such useful examples as (gtk gtk), which makes Gtk+ drawing functions available in Scheme, and (database postgres), which provides SQL access to a Postgres database.

Given the growing collection of pre-existing modules, it is quite feasible that your application could be implemented by combining a selection of these modules together with new application code written in Scheme.

If this approach is not enough, because the functionality that your application needs is not already available in this form, and it is impossible to write the new functionality in Scheme, you will need to write some C code.  If the required function is already available in C (e.g. in a library), all you need is a little glue to connect it to the world of Guile.  If not, you need both to write the basic code and to plumb it into Guile.

In either case, two general considerations are important.  Firstly, what is the interface by which the functionality is presented to the Scheme world?  Does the interface consist only of function calls (for example, a simple drawing interface), or does it need to include objects of some kind that can be passed between C and Scheme and manipulated by both worlds.  Secondly, how does the lifetime and memory management of objects in the C code relate to the garbage collection governed approach of Scheme objects?  In the case where the basic C code is not already written, most of the difficulties of memory management can be avoided by using Guile's C interface features from the start.

For the full documentation on writing C code for Guile and connecting existing C code to the Guile world, see REFFIXME.

Functional and Performance Constraints
--------------------------------------

Your Preferred Programming Style
--------------------------------

What Controls Program Execution?
--------------------------------

How About Application Users?
============================

So far we have considered what Guile programming means for an application developer.  But what if you are instead using an existing Guile-based application, and want to know what your options are for programming and extending this application?

The answer to this question varies from one application to another, because the options available depend inevitably on whether the application developer has provided any hooks for you to hang your own code on and, if there are such hooks, what they allow you to do. (Of course, in the world of free software, you always have the freedom to modify the application's source code to your own requirements.  Here we are concerned with the extension options that the application has provided for without your needing to modify its source code.)  For example...

  * If the application permits you to load and execute any Guile code, the world is your oyster.  You can extend the application in any way that you choose.

  * A more cautious application might allow you to load and execute Guile code, but only in a safe environment, where the interface available is restricted by the application from the standard Guile API.

  * Or a really fearful application might not provide a hook to really execute user code at all, but just use Scheme syntax as a convenient way for users to specify application data or configuration options.

In the last two cases, what you can do is, by definition, restricted by the application, and you should refer to the application's own manual to find out your options.

The most well known example of the first case is Emacs, with its extension language Emacs Lisp: as well as being a text editor, Emacs supports the loading and execution of arbitrary Emacs Lisp code.  The result of such openness has been dramatic: Emacs now benefits from user-contributed Emacs Lisp libraries that extend the basic editing function to do everything from reading news to psychoanalysis and playing adventure games.  The only limitation is that extensions are restricted to the functionality provided by Emacs's built-in set of primitive operations.  For example, you can interact and display data by manipulating the contents of an Emacs buffer, but you can't pop-up and draw a window with a layout that is totally different to the Emacs standard.

This situation with a Guile application that supports the loading of arbitrary user code is similar, except perhaps even more so, because Guile also supports the loading of extension libraries written in C. This last point enables user code to add new primitive operations to Guile, and so to bypass the limitation present in Emacs Lisp.

At this point, the distinction between an application developer and an application user becomes rather blurred.  Instead of seeing yourself as a user extending an application, you could equally well say that you are developing a new application of your own using some of the primitive functionality provided by the original application.  As such, all the discussions of the preceding sections of this chapter are relevant to how you can proceed with developing your extension.