Lisp on Lines : The Missing Manual. /Abstract/: Lisp on Lines is a Common Lisp based framework for rapid development of data-driven applications. It is particularly useful for producing Web based applications, but is also useful elsewhere. * Introduction Lisp on Lines (LoL) is a framework for rapid development of data-driven applications, with a particular focus on web-based applications. It builds on the UncommonWeb engine and Contextl, and uses CLOS and the MOP extensively. Most of LoL can be used both at the REPL and through the browser, offering many options for development and testing. While the target audience for LoL is developers experienced with both web technologies and common lisp, a good programmer with a little experience in either should be able to pick things up fairly quickly. * Installation LoL has a load of dependencies, which themselves depend on others, etc. The best way to deal with this is to use [[http://common-lisp.net/project/clbuild/][clbuild]], a library management tool. If you'd prefer to manage your libraries manually, the dependencies, according to clbuild, are : alexandria arnesi bordeaux-threads cl-base64 cl-fad cl-mime cl-ppcre cl-qprint closer-mop contextl iterate lift local-time lw-compat net-telent-date parenscript parse-number portable-threads puri rfc2109 slime split-sequence trivial-garbage ucw usocket yaclml All libraries should be installed from version control where available. * Describing the domain with the MAO protocol. LoL uses a protocol it calls Meta-Attributed Objects, or MAO, as the basis of its display mechanism. In MAO, we create context-aware DESCRIPTIONs of objects, and those descriptions are used to generate the display of the object itself. By having these external descriptions change based on the context in which they are used, a few generic components can come together to create complex interfaces. ** Descriptions Descriptions are a similar conceptually to classes. Every Lisp object has one, and the root description that all descriptions inherit from is known as T. FIND-DESCRIPTION is used to, well, find descriptions. #+BEGIN_SRC lisp (find-description t) => # #+END_SRC ** Attributes and Properties A description is a collection of ATTRIBUTEs, among other things. Each attribute describes a part of an object, and any number of attributes may or may not be active. The ATTRIBUTES function is used to find a the list attributes that are both active and applicable in the current context. #+BEGIN_SRC lisp (attributes (find-description t)) =>(# # #) #+END_SRC The functions DESCRIPTION-ATTRIBUTES, DESCRIPTION-ACTIVE-ATTRIBUTES and DESCRIPTION-CURRENT-ATTRIBUTES return all the descriptions attributes, Attributes that are currently active regardless of context, and attributes that exist in the current context but may or may not be active, respectively. Attributes have properties, for example ATTRIBUTE-LABEL and ATTRIBUTE-VALUE. By simply iterating through the attributes of a described object, we can create a generic display for any lisp object. This is very similar, and was inspired by the technique outlined by Adrian Lienhard in [[http://www.adrian-lienhard.ch/files/mewa.pdf][MEWA: A Meta-level Architecture for Generic Web-Application Construction_]]. For attribute properties to be useful, the description must be associated with the object it is meant to describe. The function FUNCALL-WITH-DESCRIBED-OBJECT takes care of setting up the proper context. There is some syntax for it in the form of WITH-DESCRIBED-OBJECT : #+BEGIN_SRC lisp (let ((description (find-description t)) (object "Hello World")) (with-described-object (object description) (dolist (a (attributes description)) (format t "~@[~A: ~]~A~%" (attribute-label a) (attribute-value a))))) => Hello World Type: (SIMPLE-ARRAY CHARACTER (11)) Class: # NIL #+END_SRC FUNCALL-WITH-DESCRIBED-OBJECT binds two specials, *DESCRIPTION* and *OBJECT*, to its arguments. Knowing this, we can shorten our code somewhat. Later on we'll be far away from the lexical bindings of description and object, so these special variables are essential. Another reason for the *description* variable is that WITH-DESCRIBED-OBJECT will use DESCRIPTION-OF to determine the description if the DESCRIPTION argument is NIL #+BEGIN_SRC lisp (with-described-object ("Hello World" nil) (dolist (a (attributes *description*)) (format t "~@[~A: ~]~A~%" (attribute-label a) (attribute-value a)))) Lets wrap that up in a function that we can re-use. LoL includes an entire DISPLAY mechanism that is slightly more involved, but this serves as an excellent example with not bogging us down in details. #+BEGIN_SRC lisp (defun present (object &optional description) (with-described-object (object description) (dolist (a (attributes *description*)) (format t "~@[~A: ~]~A~%" (attribute-label a) (attribute-value a))))) #+END_SRC ** Contexts MAO adds to MEWA the concept of dynamic context. By changing the context in which an object is described, we combine and specialize the generic displays, ultimately creating different views of our objects. LoL uses ContextL extensively. Descriptions are contextl layers, and attributes themselves are layered classes. Most of the exported functions are layered methods, and the idea of dynamic context-sensitivity is used throughout LoL. If you're not familiar with contextl, don't worry, LoL mostly stands on its own. Still, reading through the material on contextl won't hurt. Descriptions may have different attributes dependant on what description contexts (or layers) are currently active. Attributes themselves might have different properties. When an object is being described (using WITH-DESCRIBED-OBJECT), it is also activated as a layer context. One can also activate/deactivate contexts manually, using WITH-ACTIVE-DESCRIPTIONS and WITH-INACTIVE-DESCRIPTIONS. Hopefully a little code will make this more clear : #+BEGIN_SRC lisp (present "Hello World") => Hello World Type: (SIMPLE-ARRAY CHARACTER (11)) Class: # Simple character string ;; Now we'll activate a built-in description, INLINE. (with-active-descriptions (inline) (present "Hello World")) => Hello World #+END_SRC You can see that the behavior of PRESENT changed when the INLINE context was activated. This is the key innovation that makes LoL so useful. In the next chapter we'll create our own descriptions and demonstrate this further. * Defining and Using Descriptions ** Defining a simple description The basics of the MAO should now (hopefully) be clear, so lets start using it. First, we'll create our very own description. #+BEGIN_SRC lisp (define-description hello-world () ((title :value "Lisp on Lines Demo") (identity :label "Message") (length :label "Length" :function #'length) (active-attributes :value '(title identity length)))) #+END_SRC Descriptions are defined very much like CLOS classes, and are in fact implemented that way, inheritance rules apply. The object returned from FIND-DESCRIPTION is best described as a prototype-based singleton. In other words, there is only one instance, and it inherits attributes and properties from further up its hierarchy unless specifically overridden. Attributes can have any number of properties, (see the class STANDARD-ATTRIBUTE), but the three most important are accessed via the methods ATTRIBUTE-LABEL, ATTRIBUTE-VALUE and ATTRIBUTE-FUNCTION,and named (in DEFINE-DESCRIPTION forms and elsewhere) by the :label, :value, and :function keywords. ATTRIBUTE-LABEL is simply a textual label that describes the attribute. ATTRIBUTE-VALUE is defined to return the result of calling ATTRIBUTE-FUNCTION with the object as its argument. If ATTRIBUTE-FUNCTION is NIL, the value of the :value property is returned directly. In the example above, the IDENTITY and ACTIVE-ATTRIBUTES attributes are inherited from T, and we are simply overriding the default properties for our description. LENGTH and TITLE are specific to this description. A look at src/standard-descriptions/t.lisp may be instructive at this point. Now, we can present our object using our new description. #+BEGIN_SRC lisp (present "Hello World" (find-description 'hello-world)) => Lisp on Lines Demo Message: Hello World Length: 11 NIL #+END_SRC ** Using descriptions as and with contexts. A we mentioned earlier, when an object is being described, the 'description context' is also made active. On top of that, one can define partial descriptions that are only active when other description contexts have been activated. We'll make a ONE-LINE description similar to the INLINE description demonstrated earlier. #+BEGIN_SRC lisp (define-description one-line ()) (define-description hello-world () ((identity :label nil) (active-attributes :value '(identity))) (:in-description one-line)) #+END_SRC Here we've defined a new description, ONE-LINE, and a context-sensitive extension to our HELLO-WORLD description. This partial desription will be active only when in the context of a one-line description. One can have attributes that only exist in certain description contexts, and attributes can have different properties. #+BEGIN_SRC lisp (let ((message "Hello World!") (description (find-description 'hello-world))) (print :normal)(terpri) (present message description) (print :one-line)(terpri) (with-active-descriptions (one-line) (present message description))) => :NORMAL Lisp on Lines Demo Message: Hello World! Length: 12 :ONE-LINE Hello World! NIL #+END_SRC By activating the description ONE-LINE, we've changed the context in which our object is displayed. We can create any number of descriptions and contexts and activate/deactivate them in any order. Descriptions are implemented as ContextL 'layers', so if all this seems weird, reading the ContextL papers might help. ** T : The root of all descriptions. Because all descriptions inherit from T, we can define contexts for T and they will apply to every description. The INLINE description can be found in standard-descriptions/inline.lisp, where we define a desription for T in the context of the INLINE description : #+BEGIN_SRC lisp ;; Defined by LoL in inline.lisp : (define-description t () ((identity :label nil) (active-attributes :value '(identity)) (attribute-delimiter :value ", ") (label-formatter :value (curry #'format nil "~A: ")) (value-formatter :value (curry #'format nil "~A"))) (:in-description inline))} #+END_SRC The does for the LoL DISPLAY mechanism what ONE-LINE did for PRESENT, only with more magic. By exetending T in this way, it's easy to create contexts the redefine the behavior of LoL while still reusing the basics. ** DESCRIPTION-OF : Permanently Associate a description with a class. The LAYERED-FUNCTION DESCRIPTION-OF will return the description associated with an object. #+BEGIN_SRC lisp (description-of nil) => # (description-of t) => # (description-of '(1 2 3)) => # ;;etc #+END_SRC * The DISPLAY Protocol Our function, PRESENT, is very basic, though pretty powerful when combined with descriptions and contexts. LoL includes a superset of such functionality built-in. The main entry point into this protocol is the DISPLAY function. The signature for this functions is : #+BEGIN_SRC lisp (display DISPLAY OBJECT &REST ARGS &KEY DEACTIVATE ACTIVATE &ALLOW-OTHER-KEYS) #+END_SRC The first argument, DISPLAY, is the place where we will display to/on/in/with. It could be a stream, a UCW component, a CLIM gadget, or anything else you might want to use. One can specialize on this argument (though it's better to specialize DISPLAY-USING-DESCRIPTION... more on that later) to use generic descriptions to display objects in different environments. The second argument is simply the object to be displayed. Here's a simple example : #+BEGIN_SRC lisp (display t t) => T Type:BOOLEAN Class:# Symbol Name:T Value:T Package:# Function: ; No value #+END_SRC The two arguments specified in the lambda-list, ACTIVATE and DEACTIVATE, are used to activate and deactivate description contexts in the scope of the display function. #+BEGIN_SRC lisp (display nil t :activate '(inline)) => "t" (with-active-descriptions (inline) (display nil t :deactivate '(inline))) => "T Type:BOOLEAN Class:# Symbol Name:T Value:T Package:# Function:" #+END_SRC Any other keyword arguments passed will be used to set the value of an attribute with a :keyword property, in the dynamic context of the DISPLAY function call. Once such attribute, and a very useful one is ACTIVE-ATTRIBUTES with its :attributes keyword : #+BEGIN_SRC lisp (display t t :attributes '(class package)) => Class:# Package:# #+END_SRC The properties of attributes that do not have a :keyword property can also be set dynamically. Since :attributes is the :keyword property of the ACTIVE-ATTRIBUTES attribute, the following form is equivalent to the previous : #+BEGIN_SRC lisp (display t t :attributes '((active-attributes :value (class package)))) => Class:# Package:# #+END_SRC Setting the attributes this way is almost like creating an anonymous description context... you can express just about anything you would in a DEFINE-DESCRIPTION. Here's a more involved example : #+BEGIN_SRC lisp (display t t :attributes `((identity :label "The Object") (class :label "CLOS Class") (package :value "COMMON LISP" :function nil) (type :value-formatter ,(lambda (a) (format nil "Got a value? ~A" a))))) => The Object:T CLOS Class:# Package:COMMON LISP Type:Got a value? BOOLEAN #+END_SRC I hope that serves well to demonstrate the concepts behind LoL, as there is no API documentation available at the moment... use the source luke! * Automatic Descriptions for CLOS classes. Lisp-on-Lines includes a compose-able metaclass, DESCRIBED-CLASS. It can be combined with _any_ other metaclass without affecting the behavior of that class. DESCRIBED-CLASS has been used with the metaclasses provided by CLSQL, ROFL, Rucksack and UCW simply by defining a class that inherits from both metaclasses. DESCRIBED-CLASS creates a base description for the class, named DESCRIPTION-FOR-, and another description with the same name as the class that has the previous description as a superclass. The then defines a method on DESCRIPTION-OF that returns the second description. LoL includes DESCRIBED-STANDARD-CLASS, which is subclass of STANDARD-CLASS and DESCRIBED-CLASS. We'll use this to create a class and its description. #+BEGIN_SRC lisp (defclass person () (first-name last-name company-name date-of-birth phone fax email address city province postal-code) (:metaclass described-standard-class)) => # (display t (make-instance 'person)) => First name:# Last name:# Company name:# Date of birth:# Phone:# Fax:# Email:# Address:# City:# Province:# Postal code:# #+END_SRC ** Described CLOS objects an the EDITABLE description The slots of an object are SETF'able places, and LoL takes advantage of that to provide EDITABLE descriptions automatically. When the EDITABLE description is active, and editor will be presented. The REPL based editor is pretty basic, but still useful. The HTML based editor will be described later. #+BEGIN_SRC lisp (defun edit-object (object &rest args) (with-active-descriptions (editable) (apply #'display t object args))) (let ((object (make-instance 'person))) (edit-object object) (terpri) (display t object)) ;; What follows are prompts and the information i entered First name:Drew Last name:Crampsie Company name:The Tech Co-op Date of birth:1978-07-31 Phone:555-5555 Fax:555-5555 Email:drewc@tech.coop Address:s/v Kanu, Lower Fraser River City:Richmond Province:BC Postal code:V1V3T6 ;; And this is what was displayed. First name:Drew Last name:Crampsie Company name:The Tech Co-op Date of birth:1978-07-31 Phone:555-5555 Fax:555-5555 Email:drewc@tech.coop Address:s/v Kanu, Lower Fraser River City:Richmond Province:BC Postal code:V1V3T6 #+END_SRC ** Extending the generated description We mentioned earlier that DESCRIBED-CLASS creates two descriptions : #+BEGIN_SRC lisp (find-description 'description-for-person) => # (find-description 'person) => # (description-of (make-instance 'person)) => # #+END_SRC The reason for this is so we can redefine the description PERSON while keeping all the generated information from DESCRIPTION-FOR-PERSON. In this case, we will add an attribute, PERSON-AGE, that calculates a persons age based on the data in the date-of-birth slot. * Using Lisp-on-Lines for the Web. LoL was developed, and is primarily used, for implementing data-driven web applications. As such, it comes with a host of features for doing just that. LoL, by default, implements its web portion on top of the wonderful UnCommon Web meta-framework. The LISP-ON-LINES-UCW ASDF system should be loaded, as it provides the features we're going to discuss.