Commit | Line | Data |
---|---|---|
6e4630e0 AW |
1 | @c -*-texinfo-*- |
2 | @c This is part of the GNU Guile Reference Manual. | |
3 | @c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2005, 2010, 2011, 2013, 2014 | |
4 | @c Free Software Foundation, Inc. | |
5 | @c See the file guile.texi for copying conditions. | |
6 | ||
7 | @node Defining New Foreign Object Types | |
8 | @section Defining New Foreign Object Types | |
9 | ||
10 | The @dfn{foreign object type} facility is Guile's mechanism for | |
11 | importing object and types from C or other languages into Guile's | |
12 | system. If you have a C @code{struct foo} type, for example, you can | |
13 | define a corresponding Guile foreign object type that allows Scheme code | |
14 | to handle @code{struct foo *} objects. | |
15 | ||
16 | To define a new foreign object type, the programmer provides Guile with | |
17 | some essential information about the type --- what its name is, how many | |
18 | fields it has, and its finalizer (if any) --- and Guile allocates a | |
19 | fresh type for it. Foreign objects can be accessed from Scheme or from | |
20 | C. | |
21 | ||
22 | @menu | |
23 | * Defining Foreign Object Types:: | |
24 | * Creating Foreign Objects:: | |
25 | * Type Checking of Foreign Objects:: | |
26 | * Foreign Object Memory Management:: | |
27 | * Foreign Objects and Scheme:: | |
28 | @end menu | |
29 | ||
30 | @node Defining Foreign Object Types | |
31 | @subsection Defining Foreign Object Types | |
32 | ||
33 | To create a new foreign object type from C, call | |
34 | @code{scm_make_foreign_object_type}. It returns a value of type | |
35 | @code{SCM} which identifies the new type. | |
36 | ||
37 | Here is how one might declare a new type representing eight-bit | |
38 | gray-scale images: | |
39 | ||
40 | @example | |
41 | #include <libguile.h> | |
42 | ||
43 | struct image @{ | |
44 | int width, height; | |
45 | char *pixels; | |
46 | ||
47 | /* The name of this image */ | |
48 | SCM name; | |
49 | ||
50 | /* A function to call when this image is | |
51 | modified, e.g., to update the screen, | |
52 | or SCM_BOOL_F if no action necessary */ | |
53 | SCM update_func; | |
54 | @}; | |
55 | ||
56 | static SCM image_type image_type; | |
57 | ||
58 | void | |
59 | init_image_type (void) | |
60 | @{ | |
61 | SCM name, slots; | |
62 | scm_t_struct_finalize finalizer; | |
63 | ||
64 | name = scm_from_utf8_symbol ("image"); | |
65 | slots = scm_list_1 (scm_from_utf8_symbol ("data")); | |
66 | finalizer = NULL; | |
67 | ||
68 | image_type = | |
69 | scm_make_foreign_object_type (name, slots, finalizer); | |
70 | @} | |
71 | @end example | |
72 | ||
73 | The result is an initialized @code{image_type} value that identifies the | |
74 | new foreign object type. The next section describes how to create | |
75 | foreign objects and how to access their slots. | |
76 | ||
77 | ||
78 | @node Creating Foreign Objects | |
79 | @subsection Creating Foreign Objects | |
80 | ||
81 | Foreign objects contain zero or more ``slots'' of data. A slot can hold | |
82 | a pointer, an integer that fits into a @code{size_t} or @code{ssize_t}, | |
83 | or a @code{SCM} value. | |
84 | ||
85 | All objects of a given foreign type have the same number of slots. In | |
86 | the example from the previous section, the @code{image} type has one | |
87 | slot, because the slots list passed to | |
88 | @code{scm_make_foreign_object_type} is of length one. (The actual names | |
89 | given to slots are unimportant for most users the C interface, but can | |
90 | be used on the Scheme side to introspect on the foreign object.) | |
91 | ||
92 | To construct a foreign object and initialize its first slot, call | |
93 | @code{scm_make_foreign_object_1 (@var{type}, @var{first_slot_value})}. | |
94 | There are similarly named constructors for initializing 0, 1, 2, or 3 | |
95 | slots, or initializing @var{n} slots via an array. @xref{Foreign | |
96 | Objects}, for full details. Any fields that are not explicitly | |
97 | initialized are set to 0. | |
98 | ||
99 | To get or set the value of a slot by index, you can use the | |
100 | @code{scm_foreign_object_ref} and @code{scm_foreign_object_set_x} | |
101 | functions. These functions take and return values as @code{void *} | |
102 | pointers; there are corresponding convenience procedures like | |
103 | @code{_signed_ref}, @code{_unsigned_set_x} and so on for dealing with | |
104 | slots as signed or unsigned integers. | |
105 | ||
106 | Foreign objects fields that are pointers can be tricky to manage. If | |
107 | possible, it is best that all memory that is referenced by a foreign | |
108 | object be managed by the garbage collector. That way, the GC can | |
109 | automatically ensure that memory is accessible when it is needed, and | |
110 | freed when it becomes inaccessible. If this is not the case for your | |
111 | program -- for example, if you are exposing an object to Scheme that was | |
112 | allocated by some other, Guile-unaware part of your program -- then you | |
113 | will probably need to implement a finalizer. @xref{Foreign Object | |
114 | Memory Management}, for more. | |
115 | ||
116 | Continuing the example from above, if the global variable | |
117 | @code{image_type} contains the type returned by | |
118 | @code{scm_make_foreign_object_type}, here is how we could construct a | |
119 | foreign object whose ``data'' field contains a pointer to a freshly | |
120 | allocated @code{struct image}: | |
121 | ||
122 | @example | |
123 | SCM | |
124 | make_image (SCM name, SCM s_width, SCM s_height) | |
125 | @{ | |
126 | struct image *image; | |
127 | int width = scm_to_int (s_width); | |
128 | int height = scm_to_int (s_height); | |
129 | ||
130 | /* Allocate the `struct image'. Because we | |
131 | use scm_gc_malloc, this memory block will | |
132 | be automatically reclaimed when it becomes | |
133 | inaccessible, and its members will be traced | |
134 | by the garbage collector. */ | |
135 | image = (struct image *) | |
136 | scm_gc_malloc (sizeof (struct image), "image"); | |
137 | ||
138 | image->width = width; | |
139 | image->height = height; | |
140 | ||
141 | /* Allocating the pixels with | |
142 | scm_gc_malloc_pointerless means that the | |
143 | pixels data is collectable by GC, but | |
144 | that GC shouldn't spend time tracing its | |
145 | contents for nested pointers because there | |
146 | aren't any. */ | |
147 | image->pixels = | |
148 | scm_gc_malloc_pointerless (width * height, "image pixels"); | |
149 | ||
150 | image->name = name; | |
151 | image->update_func = SCM_BOOL_F; | |
152 | ||
153 | /* Now wrap the struct image* in a new foreign | |
154 | object, and return that object. */ | |
155 | return scm_make_foreign_object_1 (image_type, image); | |
156 | @} | |
157 | @end example | |
158 | ||
159 | We use @code{scm_gc_malloc_pointerless} for the pixel buffer to tell the | |
160 | garbage collector not to scan it for pointers. Calls to | |
161 | @code{scm_gc_malloc}, @code{scm_make_foreign_object_1}, and | |
162 | @code{scm_gc_malloc_pointerless} raise an exception in out-of-memory | |
163 | conditions; the garbage collector is able to reclaim previously | |
164 | allocated memory if that happens. | |
165 | ||
166 | ||
167 | @node Type Checking of Foreign Objects | |
168 | @subsection Type Checking of Foreign Objects | |
169 | ||
170 | Functions that operate on foreign objects should check that the passed | |
171 | @code{SCM} value indeed is of the correct type before accessing its | |
172 | data. They can do this with @code{scm_assert_foreign_object_type}. | |
173 | ||
174 | For example, here is a simple function that operates on an image object, | |
175 | and checks the type of its argument. | |
176 | ||
177 | @example | |
178 | SCM | |
179 | clear_image (SCM image_obj) | |
180 | @{ | |
181 | int area; | |
182 | struct image *image; | |
183 | ||
184 | scm_assert_foreign_object_type (image_type, image_obj); | |
185 | ||
186 | image = scm_foreign_object_ref (image_obj, 0); | |
187 | area = image->width * image->height; | |
188 | memset (image->pixels, 0, area); | |
189 | ||
190 | /* Invoke the image's update function. */ | |
191 | if (scm_is_true (image->update_func)) | |
192 | scm_call_0 (image->update_func); | |
193 | ||
194 | return SCM_UNSPECIFIED; | |
195 | @} | |
196 | @end example | |
197 | ||
198 | ||
199 | @node Foreign Object Memory Management | |
200 | @subsection Foreign Object Memory Management | |
201 | ||
202 | Once a foreign object has been released to the tender mercies of the | |
203 | Scheme system, it must be prepared to survive garbage collection. In | |
204 | the example above, all the memory associated with the foreign object is | |
205 | managed by the garbage collector because we used the @code{scm_gc_} | |
206 | allocation functions. Thus, no special care must be taken: the garbage | |
207 | collector automatically scans them and reclaims any unused memory. | |
208 | ||
209 | However, when data associated with a foreign object is managed in some | |
210 | other way---e.g., @code{malloc}'d memory or file descriptors---it is | |
211 | possible to specify a @dfn{finalizer} function to release those | |
212 | resources when the foreign object is reclaimed. | |
213 | ||
214 | As discussed in @pxref{Garbage Collection}, Guile's garbage collector | |
215 | will reclaim inaccessible memory as needed. This reclamation process | |
216 | runs concurrently with the main program. When Guile analyzes the heap | |
217 | and detemines that an object's memory can be reclaimed, that memory is | |
218 | simply put on a ``free list'' of objects that can be reclaimed. Usually | |
219 | that's the end of it -- that's all that garbage collection does, in | |
220 | Guile. However some objects can have ``finalizers'' associated with | |
221 | them -- functions that are called on reclaimable objects to effect any | |
222 | external cleanup actions. | |
223 | ||
224 | Finalizers are tricky business and it is best to avoid them. They can | |
225 | be invoked at unexpected times, or not at all -- for example, they are | |
226 | not invoked on process exit. They don't help the garbage collector do | |
227 | its job; in fact, they are a hindrance. Furthermore, they perturb the | |
228 | garbage collector's internal accounting. The GC decides to scan the | |
229 | heap when it thinks that it is necessary, after some amount of | |
230 | allocation. Finalizable objects almost always represent an amount of | |
231 | allocation that is invisible to the garbage collector. The effect can | |
232 | be that the actual resource usage of a system with finalizable objects | |
233 | is higher than what the GC thinks it should be. | |
234 | ||
235 | All those caveats aside, some foreign object types will need finalizers. | |
236 | For example, if we had a foreign object type that wrapped file | |
237 | descriptors -- and we aren't suggesting this, as Guile already has ports | |
238 | -- then you might define the type like this: | |
239 | ||
240 | @example | |
241 | static SCM file_type; | |
242 | ||
243 | static void | |
244 | finalize_file (SCM file) | |
245 | @{ | |
246 | int fd = scm_foreign_object_signed_ref (file, 0); | |
247 | scm_foreign_object_set_x (file, 0, -1); | |
248 | if (fd >= 0) | |
249 | close (fd); | |
250 | @} | |
251 | ||
252 | static void | |
253 | init_file_type (void) | |
254 | @{ | |
255 | SCM name, slots; | |
256 | scm_t_struct_finalize finalizer; | |
257 | ||
258 | name = scm_from_utf8_symbol ("file"); | |
259 | slots = scm_list_1 (scm_from_utf8_symbol ("fd")); | |
260 | finalizer = finalize_file; | |
261 | ||
262 | image_type = | |
263 | scm_make_foreign_object_type (name, slots, finalizer); | |
264 | @} | |
265 | ||
266 | static SCM | |
267 | make_file (int fd) | |
268 | @{ | |
269 | return scm_make_foreign_object_1 (file_type, (void *) fd); | |
270 | @} | |
271 | @end example | |
272 | ||
273 | @cindex finalizer | |
274 | @cindex finalization | |
275 | ||
276 | Note that the finalizer can be called in any context. In particular, if | |
277 | the user's Guile is built with support for threads, the finalizer may be | |
278 | called from any thread that is running Guile. In Guile 2.0, finalizers | |
279 | are invoked via ``asyncs'', which interleaves them with running Scheme | |
280 | code; @pxref{System asyncs}. In Guile 2.2 there will be a dedicated | |
281 | finalization thread, to ensure that the finalization doesn't run within | |
282 | the critical section of any other thread known to Guile. | |
283 | ||
284 | In either case, finalizers run concurrently with the main program, and | |
285 | so they need to be async-safe and thread-safe. If for some reason this | |
286 | is impossible, perhaps because you are embedding Guile in some | |
287 | application that is not itself thread-safe, you have a few options. One | |
288 | is to use guardians instead of finalizers, and arrange to pump the | |
289 | guardians for finalizable objects. @xref{Guardians}, for more | |
290 | information. The other option is to disable automatic finalization | |
291 | entirely, and arrange to call @code{scm_run_finalizers ()} at | |
292 | appropriate points. @xref{Foreign Objects}, for more on these | |
293 | interfaces. | |
294 | ||
295 | Finalizers are allowed to allocate memory, access GC-managed memory, and | |
296 | in general can do anything any Guile user code can do. This was not the | |
297 | case in Guile 1.8, where finalizers were much more restricted. In | |
298 | particular, in Guile 2.0, finalizers can resuscitate objects. We do not | |
299 | recommend that users avail themselves of this possibility, however, as a | |
300 | resuscitated object can re-expose other finalizable objects that have | |
301 | been already finalized back to Scheme. These objects will not be | |
302 | finalized again, but they could cause use-after-free problems to code | |
303 | that handles objects of that particular foreign object type. To guard | |
304 | against this possibility, robust finalization routines should clear | |
305 | state from the foreign object, as in the above @code{free_file} example. | |
306 | ||
307 | One final caveat. Foreign object finalizers are associated with the | |
308 | lifetime of a foreign object, not of its fields. If you access a field | |
309 | of a finalizable foreign object, and do not arrange to keep a reference | |
310 | on the foreign object itself, it could be that the outer foreign object | |
311 | gets finalized while you are working with its field. | |
312 | ||
313 | For example, consider a procedure to read some data from a file, from | |
314 | our example above. | |
315 | ||
316 | @example | |
317 | SCM | |
318 | read_bytes (SCM file, SCM n) | |
319 | @{ | |
320 | int fd; | |
321 | SCM buf; | |
322 | size_t len, pos; | |
323 | ||
324 | scm_assert_foreign_object_type (file_type, file); | |
325 | ||
326 | fd = scm_foreign_object_signed_ref (file, 0); | |
327 | if (fd < 0) | |
328 | scm_wrong_type_arg_msg ("read-bytes", SCM_ARG1, | |
329 | file, "open file"); | |
330 | ||
331 | len = scm_to_size_t (n); | |
332 | SCM buf = scm_c_make_bytevector (scm_to_size_t (n)); | |
333 | ||
334 | pos = 0; | |
335 | while (pos < len) | |
336 | @{ | |
337 | char *bytes = SCM_BYTEVECTOR_CONTENTS (buf); | |
338 | ssize_t count = read (fd, bytes + pos, len - pos); | |
339 | if (count < 0) | |
340 | scm_syserror ("read-bytes"); | |
341 | if (count == 0) | |
342 | break; | |
343 | pos += count; | |
344 | @} | |
345 | ||
346 | scm_remember_upto_here_1 (file); | |
347 | ||
348 | return scm_values (scm_list_2 (buf, scm_from_size_t (pos))); | |
349 | @} | |
350 | @end example | |
351 | ||
352 | After the prelude, only the @code{fd} value is used and the C compiler | |
353 | has no reason to keep the @code{file} object around. If | |
354 | @code{scm_c_make_bytevector} results in a garbage collection, | |
355 | @code{file} might not be on the stack or anywhere else and could be | |
356 | finalized, leaving @code{read} to read a closed (or, in a multi-threaded | |
357 | program, possibly re-used) file descriptor. The use of | |
358 | @code{scm_remember_upto_here_1} prevents this, by creating a reference | |
359 | to @code{file} after all data accesses. @xref{Garbage Collection | |
360 | Functions}. | |
361 | ||
362 | @code{scm_remember_upto_here_1} is only needed on finalizable objects, | |
363 | because garbage collection of other values is invisible to the program | |
364 | -- it happens when needed, and is not observable. But if you can, save | |
365 | yourself the headache and build your program in such a way that it | |
366 | doesn't need finalization. | |
367 | ||
368 | ||
369 | @node Foreign Objects and Scheme | |
370 | @subsection Foreign Objects and Scheme | |
371 | ||
372 | It is also possible to create foreign objects and object types from | |
373 | Scheme, and to access fields of foreign objects from Scheme. For | |
374 | example, the file example from the last section could be equivalently | |
375 | expressed as: | |
376 | ||
377 | @example | |
378 | (define-module (my-file) | |
379 | #:use-module (system foreign-object) | |
380 | #:use-module ((oop goops) #:select (make)) | |
381 | #:export (make-file)) | |
382 | ||
383 | (define (finalize-file file) | |
384 | (let ((fd (struct-ref file 0))) | |
385 | (unless (< fd 0) | |
386 | (struct-set! file 0 -1) | |
387 | (close-fdes fd)))) | |
388 | ||
389 | (define <file> | |
390 | (make-foreign-object-type '<file> '(fd) | |
391 | #:finalizer finalize-file)) | |
392 | ||
393 | (define (make-file fd) | |
394 | (make <file> #:fd fd)) | |
395 | @end example | |
396 | ||
397 | Here we see that the result of @code{make-foreign-object-type}, which is | |
398 | the equivalent of @code{scm_make_foreign_object_type}, is a struct | |
399 | vtable. @xref{Vtables}, for more information. To instantiate the | |
400 | foreign object, which is really a Guile struct, we use @code{make}. (We | |
401 | could have used @code{make-struct/no-tail}, but as an implementation | |
402 | detail, finalizer are attached in the @code{initialize} method called by | |
403 | @code{make}). To access the fields, we use @code{struct-ref} and | |
404 | @code{struct-set!}. @xref{Structure Basics}. | |
405 | ||
406 | There is a convenience syntax, @code{define-foreign-object-type}, that | |
407 | defines a type along with a constructor, and getters for the fields. An | |
408 | appropriate invocation of @code{define-foreign-object-type} for the | |
409 | file object type could look like this: | |
410 | ||
411 | @example | |
412 | (use-modules (system foreign-object)) | |
413 | ||
414 | (define-foreign-object-type <file> | |
415 | make-file | |
416 | (fd) | |
417 | #:finalizer finalize-file) | |
418 | @end example | |
419 | ||
420 | This defines the @code{<file>} type with one field, a @code{make-file} | |
421 | constructor, and a getter for the @code{fd} field, bound to @code{fd}. | |
422 | ||
423 | Foreign object types are not only vtables but are actually GOOPS | |
424 | classes, as hinted at above. @xref{GOOPS}, for more on Guile's | |
425 | object-oriented programming system. Thus one can define print and | |
426 | equality methods using GOOPS: | |
427 | ||
428 | @example | |
429 | (use-modules (oop goops)) | |
430 | ||
431 | (define-method (write (file <file>) port) | |
432 | ;; Assuming existence of the `fd' getter | |
433 | (format port "#<<file> ~a>" (fd file))) | |
434 | ||
435 | (define-method (equal? (a <file>) (b <file>)) | |
436 | (eqv? (fd a) (fd b))) | |
437 | @end example | |
438 | ||
439 | One can even sub-class foreign types. | |
440 | ||
441 | @example | |
442 | (define-class <named-file> (<file>) | |
443 | (name #:init-keyword #:name #:init-value #f #:accessor name)) | |
444 | @end example | |
445 | ||
446 | The question arises of how to construct these values, given that | |
447 | @code{make-file} returns a plain old @code{<file>} object. It turns out | |
448 | that you can use the GOOPS construction interface, where every field of | |
449 | the foreign object has an associated initialization keyword argument. | |
450 | ||
451 | @example | |
452 | (define* (my-open-file name #:optional (flags O_RDONLY)) | |
453 | (make <named-file> #:fd (open-fdes name flags) #:name name)) | |
454 | ||
455 | (define-method (write (file <named-file>) port) | |
456 | (format port "#<<file> ~s ~a>" (name file) (fd file))) | |
457 | @end example | |
458 | ||
459 | @xref{Foreign Objects}, for full documentation on the Scheme interface | |
460 | to foreign objects. @xref{GOOPS}, for more on GOOPS. | |
461 | ||
462 | As a final note, you might wonder how this system supports encapsulation | |
463 | of sensitive values. First, we have to recognize that some facilities | |
464 | are essentially unsafe and have global scope. For example, in C, the | |
465 | integrity and confidentiality of a part of a program is at the mercy of | |
466 | every other part of that program -- because any memory could potentially | |
467 | access any other. At the same time, principled access to structured | |
468 | data is organized in C on lexical boundaries; if you don't expose | |
469 | accessors for your object, you trust other parts of the program not to | |
470 | access it. | |
471 | ||
472 | The answer is similar in Scheme. Although Scheme's unsafe constructs | |
473 | are fewer in number than in C, they do exist. The @code{(system | |
474 | foreign)} module can be used to violate confidentiality and integrity, | |
475 | and shouldn't be exposed to untrusted code. Although @code{struct-ref} | |
476 | and @code{struct-set!} are less unsafe, they still have a cross-cutting | |
477 | capability of drilling through abstractions. Performing a | |
478 | @code{struct-set!} on a foreign object slot could cause unsafe foreign | |
479 | code to crash. Ultimately, structures in Scheme are capabilities for | |
480 | abstraction, and not abstractions themselves. | |
481 | ||
482 | That leaves us with the lexical capabilities, like constructors and | |
483 | accessors. Here is where encapsulation lies: the practical degree to | |
484 | which the innards of your foreign objects are exposed is the degree to | |
485 | which their accessors are lexically available in user code. If you want | |
486 | to allow users to reference fields of your foreign object, provide them | |
487 | with a getter. Otherwise you should assume that the only access to your | |
488 | object may come from your code, which has the relevant authority, or via | |
489 | code with access to cross-cutting @code{struct-ref} and such, which also | |
490 | has the cross-cutting authority. |