Commit | Line | Data |
---|---|---|
6e4630e0 AW |
1 | @c -*-texinfo-*- |
2 | @c This is part of the GNU Guile Reference Manual. | |
3 | @c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2005, 2010, 2011, 2013, 2014 | |
4 | @c Free Software Foundation, Inc. | |
5 | @c See the file guile.texi for copying conditions. | |
6 | ||
7 | @node Defining New Foreign Object Types | |
8 | @section Defining New Foreign Object Types | |
9 | ||
10 | The @dfn{foreign object type} facility is Guile's mechanism for | |
11 | importing object and types from C or other languages into Guile's | |
12 | system. If you have a C @code{struct foo} type, for example, you can | |
13 | define a corresponding Guile foreign object type that allows Scheme code | |
14 | to handle @code{struct foo *} objects. | |
15 | ||
16 | To define a new foreign object type, the programmer provides Guile with | |
17 | some essential information about the type --- what its name is, how many | |
18 | fields it has, and its finalizer (if any) --- and Guile allocates a | |
19 | fresh type for it. Foreign objects can be accessed from Scheme or from | |
20 | C. | |
21 | ||
22 | @menu | |
23 | * Defining Foreign Object Types:: | |
24 | * Creating Foreign Objects:: | |
25 | * Type Checking of Foreign Objects:: | |
26 | * Foreign Object Memory Management:: | |
27 | * Foreign Objects and Scheme:: | |
28 | @end menu | |
29 | ||
30 | @node Defining Foreign Object Types | |
31 | @subsection Defining Foreign Object Types | |
32 | ||
33 | To create a new foreign object type from C, call | |
34 | @code{scm_make_foreign_object_type}. It returns a value of type | |
35 | @code{SCM} which identifies the new type. | |
36 | ||
37 | Here is how one might declare a new type representing eight-bit | |
38 | gray-scale images: | |
39 | ||
40 | @example | |
41 | #include <libguile.h> | |
42 | ||
43 | struct image @{ | |
44 | int width, height; | |
45 | char *pixels; | |
46 | ||
47 | /* The name of this image */ | |
48 | SCM name; | |
49 | ||
50 | /* A function to call when this image is | |
51 | modified, e.g., to update the screen, | |
52 | or SCM_BOOL_F if no action necessary */ | |
53 | SCM update_func; | |
54 | @}; | |
55 | ||
56 | static SCM image_type image_type; | |
57 | ||
58 | void | |
59 | init_image_type (void) | |
60 | @{ | |
61 | SCM name, slots; | |
62 | scm_t_struct_finalize finalizer; | |
63 | ||
64 | name = scm_from_utf8_symbol ("image"); | |
65 | slots = scm_list_1 (scm_from_utf8_symbol ("data")); | |
66 | finalizer = NULL; | |
67 | ||
68 | image_type = | |
69 | scm_make_foreign_object_type (name, slots, finalizer); | |
70 | @} | |
71 | @end example | |
72 | ||
73 | The result is an initialized @code{image_type} value that identifies the | |
74 | new foreign object type. The next section describes how to create | |
75 | foreign objects and how to access their slots. | |
76 | ||
77 | ||
78 | @node Creating Foreign Objects | |
79 | @subsection Creating Foreign Objects | |
80 | ||
81 | Foreign objects contain zero or more ``slots'' of data. A slot can hold | |
82 | a pointer, an integer that fits into a @code{size_t} or @code{ssize_t}, | |
83 | or a @code{SCM} value. | |
84 | ||
85 | All objects of a given foreign type have the same number of slots. In | |
86 | the example from the previous section, the @code{image} type has one | |
87 | slot, because the slots list passed to | |
88 | @code{scm_make_foreign_object_type} is of length one. (The actual names | |
d9a4a1cd AW |
89 | given to slots are unimportant for most users of the C interface, but |
90 | can be used on the Scheme side to introspect on the foreign object.) | |
6e4630e0 AW |
91 | |
92 | To construct a foreign object and initialize its first slot, call | |
93 | @code{scm_make_foreign_object_1 (@var{type}, @var{first_slot_value})}. | |
94 | There are similarly named constructors for initializing 0, 1, 2, or 3 | |
95 | slots, or initializing @var{n} slots via an array. @xref{Foreign | |
96 | Objects}, for full details. Any fields that are not explicitly | |
97 | initialized are set to 0. | |
98 | ||
99 | To get or set the value of a slot by index, you can use the | |
100 | @code{scm_foreign_object_ref} and @code{scm_foreign_object_set_x} | |
101 | functions. These functions take and return values as @code{void *} | |
102 | pointers; there are corresponding convenience procedures like | |
103 | @code{_signed_ref}, @code{_unsigned_set_x} and so on for dealing with | |
104 | slots as signed or unsigned integers. | |
105 | ||
106 | Foreign objects fields that are pointers can be tricky to manage. If | |
107 | possible, it is best that all memory that is referenced by a foreign | |
108 | object be managed by the garbage collector. That way, the GC can | |
109 | automatically ensure that memory is accessible when it is needed, and | |
110 | freed when it becomes inaccessible. If this is not the case for your | |
111 | program -- for example, if you are exposing an object to Scheme that was | |
112 | allocated by some other, Guile-unaware part of your program -- then you | |
113 | will probably need to implement a finalizer. @xref{Foreign Object | |
114 | Memory Management}, for more. | |
115 | ||
d9a4a1cd | 116 | Continuing the example from the previous section, if the global variable |
6e4630e0 AW |
117 | @code{image_type} contains the type returned by |
118 | @code{scm_make_foreign_object_type}, here is how we could construct a | |
119 | foreign object whose ``data'' field contains a pointer to a freshly | |
120 | allocated @code{struct image}: | |
121 | ||
122 | @example | |
123 | SCM | |
124 | make_image (SCM name, SCM s_width, SCM s_height) | |
125 | @{ | |
126 | struct image *image; | |
127 | int width = scm_to_int (s_width); | |
128 | int height = scm_to_int (s_height); | |
129 | ||
130 | /* Allocate the `struct image'. Because we | |
131 | use scm_gc_malloc, this memory block will | |
132 | be automatically reclaimed when it becomes | |
133 | inaccessible, and its members will be traced | |
134 | by the garbage collector. */ | |
135 | image = (struct image *) | |
136 | scm_gc_malloc (sizeof (struct image), "image"); | |
137 | ||
138 | image->width = width; | |
139 | image->height = height; | |
140 | ||
141 | /* Allocating the pixels with | |
142 | scm_gc_malloc_pointerless means that the | |
143 | pixels data is collectable by GC, but | |
144 | that GC shouldn't spend time tracing its | |
145 | contents for nested pointers because there | |
146 | aren't any. */ | |
147 | image->pixels = | |
148 | scm_gc_malloc_pointerless (width * height, "image pixels"); | |
149 | ||
150 | image->name = name; | |
151 | image->update_func = SCM_BOOL_F; | |
152 | ||
153 | /* Now wrap the struct image* in a new foreign | |
154 | object, and return that object. */ | |
155 | return scm_make_foreign_object_1 (image_type, image); | |
156 | @} | |
157 | @end example | |
158 | ||
159 | We use @code{scm_gc_malloc_pointerless} for the pixel buffer to tell the | |
160 | garbage collector not to scan it for pointers. Calls to | |
161 | @code{scm_gc_malloc}, @code{scm_make_foreign_object_1}, and | |
162 | @code{scm_gc_malloc_pointerless} raise an exception in out-of-memory | |
163 | conditions; the garbage collector is able to reclaim previously | |
164 | allocated memory if that happens. | |
165 | ||
166 | ||
167 | @node Type Checking of Foreign Objects | |
168 | @subsection Type Checking of Foreign Objects | |
169 | ||
170 | Functions that operate on foreign objects should check that the passed | |
171 | @code{SCM} value indeed is of the correct type before accessing its | |
172 | data. They can do this with @code{scm_assert_foreign_object_type}. | |
173 | ||
174 | For example, here is a simple function that operates on an image object, | |
175 | and checks the type of its argument. | |
176 | ||
177 | @example | |
178 | SCM | |
179 | clear_image (SCM image_obj) | |
180 | @{ | |
181 | int area; | |
182 | struct image *image; | |
183 | ||
184 | scm_assert_foreign_object_type (image_type, image_obj); | |
185 | ||
186 | image = scm_foreign_object_ref (image_obj, 0); | |
187 | area = image->width * image->height; | |
188 | memset (image->pixels, 0, area); | |
189 | ||
190 | /* Invoke the image's update function. */ | |
191 | if (scm_is_true (image->update_func)) | |
192 | scm_call_0 (image->update_func); | |
193 | ||
194 | return SCM_UNSPECIFIED; | |
195 | @} | |
196 | @end example | |
197 | ||
198 | ||
199 | @node Foreign Object Memory Management | |
200 | @subsection Foreign Object Memory Management | |
201 | ||
202 | Once a foreign object has been released to the tender mercies of the | |
203 | Scheme system, it must be prepared to survive garbage collection. In | |
204 | the example above, all the memory associated with the foreign object is | |
205 | managed by the garbage collector because we used the @code{scm_gc_} | |
206 | allocation functions. Thus, no special care must be taken: the garbage | |
207 | collector automatically scans them and reclaims any unused memory. | |
208 | ||
209 | However, when data associated with a foreign object is managed in some | |
210 | other way---e.g., @code{malloc}'d memory or file descriptors---it is | |
211 | possible to specify a @dfn{finalizer} function to release those | |
212 | resources when the foreign object is reclaimed. | |
213 | ||
214 | As discussed in @pxref{Garbage Collection}, Guile's garbage collector | |
215 | will reclaim inaccessible memory as needed. This reclamation process | |
216 | runs concurrently with the main program. When Guile analyzes the heap | |
d9a4a1cd AW |
217 | and determines that an object's memory can be reclaimed, that memory is |
218 | put on a ``free list'' of objects that can be reclaimed. Usually that's | |
219 | the end of it---the object is available for immediate re-use. However | |
220 | some objects can have ``finalizers'' associated with them---functions | |
221 | that are called on reclaimable objects to effect any external cleanup | |
222 | actions. | |
6e4630e0 AW |
223 | |
224 | Finalizers are tricky business and it is best to avoid them. They can | |
d9a4a1cd | 225 | be invoked at unexpected times, or not at all---for example, they are |
6e4630e0 AW |
226 | not invoked on process exit. They don't help the garbage collector do |
227 | its job; in fact, they are a hindrance. Furthermore, they perturb the | |
228 | garbage collector's internal accounting. The GC decides to scan the | |
229 | heap when it thinks that it is necessary, after some amount of | |
230 | allocation. Finalizable objects almost always represent an amount of | |
231 | allocation that is invisible to the garbage collector. The effect can | |
232 | be that the actual resource usage of a system with finalizable objects | |
233 | is higher than what the GC thinks it should be. | |
234 | ||
235 | All those caveats aside, some foreign object types will need finalizers. | |
236 | For example, if we had a foreign object type that wrapped file | |
d9a4a1cd AW |
237 | descriptors---and we aren't suggesting this, as Guile already has ports |
238 | ---then you might define the type like this: | |
6e4630e0 AW |
239 | |
240 | @example | |
241 | static SCM file_type; | |
242 | ||
243 | static void | |
244 | finalize_file (SCM file) | |
245 | @{ | |
246 | int fd = scm_foreign_object_signed_ref (file, 0); | |
6e4630e0 | 247 | if (fd >= 0) |
d9a4a1cd AW |
248 | @{ |
249 | scm_foreign_object_signed_set_x (file, 0, -1); | |
250 | close (fd); | |
251 | @} | |
6e4630e0 AW |
252 | @} |
253 | ||
254 | static void | |
255 | init_file_type (void) | |
256 | @{ | |
257 | SCM name, slots; | |
258 | scm_t_struct_finalize finalizer; | |
259 | ||
260 | name = scm_from_utf8_symbol ("file"); | |
261 | slots = scm_list_1 (scm_from_utf8_symbol ("fd")); | |
262 | finalizer = finalize_file; | |
263 | ||
264 | image_type = | |
265 | scm_make_foreign_object_type (name, slots, finalizer); | |
266 | @} | |
267 | ||
268 | static SCM | |
269 | make_file (int fd) | |
270 | @{ | |
271 | return scm_make_foreign_object_1 (file_type, (void *) fd); | |
272 | @} | |
273 | @end example | |
274 | ||
275 | @cindex finalizer | |
276 | @cindex finalization | |
277 | ||
d9a4a1cd AW |
278 | Note that the finalizer may be invoked in ways and at times you might |
279 | not expect. In particular, if the user's Guile is built with support | |
280 | for threads, the finalizer may be called from any thread that is running | |
281 | Guile. In Guile 2.0, finalizers are invoked via ``asyncs'', which | |
282 | interleaves them with running Scheme code; @pxref{System asyncs}. In | |
283 | Guile 2.2 there will be a dedicated finalization thread, to ensure that | |
284 | the finalization doesn't run within the critical section of any other | |
285 | thread known to Guile. | |
6e4630e0 AW |
286 | |
287 | In either case, finalizers run concurrently with the main program, and | |
288 | so they need to be async-safe and thread-safe. If for some reason this | |
289 | is impossible, perhaps because you are embedding Guile in some | |
290 | application that is not itself thread-safe, you have a few options. One | |
291 | is to use guardians instead of finalizers, and arrange to pump the | |
292 | guardians for finalizable objects. @xref{Guardians}, for more | |
293 | information. The other option is to disable automatic finalization | |
294 | entirely, and arrange to call @code{scm_run_finalizers ()} at | |
295 | appropriate points. @xref{Foreign Objects}, for more on these | |
296 | interfaces. | |
297 | ||
298 | Finalizers are allowed to allocate memory, access GC-managed memory, and | |
299 | in general can do anything any Guile user code can do. This was not the | |
300 | case in Guile 1.8, where finalizers were much more restricted. In | |
301 | particular, in Guile 2.0, finalizers can resuscitate objects. We do not | |
302 | recommend that users avail themselves of this possibility, however, as a | |
303 | resuscitated object can re-expose other finalizable objects that have | |
304 | been already finalized back to Scheme. These objects will not be | |
305 | finalized again, but they could cause use-after-free problems to code | |
306 | that handles objects of that particular foreign object type. To guard | |
307 | against this possibility, robust finalization routines should clear | |
308 | state from the foreign object, as in the above @code{free_file} example. | |
309 | ||
310 | One final caveat. Foreign object finalizers are associated with the | |
311 | lifetime of a foreign object, not of its fields. If you access a field | |
312 | of a finalizable foreign object, and do not arrange to keep a reference | |
313 | on the foreign object itself, it could be that the outer foreign object | |
314 | gets finalized while you are working with its field. | |
315 | ||
316 | For example, consider a procedure to read some data from a file, from | |
317 | our example above. | |
318 | ||
319 | @example | |
320 | SCM | |
321 | read_bytes (SCM file, SCM n) | |
322 | @{ | |
323 | int fd; | |
324 | SCM buf; | |
325 | size_t len, pos; | |
326 | ||
327 | scm_assert_foreign_object_type (file_type, file); | |
328 | ||
329 | fd = scm_foreign_object_signed_ref (file, 0); | |
330 | if (fd < 0) | |
331 | scm_wrong_type_arg_msg ("read-bytes", SCM_ARG1, | |
332 | file, "open file"); | |
333 | ||
334 | len = scm_to_size_t (n); | |
335 | SCM buf = scm_c_make_bytevector (scm_to_size_t (n)); | |
336 | ||
337 | pos = 0; | |
338 | while (pos < len) | |
339 | @{ | |
340 | char *bytes = SCM_BYTEVECTOR_CONTENTS (buf); | |
341 | ssize_t count = read (fd, bytes + pos, len - pos); | |
342 | if (count < 0) | |
343 | scm_syserror ("read-bytes"); | |
344 | if (count == 0) | |
345 | break; | |
346 | pos += count; | |
347 | @} | |
348 | ||
349 | scm_remember_upto_here_1 (file); | |
350 | ||
351 | return scm_values (scm_list_2 (buf, scm_from_size_t (pos))); | |
352 | @} | |
353 | @end example | |
354 | ||
355 | After the prelude, only the @code{fd} value is used and the C compiler | |
356 | has no reason to keep the @code{file} object around. If | |
357 | @code{scm_c_make_bytevector} results in a garbage collection, | |
358 | @code{file} might not be on the stack or anywhere else and could be | |
359 | finalized, leaving @code{read} to read a closed (or, in a multi-threaded | |
360 | program, possibly re-used) file descriptor. The use of | |
361 | @code{scm_remember_upto_here_1} prevents this, by creating a reference | |
362 | to @code{file} after all data accesses. @xref{Garbage Collection | |
363 | Functions}. | |
364 | ||
365 | @code{scm_remember_upto_here_1} is only needed on finalizable objects, | |
366 | because garbage collection of other values is invisible to the program | |
367 | -- it happens when needed, and is not observable. But if you can, save | |
368 | yourself the headache and build your program in such a way that it | |
369 | doesn't need finalization. | |
370 | ||
371 | ||
372 | @node Foreign Objects and Scheme | |
373 | @subsection Foreign Objects and Scheme | |
374 | ||
375 | It is also possible to create foreign objects and object types from | |
376 | Scheme, and to access fields of foreign objects from Scheme. For | |
377 | example, the file example from the last section could be equivalently | |
378 | expressed as: | |
379 | ||
380 | @example | |
381 | (define-module (my-file) | |
382 | #:use-module (system foreign-object) | |
383 | #:use-module ((oop goops) #:select (make)) | |
384 | #:export (make-file)) | |
385 | ||
386 | (define (finalize-file file) | |
387 | (let ((fd (struct-ref file 0))) | |
388 | (unless (< fd 0) | |
389 | (struct-set! file 0 -1) | |
390 | (close-fdes fd)))) | |
391 | ||
392 | (define <file> | |
393 | (make-foreign-object-type '<file> '(fd) | |
394 | #:finalizer finalize-file)) | |
395 | ||
396 | (define (make-file fd) | |
397 | (make <file> #:fd fd)) | |
398 | @end example | |
399 | ||
400 | Here we see that the result of @code{make-foreign-object-type}, which is | |
401 | the equivalent of @code{scm_make_foreign_object_type}, is a struct | |
402 | vtable. @xref{Vtables}, for more information. To instantiate the | |
403 | foreign object, which is really a Guile struct, we use @code{make}. (We | |
404 | could have used @code{make-struct/no-tail}, but as an implementation | |
d9a4a1cd AW |
405 | detail, finalizers are attached in the @code{initialize} method called |
406 | by @code{make}). To access the fields, we use @code{struct-ref} and | |
6e4630e0 AW |
407 | @code{struct-set!}. @xref{Structure Basics}. |
408 | ||
409 | There is a convenience syntax, @code{define-foreign-object-type}, that | |
410 | defines a type along with a constructor, and getters for the fields. An | |
411 | appropriate invocation of @code{define-foreign-object-type} for the | |
412 | file object type could look like this: | |
413 | ||
414 | @example | |
415 | (use-modules (system foreign-object)) | |
416 | ||
417 | (define-foreign-object-type <file> | |
418 | make-file | |
419 | (fd) | |
420 | #:finalizer finalize-file) | |
421 | @end example | |
422 | ||
423 | This defines the @code{<file>} type with one field, a @code{make-file} | |
424 | constructor, and a getter for the @code{fd} field, bound to @code{fd}. | |
425 | ||
426 | Foreign object types are not only vtables but are actually GOOPS | |
427 | classes, as hinted at above. @xref{GOOPS}, for more on Guile's | |
428 | object-oriented programming system. Thus one can define print and | |
429 | equality methods using GOOPS: | |
430 | ||
431 | @example | |
432 | (use-modules (oop goops)) | |
433 | ||
434 | (define-method (write (file <file>) port) | |
435 | ;; Assuming existence of the `fd' getter | |
436 | (format port "#<<file> ~a>" (fd file))) | |
437 | ||
438 | (define-method (equal? (a <file>) (b <file>)) | |
439 | (eqv? (fd a) (fd b))) | |
440 | @end example | |
441 | ||
442 | One can even sub-class foreign types. | |
443 | ||
444 | @example | |
445 | (define-class <named-file> (<file>) | |
446 | (name #:init-keyword #:name #:init-value #f #:accessor name)) | |
447 | @end example | |
448 | ||
449 | The question arises of how to construct these values, given that | |
450 | @code{make-file} returns a plain old @code{<file>} object. It turns out | |
451 | that you can use the GOOPS construction interface, where every field of | |
452 | the foreign object has an associated initialization keyword argument. | |
453 | ||
454 | @example | |
455 | (define* (my-open-file name #:optional (flags O_RDONLY)) | |
456 | (make <named-file> #:fd (open-fdes name flags) #:name name)) | |
457 | ||
458 | (define-method (write (file <named-file>) port) | |
459 | (format port "#<<file> ~s ~a>" (name file) (fd file))) | |
460 | @end example | |
461 | ||
462 | @xref{Foreign Objects}, for full documentation on the Scheme interface | |
463 | to foreign objects. @xref{GOOPS}, for more on GOOPS. | |
464 | ||
465 | As a final note, you might wonder how this system supports encapsulation | |
466 | of sensitive values. First, we have to recognize that some facilities | |
467 | are essentially unsafe and have global scope. For example, in C, the | |
468 | integrity and confidentiality of a part of a program is at the mercy of | |
d9a4a1cd AW |
469 | every other part of that program -- because any part of the program can |
470 | read and write anything in its address space. At the same time, | |
471 | principled access to structured data is organized in C on lexical | |
472 | boundaries; if you don't expose accessors for your object, you trust | |
473 | other parts of the program not to work around that barrier. | |
474 | ||
475 | The situation is not dissimilar in Scheme. Although Scheme's unsafe | |
476 | constructs are fewer in number than in C, they do exist. The | |
477 | @code{(system foreign)} module can be used to violate confidentiality | |
478 | and integrity, and shouldn't be exposed to untrusted code. Although | |
479 | @code{struct-ref} and @code{struct-set!} are less unsafe, they still | |
480 | have a cross-cutting capability of drilling through abstractions. | |
481 | Performing a @code{struct-set!} on a foreign object slot could cause | |
482 | unsafe foreign code to crash. Ultimately, structures in Scheme are | |
483 | capabilities for abstraction, and not abstractions themselves. | |
6e4630e0 AW |
484 | |
485 | That leaves us with the lexical capabilities, like constructors and | |
486 | accessors. Here is where encapsulation lies: the practical degree to | |
487 | which the innards of your foreign objects are exposed is the degree to | |
488 | which their accessors are lexically available in user code. If you want | |
489 | to allow users to reference fields of your foreign object, provide them | |
490 | with a getter. Otherwise you should assume that the only access to your | |
491 | object may come from your code, which has the relevant authority, or via | |
492 | code with access to cross-cutting @code{struct-ref} and such, which also | |
493 | has the cross-cutting authority. |