On 2012-02-01, at 4:19 AM, Mikael wrote:

Btw, something I'd appreciate much to see documented on the wiki would be a complete reference on how to traverse and construct Scheme structures from C. So like, how do cdr, car, null?, cons, vector-ref, boolean values, fixnums etc.

Why do you need this?

The problem in documenting the macros in gambit.h (such as ___CONS, ___CAR, ___PAIRP, etc) is that their API is very complex when it comes to memory allocation. Their proper operation depends on invariants that are error-prone to maintain by hand, but that are automatically accounted for by the Gambit-C backend.

Let me give you an idea by explaining how ___CONS works.

At a high level, the C macro call ___CONS(x,y) performs the operation of the Scheme call (cons x y). The detailed implementation adds quite a bit of complexity to how ___CONS must be used.

1) ___CONS performs the allocation using the heap pointer ___hp (whose type is ___SCMOBJ *). For performance, ___hp is a local variable of the C function containing the call to ___CONS. At some point in the C code before the call to ___CONS (usually at the start of the C function), the global heap pointer is copied to ___hp. Also, at some point after the call to ___CONS (and other memory allocating macros), the variable ___hp is copied to the global heap pointer. Doing this makes it likely that the C compiler will allocate ___hp to a machine register.

2) ___CONS does not perform a heap overflow check because the Gambit-C backend is optimized to perform a single check for multiple allocations. So the overflow check has to be performed separately, by the ___CHECK_HEAP macro.

3) When the ___CHECK_HEAP macro detects that the heap is full, it calls the GC to free some space. Because objects might be moved by the GC, just before calling the GC, the live variables have to be saved to a safe place visited by the GC (typically the stack), and after the call to the GC, the live variables are restored to point to the new location of the objects. This live variable analysis is done automatically by the Gambit-C backend. It would be very tedious and error prone to do it by hand.

4) Finally, the call to the GC is in fact a non-tail Scheme call. This makes the saving of the Scheme context simple (the live variables are automatically saved on the stack). The Gambit-C backend can easily deal with the intricacies of performing a Scheme call, but this would be tedious and error prone to do it by hand.

Note that a pair could be allocated by the ___make_pair function. However, there are some issues here also related to memory allocation. The documentation of ___make_pair in lib/mem.c is as follows:

/* * '___make_pair (car, cdr, kind)' creates a Scheme pair having the * values 'car' and 'cdr' in its CAR and CDR fields. The 'car' and * 'cdr' arguments must not be movable objects and any still object * must be reachable some other way or have a nonzero refcount. A * permanent or still object is allocated, depending on 'kind' * (___PERM for permanent object, ___STILL for still object). A * fixnum error code is returned when there is an error. */

You could also implement your own interface using the FFI. For example, a C callable function to allocate pairs could be defined this way:

(c-define (alloc-pair car cdr) (scheme-object scheme-object) scheme-object "alloc_pair" "" (cons car cdr))

Note however that the C code calling this function must be careful not to keep the resulting pair reference in a local variable (invisible to the GC) at a point in time where the GC might be called. This problem can be eliminated by using still objects and reference counts:

(c-define (alloc-pair car cdr) (scheme-object scheme-object) scheme-object "alloc_pair" "" (##still-obj-refcount-inc! (##still-copy (cons car cdr))))

At some safe point in the C or Scheme code, you will have to decrement the reference count (otherwise the object will never be reclaimed by the GC).

I think these complexities are best addressed by writing the code in Scheme and letting the Gambit-C backend worry about the memory allocation issues. In rare cases I do write Scheme memory allocations in C, but I keep it simple and I know what I'm doing!

This is a case where documenting these internal macros will cause more trouble than not documenting them, because if I document them people will start using them, and the mailing-list (and I) will be inundated by "what am I doing wrong" questions. Moreover, the implementation of the internal macros and their invariants may have to be changed to accommodate new features (generational GC, multiprocessing, ...) and I don't want to deal with legacy code which relies on the old invariants.

I guess this is Gambit's pandora's box which I am reluctant to open.

Marc

On 2/1/12, Marc Feeley feeley@iro.umontreal.ca wrote:

On 2012-02-01, at 12:19 AM, REPLeffect wrote:

...

Would it be possible to get a configure option that will allow ___BOOL to be defined as bool instead of int? I understand at least some of the reasons behind defining it as int for compatibility with some compilers. It would just be nice to be able to chose to have it be bool instead.

Perhaps there are ramifications which I have not considered. But it makes it nice for interfacing with my C++ code to have ___BOOL actually be bool (unless, of course, unconsidered ramifications would have detrimental effects on me -- in which case, please feel free to enlighten me) :-D

I have added to the configure script an automatic detection of support for the bool type. The ___BOOL macro will be defined in gambit.h accordingly. If you want to override the default, you can now supply the definition of the ___BOOL macro on the command-line (e.g. -D___BOOL=int).

Marc

Wow, that's excellent! Thanks a lot, Marc!