[gambit-list] creating homogeneous vectors from the FFI

Marc Feeley feeley at iro.umontreal.ca
Sat Jan 12 22:31:20 EST 2008


On 12-Jan-08, at 8:07 PM, David Rush wrote:

> Hi there,
>
> I am slowly getting code for FCGI support running, but I have a small
> issue with keeping Gambit compatibility - it is unclear to me how to
> create a u8vector to bring the output of read() back into the Scheme
> world.  The FCGI protocol is a fairly ugly mix of text and binary
> components that does not map very well to a C structure interface as
> most FFIs implement. I have had fairly good success in manipulating
> the raw octet stream using Larceny and now I am looking for ways to
> bring that same style into Gambit. u8vectors look like a good map to
> Larceny's bytevector type, but I just don't see how to get to them
> from the FFI without jumping through some serious SCMOBJ hoops.
>
> Or am I missing something?

Gambit's FFI was designed to simplify memory management issues by  
avoiding the sharing of data between the Scheme world and the C  
world.  Most data types have a different representation in the Scheme  
and C worlds, and Gambit will automatically create a copy of the data  
in "the other world".  For example, when passing a Scheme string as a  
parameter to a C function expecting a C "char-string" (type char*),  
the Gambit runtime will allocate memory on the C heap, and copy the  
Scheme string into it terminated with a nul character.  The C string  
is deallocated when the C function returns.  Copying takes more time  
than sharing the data, but greatly simplifies deallocation.

Let's now focus on vectors (homogeneous or not).  Vectors are  
problematic because vectors can be quite large and it would be very  
inefficient to use copying when going from one world to the other.  So  
it is tempting to use sharing for vectors.  However, like other memory  
allocated Scheme object, vectors can be allocated as "movable" and  
"still" objects (usually small vectors are allocated as movable, and  
large ones are allocated as still).  The garbage collector may move  
movable objects in memory (to compact the heap), but still objects  
always stay at the same address.  If sharing was used for vectors it  
would be natural to convert a Scheme vector into a ___SCMOBJ* pointer  
which points to the first element of the Scheme vector.  However, if  
the garbage collector was ever called during the lifetime of the C  
function receiving the ___SCMOBJ* pointer, the pointer would point to  
the wrong address if the vector was movable.  To solve this problem we  
have to force the vectors to be allocated as still objects.  This can  
be done by creating a "still" copy of a vector using the undocumented  
##still-copy procedure, as in:

(define v (##still-copy (make-u8vector 10)))

or directly calling the C ___alloc_scmobj function:

(define alloc-still-scmobj
   (c-lambda (int long)
             scheme-object
    "___result = ___EXT(___alloc_scmobj) (___arg1, ___arg2,  
___STILL);"))

(define subtype-s8vector 20)
(define subtype-u8vector 21)

(define (make-still-u8vector len)
   (alloc-still-scmobj subtype-u8vector len))

(define v (make-still-u8vector 10))

A still vector can be passed to C using the ___SCMOBJ type, and using  
the ___BODY macro to extract a pointer to the first element:

(c-declare #<<end-of-c-declare

void u8vect_inc (___SCMOBJ u8vect, int i)
{
   ___U8* ptr = ___CAST(___U8*,___BODY(u8vect));
   ptr[i]++;
}

end-of-c-declare
)

(define u8vector-inc
   (c-lambda (scheme-object int) void "u8vect_inc"))

(u8vector-inc v 4)
(u8vector-inc v 4)
(u8vector-inc v 6)

(pp v) ;; prints: #u8(0 0 0 0 2 0 1 0 0 0)

Marc




More information about the Gambit-list mailing list