On 12-Jan-08, at 8:07 PM, David Rush wrote:

Hi there,

I am slowly getting code for FCGI support running, but I have a small issue with keeping Gambit compatibility - it is unclear to me how to create a u8vector to bring the output of read() back into the Scheme world. The FCGI protocol is a fairly ugly mix of text and binary components that does not map very well to a C structure interface as most FFIs implement. I have had fairly good success in manipulating the raw octet stream using Larceny and now I am looking for ways to bring that same style into Gambit. u8vectors look like a good map to Larceny's bytevector type, but I just don't see how to get to them from the FFI without jumping through some serious SCMOBJ hoops.

Or am I missing something?

Gambit's FFI was designed to simplify memory management issues by avoiding the sharing of data between the Scheme world and the C world. Most data types have a different representation in the Scheme and C worlds, and Gambit will automatically create a copy of the data in "the other world". For example, when passing a Scheme string as a parameter to a C function expecting a C "char-string" (type char*), the Gambit runtime will allocate memory on the C heap, and copy the Scheme string into it terminated with a nul character. The C string is deallocated when the C function returns. Copying takes more time than sharing the data, but greatly simplifies deallocation.

Let's now focus on vectors (homogeneous or not). Vectors are problematic because vectors can be quite large and it would be very inefficient to use copying when going from one world to the other. So it is tempting to use sharing for vectors. However, like other memory allocated Scheme object, vectors can be allocated as "movable" and "still" objects (usually small vectors are allocated as movable, and large ones are allocated as still). The garbage collector may move movable objects in memory (to compact the heap), but still objects always stay at the same address. If sharing was used for vectors it would be natural to convert a Scheme vector into a ___SCMOBJ* pointer which points to the first element of the Scheme vector. However, if the garbage collector was ever called during the lifetime of the C function receiving the ___SCMOBJ* pointer, the pointer would point to the wrong address if the vector was movable. To solve this problem we have to force the vectors to be allocated as still objects. This can be done by creating a "still" copy of a vector using the undocumented ##still-copy procedure, as in:

(define v (##still-copy (make-u8vector 10)))

or directly calling the C ___alloc_scmobj function:

(define alloc-still-scmobj (c-lambda (int long) scheme-object "___result = ___EXT(___alloc_scmobj) (___arg1, ___arg2, ___STILL);"))

(define subtype-s8vector 20) (define subtype-u8vector 21)

(define (make-still-u8vector len) (alloc-still-scmobj subtype-u8vector len))

(define v (make-still-u8vector 10))

A still vector can be passed to C using the ___SCMOBJ type, and using the ___BODY macro to extract a pointer to the first element:

(c-declare #<<end-of-c-declare

void u8vect_inc (___SCMOBJ u8vect, int i) { ___U8* ptr = ___CAST(___U8*,___BODY(u8vect)); ptr[i]++; }

end-of-c-declare )

(define u8vector-inc (c-lambda (scheme-object int) void "u8vect_inc"))

(u8vector-inc v 4) (u8vector-inc v 4) (u8vector-inc v 6)

(pp v) ;; prints: #u8(0 0 0 0 2 0 1 0 0 0)

Marc