[gambit-list] How allocate u8vectors from C safely&without memory leaks? (Noted ___EXT(___alloc_scmobj)(___sU8VECTOR, len, ___MOVABLE0) leaks 100%, that's used by the sqlite ffi etc.)
Jussi Piitulainen
jpiitula at ling.helsinki.fi
Sun Jun 16 09:47:45 EDT 2013
Mikael, Marc, the mention of sqlite specifically caught my eye.
I use the following code for accessing blobs in the database as any
homogeneous vector type in my own SQLite3 FFI of last summer. The code
was exercised as part of a larger program and seemed to work. __STILL
was the key.
Maybe it should decrease a refcount when __alloc_scmobj fails?
(c-declare
#<<c-declare-end
static ___SCMOBJ
resqlite3_column_blob(sqlite3_stmt* stmt, int col, int type, int size)
{
const void *buf = sqlite3_column_blob(stmt, col);
int n = sqlite3_column_bytes(stmt, col);
if (n % size) return ___FIX(___UNKNOWN_ERR);
___SCMOBJ result = ___EXT(___alloc_scmobj)(type,n,___STILL);
if (___FIXNUMP(result)) return result;
memcpy(___BODY(result), buf, n);
return result;
}
c-declare-end
)
(define sqlite3-column-u8vector-or-code
(c-lambda
(sqlite3-prepared-statement int) scheme-object
"___result = resqlite3_column_blob(___arg1,___arg2,___sU8VECTOR,1);"))
The latter definition is then repeated for each vector type and
element size.
Mikael <mikael.rcv at gmail.com> writes:
> Dear Marc,
>
> Just realized there's a memory leak in many Gambit libraries, such as the
> sqlite FFI - their u8vector allocation in C boils down to:
>
> ___result = ___EXT(___alloc_scmobj)(___sU8VECTOR,len,___MOVABLE0);
>
>
>
>
> Studying ##make-u8vector of _kernel.scm provides some inspiration: It
>
> 1) Shows that it's fine to use ___STILL
>
> 2) Uses ___still_obj_refcount_dec ,
>
> 3) Has its C code in a ##c-code and not a ##c-lambda
>
> 4) Has its surrounding Scheme code in (##declare (not interrupts-enabled))
>
> 5) Surrounds the ___alloc_scmobj call by some magic (
> ___FRAME_STORE_RA(___R0)___W_ALL ___R_ALL ___SET_R0(___FRAME_FETCH_RA) )
>
> (##make-u8vector provides a code path for small objects apart from this,
> though the use regarded here should be for u8vectors of nontrivial size so
> going with the codepath for large objects should always be fine.)
>
>
> By applying 1) and 2) I get something that works but can't know if it's
> really correct and safe:
>
>
> How do you allocate an u8vector in C code in a way that does not leak
> memory, and that safely transfers the new object reference from the C world
> to the Scheme world?
>
>
>
> Thanks!
> Mikael
>
>
> *How to reproduce*
> Running a test that repeats
>
> ((c-lambda () scheme-object "___result =
> ___EXT(___alloc_scmobj)(___sU8VECTOR,10000,___MOVABLE0);"))
>
>
> over and over e.g.
>
> (##gc-report-set! #t)
> (for-each (lambda (x) ((c-lambda () scheme-object "___result =
> ___EXT(___alloc_scmobj)(___sU8VECTOR,10000,___MOVABLE0);")))
> (string->list (make-string 100000))
> (##gc)
>
>
> shows that this memory leaks all the allocated memory (!).
>
>
> *Fix, duno if safe & complete*
>
> (##gc-report-set! #t)
> (for-each (lambda (x) ((c-lambda () scheme-object "___result =
> ___EXT(___alloc_scmobj)(___sU8VECTOR,10000,___STILL);
> if
> (!___FIXNUMP(___result)) ___still_obj_refcount_dec (___result);
> ")))
> (string->list (make-string 100000))
> (##gc)
>
>
>
> *Copy of ##make-u8vector of _kernel.scm for reference*
>
> (define-prim (##make-u8vector k #!optional (fill (macro-absent-obj)))
> (##declare (not interrupts-enabled))
> (let ((v (##c-code #<<end-of-code
>
> long i;
> long n = ___INT(___ARG1);
> long words = ___WORDS(n) + 1;
> ___SCMOBJ result;
> if (n > ___CAST(___WORD, ___LMASK>>___LF))
> result = ___FIX(___HEAP_OVERFLOW_ERR); /* requested object is too big! */
> else if (words > ___MSECTION_BIGGEST)
> {
> ___FRAME_STORE_RA(___R0)
> ___W_ALL
> result = ___alloc_scmobj (___sU8VECTOR, n, ___STILL);
> ___R_ALL
> ___SET_R0(___FRAME_FETCH_RA)
> if (!___FIXNUMP(result))
> ___still_obj_refcount_dec (result);
> }
> else
> {
> ___BOOL overflow = 0;
> ___hp += words;
> if (___hp > ___ps->heap_limit)
> {
> ___FRAME_STORE_RA(___R0)
> ___W_ALL
> overflow = ___heap_limit () && ___garbage_collect (0);
> ___R_ALL
> ___SET_R0(___FRAME_FETCH_RA)
> }
> else
> ___hp -= words;
> if (overflow)
> result = ___FIX(___HEAP_OVERFLOW_ERR);
> else
> {
> result = ___TAG(___hp, ___tSUBTYPED);
> ___HEADER(result) = ___MAKE_HD_BYTES(n, ___sU8VECTOR);
> ___hp += words;
> }
> }
> if (!___FIXNUMP(result) && ___ARG2 != ___ABSENT)
> {
> for (i=0; i<n; i++)
> ___U8VECTORSET(result,___FIX(i),___ARG2)
> }
> ___RESULT = result;
>
> end-of-code
>
> k
> fill)))
> (if (##fixnum? v)
> (begin
> (##raise-heap-overflow-exception)
> (##make-u8vector k fill))
> v)))
> _______________________________________________
> Gambit-list mailing list
> Gambit-list at iro.umontreal.ca
> https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
More information about the Gambit-list
mailing list