[gambit-list] How allocate u8vectors from C safely&without memory leaks? (Noted ___EXT(___alloc_scmobj)(___sU8VECTOR, len, ___MOVABLE0) leaks 100%, that's used by the sqlite ffi etc.)

Jussi Piitulainen jpiitula at ling.helsinki.fi
Sun Jun 16 09:47:45 EDT 2013


Mikael, Marc, the mention of sqlite specifically caught my eye.

I use the following code for accessing blobs in the database as any
homogeneous vector type in my own SQLite3 FFI of last summer. The code
was exercised as part of a larger program and seemed to work. __STILL
was the key.

Maybe it should decrease a refcount when __alloc_scmobj fails?

(c-declare
#<<c-declare-end
static ___SCMOBJ
resqlite3_column_blob(sqlite3_stmt* stmt, int col, int type, int size)
{
   const void *buf = sqlite3_column_blob(stmt, col);
   int n = sqlite3_column_bytes(stmt, col);
   if (n % size) return ___FIX(___UNKNOWN_ERR);
   ___SCMOBJ result = ___EXT(___alloc_scmobj)(type,n,___STILL);
   if (___FIXNUMP(result)) return result;
   memcpy(___BODY(result), buf, n);
   return result;
}
c-declare-end
)

(define sqlite3-column-u8vector-or-code
  (c-lambda
   (sqlite3-prepared-statement int) scheme-object
   "___result = resqlite3_column_blob(___arg1,___arg2,___sU8VECTOR,1);"))

The latter definition is then repeated for each vector type and
element size.


Mikael <mikael.rcv at gmail.com> writes:

> Dear Marc,
> 
> Just realized there's a memory leak in many Gambit libraries, such as the
> sqlite FFI - their u8vector allocation in C boils down to:
> 
> ___result = ___EXT(___alloc_scmobj)(___sU8VECTOR,len,___MOVABLE0);
> 
> 
> 
> 
> Studying ##make-u8vector of _kernel.scm provides some inspiration: It
> 
>  1) Shows that it's fine to use ___STILL
> 
>  2) Uses  ___still_obj_refcount_dec ,
> 
>  3) Has its C code in a ##c-code and not a ##c-lambda
> 
>  4) Has its surrounding Scheme code in (##declare (not interrupts-enabled))
> 
>  5) Surrounds the ___alloc_scmobj call by some magic (
> ___FRAME_STORE_RA(___R0)___W_ALL ___R_ALL ___SET_R0(___FRAME_FETCH_RA) )
> 
> (##make-u8vector provides a code path for small objects apart from this,
> though the use regarded here should be for u8vectors of nontrivial size so
> going with the codepath for large objects should always be fine.)
> 
> 
> By applying 1) and 2) I get something that works but can't know if it's
> really correct and safe:
> 
> 
> How do you allocate an u8vector in C code in a way that does not leak
> memory, and that safely transfers the new object reference from the C world
> to the Scheme world?
> 
> 
> 
> Thanks!
> Mikael
> 
> 
> *How to reproduce*
> Running a test that repeats
> 
> ((c-lambda () scheme-object "___result =
> ___EXT(___alloc_scmobj)(___sU8VECTOR,10000,___MOVABLE0);"))
> 
> 
> over and over e.g.
> 
> (##gc-report-set! #t)
> (for-each (lambda (x) ((c-lambda () scheme-object "___result =
> ___EXT(___alloc_scmobj)(___sU8VECTOR,10000,___MOVABLE0);")))
>           (string->list (make-string 100000))
> (##gc)
> 
> 
> shows that this memory leaks all the allocated memory (!).
> 
> 
> *Fix, duno if safe & complete*
> 
> (##gc-report-set! #t)
> (for-each (lambda (x) ((c-lambda () scheme-object "___result =
> ___EXT(___alloc_scmobj)(___sU8VECTOR,10000,___STILL);
>                                                    if
> (!___FIXNUMP(___result)) ___still_obj_refcount_dec (___result);
>                                                    ")))
>           (string->list (make-string 100000))
> (##gc)
> 
> 
> 
> *Copy of ##make-u8vector of _kernel.scm for reference*
> 
> (define-prim (##make-u8vector k #!optional (fill (macro-absent-obj)))
>   (##declare (not interrupts-enabled))
>   (let ((v (##c-code #<<end-of-code
> 
> long i;
> long n = ___INT(___ARG1);
> long words = ___WORDS(n) + 1;
> ___SCMOBJ result;
> if (n > ___CAST(___WORD, ___LMASK>>___LF))
>   result = ___FIX(___HEAP_OVERFLOW_ERR); /* requested object is too big! */
> else if (words > ___MSECTION_BIGGEST)
>   {
>     ___FRAME_STORE_RA(___R0)
>     ___W_ALL
>     result = ___alloc_scmobj (___sU8VECTOR, n, ___STILL);
>     ___R_ALL
>     ___SET_R0(___FRAME_FETCH_RA)
>     if (!___FIXNUMP(result))
>       ___still_obj_refcount_dec (result);
>   }
> else
>   {
>     ___BOOL overflow = 0;
>     ___hp += words;
>     if (___hp > ___ps->heap_limit)
>       {
>         ___FRAME_STORE_RA(___R0)
>         ___W_ALL
>         overflow = ___heap_limit () && ___garbage_collect (0);
>         ___R_ALL
>         ___SET_R0(___FRAME_FETCH_RA)
>       }
>     else
>       ___hp -= words;
>     if (overflow)
>       result = ___FIX(___HEAP_OVERFLOW_ERR);
>     else
>       {
>         result = ___TAG(___hp, ___tSUBTYPED);
>         ___HEADER(result) = ___MAKE_HD_BYTES(n, ___sU8VECTOR);
>         ___hp += words;
>       }
>   }
> if (!___FIXNUMP(result) && ___ARG2 != ___ABSENT)
>   {
>     for (i=0; i<n; i++)
>       ___U8VECTORSET(result,___FIX(i),___ARG2)
>   }
> ___RESULT = result;
> 
> end-of-code
> 
>             k
>             fill)))
>     (if (##fixnum? v)
>       (begin
>         (##raise-heap-overflow-exception)
>         (##make-u8vector k fill))
>       v)))
> _______________________________________________
> Gambit-list mailing list
> Gambit-list at iro.umontreal.ca
> https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list




More information about the Gambit-list mailing list