[gambit-list] Fwd: Anyone else working in a schemey wrapper for c structs/unions/types?

Estevo euccastro at gmail.com
Thu Dec 26 23:27:25 EST 2013


It's not true that the experiment below[1] runs valgrind-clean.  I must
have forgotten to compile with -g or something.  Calling a `c-define`d
function from a release functions gives me this:

https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-December/007346.html

Anyway, I worked out a proof of concept where the 'table' is in C:

https://github.com/euccastro/refcount-hacks

I think this will do the trick.

[1] (I sent the reply below only to Mikael, by mistake.)

---------- Forwarded message ----------
From: Estevo <euccastro at gmail.com>
Date: Thu, Dec 26, 2013 at 2:53 PM
Subject: Re: [gambit-list] Anyone else working in a schemey wrapper for c
structs/unions/types?
To: Mikael <mikael.rcv at gmail.com>



https://github.com/feeley/gambit/pull/61 - for a quick proof of concept,
> that's a very neat use of the ffi type!!!
>

Thanks!  I tried the approach you suggested and the code is much simpler,
at least for this particular purpose.  There are some dangling questions
left, that I explain below in more detail.


So you tried to somehow use wills to force GC order? Without having studied
> this, I guess that was not what they were intended for in the first place
> and it makes good sense that that did not work out well.
>

I completely agree.

>
>> Once user code gets hold of A, pure Scheme code can't really know when
>> it's OK to let go of that vector.  It's the same problem.  Of course you
>> can keep a reference forever, but you don't want to hold on to unneeded
>> memory either.
>>
>
> You should be able to work that out and question is just how heavy it
> would be implementation-wise to do, no??
>
> If not why not?
>

There is no way in pure Scheme to detect when an object has been
reclaimed.  This follows from the unlimited extent semantics mandated in
the Scheme Reports.  From the perspective of Scheme code, objects are never
deleted.  Gambit doesn't violate this, and that is fine.


> But these restrictions need not be showstoppers.  A scheme that might work:
>>  - on initalization, create an ___alloc_rc'ed table[1] and assign a
>> global C pointer to it
>>  - whenever you create a dependent foreign pointer F, store in the table
>> a reference to its dependent object D, keyed by the address of the F's C
>> pointer, and
>>  - set a release function for F that will clear that entry in the table.
>>
>
> Why do you need a "table" (by table here you mean resizable vector)?
>

No, I mean the type of object that (make-table) returns.  I need
 - to keep strong references to the original object in some place that the
garbage collector will scan,
 - to be able to retrieve these references by key (in particular, by
address of the C object to which they point), in order to remove them when
the dependent pointer is finalized.


> Nice!
>
> Gambit's builtin functionality for this is fully sufficient at Gambit's
> level of abstraction over the underlying system, and indeed there's plenty
> of space for higher-level abstractions that more in this area.
>

Indeed!  I should add that I don't want to hide Gambit's standard ffi
facilities.  I want to hide my own hacks so what the user gets works
seamlessly with those facilities, should they need more control than what
my library provides.  That's why I want to give the user bare foreign
objects and no scheme wrappers over them.


> So the totality of what you are looking to provide is
>
>  a) the c-struct form which is analogous to define-type with
>
>  b) Scheme-like automatic dependency GC-reference handling between c
> structure instances as discussed here
>
> ?
>

Yes, at least in a first step.  After that is working I'm thinking of doing
something similar for arrays of structs/unions/types, perhaps with a
vector-like interface.  That is, if you allocate an array of structures the
array will stick around for exactly as long as you hold a pointer to any of
its members.

Here's a proof of concept that runs valgrind-clean and prints the expected
output in an unmodified Gambit:

    (c-declare "typedef struct { int x; int y; } point;")

    (define no-err ((c-lambda () scheme-object "___result =
___FIX(___NO_ERR);")))

    (define dependencies (make-table))

    (c-define (release-dependent p) ((pointer void)) scheme-object
"release_dependent" ""
      (if p
        (begin
          (println "resetting " (foreign-address p))
          (table-set! dependencies (foreign-address p)))
        (println "Null pointer?"))
      no-err)

    (c-define-type point (type "point" (point point*)))
    (c-define-type dependent-point (type "point" (point* point)
"release_dependent"))

    (define (make-dependent-point dependency)
      (let ((p
              ((c-lambda ()
                         dependent-point
                     "___result_voidstar =
___EXT(___alloc_rc)(sizeof(point));
                     /* I would really set these using accessors. */
                     ((point*)___result_voidstar)->x = 1;
                     ((point*)___result_voidstar)->y = 2;"))))
        (println "setting " (foreign-address p))
        (table-set! dependencies (foreign-address p) dependency)
        p))

    (define point-x
      (c-lambda (point)
                int
        "___result = ((point*)___arg1_voidstar)->x;"))
    (define point-y
      (c-lambda (point)
                int
        "___result = ((point*)___arg1_voidstar)->y;"))


    (define (point->string p)
      (string-append
        "(point "
        (number->string (point-x p))

        " "
        (number->string (point-y p))
        ")"))

    (define d (cons 'depended 'upon))
    (make-will d (lambda (x) (write x) (println " checking out")))
    (define p (make-dependent-point d))
    (println "p is " (point->string p))
    (println "forgetting global d; its will should not get executed yet")
    (set! d #f)
    (##gc)
    (##gc)
    (##gc)
    (println "letting go of p; this should kill d in two garbage
collections at most")
    (set! p #f)
    (println "first garbage collection")
    ; The first collection triggers the finalization of p, which deletes it
from
    ; the table.  But by the time that happens, the table has already been
    ; scanned, and d has been found there.
    (##gc)
    (println "second garbage collection")
    ; In the second collection d will not be reached anymore, so it's
reclaimed.
    (##gc)
    (println "done.")

This is not the end of the story, though.  Some consideration must be added
for transitive dependencies and for several foreign objects depending on
the same root.  I don't think there will be any blockers here.  I'll give
it a go this afternoon, and if I get something solid I'll withdraw my pull
request.

Thanks again for bouncing ideas!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20131227/00818167/attachment.htm>


More information about the Gambit-list mailing list