It's not true that the experiment below[1] runs valgrind-clean. I must have forgotten to compile with -g or something. Calling a `c-define`d function from a release functions gives me this:
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-December/007346....
Anyway, I worked out a proof of concept where the 'table' is in C:
https://github.com/euccastro/refcount-hacks
I think this will do the trick.
[1] (I sent the reply below only to Mikael, by mistake.)
---------- Forwarded message ---------- From: Estevo euccastro@gmail.com Date: Thu, Dec 26, 2013 at 2:53 PM Subject: Re: [gambit-list] Anyone else working in a schemey wrapper for c structs/unions/types? To: Mikael mikael.rcv@gmail.com
https://github.com/feeley/gambit/pull/61 - for a quick proof of concept,
that's a very neat use of the ffi type!!!
Thanks! I tried the approach you suggested and the code is much simpler, at least for this particular purpose. There are some dangling questions left, that I explain below in more detail.
So you tried to somehow use wills to force GC order? Without having studied
this, I guess that was not what they were intended for in the first place and it makes good sense that that did not work out well.
I completely agree.
Once user code gets hold of A, pure Scheme code can't really know when it's OK to let go of that vector. It's the same problem. Of course you can keep a reference forever, but you don't want to hold on to unneeded memory either.
You should be able to work that out and question is just how heavy it would be implementation-wise to do, no??
If not why not?
There is no way in pure Scheme to detect when an object has been reclaimed. This follows from the unlimited extent semantics mandated in the Scheme Reports. From the perspective of Scheme code, objects are never deleted. Gambit doesn't violate this, and that is fine.
But these restrictions need not be showstoppers. A scheme that might work:
- on initalization, create an ___alloc_rc'ed table[1] and assign a
global C pointer to it
- whenever you create a dependent foreign pointer F, store in the table
a reference to its dependent object D, keyed by the address of the F's C pointer, and
- set a release function for F that will clear that entry in the table.
Why do you need a "table" (by table here you mean resizable vector)?
No, I mean the type of object that (make-table) returns. I need - to keep strong references to the original object in some place that the garbage collector will scan, - to be able to retrieve these references by key (in particular, by address of the C object to which they point), in order to remove them when the dependent pointer is finalized.
Nice!
Gambit's builtin functionality for this is fully sufficient at Gambit's level of abstraction over the underlying system, and indeed there's plenty of space for higher-level abstractions that more in this area.
Indeed! I should add that I don't want to hide Gambit's standard ffi facilities. I want to hide my own hacks so what the user gets works seamlessly with those facilities, should they need more control than what my library provides. That's why I want to give the user bare foreign objects and no scheme wrappers over them.
So the totality of what you are looking to provide is
a) the c-struct form which is analogous to define-type with
b) Scheme-like automatic dependency GC-reference handling between c structure instances as discussed here
?
Yes, at least in a first step. After that is working I'm thinking of doing something similar for arrays of structs/unions/types, perhaps with a vector-like interface. That is, if you allocate an array of structures the array will stick around for exactly as long as you hold a pointer to any of its members.
Here's a proof of concept that runs valgrind-clean and prints the expected output in an unmodified Gambit:
(c-declare "typedef struct { int x; int y; } point;")
(define no-err ((c-lambda () scheme-object "___result = ___FIX(___NO_ERR);")))
(define dependencies (make-table))
(c-define (release-dependent p) ((pointer void)) scheme-object "release_dependent" "" (if p (begin (println "resetting " (foreign-address p)) (table-set! dependencies (foreign-address p))) (println "Null pointer?")) no-err)
(c-define-type point (type "point" (point point*))) (c-define-type dependent-point (type "point" (point* point) "release_dependent"))
(define (make-dependent-point dependency) (let ((p ((c-lambda () dependent-point "___result_voidstar = ___EXT(___alloc_rc)(sizeof(point)); /* I would really set these using accessors. */ ((point*)___result_voidstar)->x = 1; ((point*)___result_voidstar)->y = 2;")))) (println "setting " (foreign-address p)) (table-set! dependencies (foreign-address p) dependency) p))
(define point-x (c-lambda (point) int "___result = ((point*)___arg1_voidstar)->x;")) (define point-y (c-lambda (point) int "___result = ((point*)___arg1_voidstar)->y;"))
(define (point->string p) (string-append "(point " (number->string (point-x p))
" " (number->string (point-y p)) ")"))
(define d (cons 'depended 'upon)) (make-will d (lambda (x) (write x) (println " checking out"))) (define p (make-dependent-point d)) (println "p is " (point->string p)) (println "forgetting global d; its will should not get executed yet") (set! d #f) (##gc) (##gc) (##gc) (println "letting go of p; this should kill d in two garbage collections at most") (set! p #f) (println "first garbage collection") ; The first collection triggers the finalization of p, which deletes it from ; the table. But by the time that happens, the table has already been ; scanned, and d has been found there. (##gc) (println "second garbage collection") ; In the second collection d will not be reached anymore, so it's reclaimed. (##gc) (println "done.")
This is not the end of the story, though. Some consideration must be added for transitive dependencies and for several foreign objects depending on the same root. I don't think there will be any blockers here. I'll give it a go this afternoon, and if I get something solid I'll withdraw my pull request.
Thanks again for bouncing ideas!