I'm trying to write a macro to wrap C structures and unions [1] in an interface similar to that of Scheme records. It defines an appropriate foreign type, creates a constructor, utility functions to take and dereference pointers, and member accessors and mutators.
Here is the definition [2]:
https://github.com/euccastro/gambit-SDL2/blob/master/ffi.scm
and here is a sample of how you are supposed to use it:
https://github.com/euccastro/gambit-SDL2/blob/master/test-macro-struct.scm
So far I've been letting Gambit take care of cleaning up structs, except for those that were obtained via a non-pointer accessor to an existing struct. So disposing of your last Scheme reference to a "child" struct will not try to clean up memory in the "parent" struct. I did this in an unclean way (I'm wrapping each struct twice, with different Scheme names, one of them with a bogus cleanup function; I'm also assuming that every "voidstar" member is a struct or union that has been wrapped using this macro) and I'd be thankful for any suggestions on how to improve this, but this is not my main question.
I still have the somewhat converse problem: when Gambit loses sight of your last Scheme reference to a "parent" struct, it will clean it up even if there are Scheme references to "child" structs.
I would like to somehow make a Scheme reference to the "parent" struct from the "child" structs, so that children will keep the parent alive as long as they are around. This reference needs not be visible from Scheme; it's only to help the garbage collector do the right thing.
I can think of a couple ways to do this, but they either complicate my macro considerably, or make things more cumbersome for the user -- or both. I'm asking the list just in case I'm missing an easy way to create these links.
[1] I say "structs" in the rest of this message to refer to both C `struct` and `union`s.
[2] All my code in github is under the WTFPL license (profanity warning):
Afficher les réponses par date
On 2012-03-24, at 12:31 PM, Estevo wrote:
I still have the somewhat converse problem: when Gambit loses sight of your last Scheme reference to a "parent" struct, it will clean it up even if there are Scheme references to "child" structs.
I would like to somehow make a Scheme reference to the "parent" struct from the "child" structs, so that children will keep the parent alive as long as they are around. This reference needs not be visible from Scheme; it's only to help the garbage collector do the right thing.
I can think of a couple ways to do this, but they either complicate my macro considerably, or make things more cumbersome for the user -- or both. I'm asking the list just in case I'm missing an easy way to create these links.
I haven't looked at the details, but wouldn't a doubly-linked tree work? In other words, the parent has a list of children, and each child has a "parent" pointer. Then, whichever node of the tree is being referenced, all of the nodes of the tree will be maintained by the GC.
Marc
On 2012-03-24, at 12:31 PM, Estevo wrote:
I still have the somewhat converse problem: when Gambit loses sight of your last Scheme reference to a "parent" struct, it will clean it up even if there are Scheme references to "child" structs.
I would like to somehow make a Scheme reference to the "parent" struct from the "child" structs, so that children will keep the parent alive as long as they are around. This reference needs not be visible from Scheme; it's only to help the garbage collector do the right thing.
I can think of a couple ways to do this, but they either complicate my macro considerably, or make things more cumbersome for the user -- or both. I'm asking the list just in case I'm missing an easy way to create these links.
I haven't looked at the details, but wouldn't a doubly-linked tree work? In other words, the parent has a list of children, and each child has a "parent" pointer. Then, whichever node of the tree is being referenced, all of the nodes of the tree will be maintained by the GC.
Thanks for your quick reply!
That wouldn't remove the need for children never to try and deallocate memory. If I implement a tree but don't prevent children from deallocating their memory, I'll have multiple attempts to free the same memory (and perhaps some attempts to free the addresses of child structs) when the whole tree becomes invisible to Scheme.
And since children won't try to free memory when they go away, there is no point in keeping them alive artificially.
What I'm worried about is not the topology of the graph of references, but what would be the simplest way to implement a single edge, that is, to create a Scheme reference, preferrably without exposing it to the Scheme world. So far the scheme representations of structs and unions are very thin wrappers on the C data, and I didn't even need conversion code (that is, __BEGIN_CFUN_*** and friends). I realise I'll have to change that if I want children keeping the parent alive. I just didn't want to add any more structure than necessary.
My plan, when I asked that question, was to wrap unmanaged objects in a C structure of the form
typedef struct {___SCMOBJ* parent; <actual_struct_type> actual_struct;} <actual_struct_wrapper>;
and add ___BEGIN_(C|S)FUN_*** (etc.) handlers that create the Scheme reference to the parent. This reference gets deleted in the child's finalizer.
But that looked scary to me. My current idea is to just map the C struct to a Scheme record with fields 'parent' and 'c-data'. I'll use conversion to treat the record (and not the c-data field alone) as the Scheme representation of the object, so people won't accidentally hold onto the c-data and lose the record and therefore the reference to the parent. So the struct finalizer does nothing again; the parent reference gets cleaned up by normal garbage collection of the Scheme record.
If there is nothing very obvious that I'm missing you can safely ignore this question. The latter option seems good enough to me. I wouldn't have bothered you with this if I had thought of that one in first place.
Thank you for your time,
Estevo.
On 2012-03-24, at 4:21 PM, Estevo wrote:
My plan, when I asked that question, was to wrap unmanaged objects in a C structure of the form
typedef struct {___SCMOBJ* parent; <actual_struct_type> actual_struct;} <actual_struct_wrapper>;
and add ___BEGIN_(C|S)FUN_*** (etc.) handlers that create the Scheme reference to the parent. This reference gets deleted in the child's finalizer.
But that looked scary to me. My current idea is to just map the C struct to a Scheme record with fields 'parent' and 'c-data'. I'll use conversion to treat the record (and not the c-data field alone) as the Scheme representation of the object, so people won't accidentally hold onto the c-data and lose the record and therefore the reference to the parent. So the struct finalizer does nothing again; the parent reference gets cleaned up by normal garbage collection of the Scheme record.
If there is nothing very obvious that I'm missing you can safely ignore this question. The latter option seems good enough to me. I wouldn't have bothered you with this if I had thought of that one in first place.
This reminds me that Gambit provides a memory allocation function, ___alloc_rc, for allocating C objects on the C heap which have a back-pointer to a Scheme object, and also a reference count. The API is:
void *___alloc_rc(unsigned long n); void ___release_rc(void *ptr); void ___addref_rc(void *ptr); void ___set_data_rc(void *ptr, ___SCMOBJ val); ___SCMOBJ ___data_rc(void *ptr);
___alloc_rc(n) allocates from the C heap a reference counted block of memory which is able to contain n bytes and returns a pointer to the first byte of that block. The block of memory also contains a slot of type ___SCMOBJ, the Scheme "data". The reference count is initially 1 and the data is #f. In terms of implementation, the data slot is stored immediately before the first byte of the block.
___release_rc(ptr) decrements the reference count and reclaims the block of memory when the reference count reaches 0. So ___alloc_rc and ___release_rc are drop-in replacements for malloc and free (but you must not mix ___alloc_rc with free!).
___addref_rc(ptr) increments the reference count.
___set_data_rc(ptr, val) sets the data slot to val. As long as the reference count is positive, the GC will consider the data slot to be a root (in other words the data will remain live and will not be reclaimed by the GC).
___data_rc(ptr) returns the data slot.
The C interface uses ___alloc_rc for all of the C structures that it allocates. In particular, when a Scheme string is passed to a C function, the Scheme string is copied to a block of memory allocated with ___alloc_rc. After returning from the C function, the C interface will execute ___release_rc on the pointer to the C string. Normally this will reclaim the C string. However, the C function can call ___addref_rc to prevent this ___release_rc from deallocating the C string. A subsequent call to ___release_rc (somewhere else in the logic of the program) will reclaim the C string.
I haven't analyzed your problem in detail, but perhaps the "data" pointer in these reference counted objects can solve your problem.
Marc
This reminds me that Gambit provides a memory allocation function, ___alloc_rc,
for allocating C objects on the C heap which have a back-pointer to a Scheme object, and also a reference count. The API is:
void *___alloc_rc(unsigned long n); void ___release_rc(void *ptr); void ___addref_rc(void *ptr); void ___set_data_rc(void *ptr, ___SCMOBJ val); ___SCMOBJ ___data_rc(void *ptr);
This might be just what I need, thanks! Paradoxically enough, I was allocating my structs with ___alloc_rc, but I don't think I really understood how that works.
I'll ping the list when I have something that I believe is solid, and/or I'll put it in the dumping grounds, just in case it might be of use for anyone else.
Thanks again,
Estevo.
Estevo, if this delivered for you, please remember to document this mechanism (___alloc_rc and the others) in the Wiki so it'll be clear also for others. Mikael
Den 25 mars 2012 04:02 skrev Estevo euccastro@yahoo.com:
This reminds me that Gambit provides a memory allocation function,
___alloc_rc,
for allocating C objects on the C heap which have a back-pointer to a
Scheme
object, and also a reference count. The API is:
void *___alloc_rc(unsigned long n); void ___release_rc(void *ptr); void ___addref_rc(void *ptr); void ___set_data_rc(void *ptr, ___SCMOBJ val); ___SCMOBJ ___data_rc(void *ptr);
This might be just what I need, thanks! Paradoxically enough, I was allocating my structs with ___alloc_rc, but I don't think I really understood how that works.
I'll ping the list when I have something that I believe is solid, and/or I'll put it in the dumping grounds, just in case it might be of use for anyone else.
Thanks again,
Estevo.
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
I'll make sure to, as soon as I'm confident I'm doing it right.
De: Mikael mikael.rcv@gmail.com Para: gambit-list@iro.umontreal.ca; euccastro@yahoo.com Enviadas: Segunda-feira, 26 de Março de 2012 6:03 Assunto: Re: [gambit-list] References between 'foreign' objects.
Estevo, if this delivered for you, please remember to document this mechanism (___alloc_rc and the others) in the Wiki so it'll be clear also for others. Mikael
Den 25 mars 2012 04:02 skrev Estevo euccastro@yahoo.com:
This reminds me that Gambit provides a memory allocation function, ___alloc_rc,
for allocating C objects on the C heap which have a back-pointer to a Scheme object, and also a reference count. The API is:
void *___alloc_rc(unsigned long n); void ___release_rc(void *ptr); void ___addref_rc(void *ptr); void ___set_data_rc(void *ptr, ___SCMOBJ val); ___SCMOBJ ___data_rc(void *ptr);
This might be just what I need, thanks! Paradoxically enough, I was allocating my structs with ___alloc_rc, but I don't think I really understood how that works.
I'll ping the list when I have something that I believe is solid, and/or I'll put it in the dumping grounds, just in case it might be of use for anyone else.
Thanks again,
Estevo.
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
I think I have finally built a mental model of how foreign objects work. In order to confirm or correct this, I'll illustrate my current beliefs with an example.
Let's say I have these C structures somewhere visible to my Scheme code via c-declare:
typedef struct {int i, j;} child; typedef struct {child c; int k;} parent;
And I wrap them like this:
(c-define-type child (struct "child" child "do_nothing"))
(c-define-type parent (struct "parent"))
(define make-parent (c-lambda () parent
"___result_voidstar = ___EXT(___alloc_rc)(sizeof(parent));")) ; [1]
Where the dummy `release-function` `do_nothing` is `c-declare`d:
___SCMOBJ do_nothing(void *p) { return ___FIX(___NO_ERR);
}
Then I write an accessor:
(define (parent-c p) (let ((ret ((c-lambda (parent) child "___result_voidstar = &(((parent*)___arg1_voidstar)->c);")))) ((c-lambda (child scheme-object) void "___EXT(___set_data_rc)(___arg1_voidstar, ___arg2);") ; [2]
ret parent)
ret)
Let's say I have the above correctly compiled and say:
(define par (make-parent)) (define ch1 (parent-c p)) (define ch2 (parent-c p))
At this moment, I believe all the following sentences to be true:
(i) `par` is a Scheme foreign object that holds a pointer to the very memory I allocated in the line marked as [1], not to a copy of it. (ii) The pointer referred to in (i) is managed by Gambit, via reference counting (Gambit maintains some bookkeeping data in the address right before the pointer). At this moment, its reference count is 1.
(iii) `ch1` is a Scheme foreign object that holds a pointer to the very position of the `c` member in the struct `par`, not to a copy of it. As it happens, it points to the same memory as `par` itself. (iv) the pointer mentioned in (iii) is, too, managed by Gambit. Its reference count is, too, 1. (v) additionally, the `data` field of the 'managed pointer' mentioned in (iii) holds, by virtue of [2], a root reference to `par` (the meaning of this will be made more explicit later). (vi) I haven't accessed memory in an illegal or undefined way in doing (v).
(vii) (iii) to (vi) hold for `ch2` too.
Now I forget ch2:
(set! ch2 #f)
At this moment, I believe:
(vii) `ch2` will be disposed of in the next garbage collection. (viii) the pointer wrapped by `ch2` will get a reference count of 0 and thus be disposed of at next opportunity.
(ix) Gambit will not try and free any memory allocated in [1] when either of the above happens, because `ch2` has a bogus `release-function`.
Now I remove my only Scheme-visible reference to `par`:
(set! par #f)
At this moment, I believe:
(x) `par` would not be garbage collected now. The `data` reference from the `ch1` pointer is keeping it alive. (xi) the pointer wrapped by `par` will keep its reference count of 1.
Now I forget ch1:
(set! ch1 #f)
At this moment, I believe:
(xii) (vii) and (viii) apply for `ch1` now. (xiii) nobody is holding a reference to `par` anymore, so `par` will be garbage collected at next opportunity. (xiv) when the above happens, the pointer referred to in (i) will be left with a reference count of 0, and thus it will be disposed of soon. (xv) `par`s default `release-function` will free the memory allocated at [1].
My current implementation of the struct/union wrapping macro relies on these beliefs. I have tested some of these, but I haven't yet figured out how to confirm or falsify others (e.g. how do you check whether a memory block has been deallocated?). Anyway, if you know any of these to be false, I'd be grateful to hear from you.
Thanks in advance,
Estevo.
De: Estevo euccastro@yahoo.com Para: Marc Feeley feeley@iro.umontreal.ca Cc: "gambit-list@iro.umontreal.ca" gambit-list@iro.umontreal.ca Enviadas: Sábado, 24 de Março de 2012 21:02 Assunto: Re: [gambit-list] References between 'foreign' objects.
This reminds me that Gambit provides a memory allocation function, ___alloc_rc,
for allocating C objects on the C heap which have a back-pointer to a Scheme object, and also a reference count. The API is:
void *___alloc_rc(unsigned long n); void ___release_rc(void *ptr); void ___addref_rc(void *ptr); void ___set_data_rc(void *ptr, ___SCMOBJ val); ___SCMOBJ ___data_rc(void *ptr);
This might be just what I need, thanks! Paradoxically enough, I was allocating my structs with ___alloc_rc, but I don't think I really understood how that works.
I'll ping the list when I have something that I believe is solid, and/or I'll put it in the dumping grounds, just in case it might be of use for anyone else.
Thanks again,
Estevo.
Correction:
Then I write an accessor:
(define (parent-c p) (let ((ret ((c-lambda (parent) child "___result_voidstar = &(((parent*)___arg1_voidstar)->c);")))) ((c-lambda (child scheme-object) void "___EXT(___set_data_rc)(___arg1_voidstar, ___arg2);") ; [2]
ret parent)
ret)
In this example I forgot to pass `p` to the first c-lambda and I passed `parent` to the second one instead. The corrected version would be:
(define (parent-c p) (let ((ret ((c-lambda (parent) child "___result_voidstar = &(((parent*)___arg1_voidstar)->c);") p))) ((c-lambda (child scheme-object) void "___EXT(___set_data_rc)(___arg1_voidstar, ___arg2);") ; [2] ret p) ret)