Joel J. Adamson adamsonj@email.unc.edu wrote:
I have used (generate-proper-tail-calls #f): is there more to it than that? Where can I read about it?
It's in the Gambit manual. You should just know that it does only have any effect in the interpreter and only for subsequently loaded code and that if you switch it to #f you almost *certainly* will leak memory but it can make it easier to see what a program is doing (especially when getting an error, since you can see the full call chain).
BTW if it's really the kernel killing your program (aka OOM killer, and not just Gambit unable to get more memory), then limit the virtual memory to a low enough value (ulimit -v). In many circumstances Gambit will just throw an out of memory exception then; if you're starting your program so that it runs the repl ("enters the debugger") when getting uncaught exceptions, you'll then be right at the spot where it has allocated too much memory.
chj> - find out what the reason of the out of memory situation could chj> be: is (a) your problem maybe just asking for more memory than chj> you've got?, or (a2) your problem doesn't necessarily ask for chj> more memory than you've got in principle, but the way you're chj> evaluating it (e.g. calculate to eagerly) requires too much chj> memory at once, or (b) you've got an error in the program which chj> leads to an infinite loop allocating memory, or (c) your chj> program is holding on to memory that it doesn't need anymore?
I would find (c) to be the most plausible choice; can I rule out the other two by tweaking my parameters? If I run fewer iterations, I don't have the problem and the program completes successfully --- that means if I create fewer structs, 1000 versus 2000 --- I don't have the problem.
Well, you could check whether the amount of needed memory is going linearly or quadratic with the input value, for example.
(a)/(a2) are a question of understanding the algorithm.
chj> For (c) check your assumptions about the lifetime of memory: chj> particularly, be aware that structs (as defined using chj> define-structure or define-type), or in fact any data structure chj> like cons cells and vectors, will hold on to every location in chj> them
Okay, so am I screwed here? Just kidding: if I bind them lexically, I would need to get the data out of them before exiting that closure (tail-calling my "data-collection" function) --- I can do that.
chj> (this is unlike lexical bindings, which, as I've told above, chj> will be analyzed by the compiler and only live as long as the chj> program will possibly refer to them).
Just to make sure I'm understanding: if I do everything with a particular data structure within a lexical closure, then I can use whichever sort of data-structure I want (struct, list, vector, etc).
Hm. This sentence isn't precise enough to say whether it's right.
What can be said is that if you do not group your values artificially (by using vectors/structures/lists) but keep and pass them as individual items in individual lexical bindings, then the compiler will help avoid memory retention issues because those items which the compiler sees that they won't be accessed anymore will be released as soon as possible (i.e. the generated code will not keep a reference to them), whereas if you group them yourself, you are also responsible for yourself to decide when you don't need all values from a group anymore and thus need to create a copy of only those values of the group that you still need.
As an example,
(define-structure female mating-status mated not-mated strategy age times-mated)
(define (do-struct i) (let ((struct (make-female #f '() '() (random-strategy) 1 0))) (do ((j 0 (+ 1 j))) ((= j 5) (print (female-age struct) "\n")) (female-age-set struct (+ j (female-age female))))))
This `female' structure is going to get gc'ed when do-struct exits?
Well, the above code is not a complete program (random-strategy, female-age-set, and female are all missing).
I guess you meant to use female-age-set!.
I'm not sure I can see the point you are trying to make.
If the female structure is not being referenced anymore it will be gc'd, of course. The retention problem is one where you are still holding a reference to a structure, because you are interested in one (or more) of the values in it, but not in other values, and those other values won't be gc'd.
Christian.