[gambit-list] Help With Memory

Christian Jaeger christian at pflanze.mine.nu
Wed Sep 24 14:43:59 EDT 2008


Joel J. Adamson <adamsonj at email.unc.edu> wrote:
> I have used (generate-proper-tail-calls #f): is there more to it than
> that?  Where can I read about it?
>   

It's in the Gambit manual. You should just know that it does only have 
any effect in the interpreter and only for subsequently loaded code and 
that if you switch it to #f you almost *certainly* will leak memory but 
it can make it easier to see what a program is doing (especially when 
getting an error, since you can see the full call chain).

BTW if it's really the kernel killing your program (aka OOM killer, and 
not just Gambit unable to get more memory), then limit the virtual 
memory to a low enough value (ulimit -v). In many circumstances Gambit 
will just throw an out of memory exception then; if you're starting your 
program so that it runs the repl ("enters the debugger") when getting 
uncaught exceptions, you'll then be right at the spot where it has 
allocated too much memory.

>     chj> - find out what the reason of the out of memory situation could
>     chj> be: is (a) your problem maybe just asking for more memory than
>     chj> you've got?, or (a2) your problem doesn't necessarily ask for
>     chj> more memory than you've got in principle, but the way you're
>     chj> evaluating it (e.g. calculate to eagerly) requires too much
>     chj> memory at once, or (b) you've got an error in the program which
>     chj> leads to an infinite loop allocating memory, or (c) your
>     chj> program is holding on to memory that it doesn't need anymore?
>
> I would find (c) to be the most plausible choice; can I rule out the
> other two by tweaking my parameters?  If I run fewer iterations, I don't
> have the problem and the program completes successfully --- that means
> if I create fewer structs, 1000 versus 2000 --- I don't have the problem.
>   

Well, you could check whether the amount of needed memory is going 
linearly or quadratic with the input value, for example.

(a)/(a2) are a question of understanding the algorithm.

>     chj> For (c) check your assumptions about the lifetime of memory:
>     chj> particularly, be aware that structs (as defined using
>     chj> define-structure or define-type), or in fact any data structure
>     chj> like cons cells and vectors, will hold on to every location in
>     chj> them
>
> Okay, so am I screwed here?  Just kidding: if I bind them lexically, I
> would need to get the data out of them before exiting that closure
> (tail-calling my "data-collection" function) --- I can do that.
>
>     chj> (this is unlike lexical bindings, which, as I've told above,
>     chj> will be analyzed by the compiler and only live as long as the
>     chj> program will possibly refer to them). 
>
> Just to make sure I'm understanding: if I do everything with a
> particular data structure within a lexical closure, then I can use
> whichever sort of data-structure I want (struct, list, vector, etc).
>   

Hm. This sentence isn't precise enough to say whether it's right.

What can be said is that if you do not group your values artificially 
(by using vectors/structures/lists) but keep and pass them as individual 
items in individual lexical bindings, then the compiler will help avoid 
memory retention issues because those items which the compiler sees that 
they won't be accessed anymore will be released as soon as possible 
(i.e. the generated code will not keep a reference to them), whereas if 
you group them yourself, you are also responsible for yourself to decide 
when you don't need all values from a group anymore and thus need to 
create a copy of only those values of the group that you still need.

> As an example,
>
> (define-structure female
>   mating-status
>   mated
>   not-mated
>   strategy
>   age
>   times-mated)
>
>
> (define (do-struct i)
> 	(let ((struct (make-female
> 	     #f
> 	    '()
> 	    '()
> 	     (random-strategy) 1 0)))
> 	     (do ((j 0 (+ 1 j)))
> 	         ((= j 5) (print (female-age struct) "\n"))
> 		 (female-age-set struct (+ j (female-age female))))))
>
> This `female' structure is going to get gc'ed when do-struct exits?
>   

Well, the above code is not a complete program (random-strategy, 
female-age-set, and female are all missing).

I guess you meant to use female-age-set!.

I'm not sure I can see the point you are trying to make.

If the female structure is not being referenced anymore it will be gc'd, 
of course. The retention problem is one where you are still holding a 
reference to a structure, because you are interested in one (or more) of 
the values in it, but not in other values, and those other values won't 
be gc'd.

Christian.




More information about the Gambit-list mailing list