[gambit-list] Re: Regarding garbage collection

peter lo peter19852001 at yahoo.com.hk
Wed Sep 9 08:51:24 EDT 2009


Dear all,
    Thanks for the replies and clarifications. After some investigations, I believe that the cause of heap overflow is not memory leakage. The whole program is in Scheme, so it cannot be leakage due to external C libraries. And I have checked the program, it seems that there are no "really useless" data hanging around that cannot be garbage collected. The real reason is simply that the input data size is too large, so there are too many intermediate results, therefore needing > 1.6 GB of live memory.

    It was my fault, as I do not expect the program to need that much memory, so the representation of the intermediate result is not particularly compact. Previously I used a structure with 6 members, but in fact I really need 2 of them to do the computations. After I have changed the representation to use a simple cons cell instead, the program seem to manage to continue to run using only a peak amount of 1700MB ram.

    Another thing that I have noticed is that the system holds at most ~1700MB ram for heap, even though 100% of them are live objects, is this by design or just an accident in Windows XP? When the precentage of live objects gets closer to 100%, the GC's become more frequent as each time there is little memory reclaimed, and each one takes around 2 seconds, because a lot of memory objects are examined to determine the reachability, therefore the system is doing less useful work.  Now I am trying to reduce the allocation of short-lived data, so that there will be less GC's. I am also considering switching to a linux system.

Anyway, thanks for the help and sorry for bothering you.

Thanks.
Peter




----- 郵件原件 ----
寄件人﹕ Marc Feeley <feeley at iro.umontreal.ca>
收件人 David Rush <kumoyuki at gmail.com>
副本(CC) peter lo <peter19852001 at yahoo.com.hk>; gambit-list at iro.umontreal.ca
傳送日期﹕ 2009 年 9月 9 日 星期三 下午 8:11:51
主題: Re: ReĄG [gambit-list] Regarding garbage collection


On 9-Sep-09, at 5:36 AM, David Rush wrote:

>> 
>> As for the possibility of memory leaks, this puzzles me. In a language with garbage collection, what does it mean to have memory leaks?
> 
> Exactly what you said yourself. Code that inadvertantly keeps memory
> around. Heavy use of symbols (in the sense of memoized strings) is
> probably the easiest way to leak memory. Just about anything that is
> linked to a a global data structure is a good candidate for leaking
> memory.
> 
> If the garbage collector thinks your program could possibly ever use a
> piece of memory again, it will not free that memory. The collector is
> way more conservative than your brain, so it is certainly possible to
> fool yourself about whether or not you *intend*  to use data when you
> have in fact kept it around.

Nice explanation.

Simply: garbage collection solves the "dangling pointer" problem completely, and it only helps with the "memory leak" problem because you can write programs that keep references to data that will never be used by the program.  Garbage collectors are "conservative" in the sense that they use "reachability" to determine "usefullness" of data.

Marc


      Yahoo!香港提供網上安全攻略,教你如何防範黑客! 請前往 http://hk.promo.yahoo.com/security/ 了解更多!




More information about the Gambit-list mailing list