Optimizing the parser is not the solution. Even if we rewrite it to allocate very little memory, the other parts of the compiler will be allocating lots more memory. Recall that the parsing takes a small fraction of the total execution time, and the execution time is a good measure of the allocation rate, so the rest of the compiler probably allocates a lot.
We need a global optimization called "garbage collection".
Marc
P.S. are we sure the problem is not that repeated array operations, such as series of pushes, generate quadratic space usage? That could explain part of the problem...
On 2011-02-28, at 10:35 PM, chevalma@iro.umontreal.ca wrote:
The compilation for fib uses less than 50 MB on v8 (The total heap size before compiling fib is around 45 MB and no gc is triggered during fib compilation). It means that our object representation uses at least 5X more memory than v8, only on parsing.
I don't think the issue is really one of object size. I would guess that our objects are about as big as theirs by default.
The real issue is probably that we create alot of temporary objects during parsing. Our standard library code, for example, converts strings to char code arrays before processing them. If the parser uses the String functions alot, it may be creating temporary arrays and strings every time it does so, which at the time, we have no way of reclaiming since we have no GC.
V8 probably has standard library code that makes less spurious allocations. Our current library code, particularly the string code, does alot of allocations that could probably be avoided. I think just calling String.prototype.charAt will allocate memory for each call, for example.
If we look more carefully at what string functions the parser uses, and optimize those, we may be able to get farther in the compilation. If there are about 650 MB of allocations to initialize the standard library while running in V8, though, that's a little steep to run without a GC, but still possible.
- Maxime
Tachyon-list mailing list Tachyon-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/tachyon-list