How to compile large one-file programs with small RAM size?
Hi all. Does anybody have any hints how to compile a large one-file program (10K - 100K lines) on a machine with only 3 GB RAM? (Splitting the file is no option because it would hurt performance of the compiled program so much that I would better stick with other Scheme compilers; I am extrapolating here from Gambit 3.0 to Gambit 4.0.) Some details: - gcc 4.2.1, Linux, 32bit - gsc command:
gsc n.scm virtual memory exhausted: Cannot allocate memory
memtime reports: 412.94 user, 20.03 system, 470.08 elapsed -- Max VSize = 2331356KB, Max RSS = 2001108KB - my 'declare': (declare (r5rs-scheme) (block) ;(inline) ;(inlining-limit 500) ;(lambda-lift) ;(constant-fold) (standard-bindings) (extended-bindings) (not safe) (not interrupts-enabled) (generic) (mostly-fixnum) ) Greetings Sven
Afficher les réponses par date
On 28-Aug-07, at 4:04 AM, Sven.Hartrumpf@FernUni-Hagen.de wrote:
Hi all.
Does anybody have any hints how to compile a large one-file program (10K - 100K lines) on a machine with only 3 GB RAM?
That's a pretty large program! Is this code generated automatically? What does the code look like?
(Splitting the file is no option because it would hurt performance of the compiled program so much that I would better stick with other Scheme compilers; I am extrapolating here from Gambit 3.0 to Gambit 4.0.)
Some details:
- gcc 4.2.1, Linux, 32bit
It is possible that an earlier version of gcc can deal with larger C files. I have never tried gcc 4.2.1, but Brad Lucier may be able to help you because he has experience with almost all versions of gcc since version 3.0.
- gsc command:
gsc n.scm virtual memory exhausted: Cannot allocate memory
memtime reports: 412.94 user, 20.03 system, 470.08 elapsed -- Max VSize = 2331356KB, Max RSS = 2001108KB
Which version of gsc? By default v4.0.0 uses the compile option ___USE_INLINE_JUMPS that generates tighter and faster code than earlier versions.
- my 'declare': (declare (r5rs-scheme) (block) ;(inline) ;(inlining-limit 500)
You'll get tighter code with (not inline). The default is (inline).
;(lambda-lift) ;(constant-fold) (standard-bindings) (extended-bindings) (not safe) (not interrupts-enabled)
It is not a good idea to declare (not interrupts-enabled) at the top- level. The compiler uses interrupt checks to detect stack overflows, so you risk overrunning the stack. Generally (not interrupts- enabled) should be used in tight tail-recursive loops.
(generic) (mostly-fixnum)
You'll get tighter code with (mostly-generic). Unfortunately the performance will also drop. You should profile your program to see which parts need to go fast, and use (mostly-fixnum) for those. You should also consider creating a call graph profile. Using the call graph profile you may discover that some parts of the program call other parts of the program infrequently. You can compile these parts separately with little impact on the performance. Marc
Tue, 28 Aug 2007 09:31:40 -0400, feeley wrote:
On 28-Aug-07, at 4:04 AM, Sven.Hartrumpf@FernUni-Hagen.de wrote:
Hi all.
Does anybody have any hints how to compile a large one-file program (10K - 100K lines) on a machine with only 3 GB RAM?
That's a pretty large program! Is this code generated automatically?
No, normal manually written code.
It is possible that an earlier version of gcc can deal with larger C files. I have never tried gcc 4.2.1, but Brad Lucier may be able to help you because he has experience with almost all versions of gcc since version 3.0.
Oh, yes. Your response made clear that the message below ("virtual ...") is not from gsc, but from gcc. Could this be made clearer in the output so that we can avoid dumb questions like mine? :-)
- gsc command:
gsc n.scm virtual memory exhausted: Cannot allocate memory
memtime reports: 412.94 user, 20.03 system, 470.08 elapsed -- Max VSize = 2331356KB, Max RSS = 2001108KB
Which version of gsc? By default v4.0.0 uses the compile option ___USE_INLINE_JUMPS that generates tighter and faster code than earlier versions.
Yes, 4.0.0.
You'll get tighter code with (not inline). You'll get tighter code with (mostly-generic).
These two hints brought the RAM use down by a factor of 2. Thanks! Now I have a runnable program compiled by gsc. Greetings Sven
participants (2)
-
Marc Feeley -
Sven.Hartrumpf＠FernUni-Hagen.de