I believe the problem is that Gerbil is doing some inlining of sorts (for example tail duplication in the matcher) and then passing this to gsc which will also do some inlining of the functions in the code, and probably gcc also with the -O2 option.
Its understandable that this layering of languages will cause code bloat.
Marc
On Mar 19, 2018, at 3:05 PM, Dimitris Vyzovitis vyzo@hackzen.org wrote:
Sure, I'd like to get to the bottom of this because it's such an ugly failure mode!
For now, I changed the gerbil compiler to emit an inlining-limit declaration in meta phases (that's where the syntax-case monsters reside).
-- vyzo
On Mon, Mar 19, 2018 at 9:02 PM, Marc Feeley feeley@iro.umontreal.ca wrote: I think you should figure out which of the configure options
--enable-single-host --enable-c-opt --enable-gcc-opts
are the most useful/beneficial for the kind of code generated by Gerbil. I suspect that --enable-single-host is the most performance-enhancing option, and --enable-c-opt only gives a marginal speed increase at the cost of a much higher C compile time.
If you do try the various combinations, please report your results here. I’d like to know if my intuition is correct.
Marc
On Mar 19, 2018, at 2:56 PM, Dimitris Vyzovitis vyzo@hackzen.org wrote:
My gambit is configured with --enable-single-host --enable-c-opt --enable-gcc-opts. I have 8G on my current laptop, but I run without a swap; the death occurs at around 6G. It's not only the memory usage though, it takes forever too. clang on travis didn't OOM, but it took 15min on the file.
I think it might be a case of really bad interaction between the various optimizers in the 3 compilers involved. The gerbil emitted code is already heavily optimized to perform match tree linearization (I have a shiny new optimizer that optimizes match and syntax- case expansions). That means you can't reasonably inline anything other than single use procedures within the optimized blocks.
-- vyzo
On Mon, Mar 19, 2018 at 8:41 PM, Marc Feeley feeley@iro.umontreal.ca wrote: The default inlining-limit is 350, so the expansion from 100 is quite possible.
But the problem here is that gcc chokes on the compilation of the C file. So… what are the compilation options passed to gcc?
- are you using --enable-single-host ?
- are you using a higher level of optimization such as -O2 or -O3 rather than the default -O1 ?
These will definitely increase the pressure on the C compiler. Also, some versions of gcc do a better job at compiling large C files. The file lib/_io.c in the Gambit distribution is about 90kloc and I have never gotten an OOM error from gcc while compiling it, even though I use a “make -j 8” (8 C compilations in parallel). I do have 16 GB of RAM on my machine… how much RAM do you have on yours?
Marc
On Mar 19, 2018, at 2:31 PM, Dimitris Vyzovitis vyzo@hackzen.org wrote:
It's 140kloc without the inlining declaration and just 22Kloc with the declaration.
-- vyzo
On Mon, Mar 19, 2018 at 8:27 PM, Marc Feeley feeley@iro.umontreal.ca wrote: Out of curiosity, what is the number of LOC of C with and without the inlining-limit?
I’m just wondering if this should be classified as an issue, or if the inliner is just doing its work as expected.
Marc
On Mar 19, 2018, at 2:23 PM, Dimitris Vyzovitis vyzo@hackzen.org wrote:
It seems it's the inliner going haywire -- if I add a (declare (inlining-limit 100)), then it compiles in 20s.
-- vyzo
On Mon, Mar 19, 2018 at 7:29 PM, Dimitris Vyzovitis vyzo@hackzen.org wrote: The attached file results in a 140kloc monster that results in gcc dying with OOM after several minutes of effort, and I would like to understand why. Any ideas?
-- vyzo