[gambit-list] The monster that killed gcc

Marc Feeley feeley at iro.umontreal.ca
Mon Mar 19 15:15:29 EDT 2018


I believe the problem is that Gerbil is doing some inlining of sorts (for example tail duplication in the matcher) and then passing this to gsc which will also do some inlining of the functions in the code, and probably gcc also with the -O2 option.

Its understandable that this layering of languages will cause code bloat.

Marc



> On Mar 19, 2018, at 3:05 PM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:
> 
> Sure, I'd like to get to the bottom of this because it's such an ugly failure mode!
> 
> For now, I changed the gerbil compiler to emit an inlining-limit declaration in meta phases (that's where the syntax-case monsters reside).
> 
> -- vyzo
> 
> On Mon, Mar 19, 2018 at 9:02 PM, Marc Feeley <feeley at iro.umontreal.ca> wrote:
> I think you should figure out which of the configure options
> 
>   --enable-single-host
>   --enable-c-opt
>   --enable-gcc-opts
> 
> are the most useful/beneficial for the kind of code generated by Gerbil.  I suspect that --enable-single-host is the most performance-enhancing option, and --enable-c-opt only gives a marginal speed increase at the cost of a much higher C compile time.
> 
> If you do try the various combinations, please report your results here.  I’d like to know if my intuition is correct.
> 
> Marc
> 
> 
> 
> > On Mar 19, 2018, at 2:56 PM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:
> >
> > My gambit is configured with  --enable-single-host --enable-c-opt --enable-gcc-opts.
> > I have 8G on my current laptop, but I run without a swap; the death occurs at around 6G.
> > It's not only the memory usage though, it takes forever too. clang on travis didn't OOM, but it took 15min on the file.
> >
> > I think it might be a case of really bad interaction between the various optimizers in the 3 compilers involved. The gerbil emitted code is already heavily optimized to perform match tree linearization (I have a shiny new optimizer that optimizes match and syntax- case expansions).
> > That means you can't reasonably inline anything other than single use procedures within the optimized blocks.
> >
> > -- vyzo
> >
> >
> > On Mon, Mar 19, 2018 at 8:41 PM, Marc Feeley <feeley at iro.umontreal.ca> wrote:
> > The default inlining-limit is 350, so the expansion from 100 is quite possible.
> >
> > But the problem here is that gcc chokes on the compilation of the C file.  So… what are the compilation options passed to gcc?
> >
> > - are you using --enable-single-host ?
> > - are you using a higher level of optimization such as -O2 or -O3 rather than the default -O1 ?
> >
> > These will definitely increase the pressure on the C compiler.  Also, some versions of gcc do a better job at compiling large C files.  The file lib/_io.c in the Gambit distribution is about 90kloc and I have never gotten an OOM error from gcc while compiling it, even though I use a “make -j 8” (8 C compilations in parallel).  I do have 16 GB of RAM on my machine… how much RAM do you have on yours?
> >
> > Marc
> >
> >
> >
> > > On Mar 19, 2018, at 2:31 PM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:
> > >
> > > It's 140kloc without the inlining declaration and just 22Kloc with the declaration.
> > >
> > > -- vyzo
> > >
> > > On Mon, Mar 19, 2018 at 8:27 PM, Marc Feeley <feeley at iro.umontreal.ca> wrote:
> > > Out of curiosity, what is the number of LOC of C with and without the inlining-limit?
> > >
> > > I’m just wondering if this should be classified as an issue, or if the inliner is just doing its work as expected.
> > >
> > > Marc
> > >
> > >
> > >
> > > > On Mar 19, 2018, at 2:23 PM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:
> > > >
> > > > It seems it's the inliner going haywire -- if I add a (declare (inlining-limit 100)), then it compiles in 20s.
> > > >
> > > > -- vyzo
> > > >
> > > > On Mon, Mar 19, 2018 at 7:29 PM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:
> > > > The attached file results in a 140kloc monster that results in gcc dying with OOM after several minutes of effort, and I would like to understand why.
> > > > Any ideas?
> > > >
> > > > -- vyzo
> > > >
> > >
> > >
> >
> >
> 
> 




More information about the Gambit-list mailing list