[gambit-list] The monster that killed gcc

Dimitris Vyzovitis vyzo at hackzen.org
Mon Mar 19 15:27:25 EDT 2018


The match optimizer is very careful to avoid inlining code that doesn't
benefit from the current match tree, so there is very little tail
duplication.
I think it's the subsequent inlining and optimization from gsc/gcc that
results in the blow up, as the code really cannot be optimized further,
But yeah, layers of language will do that to you.

Note that in general I don't do optimizations that gsc already does, but we
can't reasonably expect gsc to understand and properly optimize
match/syntax-case expansions.

-- vyzo

On Mon, Mar 19, 2018 at 9:15 PM, Marc Feeley <feeley at iro.umontreal.ca>
wrote:

> I believe the problem is that Gerbil is doing some inlining of sorts (for
> example tail duplication in the matcher) and then passing this to gsc which
> will also do some inlining of the functions in the code, and probably gcc
> also with the -O2 option.
>
> Its understandable that this layering of languages will cause code bloat.
>
> Marc
>
>
>
> > On Mar 19, 2018, at 3:05 PM, Dimitris Vyzovitis <vyzo at hackzen.org>
> wrote:
> >
> > Sure, I'd like to get to the bottom of this because it's such an ugly
> failure mode!
> >
> > For now, I changed the gerbil compiler to emit an inlining-limit
> declaration in meta phases (that's where the syntax-case monsters reside).
> >
> > -- vyzo
> >
> > On Mon, Mar 19, 2018 at 9:02 PM, Marc Feeley <feeley at iro.umontreal.ca>
> wrote:
> > I think you should figure out which of the configure options
> >
> >   --enable-single-host
> >   --enable-c-opt
> >   --enable-gcc-opts
> >
> > are the most useful/beneficial for the kind of code generated by
> Gerbil.  I suspect that --enable-single-host is the most
> performance-enhancing option, and --enable-c-opt only gives a marginal
> speed increase at the cost of a much higher C compile time.
> >
> > If you do try the various combinations, please report your results
> here.  I’d like to know if my intuition is correct.
> >
> > Marc
> >
> >
> >
> > > On Mar 19, 2018, at 2:56 PM, Dimitris Vyzovitis <vyzo at hackzen.org>
> wrote:
> > >
> > > My gambit is configured with  --enable-single-host --enable-c-opt
> --enable-gcc-opts.
> > > I have 8G on my current laptop, but I run without a swap; the death
> occurs at around 6G.
> > > It's not only the memory usage though, it takes forever too. clang on
> travis didn't OOM, but it took 15min on the file.
> > >
> > > I think it might be a case of really bad interaction between the
> various optimizers in the 3 compilers involved. The gerbil emitted code is
> already heavily optimized to perform match tree linearization (I have a
> shiny new optimizer that optimizes match and syntax- case expansions).
> > > That means you can't reasonably inline anything other than single use
> procedures within the optimized blocks.
> > >
> > > -- vyzo
> > >
> > >
> > > On Mon, Mar 19, 2018 at 8:41 PM, Marc Feeley <feeley at iro.umontreal.ca>
> wrote:
> > > The default inlining-limit is 350, so the expansion from 100 is quite
> possible.
> > >
> > > But the problem here is that gcc chokes on the compilation of the C
> file.  So… what are the compilation options passed to gcc?
> > >
> > > - are you using --enable-single-host ?
> > > - are you using a higher level of optimization such as -O2 or -O3
> rather than the default -O1 ?
> > >
> > > These will definitely increase the pressure on the C compiler.  Also,
> some versions of gcc do a better job at compiling large C files.  The file
> lib/_io.c in the Gambit distribution is about 90kloc and I have never
> gotten an OOM error from gcc while compiling it, even though I use a “make
> -j 8” (8 C compilations in parallel).  I do have 16 GB of RAM on my
> machine… how much RAM do you have on yours?
> > >
> > > Marc
> > >
> > >
> > >
> > > > On Mar 19, 2018, at 2:31 PM, Dimitris Vyzovitis <vyzo at hackzen.org>
> wrote:
> > > >
> > > > It's 140kloc without the inlining declaration and just 22Kloc with
> the declaration.
> > > >
> > > > -- vyzo
> > > >
> > > > On Mon, Mar 19, 2018 at 8:27 PM, Marc Feeley <
> feeley at iro.umontreal.ca> wrote:
> > > > Out of curiosity, what is the number of LOC of C with and without
> the inlining-limit?
> > > >
> > > > I’m just wondering if this should be classified as an issue, or if
> the inliner is just doing its work as expected.
> > > >
> > > > Marc
> > > >
> > > >
> > > >
> > > > > On Mar 19, 2018, at 2:23 PM, Dimitris Vyzovitis <vyzo at hackzen.org>
> wrote:
> > > > >
> > > > > It seems it's the inliner going haywire -- if I add a (declare
> (inlining-limit 100)), then it compiles in 20s.
> > > > >
> > > > > -- vyzo
> > > > >
> > > > > On Mon, Mar 19, 2018 at 7:29 PM, Dimitris Vyzovitis <
> vyzo at hackzen.org> wrote:
> > > > > The attached file results in a 140kloc monster that results in gcc
> dying with OOM after several minutes of effort, and I would like to
> understand why.
> > > > > Any ideas?
> > > > >
> > > > > -- vyzo
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20180319/0f6accbd/attachment.htm>


More information about the Gambit-list mailing list