Leopard bug causing extremely slow parsing times
I recently upgraded from OS X Tiger to Leopard and suddenly Gambit's "parsing" time has slowed to a crawl (I have confirmed this on another upgraded machine). I'm not sure if "parsing" is the correct term. But any time where Gambit reads a file and parses the code, gsc/gsi sits there and thrashes the CPU for some time linear to the size of the program. It takes 2.54s just to `include` a 300 line file. Execution speed is not affected. I tried recompiling Gambit but nothing changed. I know you can compile Gambit with --enable-debug and --enable-profile, but I'm not sure how to debug/profile the system. I think a `make check` does some timing checks, but I waited 2 hours for it to get past the first couple checks (because of the parsing bug) and then gave up. I wish I could give a better analysis of the problem, but after trying a couple methods of debugging (dtrace, testing different files, etc.), I'm not sure what the problem is. Anyone have Leopard and would be able to shed some light on this? Gambit-C v4.0.1, OS X 10.5 (Leopard) James
Afficher les réponses par date
Are you using the prebuilt version, or did you compile Gambit yourself? If so, what version of gcc are you using (gcc --version)? Please post your bug report to http://www.iro.umontreal.ca/~gambit/ bugzilla/ . Marc On 29-Oct-07, at 11:48 AM, James Long wrote:
I recently upgraded from OS X Tiger to Leopard and suddenly Gambit's "parsing" time has slowed to a crawl (I have confirmed this on another upgraded machine). I'm not sure if "parsing" is the correct term. But any time where Gambit reads a file and parses the code, gsc/gsi sits there and thrashes the CPU for some time linear to the size of the program. It takes 2.54s just to `include` a 300 line file. Execution speed is not affected.
I tried recompiling Gambit but nothing changed. I know you can compile Gambit with --enable-debug and --enable-profile, but I'm not sure how to debug/profile the system. I think a `make check` does some timing checks, but I waited 2 hours for it to get past the first couple checks (because of the parsing bug) and then gave up.
I wish I could give a better analysis of the problem, but after trying a couple methods of debugging (dtrace, testing different files, etc.), I'm not sure what the problem is. Anyone have Leopard and would be able to shed some light on this?
Gambit-C v4.0.1, OS X 10.5 (Leopard)
I was initially using the prebuilt version, but I recompiled v4.0.1 from source (with --enable-single-host and --enable-gcc-opts) and nothing changed. I just installed the newest version of the Apple Developer Tools, gcc reports: i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5465) Bug has been posted. Tell me if you need any more info about it. http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=15 James On 10/29/07, Marc Feeley <feeley@iro.umontreal.ca> wrote:
Are you using the prebuilt version, or did you compile Gambit yourself? If so, what version of gcc are you using (gcc --version)?
Please post your bug report to http://www.iro.umontreal.ca/~gambit/ bugzilla/ .
Marc
On 29-Oct-07, at 11:48 AM, James Long wrote:
I recently upgraded from OS X Tiger to Leopard and suddenly Gambit's "parsing" time has slowed to a crawl (I have confirmed this on another upgraded machine). I'm not sure if "parsing" is the correct term. But any time where Gambit reads a file and parses the code, gsc/gsi sits there and thrashes the CPU for some time linear to the size of the program. It takes 2.54s just to `include` a 300 line file. Execution speed is not affected.
I tried recompiling Gambit but nothing changed. I know you can compile Gambit with --enable-debug and --enable-profile, but I'm not sure how to debug/profile the system. I think a `make check` does some timing checks, but I waited 2 hours for it to get past the first couple checks (because of the parsing bug) and then gave up.
I wish I could give a better analysis of the problem, but after trying a couple methods of debugging (dtrace, testing different files, etc.), I'm not sure what the problem is. Anyone have Leopard and would be able to shed some light on this?
Gambit-C v4.0.1, OS X 10.5 (Leopard)
-- James Long Coptix, Inc. longster@gmail.com
On 29-Oct-07, at 12:21 PM, James Long wrote:
I was initially using the prebuilt version, but I recompiled v4.0.1 from source (with --enable-single-host and --enable-gcc-opts) and nothing changed.
I just installed the newest version of the Apple Developer Tools, gcc reports: i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5465)
Bug has been posted. Tell me if you need any more info about it. http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=15
James
This bug is sufficiently severe that I am sending the solution to the whole list. It might affect other operating systems than MacOS Leopard so you might find the patch useful. Marc The problem is not with I/O. Apparently MacOS X has a very high resolution interval timer (setitimer). When the Gambit thread system is initialized, it sets up the interval timer for implementing the thread quantum. In _thread.scm you will find this call: (##thread-heartbeat-interval-set! (macro-inexact-+0)) which is the same as (##thread-heartbeat-interval-set! 0.0) The value 0.0 is a special value indicating that the shortest possible time interval should be used. On MacOS 10.4 this time is 1e-2 or 1e-3 seconds (I don't remember which, but fairly coarse). On MacOS 10.5 the time is 1e-6, i.e. one microsecond! Because the quantum is so small, the system is spending a lot of time being interrupted by the timer for no good reason. I just installed MacOS 10.5 and haven't yet installed Mercurial (so I can't upload a patch). In the meantime simply replace the problematic line in lib/_thread.scm by (##thread-heartbeat-interval-set! 0.01) and run "make". Note that the compiler will take forever to compile lib/_thread.scm (because gsc-comp will be running with a heartbeat interval of 1e-6 second). If you want to save some time do this: % cd lib % echo "(##thread-heartbeat-interval-set! 0.01)" > gambcini.scm % ../gsc-comp -:=.. -c -check _thread.scm % ../gsc-comp -:=.. -link -flat -o _gambc.c _kernel.c _system.c _num.c _std.c _eval.c _io.c _nonstd.c _thread.c _repl.c % make % cd ../gsc % echo "(##thread-heartbeat-interval-set! 0.01)" > gambcini.scm % ../gsc-comp -:=".." -link _host.c _utils.c _source.c _parms.c _env.c _ptree1.c _ptree2.c _gvm.c _back.c _front.c _prims.c _t-c-1.c _t-c-2.c _t-c-3.c _gsc.c % make % cp gsc ../gsc-comp % cd .. % make
I think I did this right. It took me a while to realize that I needed to make the change in _thread.scm, AND do all the "save some time" steps. Anyway, now I'm trying to run "make check" and it hangs on ../gsi/gsi -f mix.scm > test4.out I saw this in tests/mix.scm: (define heartbeat-interval (##thread-heartbeat-interval-set! 0.0)) ; set and get smallest interval I tried changing to 0.01, but that caused a pretty nasty blowout. Is there a way to get this test to pass, or to skip it? thanks On Oct 29, 2007 8:56 PM, Marc Feeley <feeley@iro.umontreal.ca> wrote:
The problem is not with I/O. Apparently MacOS X has a very high resolution interval timer (setitimer). When the Gambit thread system is initialized, it sets up the interval timer for implementing the thread quantum. In _thread.scm you will find this call:
(##thread-heartbeat-interval-set! (macro-inexact-+0))
which is the same as
(##thread-heartbeat-interval-set! 0.0)
The value 0.0 is a special value indicating that the shortest possible time interval should be used. On MacOS 10.4 this time is 1e-2 or 1e-3 seconds (I don't remember which, but fairly coarse). On MacOS 10.5 the time is 1e-6, i.e. one microsecond! Because the quantum is so small, the system is spending a lot of time being interrupted by the timer for no good reason.
I just installed MacOS 10.5 and haven't yet installed Mercurial (so I can't upload a patch). In the meantime simply replace the problematic line in lib/_thread.scm by
(##thread-heartbeat-interval-set! 0.01)
and run "make". Note that the compiler will take forever to compile lib/_thread.scm (because gsc-comp will be running with a heartbeat interval of 1e-6 second). If you want to save some time do this:
% cd lib % echo "(##thread-heartbeat-interval-set! 0.01)" > gambcini.scm % ../gsc-comp -:=.. -c -check _thread.scm % ../gsc-comp -:=.. -link -flat -o _gambc.c _kernel.c _system.c _num.c _std.c _eval.c _io.c _nonstd.c _thread.c _repl.c % make % cd ../gsc % echo "(##thread-heartbeat-interval-set! 0.01)" > gambcini.scm % ../gsc-comp -:=".." -link _host.c _utils.c _source.c _parms.c _env.c _ptree1.c _ptree2.c _gvm.c _back.c _front.c _prims.c _t-c-1.c _t-c-2.c _t-c-3.c _gsc.c % make % cp gsc ../gsc-comp % cd .. % make
It only fails when you change it to 0.01 because the compiled C code is different, and test 5 diff's the generated c code with a precompiled file. So make the change, compile the file, and copy it to test5.ok $ ../gsc/gsc -:=.. -f -c mix.scm $ cp mix.c test5.ok Of course, that kind of nullifies test 5, but only until the Leopard fix is incorporated into gambit. On Nov 10, 2007 12:26 AM, Patrick Kelly <e@pdk.to> wrote:
I think I did this right. It took me a while to realize that I needed to make the change in _thread.scm, AND do all the "save some time" steps.
Anyway, now I'm trying to run "make check" and it hangs on
../gsi/gsi -f mix.scm > test4.out
I saw this in tests/mix.scm:
(define heartbeat-interval (##thread-heartbeat-interval-set! 0.0)) ; set and get smallest interval
I tried changing to 0.01, but that caused a pretty nasty blowout.
Is there a way to get this test to pass, or to skip it?
thanks
On Oct 29, 2007 8:56 PM, Marc Feeley <feeley@iro.umontreal.ca> wrote:
The problem is not with I/O. Apparently MacOS X has a very high resolution interval timer (setitimer). When the Gambit thread system is initialized, it sets up the interval timer for implementing the thread quantum. In _thread.scm you will find this call:
(##thread-heartbeat-interval-set! (macro-inexact-+0))
which is the same as
(##thread-heartbeat-interval-set! 0.0)
The value 0.0 is a special value indicating that the shortest possible time interval should be used. On MacOS 10.4 this time is 1e-2 or 1e-3 seconds (I don't remember which, but fairly coarse). On MacOS 10.5 the time is 1e-6, i.e. one microsecond! Because the quantum is so small, the system is spending a lot of time being interrupted by the timer for no good reason.
I just installed MacOS 10.5 and haven't yet installed Mercurial (so I can't upload a patch). In the meantime simply replace the problematic line in lib/_thread.scm by
(##thread-heartbeat-interval-set! 0.01)
and run "make". Note that the compiler will take forever to compile lib/_thread.scm (because gsc-comp will be running with a heartbeat interval of 1e-6 second). If you want to save some time do this:
% cd lib % echo "(##thread-heartbeat-interval-set! 0.01)" > gambcini.scm % ../gsc-comp -:=.. -c -check _thread.scm % ../gsc-comp -:=.. -link -flat -o _gambc.c _kernel.c _system.c _num.c _std.c _eval.c _io.c _nonstd.c _thread.c _repl.c % make % cd ../gsc % echo "(##thread-heartbeat-interval-set! 0.01)" > gambcini.scm % ../gsc-comp -:=".." -link _host.c _utils.c _source.c _parms.c _env.c _ptree1.c _ptree2.c _gvm.c _back.c _front.c _prims.c _t-c-1.c _t-c-2.c _t-c-3.c _gsc.c % make % cp gsc ../gsc-comp % cd .. % make
-- James Long Coptix, Inc. longster@gmail.com
participants (3)
-
James Long -
Marc Feeley -
Patrick Kelly