[gambit-list] Need help to track down tight loop

Marc Feeley feeley at iro.umontreal.ca
Sun Aug 2 13:29:32 EDT 2020


Hello Jorg.  The heartbeat interrupts are not necessarily problematic.  They will occur if your program is running, for example in a busy loop (in your program or the runtime library).  Can you interrupt your program to get a REPL and see what it is doing?

A convenient way to do this is to start your program with the -:daR runtime option.  When you hit ctrl-C a REPL will be started in the context of what the program was executing, so a “,b” command will give you a backtrace.

Another way is to start your program with the -:d$ runtime option that starts a REPL server listening on port 44555.  Then you can start a REPL by connecting to that port (for example with the “nc” program) and use the “top” procedure to see what the threads are doing.  For example, in one shell do

% gsi -:d$ -e '(let loop () (thread-sleep! 5) (loop))'

and in another shell do

% $ nc localhost 44555
Gambit v4.9.3-1201-g3b320533

> (top 20 (thread-thread-group #1))
*** THREAD LIST:
#<thread #1 primordial>   SLEEPING 3.8s

The “top” procedure will monitor the threads for 20 seconds (the default is 10 seconds in the current thread’s thread group) and refresh the status every second.  In the above the primordial thread was sleeping (and 3.8 seconds were left to sleep).  If you remove the (thread-sleep! 5) to get an infinite loop, the status would be

#<thread #1 primordial>   RUNNING P0

which indicates that the primordial thread is running (on processor #0).  If a thread is blocked reading a port you will see something like

#<thread #1 primordial>   WAITING #<input-port #2 (stdin)>

If on the other hand your program is totally unresponsive, it could be a bug in Gambit’s runtime system.  This is much harder to debug, especially if it only happens after a long time.  In that case you could compile Gambit with --enable-debug and not --enable-single-host and use gdb or lldb to attach to the process and interrupt it.  If you also configure Gambit with --enable-debug-c-backtrace and --enable-debug-ctrl-flow-history you can “kill -3 <pid>” the program when it seems to be stuck and you will get a history of the 1024 last Scheme procedure calls, plus a C backtrace, on the gambit.log file.  This should improve your understanding of the issue…

Marc



> On Aug 2, 2020, at 9:53 AM, Jörg F. Wittenberger <Joerg.Wittenberger at softeyes.net> wrote:
> 
> Hello,
> 
> on Linux I observe gambit entering a tight loop.
> 
> Attached the result of "strace -r -p <pid> -o ot0.trace".
> 
> To me this looks like pattern of roughly 15/80/15/118... repeating the
> heartbeat operation.  In this state the program takes almost 100% CPU
> and does not respond to anything.  Normally it runs on around 40-60
> open file descriptors (most from UDP and TCP).
> 
> The problem arises essentially for sure.  Depending on the hardware
> sooner or later.  (On 32bit armel after hours to few days, aarch64 takes
> sometimes almost week, AMD64 even longer.)
> 
> Anybody having an idea how to track this down?
> 
> Thanks so much
> 
> Jörg
> <ot0.trace>_______________________________________________
> Gambit-list mailing list
> Gambit-list at iro.umontreal.ca
> https://mailman.iro.umontreal.ca/cgi-bin/mailman/listinfo/gambit-list





More information about the Gambit-list mailing list