Hello,
I am experiencing a strange bug in my code that would be long to explain. To resume the context, I have many threads running in parallel, some open-process IO, some tcp-server IO and some termite processes. At some point all the threads become blocked indefinitely and the program sleeps forever.
I stumbled on the following paragraph in the threads section of the Gambit doc:
``A thread is blocked if it is waiting for a mutex to become unlocked, an I/O operation to become possible, the end of a “sleep” period, etc. ''
I think this may lead me to a solution to my bug but I don't clearly understand the statement. Could there be an IO operation that blocks all my threads? Any help would be greatly appreciated.
Thank you, Francois Magnan
Afficher les réponses par date
Hi François,
On Jan 29, 2008 3:27 PM, François Magnan francois.magnan@licef.ca wrote:
I think this may lead me to a solution to my bug but I don't clearly understand the statement. Could there be an IO operation that blocks all my threads? Any help would be greatly appreciated.
My understanding is that Gambit uses non-blocking I/O internally combined with appropriate "wait until resource is ready or there's a timeout" system calls (e.g., select() for Unix, WaitForMultipleObjects() for Windows) to simulate blocking I/O with multiple user-space threads. So an I/O operation that blocks all threads is theoretically possible -- any I/O that is blocking *at the system level* will block all threads. So things like Gambit's host-info (which calls gethostbyname() -- blocking socket I/O) will block all threads but only until it returns the host info or errors.
I hope this info helps you find your bug.
--Jeff
François Magnan wrote:
``A thread is blocked if it is waiting for a mutex to become unlocked, an I/O operation to become possible, the end of a “sleep” period, etc. ''
Hi: A threaded system has a state change model which typically includes the above items. When any task/thread becomes blocked, the cpu chooses another from it's list of waiting tasks such that it is always doing something.
A mutex is a simple mechanism for managing shared memory. But priority inversion or deadly embrace can happen if A blocks on a mutex set by B but B needs some I/O to be done by A.
It's possible that your app may run ok on one of Linux, Windows, or SunOs but not on all because each has it's own threading model.
Make your design as simple as possible. Perhaps run 2 Gambit sessions, with the slow stuff such as network connects in one, and the fast stuff in another, with a pipe or ring buffer in between. Consider the QNX message passing model. All transactions are atomic. Blocking tasks and data manger tasks are separated in a hierarchy.
It's a complex topic that may require some determined research.
Cheers, -Bob-
On 29-Jan-08, at 3:27 PM, François Magnan wrote:
Hello,
I am experiencing a strange bug in my code that would be long to explain. To resume the context, I have many threads running in parallel, some open-process IO, some tcp-server IO and some termite processes. At some point all the threads become blocked indefinitely and the program sleeps forever.
Are these problems occurring on Mac OS X? Could you describe what processes you start with open-process? I'm asking because I recently fixed a bug in open-process.
I stumbled on the following paragraph in the threads section of the Gambit doc:
``A thread is blocked if it is waiting for a mutex to become unlocked, an I/O operation to become possible, the end of a “sleep” period, etc. ''
I think this may lead me to a solution to my bug but I don't clearly understand the statement. Could there be an IO operation that blocks all my threads? Any help would be greatly appreciated.
You have to beware of C functions that are called from the Scheme world. If the C function performs an I/O operation, or a "sleep", or a "wait" on a process, or acquire a mutex, and the operation blocks then the Scheme thread scheduler will never get a chance to schedule another Scheme thread. Some Scheme functions which rely on C library functions that can block, such as host-info, also have the same issue.
Marc
Hi,
Thank you all for your answers. The process I start with open-process is a java application that uses it's standard input and standard output for interacting with java classes through reflection. It is some kind of bridge between the Gambit world and the Java world. The bug appears on both Windows (MIngW) and MacOSX (Leopard).
Can IO with external processes (through open-process) block all threads?
I will try to get the latest patches if it fixes the bug. I will also try wo put timeouts on all IO operations to isolate the faulty one.
I also have a (separate?) problem strictly on WinXP where the open-tcp- server implementation seems to miss some client connections when there are too much concurrent connections. I have a web services server implemented in Gambit and the client browers make some Ajax calls when the page initially loads. One of them is consistently never received by the server. When there is let load on the server, the same request is always received and answered properly.
I tested this with many different clients, some requests are just dropped by the server.
Could this be related to the WaitForMultipleObjects() that Jeff Read was mentioning in his answer to my post?
Thanks, Francois Magnan
On 29-Jan-08, at 11:47 PM, Marc Feeley wrote:
On 29-Jan-08, at 3:27 PM, François Magnan wrote:
Hello,
I am experiencing a strange bug in my code that would be long to explain. To resume the context, I have many threads running in parallel, some open-process IO, some tcp-server IO and some termite processes. At some point all the threads become blocked indefinitely and the program sleeps forever.
Are these problems occurring on Mac OS X? Could you describe what processes you start with open-process? I'm asking because I recently fixed a bug in open-process.
I stumbled on the following paragraph in the threads section of the Gambit doc:
``A thread is blocked if it is waiting for a mutex to become unlocked, an I/O operation to become possible, the end of a “sleep” period, etc. ''
I think this may lead me to a solution to my bug but I don't clearly understand the statement. Could there be an IO operation that blocks all my threads? Any help would be greatly appreciated.
You have to beware of C functions that are called from the Scheme world. If the C function performs an I/O operation, or a "sleep", or a "wait" on a process, or acquire a mutex, and the operation blocks then the Scheme thread scheduler will never get a chance to schedule another Scheme thread. Some Scheme functions which rely on C library functions that can block, such as host-info, also have the same issue.
Marc
François Magnan wrote:
Hi,
Thank you all for your answers. The process I start with open-process is a java application that uses it's standard input and standard output for interacting with java classes through reflection. It is some kind of bridge between the Gambit world and the Java world. The bug appears on both Windows (MIngW) and MacOSX (Leopard).
For me that smells like the unix O_NONBLOCK problem (namely that toggling the nonblocking behaviour of an fd effects all processes sharing it; it could be that java turns off the nonblocking flag on a file descriptor it inherited from before the open-process, while gambit is still also using it; then when the Gambit runtime gets notification from the OS that IO is ready, java/whatever other process may be ready quicker than Gambit calling read or write on it, and when it's Gambit's turn to read/write, the system call blocks since there's no IO ready anymore (*)).
I'm not sure why that should happen on Windows though, do they actually have that same/equivalent problem as well?
If you were on Linux, I'd suggest you to run strace -fF around your program, then you'd easily see on which operation Gambit hangs, and with lsof you could see if the filehandles involved are in use by the other programs as well; I don't know what the equivalent tools on OSX and Windows are.
Christian.
(*) see my mail about a similar situation at https://webmail.iro.umontreal.ca/pipermail/gambit-list/2006-January/000607.h... for more details. Also note the way D.J. Bernstein solves it, by using a timer (a bit painful I'd say, since that assumes nobody else in the program will be making use of the timer, so either this would have to be optional (good idea) or SIGALRM handling would have to be virtualized (probably not worthwhile)).
-
Bob McIsaac -
Christian Jaeger -
François Magnan -
Jeff Read -
Marc Feeley