Hi Marc!
How well does SMP work now?
Using SMP code that not uses any network/file IO (only string/u8vector port IO) except for in the root processor, may that be a stable usecase this year?
Thanks! Adam
Afficher les réponses par date
Hello Adam. Sorry for the delayed response…
The SMP work has been on hold and I plan to return to it in January. The usecase you describe should be usable right now. As I have said, the main issue remaining is thread priorities (currently they are not fully working when using SMP) and fine tuning the scheduler for fine grain load balancing. Of course I’d be interested in any stress testing you can do.
Marc
On Oct 17, 2017, at 1:59 AM, Adam adam.mlmb@gmail.com wrote:
Hi Marc!
How well does SMP work now?
Using SMP code that not uses any network/file IO (only string/u8vector port IO) except for in the root processor, may that be a stable usecase this year?
Thanks! Adam
Hi Marc,
Wonderful!
If you can please take three minutes to explain the following, so that I and others can get going with the SMP, would be great:
Can you explain what uses of the Gambit runtime are *not* safe currently (so I know what I can do in the light of what I can't do)?
Can I do TCP/console/file IO from any OS & green thread, or from some particular one, which and how do I ensure that?
Can I send messages between threads (that run on different OS threads), how?
Also, if there would be any bugs now then in what area would they likely be / how would they likely manifest?
(http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Threads says "Gambit supports the execution of multiple Scheme threads. These threads are managed entirely by Gambit’s runtime and are not related to the host operating system’s threads. Gambit’s runtime does not currently take advantage of multiprocessors (i.e. at most one thread is running).", will need update.)
Adam
2017-11-11 22:45 GMT+08:00 Marc Feeley feeley@iro.umontreal.ca:
Hello Adam. Sorry for the delayed response…
The SMP work has been on hold and I plan to return to it in January. The usecase you describe should be usable right now. As I have said, the main issue remaining is thread priorities (currently they are not fully working when using SMP) and fine tuning the scheduler for fine grain load balancing. Of course I’d be interested in any stress testing you can do.
Marc
On Oct 17, 2017, at 1:59 AM, Adam adam.mlmb@gmail.com wrote:
Hi Marc!
How well does SMP work now?
Using SMP code that not uses any network/file IO (only string/u8vector
port IO) except for in the root processor, may that be a stable usecase this year?
Thanks! Adam
On Nov 11, 2017, at 5:00 PM, Adam adam.mlmb@gmail.com wrote:
Hi Marc,
Wonderful!
If you can please take three minutes to explain the following, so that I and others can get going with the SMP, would be great:
Can you explain what uses of the Gambit runtime are not safe currently (so I know what I can do in the light of what I can't do)?
Thread priorities can’t be relied upon. In the non-SMP version (without --enable-smp) if a thread becomes runnable and its priority is higher than the currently running thread, the scheduler will immediately give the CPU to the higher priority thread. This can’t be done efficiently with SMP because there are multiple running threads and it would be expensive to determine if there is a running thread on another processor that has a lower priority than a thread that has just become runnable. The semantics of priorities will have to change (probably: if a higher priority thread becomes runnable it will eventually run, i.e. its execution can’t be blocked by a lower priority thread).
For I/O, there may be missing locks to ensure that separate processors don’t do I/O on a port at the same time. I haven’t tested this thoroughly.
Can I do TCP/console/file IO from any OS & green thread, or from some particular one, which and how do I ensure that?
Keeping all I/O in a single thread is definitely the safest scenario to avoid locking issues. Just assign these tasks to a single thread. All other threads should not do I/O.
Can I send messages between threads (that run on different OS threads), how?
Yes… thread-send
Also, if there would be any bugs now then in what area would they likely be / how would they likely manifest?
Deadlocks, segmentation faults, wrong result (due to memory consistency issues).
(http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Threads says "Gambit supports the execution of multiple Scheme threads. These threads are managed entirely by Gambit’s runtime and are not related to the host operating system’s threads. Gambit’s runtime does not currently take advantage of multiprocessors (i.e. at most one thread is running).", will need update.)
Will update when this is no longer true. I view --enable-smp as experimental…
I need feedback from people using SMP to weed out problems.
Marc
Adam
2017-11-11 22:45 GMT+08:00 Marc Feeley feeley@iro.umontreal.ca: Hello Adam. Sorry for the delayed response…
The SMP work has been on hold and I plan to return to it in January. The usecase you describe should be usable right now. As I have said, the main issue remaining is thread priorities (currently they are not fully working when using SMP) and fine tuning the scheduler for fine grain load balancing. Of course I’d be interested in any stress testing you can do.
Marc
On Oct 17, 2017, at 1:59 AM, Adam adam.mlmb@gmail.com wrote:
Hi Marc!
How well does SMP work now?
Using SMP code that not uses any network/file IO (only string/u8vector port IO) except for in the root processor, may that be a stable usecase this year?
Thanks! Adam
Wow!
(I'll try to start using the correct terminology "processor" for "OS thread" and "thread" for "green thread", soon.)
2017-11-13 21:41 GMT+08:00 Marc Feeley feeley@iro.umontreal.ca: [..]
For I/O, there may be missing locks to ensure that separate processors don’t do I/O on a port at the same time. I haven’t tested this thoroughly.
(At least the user code-facing Gambit IO routines, have looked since before to me like having the locking well implemented. And I guess now those same mutex locks, are automatically SMP-safe - err, I should ask about that below. So you must mean some kind of runtime-internal locking here somehow.)
Is |mutex-lock!| SMP-safe now - also without costing much more now than before?
If the SMP-safe |mutex-lock!| is more expensive only: is there then any variant that would only work between green threads within one and the same OS thread.
Can I do TCP/console/file IO from any OS & green thread, or from some particular one, which and how do I ensure that?
Keeping all I/O in a single thread is definitely the safest scenario to avoid locking issues. Just assign these tasks to a single thread. All other threads should not do I/O.
Wait - say you do console IO in one OS thread, TCP IO in one OS thread, and file IO in one OS thread - how could that be a problem?
Also, when you say it's best to assign the IO tasks to one thread, do you mean one OS thread or one green thread - if one OS thread, then, how do I stick and unstick a given set of green threads (ordinary thread objects) to be executed in one particular OS thread only?
(http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Threads says
"Gambit supports the execution of multiple Scheme threads. These threads are managed entirely by Gambit’s runtime and are not related to the host operating system’s threads. Gambit’s runtime does not currently take advantage of multiprocessors (i.e. at most one thread is running).", will need update.)
Will update when this is no longer true. I view --enable-smp as experimental…
I need feedback from people using SMP to weed out problems.
Noted.
Thanks!
Too many questions! The SMP work is experimental and I will return to it in January…
Marc
On Nov 13, 2017, at 10:58 AM, Adam adam.mlmb@gmail.com wrote:
Wow!
(I'll try to start using the correct terminology "processor" for "OS thread" and "thread" for "green thread", soon.)
2017-11-13 21:41 GMT+08:00 Marc Feeley feeley@iro.umontreal.ca: [..] For I/O, there may be missing locks to ensure that separate processors don’t do I/O on a port at the same time. I haven’t tested this thoroughly.
(At least the user code-facing Gambit IO routines, have looked since before to me like having the locking well implemented. And I guess now those same mutex locks, are automatically SMP-safe - err, I should ask about that below. So you must mean some kind of runtime-internal locking here somehow.)
Is |mutex-lock!| SMP-safe now - also without costing much more now than before?
If the SMP-safe |mutex-lock!| is more expensive only: is there then any variant that would only work between green threads within one and the same OS thread.
Can I do TCP/console/file IO from any OS & green thread, or from some particular one, which and how do I ensure that?
Keeping all I/O in a single thread is definitely the safest scenario to avoid locking issues. Just assign these tasks to a single thread. All other threads should not do I/O.
Wait - say you do console IO in one OS thread, TCP IO in one OS thread, and file IO in one OS thread - how could that be a problem?
Also, when you say it's best to assign the IO tasks to one thread, do you mean one OS thread or one green thread - if one OS thread, then, how do I stick and unstick a given set of green threads (ordinary thread objects) to be executed in one particular OS thread only?
(http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Threads says "Gambit supports the execution of multiple Scheme threads. These threads are managed entirely by Gambit’s runtime and are not related to the host operating system’s threads. Gambit’s runtime does not currently take advantage of multiprocessors (i.e. at most one thread is running).", will need update.)
Will update when this is no longer true. I view --enable-smp as experimental…
I need feedback from people using SMP to weed out problems.
Noted.
Thanks!