Hi Marc,
Can you give some introduction to how the new SMP features are used?
So the gsi/gsc argument -:pN specifies how many GVM processors (i.e. OS threads) should be spawned. That one is not yet in the manual ( http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Threads).
And then the green threads run spread over the GVM processors?
How do you enumerate the GVM processors? How do you glue particular green threads or green thread groups to a particular GVM processor, if desired?
What more SMP-specific forms and features are there?
Thanks!
Afficher les réponses par date
Ah right, and it's the "smp" branch one is supposed to use? Until when?
2017-03-24 16:00 GMT+03:00 Adam adam.mlmb@gmail.com:
Hi Marc,
Can you give some introduction to how the new SMP features are used?
So the gsi/gsc argument -:pN specifies how many GVM processors (i.e. OS threads) should be spawned. That one is not yet in the manual ( http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Threads).
And then the green threads run spread over the GVM processors?
How do you enumerate the GVM processors? How do you glue particular green threads or green thread groups to a particular GVM processor, if desired?
What more SMP-specific forms and features are there?
Thanks!
The smp branch is currently a WIP with some unimplemented features (such as thread priorities and priority inheritance). There are also some infrequent bugs that show up when stress testing. Nevertheless it is quite usable for experimentation and I am actively working on ironing out the bugs. I expect a stable smp branch by the end of April. Eventually I want the smp branch to be merged with the master branch and to make --enable-multiple-threaded-vms the default. This will only happen after extensive portability testing so that the master branch remains very stable and can be built “out of the box” without any issues and tinkering.
So to answer your questions… You need to build the smp branch to get parallel execution. These steps should work:
git clone https://github.com/gambit/gambit.git cd gambit git checkout smp ./configure make -j4 current-gsc-boot ./configure --enable-single-host --enable-multiple-threaded-vms make -j4 from-scratch make check
When a program built with Gambit (including gsi and gsc) is started, the -:pN runtime option determines how many OS threads are used to run the code. Each OS thread is called a “processor” because the runtime system contains a thread scheduler that dynamically assigns the Scheme threads to the OS threads, as though they were hardware processors of a multiprocessor computer.
If the -:pN runtime option is not provided, then 50% of the computer’s CPUs are used. If the -:pN runtime option is provided, the N is the number of OS threads to start, with the special value 0 meaning “all of the CPUs”, -1 meaning “all but one of the CPUs”, etc. Also you can add a percent sign after the N to indicate a proportion of all the CPUs, so -:p25% means 1/4 of all the CPUs.
(##cpu-count) returns total number of CPUs on the computer (##current-vm-processor-count) returns the number of “processors” (OS threads) (current-processor) returns the processor running the current thread (a Scheme object with the state of the processor) (processor-id processor) returns the id of the processor object
Each “processor” has a run queue of threads ready to run. A processor will focus its attention on the execution of the threads in its own run queue. If a processor goes idle, it will steal a thread from another processor’s run queue. If it can’t find a thread to steal, the processor will go into a wait state that effectively suspends the OS thread (so that it doesn’t consume power needlessly). Note that waiting processors are woken up when a garbage collection is started, so that all processors participate in the parallel GC.
The thread scheduling algorithm will surely evolve and gain features as more experience with it is accumulated. Currently there is no way to “pin” a Scheme thread to a particular processor.
If you want to monitor what the threads are doing, you can call (top) to get a list of the threads and what they are doing (updated every second for a period of 10 seconds by default). When a thread is running, the processor it is running on will be indicated.
Another interesting procedure is thread-interrupt! which can be used to request that a thread execute a thunk at any moment, even when it is sleeping or waiting (for a mutex, etc). This can be combined with current-processor to determine on which processor a thread is running:
(define t (make-thread (lambda () (let loop () (loop))))) (thread-start! t)
#<thread #2>
(thread-interrupt! t current-thread)
#<thread #2>
(thread-interrupt! t current-processor)
#<processor #3>
(processor-id (thread-interrupt! t current-processor))
2
(thread-terminate! t) (##current-vm-processor-count)
4
(##cpu-count)
8
Marc
On Mar 24, 2017, at 9:00 AM, Adam adam.mlmb@gmail.com wrote:
Hi Marc,
Can you give some introduction to how the new SMP features are used?
So the gsi/gsc argument -:pN specifies how many GVM processors (i.e. OS threads) should be spawned. That one is not yet in the manual (http://www.iro.umontreal.ca/~gambit/doc/gambit.html#Threads).
And then the green threads run spread over the GVM processors?
How do you enumerate the GVM processors? How do you glue particular green threads or green thread groups to a particular GVM processor, if desired?
What more SMP-specific forms and features are there?
Thanks!
2017-03-24 15:02 GMT+01:00 Marc Feeley feeley@iro.umontreal.ca:
The smp branch is currently a WIP with some unimplemented features (such as thread priorities and priority inheritance). There are also some infrequent bugs that show up when stress testing. Nevertheless it is quite usable for experimentation and I am actively working on ironing out the bugs. I expect a stable smp branch by the end of April.
Great!
Eventually I want the smp branch to be merged with the master branch and to make --enable-multiple-threaded-vms the default. This will only happen after extensive portability testing so that the master branch remains very stable and can be built “out of the box” without any issues and tinkering.
Ok!
So to answer your questions… You need to build the smp branch to get parallel execution. These steps should work:
git clone https://github.com/gambit/gambit.git cd gambit git checkout smp ./configure make -j4 current-gsc-boot ./configure --enable-single-host --enable-multiple-threaded-vms make -j4 from-scratch make check
Great - building either branch of latest commit 210c66cc1d885734bb62a8025fef72801d142d84 as follows, did not build, will try the way you suggested.
$ cd gambit-smp $ git checkout smp $ ./configure --enable-single-host --enable-multiple-threaded-vms $ make [..] gcc -pthread -Wno-unused -Wno-write-strings -O1 -fwrapv -fno-strict-aliasing -fno-trapping-math -fno-math-errno -fschedule-insns2 -fomit-frame-pointer -fPIC -fno-common -mieee-fp -mpc64 -pthread -I"../include" -c -o "_kernel.o" -I. -DHAVE_CONFIG_H -D___GAMBITDIR=""/usr/local/Gambit"" -D___SYS_TYPE_CPU=""x86_64"" -D___SYS_TYPE_VENDOR=""unknown"" -D___SYS_TYPE_OS=""linux-gnu"" -D___CONFIGURE_COMMAND=""./configure '--enable-single-host' '--enable-multiple-threaded-vms'"" -D___OBJ_EXTENSION="".o"" -D___EXE_EXTENSION="""" -D___BAT_EXTENSION="""" -D___PRIMAL _kernel.c -D___LIBRARY _kernel.c: In function ‘___H__20___kernel’: _kernel.c:6346:27: error: void value not ignored as it ought to be ___FLONUM_VAL(___ARG2) = ___set_heartbeat_interval (___FLONUM_VAL(___ARG1)); ^ makefile:164: recipe for target '_kernel.o' failed make[1]: *** [_kernel.o] Error 1 $ gcc --version gcc (Debian 4.9.2-10) 4.9.2
The process as given works for me here (albeit with a more modern GCC 6.3.0).
James
On Sat, Mar 25, 2017 at 6:15 AM, Adam adam.mlmb@gmail.com wrote:
2017-03-24 15:02 GMT+01:00 Marc Feeley feeley@iro.umontreal.ca:
The smp branch is currently a WIP with some unimplemented features (such as thread priorities and priority inheritance). There are also some infrequent bugs that show up when stress testing. Nevertheless it is quite usable for experimentation and I am actively working on ironing out the bugs. I expect a stable smp branch by the end of April.
Great!
Eventually I want the smp branch to be merged with the master branch and to make --enable-multiple-threaded-vms the default. This will only happen after extensive portability testing so that the master branch remains very stable and can be built “out of the box” without any issues and tinkering.
Ok!
So to answer your questions… You need to build the smp branch to get parallel execution. These steps should work:
git clone https://github.com/gambit/gambit.git cd gambit git checkout smp ./configure make -j4 current-gsc-boot ./configure --enable-single-host --enable-multiple-threaded-vms make -j4 from-scratch make check
Great - building either branch of latest commit 210c66cc1d885734bb62a8025fef72801d142d84 as follows, did not build, will try the way you suggested.
$ cd gambit-smp $ git checkout smp $ ./configure --enable-single-host --enable-multiple-threaded-vms $ make [..] gcc -pthread -Wno-unused -Wno-write-strings -O1 -fwrapv -fno-strict-aliasing -fno-trapping-math -fno-math-errno -fschedule-insns2 -fomit-frame-pointer -fPIC -fno-common -mieee-fp -mpc64 -pthread -I"../include" -c -o "_kernel.o" -I. -DHAVE_CONFIG_H -D___GAMBITDIR=""/usr/local/Gambit"" -D___SYS_TYPE_CPU=""x86_64"" -D___SYS_TYPE_VENDOR=""unknown"" -D___SYS_TYPE_OS=""linux-gnu"" -D___CONFIGURE_COMMAND=""./configure '--enable-single-host' '--enable-multiple-threaded-vms'"" -D___OBJ_EXTENSION="".o"" -D___EXE_EXTENSION="""" -D___BAT_EXTENSION="""" -D___PRIMAL _kernel.c -D___LIBRARY _kernel.c: In function ‘___H__20___kernel’: _kernel.c:6346:27: error: void value not ignored as it ought to be ___FLONUM_VAL(___ARG2) = ___set_heartbeat_interval (___FLONUM_VAL(___ARG1)); ^ makefile:164: recipe for target '_kernel.o' failed make[1]: *** [_kernel.o] Error 1 $ gcc --version gcc (Debian 4.9.2-10) 4.9.2
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list