[gambit-list] SMP works up to what point in GSC, any practical use possible currently?
Marc Feeley
feeley at iro.umontreal.ca
Mon Aug 21 09:13:16 EDT 2017
> On Aug 21, 2017, at 8:11 AM, Adam <adam.mlmb at gmail.com> wrote:
>
> Hi Marc,
>
> Aha - so - the garbage collector can run in parallell as of today.
>
> Here are some more in depth questions (also hopefully more in line with your terminology):
>
>
> Can objects/heap be shared between (processors in different) VM:s?
>
No, by design. Each VM has its independent GC and runtime system. Very few things are shared… the heartbeat timer (this is an OS constraint) and symbols.
>
> How far has the work on multiple processors in one VM (--enable-multiple-threaded-vms) gone? Specifically:
>
> Can different processors which are running in one VM, share heap/objects reliably?
Yes this is reliable. In other words, the GC is reliable on multiple threaded VMs.
>
> Can multiple processors which are running in one VM execute [code] in parallel reliably, presuming all but the root/first processor only utilizes some given subset of the runtime (e.g. math and list processing is fine but no IO as the IO not is multiprocessor-safe yet)?
Yes. The subset is very large… mainly stay away from thread priorities… for now.
>
> Do routines to allocate/launch and stop/collect/deallocate (more) processors in one VM exist & work reliably?
Yes. The only part that is not thoroughly tested is the handling of errors when the VM is resized (i.e. resizing the VM from 1 to 8 processors, what happens when the OS fails to allocate the OS thread for processor #4?). The handling code is there to detect this but it isn’t graceful (fatal error) and it hasn’t been tested much.
The call (##current-vm-resize ##startup-processor! N) where N is the new size will do this, but it is an internal function for implementing the VM that shoudn’t be used if you don’t know what you are doing…
>
> Do routines to send messages between multiple processors in one VM exist & work reliably?
Yes. Here’s a simple example with thread-send :
(declare (standard-bindings) (block))
(define (busy-sleep n)
(if (> n 0)
(busy-sleep (- n 1))
#f))
(define (short-delay)
(busy-sleep 100000)) ;; about 100 microseconds
(define (go n)
(define (ring next-thread)
(let loop ()
(let ((msg (thread-receive)))
(thread-send next-thread (- msg 1))
(short-delay)
(if (> msg 0)
(loop)))))
(letrec ((t1 (make-thread (lambda () (ring t2))))
(t2 (make-thread (lambda () (ring t3))))
(t3 (make-thread (lambda () (ring t4))))
(t4 (make-thread (lambda () (ring t1)))))
(for-each thread-start! (list t1 t2 t3 t4))
(thread-send t1 n)
(for-each thread-join! (list t1 t2 t3 t4))))
(time (go 1000000))
When this is run in a 4 processor VM, all the processors are kept working at a high percentage (~ 80%), so the speedup over a 1 processor VM is close to 4. See the parallelism profile generated by xactlog below.
Other interprocessor communication mechanisms also work (mutexes, condition variables, etc).
>
> Do routines to facilitate work stealing between multiple processors in one VM exist & work reliably?
>
Yes work stealing is implemented.
>
> I presume all the above is in the smp branch. Given some common sense testing for a particular git commit, can the smp branch be counted as reliable?
The SMP branch was combined with the master branch yesterday. The configure option --enable-smp will enable the SMP Scheme thread scheduler (but you have to “make;make bootclean;make” to activate it). You also need --enable-multiple-threaded-vms.
While I’m on this subject, --enable-multiple-threaded-vms will become the new default after the next release. So those who prefer to not use the parallel GC should start adding --disable-multiple-threaded-vms in their build process.
>
> Also given common sense testing, can the main branch be counted as reliable?
>
The master branch contains the latest development patches, so it should not be considered maximally reliable. It happens on occasion that a patch breaks some existing infrequently used feature. If you want the highest reliability use a release.
>
> When in the future do you think the runtime including IO will be proofed for use across multiple processors in one VM?
On my TODO over the next 6 months. The SMP Scheme thread scheduler is already in good shape. The time consuming part (yet to be done) is implementing thread priorities and fine tuning and testing the runtime system.
>
> Thanks and best regards,
> Adam
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gambit.actlog.pdf
Type: application/pdf
Size: 42706 bytes
Desc: not available
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20170821/61d4aa30/attachment.pdf>
-------------- next part --------------
More information about the Gambit-list
mailing list