[gambit-list] SMP works up to what point in GSC, any practical use possible currently?

Adam adam.mlmb at gmail.com
Mon Aug 21 11:01:53 EDT 2017


Wow. This is a big day. Thank you.


2017-08-21 21:13 GMT+08:00 Marc Feeley <feeley at iro.umontreal.ca>:

>
> > On Aug 21, 2017, at 8:11 AM, Adam <adam.mlmb at gmail.com> wrote:
> >
> > Hi Marc,
> >
> > Aha - so - the garbage collector can run in parallell as of today.
> >
> > Here are some more in depth questions (also hopefully more in line with
> your terminology):
> >
> >
> > Can objects/heap be shared between (processors in different) VM:s?
> >
>
> No, by design.  Each VM has its independent GC and runtime system.  Very
> few things are shared… the heartbeat timer (this is an OS constraint) and
> symbols.
>
> >
> > How far has the work on multiple processors in one VM
> (--enable-multiple-threaded-vms) gone? Specifically:
> >
> > Can different processors which are running in one VM, share heap/objects
> reliably?
>
> Yes this is reliable.  In other words, the GC is reliable on multiple
> threaded VMs.
>
> >
> > Can multiple processors which are running in one VM execute [code] in
> parallel reliably, presuming all but the root/first processor only utilizes
> some given subset of the runtime (e.g. math and list processing is fine but
> no IO as the IO not is multiprocessor-safe yet)?
>
> Yes.  The subset is very large… mainly stay away from thread priorities…
> for now.
>
> >
> > Do routines to allocate/launch and stop/collect/deallocate (more)
> processors in one VM exist & work reliably?
>
> Yes. The only part that is not thoroughly tested is the handling of errors
> when the VM is resized (i.e. resizing the VM from 1 to 8 processors, what
> happens when the OS fails to allocate the OS thread for processor #4?).
> The handling code is there to detect this but it isn’t graceful (fatal
> error) and it hasn’t been tested much.
>
> The call (##current-vm-resize ##startup-processor! N) where N is the new
> size will do this, but it is an internal function for implementing the VM
> that shoudn’t be used if you don’t know what you are doing…
>
> >
> > Do routines to send messages between multiple processors in one VM exist
> & work reliably?
>
> Yes.  Here’s a simple example with thread-send :
>
> (declare (standard-bindings) (block))
>
> (define (busy-sleep n)
>   (if (> n 0)
>       (busy-sleep (- n 1))
>       #f))
>
> (define (short-delay)
>   (busy-sleep 100000)) ;; about 100 microseconds
>
> (define (go n)
>
>   (define (ring next-thread)
>     (let loop ()
>       (let ((msg (thread-receive)))
>         (thread-send next-thread (- msg 1))
>         (short-delay)
>         (if (> msg 0)
>             (loop)))))
>
>   (letrec ((t1 (make-thread (lambda () (ring t2))))
>            (t2 (make-thread (lambda () (ring t3))))
>            (t3 (make-thread (lambda () (ring t4))))
>            (t4 (make-thread (lambda () (ring t1)))))
>
>     (for-each thread-start! (list t1 t2 t3 t4))
>
>     (thread-send t1 n)
>
>     (for-each thread-join! (list t1 t2 t3 t4))))
>
> (time (go 1000000))
>
> When this is run in a 4 processor VM, all the processors are kept working
> at a high percentage (~ 80%), so the speedup over a 1 processor VM is close
> to 4. See the parallelism profile generated by xactlog below.
>
> Other interprocessor communication mechanisms also work (mutexes,
> condition variables, etc).
>
> >
> > Do routines to facilitate work stealing between multiple processors in
> one VM exist & work reliably?
> >
>
> Yes work stealing is implemented.
>
> >
> > I presume all the above is in the smp branch. Given some common sense
> testing for a particular git commit, can the smp branch be counted as
> reliable?
>
> The SMP branch was combined with the master branch yesterday.  The
> configure option --enable-smp will enable the SMP Scheme thread scheduler
> (but you have to “make;make bootclean;make” to activate it).  You also need
> --enable-multiple-threaded-vms.
>
> While I’m on this subject, --enable-multiple-threaded-vms will become the
> new default after the next release.  So those who prefer to not use the
> parallel GC should start adding --disable-multiple-threaded-vms in their
> build process.
>
> >
> > Also given common sense testing, can the main branch be counted as
> reliable?
> >
>
> The master branch contains the latest development patches, so it should
> not be considered maximally reliable.  It happens on occasion that a patch
> breaks some existing infrequently used feature.  If you want the highest
> reliability use a release.
>
> >
> > When in the future do you think the runtime including IO will be proofed
> for use across multiple processors in one VM?
>
> On my TODO over the next 6 months.  The SMP Scheme thread scheduler is
> already in good shape.  The time consuming part (yet to be done) is
> implementing thread priorities and fine tuning and testing the runtime
> system.
>
> >
> > Thanks and best regards,
> > Adam
>
> Marc
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20170821/f11bef80/attachment.htm>


More information about the Gambit-list mailing list