[gambit-list] Qwaq releases Hydra multi-core Squeak VM « The

Joel Borggrén-Franck joel.borggren.franck at gmail.com
Thu Feb 28 05:21:16 EST 2008


On Tue, Feb 26, 2008 at 12:30 PM, Christian Jaeger
<christian at pflanze.mine.nu> wrote:

> Joel Borggrén-Franck wrote:
>  > I wouldn't go as far as claiming that Python has failed, rather that
>  > the solutions is far from perfect. The Python interpreter suffers from
>  > its Global Interpreter Lock that hinders the smooth operation of large
>  > amounts of threads.
>
>  How are you going to improve it? What will happen with an
>  ever-increasing number of cores?
>

You could for example use a more fine-grained locks. This will allow
you to use more threads, but won't change the time it takes to spawn
them, so it solves part of the problem. Now I'm not going to argue
that using one os-threads per language-thread is a good thing, but it
can be made to work for large number of cores (presumably with a lot
of work), there are instances of java running on 700+ cores using
thousands of threads in a single image. This is not how I envision the
future of concurrent programming, but it is working, right now, hence
I won't name it 'failure'.

>
>  > Gambits model can be different though. In gambit, in theory, you only
>  > need a fixed amount of pthreds, maybe 1-2 per core, then you multiplex
>  > your gambit processes over the pthreads.
>
>  This sounds like the classical N:M threading idea. It's not clear this
>  is an advantage over just only using native threads, as the NPTL people
>  have shown. Letting *one* scheduler (the one in the OS kernel) handle
>  everything was more efficient.
>

>From what I can see, the gambit model promises fast thread spawning,
and this is something we want to keep. Now this can not be done with
os-threads, so you simply can't use a 1:1 model, because no OS today
delivers the spawn-times we want. Also NPTL is supposed to be a
general purpose thread-library, I imagine you can cut some corners
when you know more of your target system.

Also there is the question of relative efficiency and
programmer/machine efficiency. I do believe there can be a place for a
N:M model, at least in the short run.

>
>  OS threads have the disadvantage of not knowing about your language. And
>   I'm not sure how OS threads (aka pthreads) will solve the problem of
>  the ever-increasing number of cpus.
>

And green threads usually have the disadvantage of not beeing
schedulable over different processors. This leaves us with the only
option of using multiple gambit processes scheduled over the cores
(presumably glued together by termite), OR use an N:M scheme OR using
both of them.

Also in the foreseeable future we will only have a bounded number of cores.

>
>  > I think this is the model the
>  > Erlang VM uses.
>
>  Ignorant question: does Erlang have a shared heap? Any shared data at
>  all? Except in Mnesia maybe?
>

Though far from an Erlang expert, i believe it's supposed to not share
anything. The native thread implementation uses one erlang scheduler
per thread, and in the initial release only one os-thread could have
outstanding I/O at anytime (if I got my facts straight).



More information about the Gambit-list mailing list