[gambit-list] Qwaq releases Hydra multi-core Squeak VM « The

Christian Jaeger christian at pflanze.mine.nu
Tue Feb 26 06:30:15 EST 2008


Joel Borggrén-Franck wrote:
> I wouldn't go as far as claiming that Python has failed, rather that
> the solutions is far from perfect. The Python interpreter suffers from
> its Global Interpreter Lock that hinders the smooth operation of large
> amounts of threads.

How are you going to improve it? What will happen with an 
ever-increasing number of cores?

> Gambits model can be different though. In gambit, in theory, you only
> need a fixed amount of pthreds, maybe 1-2 per core, then you multiplex
> your gambit processes over the pthreads.

This sounds like the classical N:M threading idea. It's not clear this 
is an advantage over just only using native threads, as the NPTL people 
have shown. Letting *one* scheduler (the one in the OS kernel) handle 
everything was more efficient. ("Machine"/"C style" threads require 
separate continguous stacks, though, and this always pays the overhead 
of in average 1/2 block of non-used data per thread. Gambit's kernel 
does better than that. You'd have to give up that advantage when using 
only those native threads. But could you keep the advantage even with a 
N:M model?)

OS threads have the disadvantage of not knowing about your language. And 
  I'm not sure how OS threads (aka pthreads) will solve the problem of 
the ever-increasing number of cpus. To prevent 
communication/synchronization overhead between cpus becoming a 
bottleneck quickly, some hierarchical interconnection mechanism is 
necessary. It's not that just magically in the future every pthread can 
communicate with any other running on another cpu as efficiently as it 
can do in your dual core machine now, I think.

I suppose, the more cores you have, the more will communication between 
cpu's start looking like socket communication between processes. After 
ignoring all the terminology, a multicore machine is basically nothing 
else than a more closely connected cluster. Thus afaict, the multicore 
architectures, once reaching a big number of cores, always ever only 
have a constant advantage over clusters. Only creating a hierarchy is 
making more than a constant advantage (multicore for fast local 
communication, clustering for the next layer), so I suppose some time in 
the future one machine will contain *several* layers of calculation 
engines, some more closely tied to each other than others, to continue 
this pyramide.

I don't know very much about clustering, and even less about using 
clusters of smp machines, but I guess if you want to make best use of 
both layers (cluster and smp, in the mentioned pyramide-like way), this 
is manual work.

> I think this is the model the
> Erlang VM uses.

Ignorant question: does Erlang have a shared heap? Any shared data at 
all? Except in Mnesia maybe?

Christian.



More information about the Gambit-list mailing list