Joel Borggrén-Franck wrote:
I wouldn't go as far as claiming that Python has failed, rather that the solutions is far from perfect. The Python interpreter suffers from its Global Interpreter Lock that hinders the smooth operation of large amounts of threads.
How are you going to improve it? What will happen with an ever-increasing number of cores?
Gambits model can be different though. In gambit, in theory, you only need a fixed amount of pthreds, maybe 1-2 per core, then you multiplex your gambit processes over the pthreads.
This sounds like the classical N:M threading idea. It's not clear this is an advantage over just only using native threads, as the NPTL people have shown. Letting *one* scheduler (the one in the OS kernel) handle everything was more efficient. ("Machine"/"C style" threads require separate continguous stacks, though, and this always pays the overhead of in average 1/2 block of non-used data per thread. Gambit's kernel does better than that. You'd have to give up that advantage when using only those native threads. But could you keep the advantage even with a N:M model?)
OS threads have the disadvantage of not knowing about your language. And I'm not sure how OS threads (aka pthreads) will solve the problem of the ever-increasing number of cpus. To prevent communication/synchronization overhead between cpus becoming a bottleneck quickly, some hierarchical interconnection mechanism is necessary. It's not that just magically in the future every pthread can communicate with any other running on another cpu as efficiently as it can do in your dual core machine now, I think.
I suppose, the more cores you have, the more will communication between cpu's start looking like socket communication between processes. After ignoring all the terminology, a multicore machine is basically nothing else than a more closely connected cluster. Thus afaict, the multicore architectures, once reaching a big number of cores, always ever only have a constant advantage over clusters. Only creating a hierarchy is making more than a constant advantage (multicore for fast local communication, clustering for the next layer), so I suppose some time in the future one machine will contain *several* layers of calculation engines, some more closely tied to each other than others, to continue this pyramide.
I don't know very much about clustering, and even less about using clusters of smp machines, but I guess if you want to make best use of both layers (cluster and smp, in the mentioned pyramide-like way), this is manual work.
I think this is the model the Erlang VM uses.
Ignorant question: does Erlang have a shared heap? Any shared data at all? Except in Mnesia maybe?
Christian.