On Tue, Feb 26, 2008 at 12:30 PM, Christian Jaeger christian@pflanze.mine.nu wrote:
Joel Borggrén-Franck wrote:
I wouldn't go as far as claiming that Python has failed, rather that the solutions is far from perfect. The Python interpreter suffers from its Global Interpreter Lock that hinders the smooth operation of large amounts of threads.
How are you going to improve it? What will happen with an ever-increasing number of cores?
You could for example use a more fine-grained locks. This will allow you to use more threads, but won't change the time it takes to spawn them, so it solves part of the problem. Now I'm not going to argue that using one os-threads per language-thread is a good thing, but it can be made to work for large number of cores (presumably with a lot of work), there are instances of java running on 700+ cores using thousands of threads in a single image. This is not how I envision the future of concurrent programming, but it is working, right now, hence I won't name it 'failure'.
Gambits model can be different though. In gambit, in theory, you only need a fixed amount of pthreds, maybe 1-2 per core, then you multiplex your gambit processes over the pthreads.
This sounds like the classical N:M threading idea. It's not clear this is an advantage over just only using native threads, as the NPTL people have shown. Letting *one* scheduler (the one in the OS kernel) handle everything was more efficient.
From what I can see, the gambit model promises fast thread spawning,
and this is something we want to keep. Now this can not be done with os-threads, so you simply can't use a 1:1 model, because no OS today delivers the spawn-times we want. Also NPTL is supposed to be a general purpose thread-library, I imagine you can cut some corners when you know more of your target system.
Also there is the question of relative efficiency and programmer/machine efficiency. I do believe there can be a place for a N:M model, at least in the short run.
OS threads have the disadvantage of not knowing about your language. And I'm not sure how OS threads (aka pthreads) will solve the problem of the ever-increasing number of cpus.
And green threads usually have the disadvantage of not beeing schedulable over different processors. This leaves us with the only option of using multiple gambit processes scheduled over the cores (presumably glued together by termite), OR use an N:M scheme OR using both of them.
Also in the foreseeable future we will only have a bounded number of cores.
I think this is the model the Erlang VM uses.
Ignorant question: does Erlang have a shared heap? Any shared data at all? Except in Mnesia maybe?
Though far from an Erlang expert, i believe it's supposed to not share anything. The native thread implementation uses one erlang scheduler per thread, and in the initial release only one os-thread could have outstanding I/O at anytime (if I got my facts straight).