Blake Winton wrote:
Andrew Lentvorski wrote:
Blake Winton wrote:
My understanding (as a long time Python user) is that Python isn't interested in the billion-threads "solution" to multi-processing, (Guido seems to think, and I agree with him, that multi-threaded programs are too hard/impossible to get correct) and would rather see a different "fix" for the problem, whatever that may be. Shared-nothing/Multiple-processes seems like a contender.
Starting up multiple Python processes is *very* heavyweight. Communicating between those processes is also heavyweight.
The Python "solution" of starting multiple processes and talking through sockets is, quite simply, garbage.
Let's take a step back here. I'm not Python. Saying that something may be a contender is a far cry from calling it a "solution". I never said that the processes would communicate through sockets. (Unix pipes or shared memory would be far better for things on the same box!)
Sorry, I was under the impression that you had witnessed one of these arguments firsthand from the Python community. I have seen the Python community propose that as a solution multiple times.
Unix pipes or shared memory are not obviously better on the same box anymore. Both of those solutions require significant intervention from the OS kernel and are usually fairly unoptimized(read: painfully slow and limited). By contrast, network stacks are *heavily* optimized an often have zero-copy modes in which pointers to memory are passed around with very little overhead.
Neither of us has any idea what the startup time of Python will be in the future, nor how heavyweight inter-process communication will be.
That's disingenuous. Of course we do. It's some factor within what it is now barring miracles. Aka: painfully slow.
You seem to have an axe to grind, and while I appreciate that none of the current languages do concurrency particularly well, I think you've missed the mark in this particular criticism.
I absolutely have an axe to grind. I want a good language that handles concurrency well. Nothing particularly fits right now.
The issue isn't "threads" or processes per se. Threads or processes could be fine, it's just that they're too heavy on an OS. Look at all the optimization and work people put in to make Apache scale on an OS.
Which OS is that? ;) You have to believe that, as processors gain more and more cores, OSes will react by making process creation and communication far lighter-weight, just like JVM implementors made synchronizing on objects lighter and lighter-weight as people used it more.
Why do I have to believe that? None of the modern OS's seem to be heading that direction. User responsiveness is a far more compelling application (think uninterrupted audio and video) than concurrency communication. OS's have plenty of background tasks to soak up those cores so they don't really care.
And if you're going to look at web servers, why not look at some which use Shared-nothing/Multiple-processes, like Yaws (found at http://yaws.hyber.org/ )? They seem to have beaten the pants off of Apache, according to http://www.sics.se/~joe/apachevsyaws.html
I have, but these systems have their own quirks. In particular, everything is in the Erlang world. The moment you have to leave it (think mod_XXXX), your performance drops to the same problems as Apache.
The real issue is that no current language except Erlang supports message passing as an easy primitive between "threads".
So you do like Erlang's shared-nothing/multiple-processes approach?
Yes. But I'd like to see other languages (thing like Python, Ruby, etc.) at least *try*--and maybe even try other solutions. Python has simply dismissed the problem.
A solution to concurrency will be the issue that dethrones the current languages. I know that I'm looking.
I'm sure the entire programming language community will be interested in anything you find. Seriously. It seems to be considered one of the major problems of our age on every language-related mailing list I'm on, and everyone is looking for a halfway-decent answer. (Not even _the_ answer, just something that's better than what we have now.)
Right now, Erlang is it. It's okay, but I didn't really like it for general programming. It's kind of a strange combination of static and dynamic (records are static). It works *great* for network stack-like programs, though. It reduced about 10,000 lines of Java newtork stack code to about 1000 lines of Erlang. Big, *BIG* win.
I just wish there was another Erlang implementation. I've looked at the VM code for Erlang, and it's painful. Reimplementing from scratch is probably going to be easier.
-a