[gambit-list] Qwaq releases Hydra multi-core Squeak VM « The

Blake Winton bwinton at latte.ca
Thu Feb 28 16:36:23 EST 2008


Andrew Lentvorski wrote:
> Blake Winton wrote:
>>> The Python "solution" of starting multiple processes and talking 
>>> through sockets is, quite simply, garbage.
>> Saying that something may be a contender is a far cry from calling it
 >> a "solution".
> Sorry, I was under the impression that you had witnessed one of these 
> arguments firsthand from the Python community.  I have seen the Python 
> community propose that as a solution multiple times.

Urls?  I can believe that they suggest it as preferable to multi-threading 
(working and slow is always better than fast and broken ;), or even the best 
solution for now, but I'm sure we both agree that it's hardly the best solution 
possible.

>> Unix pipes or shared memory would be far better for things on the same box!)
> Unix pipes or shared memory are not obviously better on the same box 
> anymore.
[snip...]

Cool...  It's good to hear that network-y things are getting faster.

>> Neither of us has any idea what the startup time of Python will be in 
>> the future, nor how heavyweight inter-process communication will be.
> That's disingenuous.  Of course we do.  It's some factor within what it 
> is now barring miracles.  Aka: painfully slow.

Perhaps, but if the future ends up being multi-process and sockets, it would 
hardly surprise me to see heavy work get done on optimizing Python's startup, or 
native-code compilation, or whatever else will relieve that particular pain.

>> You seem to have an axe to grind, and while I appreciate that none of 
>> the current languages do concurrency particularly well, I think you've 
>> missed the mark in this particular criticism.
> I absolutely have an axe to grind.  I want a good language that handles 
> concurrency well.  Nothing particularly fits right now.

We completely agree on that.  I'ld love to see something that works better than 
what we've got, and if it could manage to be a library on top of an existing 
language, so much the better.

>> You have to believe that, as processors gain more and more cores, OSes
 >> will react by making process creation and communication far lighter-weight
> Why do I have to believe that?  None of the modern OS's seem to be 
> heading that direction.

Yet.  But the multi-core era is just starting.  Also, didn't BeOS go that 
direction?  I thought they concentrated on allowing you to create billions of 
threads, and had some impressive demos as a result.

 > User responsiveness is a far more compelling
> application (think uninterrupted audio and video) than concurrency 
> communication.  OS's have plenty of background tasks to soak up those 
> cores so they don't really care.

How many background tasks do they have?  My WinXP box seems to have 32-ish.  My 
OSX box has 72.  They're both fewer than the 64 and 128 core processors I expect 
to see in my lifetime.  At some point they're going to run out, and then what?

>> And if you're going to look at web servers, why not look at some which 
>> use Shared-nothing/Multiple-processes, like Yaws (found at 
>> http://yaws.hyber.org/ )?  They seem to have beaten the pants off of 
>> Apache, according to http://www.sics.se/~joe/apachevsyaws.html
> I have, but these systems have their own quirks.  In particular, 
> everything is in the Erlang world.  The moment you have to leave it 
> (think mod_XXXX), your performance drops to the same problems as Apache.

I'm sure I've heard of a library that talks to Erlang nodes, so that you could 
write your programs in whatever you want, and hook them in to Erlang...

My point was more that the work the Apache people are doing to make Apache scale 
is beside the point, since a better concurrency paradigm is beating the pants 
off of them in terms of scalability.

>>> The real issue is that no current language except Erlang supports 
>>> message passing as an easy primitive between "threads".
>> So you do like Erlang's shared-nothing/multiple-processes approach?
> Yes.  But I'd like to see other languages (thing like Python, Ruby, 
> etc.) at least *try*--and maybe even try other solutions.

What (other than STM) are the other solutions in this space?

 > Python has simply dismissed the problem.

I don't see that.  My reading of it is that the Python community has dismissed 
the "solution" of shared-state multi-threading.  (And if you're not sharing 
state, are they really threads anymore?)  But if someone comes up with another 
solution, I'm sure the Python community would be more than happy to implement it.

>>> A solution to concurrency will be the issue that dethrones the 
>>> current languages.  I know that I'm looking.
>> I'm sure the entire programming language community will be interested 
>> in anything you find.
> Right now, Erlang is it.  It's okay, but I didn't really like it for 
> general programming.

Hmm.  I haven't used it for general programming, but that doesn't surprise me 
terribly.  It looked neat, but not necessarily practical.

 > It's kind of a strange combination of static and dynamic (records are static).

It's my understanding that that's where it gets its main wins.  The data is 
static, so you don't need to copy it all over the place, or worry about thread 
contention, but the code is dynamic so you can do something...  Uh, that would 
be the point where I would start waving my hands.  ;)

> I just wish there was another Erlang implementation.  I've looked at the 
> VM code for Erlang, and it's painful.  Reimplementing from scratch is 
> probably going to be easier.

If only there was an implementation of the Erlang ideas in, say, Scheme.  ;)

Later,
Blake.



More information about the Gambit-list mailing list