[gambit-list] Google Summer of Code

Christian Jaeger christian at pflanze.mine.nu
Tue Mar 11 16:49:57 EDT 2008


Andrew Lentvorski wrote:
> Christian Jaeger wrote:
>
>   
>>  From what I've seen, the experiences of the GHC implementors working on 
>> a parallel system (and at first, iirc, a concurrent GC?) may be 
>> worthwhile readings.
>>     
>
> How does Erlang handle this?  The latest versions handle 
> multicore/multiprocessor if I remember correctly.
>   

Just to make clear about what we are talking: Erlang "processes" do not 
share state (in the form of mutations that could be directly visible in 
other processes), from the point of view of a user they are *not* shared 
memory threads. And, not only does the language not offer direct 
cross-process mutations, but their implementation traditionally did use 
a separate heaps for every process, even in their user-space scheduled 
implementation, so not even the *implementation* is/has been using 
shared memory (except probably for (immutable) program code). As long as 
they keep it that way, it's dead easy running multiple OS threads, you 
simply use the OS communication instruments for passing messages between 
the threads. Whether using OS threads is significantly faster than OS 
processes in this case is what I have been wondering about several 
times, numbers would be interesting here (I guess it would eliminate (or 
reduce) the costs of setting up MMU tables during context switches 
between OS threads of the same process, other than that, I see no reason 
for a speedup).

Now Scheme is obviously different in that it offers mutation and if you 
want to make the mutations been passed over to other processes (like the 
heavy math vector calculation people would like), the shared-nothing 
approach doesn't work. (I've already suggested using separate shared 
memory for such tasks.)

But on top of that, even if you're using Termite (which does not support 
mutation): even if you strip shared-memory mutation from Scheme 
completely (e.g. apply the Termite semantics to the underlying Scheme 
system), there is one thing left: Termite is so fast in passing big data 
structures between Termite "processes" (Scheme threads) because Termite 
"processes" are employing underlying shared memory storage. So no 
serialization/deserialization has to be done for local data transfers. 
Now you can't simply run multiple Termite "threads" (i.e. underlying 
Scheme threads) on different OS threads anymore, because you've got a 
heap that needs synchronization. You now need a concurrent GC etc., 
something like what non-green threaded Java virtual machines do (or 
something less scalable like the global interpreter lock of Python).

Now that being said, Erlang people have been discussing moving to using 
shared heaps, just like Termite. Now the interesting question is: have 
they managed to efficiently implements heaps being shared *across OS 
threads*? Note that if they did not (e.g. one heap is only ever 
green-threaded), they are simply using the Gambit approach, but saving 
the above mentioned MMU overhead. This would be the approach of running 
multiple Gambit interpreters in separate pthreads in the same OS 
process. This should be quite easy (but, as mentioned, I wonder how much 
of a difference that will make?). Do you have pointers to corresponding 
Erlang posts/docs?

>   
>> BTW the chicken people have also had some interesting discussion about 
>> GSoC participation in the last few days (don't be sad about the 
>> difference in volume), also including a few points about threading. 
>> Gambit does have a better chance of doing fancy stuff with garbage 
>> collection (because of it handling the continuation stack explicitely), 
>> of course.
>>     
>
> Is there some documentation about this somewhere?  I'd love to read it.
>   

(I've got that knowledge gradually over time from experiments / mailing 
list posts / reading the sources. I suggest to start by reading the 
paper Marc has posted on March 5th.)

>>> - web framework
>>>   
>>>       
>> Should I mentor on this?
>>   I've actually got some actual professional (well, that doesn't mean 
>> particularly successful) experience in this area (with an XML 
>> background). But I'd rather prefer having a module system done before 
>> continuing writing suboptimally modularized code as I did up to now.
>>     
>
> I'm not sure a "web framework" is that great an idea.  "Frameworks" work 
> best when they are scratching a particular itch.  In addition, any good 
> "web framework" for Scheme is likely to be not very Gambit specific.
>   

Well, of course I've got some ideas on how that framework should look like.

It could be open to plug Termite in.

Of course you've got a point in that you could most probably port it to 
other Scheme systems, and the same of course applies to frameworks 
developed on other systems, they could be ported to Gambit. But if there 
is no Scheme system agnostic organization out there willing to take on 
the task of implementing such libraries, then this is being left to the 
individual system development organizations. I see no reason for Gambit 
playing purely a library consumer, as opposed to contributor as everyone 
else. Of course creating such a framework could also leverage (by 
porting) *parts* of other projects. Two things to keep in mind are: (a) 
system specifics to make something efficient; (b) module system 
specifics to make something particularly elegant. Part (a) could help 
show off Gambit's performance, of course (but yeah, is there a user out 
there right now who wants this?), part (b) is why I'd prefer seeing 
module stuff done first.

Christian.




More information about the Gambit-list mailing list