[gambit-list] Google Summer of Code

Christian Jaeger christian at pflanze.mine.nu
Sun Mar 9 23:12:01 EDT 2008


(I've stripped the snow and mslug list addresses, but added François 
Pinard because of the Pymacs reference.)


(I'm not a student.)

> If you cannot participate (e.g. you are not a student) but have  
> project suggestions, then please speak up on the mailing list.
>
> Some projects ideas that come to mind:
>
> - native thread support
> - real-time/parallel GC
>   

It's difficult for me to say anything about threading. I think it's a 
challenging topic (and I've got no multiprocessor programming experience).

There are two things that I think could be useful for someone like me 
who was thinking about using threading on top of the Gambit core (as 
libraries), maybe this will also be helping integration with whatever 
grows up in the core:

- it would be useful to have a little portable framework for 
synchronization of memory contents (barriers) and atomic ops, including 
documentation. I guess gcc 4.2(.2+) may already offer all we need, but 
maybe alternatives for users of older gcc's or non-gcc compilers could 
be collected, and especially the knowhow about correct memory handling, 
and the cost patterns involved would be useful (with comparisons between 
different architectures, esp. between NUMA (e.g. multi-Opteron) and 
uniform memory access (SMP like Athlon 64 X2 or FX or Intel Core 2) 
machines).

- maybe a possibility to integrate mutexes into the Gambit scheduler 
(especially Linux' futex'es, which may already be used by the pthread 
mutex implementation in glibc now); i.e. make it possible for a Gambit 
green thread to block on such a mutex without also blocking other green 
threads. On top of that, one could implement an extension of the FFI 
(even as a library) which does FFI calls "into" a separate pthread, in 
other words: the Gambit green thread would kick off a (potentionally 
blocking) call in another (possibly pooled) pthread, and wait on the 
mutex for completion, during which time other green threads are being 
run by the Gambit scheduler. That's achieving the same thing as using 
pairs of unix pipe(2)'s or a socket(2) pair (and writing the procedure 
and argument addresses out to the foreign thread and reading the 
(address of the) result back after the call) and registering the 
pipe/socket in the Gambit runtime, but using futexes instead of pipes 
*might* make it faster (but then maybe not); obviously such a layer 
would solve the problem of FFI calls blocking green threads. BTW iirc 
GHC(?) does implement this (I don't know details) in it's FFI (including 
a flag passed to each foreign function definition whether it's a quick 
or potentionally blocking call).

(PS. Brad, in case you didn't know already: gcc 4.2(.2+) also has 
"vector instructions through built-in functions" (access to MMX, 3Dnow! 
and SSE), which might possibly be interesting for you. But then, maybe 
not if your problems are memory bandwidth limited.)

 From what I've seen, the experiences of the GHC implementors working on 
a parallel system (and at first, iirc, a concurrent GC?) may be 
worthwhile readings.

BTW the chicken people have also had some interesting discussion about 
GSoC participation in the last few days (don't be sad about the 
difference in volume), also including a few points about threading. 
Gambit does have a better chance of doing fancy stuff with garbage 
collection (because of it handling the continuation stack explicitely), 
of course.

> - Gambit packages
>   

Hm, should I try to offer mentoring:
  * working on chjmodule (renaming it at that occasion)?
  * working on some other module system?

I see that mentors must be listed already at the organization 
application. I'll release my current code quickly now (give me 24h). 
Sorry about being so late.

> - web framework
>   

Should I mentor on this?
  I've actually got some actual professional (well, that doesn't mean 
particularly successful) experience in this area (with an XML 
background). But I'd rather prefer having a module system done before 
continuing writing suboptimally modularized code as I did up to now.

> - documentation
>   

What comes to mind re documentation:

- make Gambit internals more approachable. (Documentation, compilation 
in safe mode, runtime reloading of separately compiled system parts.)

Other ideas:

- implement ability to 'migrate' continuations from compiled code to 
interpreted code. I.e. you would capture (and possibly serialize) a 
continuation of an error situation which happened in compiled code, 
store it away, then when you get to look at the problem, you load all 
code that you think is relevant for debugging into the *interpreter* 
(uncompiled) and then let the system turn that continuation into one 
that calls the interpreted procedures instead of the compiled ones. The 
idea is to be able to debug the problem much better (like use the 
stepper, change procedures on the fly etc.). How impossible would that be?..

- especially if the above won't work at all: change the interpreter so 
that it analyzes the lifetime of lexical variables and doesn't hold on 
to objects in variables longer than up to their last reference, to 
prevent the memory retention problems that otherwise plague lazy code. 
And possibly while doing so: try to reduce the size of serialized 
continuations from interpreted code.

- optional step-back debugging ("reverse in time"), possibly as global 
syntax transformation hook, by storing away the last n continuations 
seen by the interpreter (or also compiled programs; that's why 
implementing it as transformation might be preferrable).

> - François Pinard's projects (see MSLUG mailing-list archive)
>   

- regarding the Gambit<->Python interaction over a pipe/socket: having 
written Gambit->Perl bindings (although still missing callbacks from 
Perl into Scheme), I can say that linking two languages in the same 
process through the FFI is certainly possible. But it currently misses 
the above-mentioned FFI-in-a-separate-pthread approach to prevent Perl 
code from blocking Gambit green threads. Also running the other language 
in a separate process is safer against crashes, of course (although this 
hasn't been a problem for me so far). I did it in-process because I 
didn't want to risk performance problems. For example I don't normally 
convert a perl hashtable into a Gambit table when being returned from a 
call to perl code, but instead return a foreign object around the perl 
object, on which the Scheme code can then call perl's hashtable 
operators to get at the individual values. That saves the overhead of 
conversion of the whole table if you're only interested in a few values. 
The round trip times over a socket may make that relatively costly. (It 
will be better with Termite+multiple processes and a common shared 
read-only storage, since you can then read from the common storage and 
save the round trips :). I'll help doing some benchmark if you want to 
compare.

- speaking of pymacs: wouldn't it be nice to have a Elisp<->Gambit 
interface? (schemacs or something). But about the same thing might also 
be achievable through a SLIME48 ported to Gambit, which itself might be 
another good candidate for a GSoC project.


Christian.




More information about the Gambit-list mailing list