[gambit-list] Google Summer of Code
Christian Jaeger
christian at pflanze.mine.nu
Sun Mar 9 23:12:01 EDT 2008
(I've stripped the snow and mslug list addresses, but added François
Pinard because of the Pymacs reference.)
(I'm not a student.)
> If you cannot participate (e.g. you are not a student) but have
> project suggestions, then please speak up on the mailing list.
>
> Some projects ideas that come to mind:
>
> - native thread support
> - real-time/parallel GC
>
It's difficult for me to say anything about threading. I think it's a
challenging topic (and I've got no multiprocessor programming experience).
There are two things that I think could be useful for someone like me
who was thinking about using threading on top of the Gambit core (as
libraries), maybe this will also be helping integration with whatever
grows up in the core:
- it would be useful to have a little portable framework for
synchronization of memory contents (barriers) and atomic ops, including
documentation. I guess gcc 4.2(.2+) may already offer all we need, but
maybe alternatives for users of older gcc's or non-gcc compilers could
be collected, and especially the knowhow about correct memory handling,
and the cost patterns involved would be useful (with comparisons between
different architectures, esp. between NUMA (e.g. multi-Opteron) and
uniform memory access (SMP like Athlon 64 X2 or FX or Intel Core 2)
machines).
- maybe a possibility to integrate mutexes into the Gambit scheduler
(especially Linux' futex'es, which may already be used by the pthread
mutex implementation in glibc now); i.e. make it possible for a Gambit
green thread to block on such a mutex without also blocking other green
threads. On top of that, one could implement an extension of the FFI
(even as a library) which does FFI calls "into" a separate pthread, in
other words: the Gambit green thread would kick off a (potentionally
blocking) call in another (possibly pooled) pthread, and wait on the
mutex for completion, during which time other green threads are being
run by the Gambit scheduler. That's achieving the same thing as using
pairs of unix pipe(2)'s or a socket(2) pair (and writing the procedure
and argument addresses out to the foreign thread and reading the
(address of the) result back after the call) and registering the
pipe/socket in the Gambit runtime, but using futexes instead of pipes
*might* make it faster (but then maybe not); obviously such a layer
would solve the problem of FFI calls blocking green threads. BTW iirc
GHC(?) does implement this (I don't know details) in it's FFI (including
a flag passed to each foreign function definition whether it's a quick
or potentionally blocking call).
(PS. Brad, in case you didn't know already: gcc 4.2(.2+) also has
"vector instructions through built-in functions" (access to MMX, 3Dnow!
and SSE), which might possibly be interesting for you. But then, maybe
not if your problems are memory bandwidth limited.)
From what I've seen, the experiences of the GHC implementors working on
a parallel system (and at first, iirc, a concurrent GC?) may be
worthwhile readings.
BTW the chicken people have also had some interesting discussion about
GSoC participation in the last few days (don't be sad about the
difference in volume), also including a few points about threading.
Gambit does have a better chance of doing fancy stuff with garbage
collection (because of it handling the continuation stack explicitely),
of course.
> - Gambit packages
>
Hm, should I try to offer mentoring:
* working on chjmodule (renaming it at that occasion)?
* working on some other module system?
I see that mentors must be listed already at the organization
application. I'll release my current code quickly now (give me 24h).
Sorry about being so late.
> - web framework
>
Should I mentor on this?
I've actually got some actual professional (well, that doesn't mean
particularly successful) experience in this area (with an XML
background). But I'd rather prefer having a module system done before
continuing writing suboptimally modularized code as I did up to now.
> - documentation
>
What comes to mind re documentation:
- make Gambit internals more approachable. (Documentation, compilation
in safe mode, runtime reloading of separately compiled system parts.)
Other ideas:
- implement ability to 'migrate' continuations from compiled code to
interpreted code. I.e. you would capture (and possibly serialize) a
continuation of an error situation which happened in compiled code,
store it away, then when you get to look at the problem, you load all
code that you think is relevant for debugging into the *interpreter*
(uncompiled) and then let the system turn that continuation into one
that calls the interpreted procedures instead of the compiled ones. The
idea is to be able to debug the problem much better (like use the
stepper, change procedures on the fly etc.). How impossible would that be?..
- especially if the above won't work at all: change the interpreter so
that it analyzes the lifetime of lexical variables and doesn't hold on
to objects in variables longer than up to their last reference, to
prevent the memory retention problems that otherwise plague lazy code.
And possibly while doing so: try to reduce the size of serialized
continuations from interpreted code.
- optional step-back debugging ("reverse in time"), possibly as global
syntax transformation hook, by storing away the last n continuations
seen by the interpreter (or also compiled programs; that's why
implementing it as transformation might be preferrable).
> - François Pinard's projects (see MSLUG mailing-list archive)
>
- regarding the Gambit<->Python interaction over a pipe/socket: having
written Gambit->Perl bindings (although still missing callbacks from
Perl into Scheme), I can say that linking two languages in the same
process through the FFI is certainly possible. But it currently misses
the above-mentioned FFI-in-a-separate-pthread approach to prevent Perl
code from blocking Gambit green threads. Also running the other language
in a separate process is safer against crashes, of course (although this
hasn't been a problem for me so far). I did it in-process because I
didn't want to risk performance problems. For example I don't normally
convert a perl hashtable into a Gambit table when being returned from a
call to perl code, but instead return a foreign object around the perl
object, on which the Scheme code can then call perl's hashtable
operators to get at the individual values. That saves the overhead of
conversion of the whole table if you're only interested in a few values.
The round trip times over a socket may make that relatively costly. (It
will be better with Termite+multiple processes and a common shared
read-only storage, since you can then read from the common storage and
save the round trips :). I'll help doing some benchmark if you want to
compare.
- speaking of pymacs: wouldn't it be nice to have a Elisp<->Gambit
interface? (schemacs or something). But about the same thing might also
be achievable through a SLIME48 ported to Gambit, which itself might be
another good candidate for a GSoC project.
Christian.
More information about the Gambit-list
mailing list