[gambit-list] Multiprocessor support
Christian
christian at pflanze.mine.nu
Wed Dec 6 05:47:27 EST 2006
At 9:57 Uhr +0100 06.12.2006, Daniel Sadilek wrote:
>Hello,
>
>>(Generally: multiple processes and data passing (open-process, or on
>>unix fork() and pipe() or possibly mmap() (poke me if you want me to
>>release an updated cj-posix library), or (as mentioned) Termite).)
>
>I had a look at your cj-posix library. Maybe that's a starting point
>for me. As I understand, when I fork, all (not just the current)
>Gambit-threads will be forked and the communication has to be done by
>pipes. Is it possible to have only the current thread forked
Yes, by using the
(fork* thunk) => pid
procedure in my current cj-posix module
( http://scheme.mine.nu/gambit/scratch/cj-posix/
I didn't get around implementing mmap yet).
> and to
>have shared data structures
Files and mmaps: using the C interface it's quite easy to write
homogenous number vectors which are based on mmap'ed files. There has
been a project for which I've planned to implement those, the project
has been stopped (for the time being), but I want to implement those
anyway (without fixed timeline).
So one could store the data in file based vectors, then fork off
children which process some (or parts) of them and produce output
vectors, which can then be read by the parent or another child which
processes them further etc.; using mmap (and e.g. a fill indicator
being polled) one could even start reading such vectors from reader
processes before they have been finished being written by the writer
process.
If you're interested in this framework, I'll try to write it soon.
Comments welcome.
> or do you have any idea how that could be
>emulated (and elegantly abstracted)? Do you have a simple hello world
>example?
>
>>I've started writing bindings for PETSc[1] (without a timeline for
>>completion). That library can do multiprocessing (even across a whole
>>cluster). Are you interested in those?
>
>I'm not sure. Can this library help me to run the _Scheme_code_ on
>multiple processors?
Well, PETSc is using MPI underneath, and if I bind this as well, that
should offer a way to start processes on other nodes and pass data
around. But PETSc (afaik) is not meant to help you write parallel
programs - is is meant to write sequential programs where the
calculations are processed in parallel transparently.
The point is that if you're writing simulations using nodes, you
could probably implement this, let's say, as vectors and the
processing steps as multiplicatios of those vectors with matrices.
PETSc offers sparse matrices, so it is supposed to be efficient even
if only few nodes have relationships between each other.
Christian.
More information about the Gambit-list
mailing list