[gambit-list] Multiprocessor support

Christian christian at pflanze.mine.nu
Wed Dec 6 05:47:27 EST 2006


At 9:57 Uhr +0100 06.12.2006, Daniel Sadilek wrote:
>Hello,
>
>>(Generally: multiple processes and data passing (open-process, or on
>>unix fork() and pipe() or possibly mmap() (poke me if you want me to
>>release an updated cj-posix library), or (as mentioned) Termite).)
>
>I had a look at your cj-posix library. Maybe that's a starting point
>for me. As I understand, when I fork, all (not just the current)
>Gambit-threads will be forked and the communication has to be done by
>pipes. Is it possible to have only the current thread forked

Yes, by using the
    (fork* thunk) => pid
procedure in my current cj-posix module
( http://scheme.mine.nu/gambit/scratch/cj-posix/
  I didn't get around implementing mmap yet).

>  and to
>have shared data structures

Files and mmaps: using the C interface it's quite easy to write 
homogenous number vectors which are based on mmap'ed files. There has 
been a project for which I've planned to implement those, the project 
has been stopped (for the time being), but I want to implement those 
anyway (without fixed timeline).

So one could store the data in file based vectors, then fork off 
children which process some (or parts) of them and produce output 
vectors, which can then be read by the parent or another child which 
processes them further etc.; using mmap (and e.g. a fill indicator 
being polled) one could even start reading such vectors from reader 
processes before they have been finished being written by the writer 
process.

If you're interested in this framework, I'll try to write it soon. 
Comments welcome.

>  or do you have any idea how that could be
>emulated (and elegantly abstracted)? Do you have a simple hello world
>example?
>
>>I've started writing bindings for PETSc[1] (without a timeline for
>>completion). That library can do multiprocessing (even across a whole
>>cluster). Are you interested in those?
>
>I'm not sure. Can this library help me to run the _Scheme_code_ on
>multiple processors?

Well, PETSc is using MPI underneath, and if I bind this as well, that 
should offer a way to start processes on other nodes and pass data 
around. But PETSc (afaik) is not meant to help you write parallel 
programs - is is meant to write sequential programs where the 
calculations are processed in parallel transparently.

The point is that if you're writing simulations using nodes, you 
could probably implement this, let's say, as vectors and the 
processing steps as multiplicatios of those vectors with matrices. 
PETSc offers sparse matrices, so it is supposed to be efficient even 
if only few nodes have relationships between each other.

Christian.



More information about the Gambit-list mailing list