At 9:57 Uhr +0100 06.12.2006, Daniel Sadilek wrote:

Hello,

...

(Generally: multiple processes and data passing (open-process, or on unix fork() and pipe() or possibly mmap() (poke me if you want me to release an updated cj-posix library), or (as mentioned) Termite).)

I had a look at your cj-posix library. Maybe that's a starting point for me. As I understand, when I fork, all (not just the current) Gambit-threads will be forked and the communication has to be done by pipes. Is it possible to have only the current thread forked

Yes, by using the (fork* thunk) => pid procedure in my current cj-posix module ( http://scheme.mine.nu/gambit/scratch/cj-posix/ I didn't get around implementing mmap yet).

and to have shared data structures

Files and mmaps: using the C interface it's quite easy to write homogenous number vectors which are based on mmap'ed files. There has been a project for which I've planned to implement those, the project has been stopped (for the time being), but I want to implement those anyway (without fixed timeline).

So one could store the data in file based vectors, then fork off children which process some (or parts) of them and produce output vectors, which can then be read by the parent or another child which processes them further etc.; using mmap (and e.g. a fill indicator being polled) one could even start reading such vectors from reader processes before they have been finished being written by the writer process.

If you're interested in this framework, I'll try to write it soon. Comments welcome.

or do you have any idea how that could be emulated (and elegantly abstracted)? Do you have a simple hello world example?

...

I've started writing bindings for PETSc[1] (without a timeline for completion). That library can do multiprocessing (even across a whole cluster). Are you interested in those?

I'm not sure. Can this library help me to run the _Scheme_code_ on multiple processors?

Well, PETSc is using MPI underneath, and if I bind this as well, that should offer a way to start processes on other nodes and pass data around. But PETSc (afaik) is not meant to help you write parallel programs - is is meant to write sequential programs where the calculations are processed in parallel transparently.

The point is that if you're writing simulations using nodes, you could probably implement this, let's say, as vectors and the processing steps as multiplicatios of those vectors with matrices. PETSc offers sparse matrices, so it is supposed to be efficient even if only few nodes have relationships between each other.

Christian.