Hi,
I've just started working with Termite to distribute the processing for a large data mining system over a multi core CPU and I seem to be hitting a few problems for which any advice is welcome.
Small scale examples seem to work well but once I scale up I hit problems. I'm running on a 1Gig Mac using OS X 10.4 with the latest distribution of Gambit and the current tarball from the Termite site together with the slib library.
At first I built a two process client server example but it hung. The problem turned out to be the array.scm file in the slib distribution redefining the equal? procedure. Once the redefinition was removed all worked well. So long as I used tags and not pids to identify one remote process to another. The pid returned by remote-spawn is not the same as that returned by the process calling (self).
I then moved on to trying a parallel version of map, akin to that given at http://theschemeway.blogspot.com/2007/06/parallel-map-with-termite.html . That worked fine for small simple examples but when I threw in a larger example, evaluating forty instances of a k-means cluster procedure in parallel in one process, it consistently dumps core but not in any predictable manner. All forty threads kick off fine but while some succeed others fail catastrophically and at no particular point resulting in Gambit dumping core. I've run the code using map and all was well, and the code has nothing in it which seems likely to cause the problem. I've run the code both compiled and fully interpreted and the behavior is the same.
First is there any changes to the Gambit build which might help identifying where the fault is occurring? Secondly are there known problems with Termite especially when spawning moderately large processes.?
Many Thanks
Nick Walton