[gambit-list] gcc stage timing

Lang Martin lang-gb at coptix.com
Tue Oct 17 13:13:56 EDT 2006


quoth Christian:

> In our case (parallel gcc), it's problematic since running right 
> below the process limit will make gcc fail when it tries to run 
> subprocesses itself.

quite.

> Will you be able to deduce gcc's reason of 
> failure from it's exit status or will you have to parse it's output? 
> Will you patch it to make it take an option telling it to retry 
> forking his own children? Maybe we need a new GNU system to implement 
> this idea right everywhere?

well, it'd be nice. I get your point, but that's exactly what you're
doing, right? deducing gcc's limit from it's failure, and tweaking a
parameter until you've manually constrained scheme to the point where
it runs correctly. It's a fine solution for your problem, but it would
scale better (up & down) with a restarting kind of approach.

> The semaphore approach has at least the advantage of being efficient 
> in the sense that new tasks are started as soon as older ones finish 
> (no retry/polling needed, so no phases of inactivity).

that's true.

> In the end, it's really just a question whether the limit should be 
> in the OS or in the app. Lispers think "the lisp image is the 
> system", and Gambit goes that route insofar as it implements such 
> things as code reloading or threading. So it's reasonable to accept 
> the limit is in this sub-OS, too.

That makes sense. I'll see where I get with my train of thought on
this, and keep an eye on your objections. I'm still in the midst of
moving from DJB-style systems to a lisp style, so it'll probably take
a few attempts to reconcile what I want.

> Heh, interesting approach; turning an imperative (mutating state) 
> generator into a stream, right?

Correct. The generator in my program makes an SQL call. I like the
metaphor, and I've used it a few times now for ports, etc.

> Actually seems to make sense if you've got an imperative generator. 
> (But even on Gambit, call/cc costs a little bit more than a cons or 
> two, so rewriting that into a straight functional stream generator 
> may be a little bit more efficient.)
> 
> (BTW you should probably rename |fn| to something else here as well.)

So, something like this:

(define (make-lazy proc)
  (letrec ((nothing (gensym))
           (exit (lambda () nothing))
           (me (lambda (proc)
                 (let ((value (proc exit)))
                   (if (eqv? value nothing)
                       '()
                       (cons value
                             (delay (me))))))
               ))
    (delay (me))))

That's just the one call to gensym, which vaguely seems like the most
expensive part of doing it this way, and should be as correct. Could
write a generator to break it, but I don't suppose it'd happen
accidently.
               
> So the order of the values in the output is not the same as in the 
> stream, right? So it's not a "parallel map" really.

Indeed. Sorry about that, I should have mentioned it. The endpoint of
all this is a for-each loop that inserts results through mutation.
It's SQL, again. So, for my purposes, I wanted it to send back results
in any order, but as quickly as possible.

Thanks for the feedback, and sorry for getting off-topic in this
thread.

Lang



More information about the Gambit-list mailing list