[gambit-list] A rudimentary pipeline implementation with bridging threads

Jussi Piitulainen jpiitula at ling.helsinki.fi
Mon Jan 28 05:02:45 EST 2013


Assuming my previous question came through, I asked about implementing
pipelines of processes in Gambit-C. The problem was that I could not
seem to make the output of one process to connect to the input of the
next on their own.

I'm now experimenting with Gambit-C threads to bridge those gaps.
Here is a rudimentary implementation of what I want. The comments
indicate things that I think I know I still need to do. I haven't used
threads much before, and never in Scheme.

The original question stands: am I overlooking some obvious way to do
such pipelines that Gambit-C already has? Because this is the easy 80%
of the work and another 80% sheer tedium probably remains :)

(define (echo from to) ;should use a buffer
  (do ((line (read-line from) (read-line from)))
      ((eof-object? line))
    (display line to)
    (newline to)))

(define (piece from to)
  (lambda ()
    (echo from to) ;should allocate a private buffer for echo
    (close-output-port to)  ;stopped using it as output port
    (process-status from))) ;wait for it - maybe time it out?

(define (end from) ;not closing current output port
  (lambda ()
    (echo from (current-output-port))
    (process-status from)))

;actually, the end should be controlled by the called altogether! or
;is it now?

;caller needs to close the first pipe segment as output port, and do I
;need to force some output or close input ports? and errors should be
;handled, including diagnostic output from the processes and failed
;status of any, and not leave processes around on any sort of error if
;possible. oh and do all those threads terminate nicely and always?

(define (pipe first . rest)
  (let ((first (open-process first)))
    (do ((last first (car rest))
	 (rest (map open-process rest) (cdr rest)))
	((null? rest)
	 (thread-start! (make-thread (end last)))
	 ;;^altogether mistaken - leave it to the caller
	 (values first last))
      (thread-start! (make-thread (piece last (car rest)))))))

(define (test) ;has hardwired end to echo the output in current output
  (call-with-values ;^fix it so the called handles reading and waiting
      (lambda ()
	(pipe "date" '(path: "tr" arguments: ("0-9" "D"))))
    (lambda (first last)
      (close-output-port first))))




More information about the Gambit-list mailing list