[gambit-list] A rudimentary pipeline implementation with bridging threads

Mikael mikael.rcv at gmail.com
Mon Jan 28 06:20:09 EST 2013


Aha.

Check what API:s Unix actually provides to do that (that sh uses) as for sh
not to need to proxy any data itself but just interconnect the started
binaries that way, and then mimick it yourself, by making c-lambdas for the
respective API:s,  compiled with compile-file and loaded with load, and
then putting it together yourself.

Find a way to merge this setup together with open-process' underlying code
so you get stdout access to the first-started binary and stdin access from
the last started one. All the ##-prefixed *procedures* in Gambit's
sourcecode are reachable from your sourcecode.

Probably several would be curious to see the particular solution you come
up with, feel free to share it here on the ml!

Brgds,
Mikael

2013/1/28 Jussi Piitulainen <jpiitula at ling.helsinki.fi>

> There's (shell-command "date | tr 0-9 D") that works but requires me
> to build a string for the shell to parse and I don't think it lets me
> control the input to and output from the pipe from Gambit.
>
> Then there's
> (read-line (open-process '(path: "sh" arguments: ("-c" "date | tr 0-9
> D"))))
> that probably does what I want except again I don't want to build that
> string in the first place.
>
> I want to do in Gambit what the shell does when it launches a
> pipeline, while being able to write to and read from the resulting
> pipeline. I know how to mix the two, but my goal is to use gsi
> _instead_ of the shell.
>
> (And no, I don't mean to create a cycle. Just a straight pipe.)
>
> Thanks.
>
>
> Mikael <mikael.rcv at gmail.com> writes:
>
> > So you have a Gambit process A that you want to launch Unix processes B
> and
> > C, such that A outputs to B, which outputs to C, which outputs back to A?
> >
> > Probably you can just rely on Unix' piping primitives for that, something
> > akin to open-process `(app: "/bin/sh" args: ("B" "|" "C"  or "B | C")
> > ..redirection.. #t ?
> >
> > 2013/1/28 Jussi Piitulainen <jpiitula at ling.helsinki.fi>
> >
> > > Assuming my previous question came through, I asked about implementing
> > > pipelines of processes in Gambit-C. The problem was that I could not
> > > seem to make the output of one process to connect to the input of the
> > > next on their own.
> > >
> > > I'm now experimenting with Gambit-C threads to bridge those gaps.
> > > Here is a rudimentary implementation of what I want. The comments
> > > indicate things that I think I know I still need to do. I haven't used
> > > threads much before, and never in Scheme.
> > >
> > > The original question stands: am I overlooking some obvious way to do
> > > such pipelines that Gambit-C already has? Because this is the easy 80%
> > > of the work and another 80% sheer tedium probably remains :)
> > >
> > > (define (echo from to) ;should use a buffer
> > >   (do ((line (read-line from) (read-line from)))
> > >       ((eof-object? line))
> > >     (display line to)
> > >     (newline to)))
> > >
> > > (define (piece from to)
> > >   (lambda ()
> > >     (echo from to) ;should allocate a private buffer for echo
> > >     (close-output-port to)  ;stopped using it as output port
> > >     (process-status from))) ;wait for it - maybe time it out?
> > >
> > > (define (end from) ;not closing current output port
> > >   (lambda ()
> > >     (echo from (current-output-port))
> > >     (process-status from)))
> > >
> > > ;actually, the end should be controlled by the called altogether! or
> > > ;is it now?
> > >
> > > ;caller needs to close the first pipe segment as output port, and do I
> > > ;need to force some output or close input ports? and errors should be
> > > ;handled, including diagnostic output from the processes and failed
> > > ;status of any, and not leave processes around on any sort of error if
> > > ;possible. oh and do all those threads terminate nicely and always?
> > >
> > > (define (pipe first . rest)
> > >   (let ((first (open-process first)))
> > >     (do ((last first (car rest))
> > >          (rest (map open-process rest) (cdr rest)))
> > >         ((null? rest)
> > >          (thread-start! (make-thread (end last)))
> > >          ;;^altogether mistaken - leave it to the caller
> > >          (values first last))
> > >       (thread-start! (make-thread (piece last (car rest)))))))
> > >
> > > (define (test) ;has hardwired end to echo the output in current output
> > >   (call-with-values ;^fix it so the called handles reading and waiting
> > >       (lambda ()
> > >         (pipe "date" '(path: "tr" arguments: ("0-9" "D"))))
> > >     (lambda (first last)
> > >       (close-output-port first))))
> > >
> > > _______________________________________________
> > > Gambit-list mailing list
> > > Gambit-list at iro.umontreal.ca
> > > https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
> > >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20130128/8006827c/attachment.htm>


More information about the Gambit-list mailing list