[gambit-list] Does gambit support flow control on ports?

Jörg F. Wittenberger Joerg.Wittenberger at softeyes.net
Mon Nov 16 14:22:10 EST 2020


thank you Marc!

Summary:
a) issue solved
b) discovered more surprising behavior (bug?)
b) actually I'm now interested in something related: a different way to
  create pipes.

# (a) Trick Done

> If you want to limit the buffering done by the in-memory FIFO, for
> example if you want to avoid a form of memory leak when the consumer
> is slower than the producer, then you can set a buffering limit with
> “macro-u8vector-port-buffering-limit-set!” like this:
> 
>    (include "~~lib/_gambit#.scm")
> 
>    (define (make-pipe)
>      (open-u8vector-pipe '(buffering: #f) '(buffering: #f)))
> 
>    (receive (in out) (make-pipe)
> 
>      ;; set buffering limit... actual limit = (* 64 (+ 2 (quotient
> lim 64))) (let ((lim 1000))
>        (macro-u8vector-port-buffering-limit-set! out lim))
> 
>      (let ((i 0))
>        (with-exception-catcher
>         (lambda (e)
>           (if (deadlock-exception? e)
>               (pp `(deadlock after writing ,i bytes))
>               (raise e)))
>         (lambda ()
>           (let loop ()
>             (if (< i 1000000)
>                 (begin
>                   (write-u8 42 out)
>                   (set! i (+ i 1))
>                   (loop))
>                 (pp `(wrote ,i bytes without deadlock))))))))

This helped A LOT!

Thanks soo much!

# (b) Unexpected: successfully writing to closed ports!

Example:

    > (define in)
    > (define out (receive (a b) (set! i a) b)
    > (close-output-port out)
    > (write-subu8vector '#u8(1 2 3 4) 0 2 out)
    2
    >

(Unhelpful when detection closed connections. :-/)

# (c) WANTED

The context I'm working on is essentially a proxy dispatching
connections between ports; be them tcp, processes, files or - important
in the context - custom I/O via Gambits C interface.

Using these u8vector-pipes - with these buffering limits - it does now
what it is functionally supposed to do.

But:  Most of the time the processing is like SOCKS or HTTP/S proxy
work: handle the first (few) protocol dependent things and then connect
the ports.  Currently the latter is implemented using two threads
copying one directions each.  Plus some overhead:

> Note that the actual limit that you get is really (* 64 (+ 2
> (quotient limit 64))) because the in-memory FIFO is represented by a
> list of 64 byte chunks.

In fact I do actually not need that pipe, I feel.

Would there be a way to connect two (bi-directional) ports "from now
until closed" and return.  (I assume that there should be at least
one thread to be joined in order to learn when the connection is done.
Alternative hints, like callbacks etc. welcome.)

To be clear: I'd like to replace my procedure:

    (ports-connect! in1 out1 in2 out2)

Which spawns a second thread to copy inA to outB and vice versa and
returns once both threads terminated.

I feel gambit can do much better than what I have so far.

Just how?  Once the connections are dispatched, all overhead is
evil.  ;-)

Thanks

/Jörg






More information about the Gambit-list mailing list