On Nov 8, 2020, at 6:49 AM, Jörg F. Wittenberger Joerg.Wittenberger@softeyes.net wrote:
Hi,
the following I wrote assuming the `no buffering` of pipe ports would block the writer until there is a reader thread ready to receive the data.
Apparently that's not the case.
Does Gambit support such a flow control?
(Or alternatively: is there a way to create my own ports, which would?)
Thanks soo much
/Jörg
(define (make-pipe) (open-u8vector-pipe '(buffering: #f) '(buffering: #f)))
(receive (in out) (make-pipe) (display "foo" out) ;; I'd expect this to block, (force-output out) ;; but even this does not. "too bad")
The “buffering:” setting on byte ports indicates how promptly the data is transferred to the “next consumer” in the I/O pipeline. When using buffering: #f each byte is immediately transferred to the next stage of the pipeline. For example this code
(define p (open-output-file (list path: "/dev/stdout" buffering: #f)))
(let loop ((i 0)) (if (< i 5000) (begin (write-u8 42 p) (thread-sleep! 0.001) (loop (+ i 1)))))
will display a “*” (code 42) every millisecond. But if the buffering setting is changed to #t the data will be output in chunks of 1024 bytes, which is the size of the byte buffer. Note that in this example the “next consumer” in the I/O pipeline is the operating system (that moves the data to the process’ stdout).
When using open-u8vector-pipe two ports are created, an output port and an input port. The “next consumer” in the I/O pipeline of the output port is an in-memory FIFO that accumulates the data received. The input port consumes data from this in-memory FIFO to fill its buffers.
So in your example, using a buffering: #f has no effect on the buffering being done by the in-memory FIFO.
If you want to limit the buffering done by the in-memory FIFO, for example if you want to avoid a form of memory leak when the consumer is slower than the producer, then you can set a buffering limit with “macro-u8vector-port-buffering-limit-set!” like this:
(include "~~lib/_gambit#.scm")
(define (make-pipe) (open-u8vector-pipe '(buffering: #f) '(buffering: #f)))
(receive (in out) (make-pipe)
;; set buffering limit... actual limit = (* 64 (+ 2 (quotient lim 64))) (let ((lim 1000)) (macro-u8vector-port-buffering-limit-set! out lim))
(let ((i 0)) (with-exception-catcher (lambda (e) (if (deadlock-exception? e) (pp `(deadlock after writing ,i bytes)) (raise e))) (lambda () (let loop () (if (< i 1000000) (begin (write-u8 42 out) (set! i (+ i 1)) (loop)) (pp `(wrote ,i bytes without deadlock))))))))
Note that the actual limit that you get is really (* 64 (+ 2 (quotient limit 64))) because the in-memory FIFO is represented by a list of 64 byte chunks.
Marc