[gambit-list] newbie question: output from one port as input to another port

[Christian]

At 22:40 Uhr -0500 11.03.2006, Samuel Montgomery-Blinn wrote:
>So, I have been trying to learn Gambit-C's port mechanisms. This is 
>likely an infantile question, but it seems that it should be the 
>most basic thing to be able to take one input port and "connect" it 
>to an output port. The specific use for me is sending the contents 
>of a file over a network (tcp) port. I have tried a few methods, 
>mostly "busy" and likely lossy methods.
>
>1. read the entire contents of the file port, then output the file 
>to the network port. (wastes time in which earlier parts of the file 
>could be written to the network port)
>2. read a line from the file port, and output the line to the 
>network port. repeat until eof. (wastes time for many reasons)

Since there is no builtin, your second suggestion is not far from the 
optimum. I would use {read,write}-subu8vector or 
{read,write}-substring instead (the latter should be necessary if the 
input and output ports use different charset [or end-of-line?] 
encodings):

(define bufsiz 4096)

(define (port-copy-bytes from to)
   (let ((buf (make-u8vector bufsiz)))
     (let lp ((total-bytes 0))
       (let ((n-read (read-subu8vector buf 0 bufsiz from)))
	(if (= n-read 0) total-bytes ;; eof (or possibly timeout) case
	    (let ((n-written (write-subu8vector buf 0 n-read to)))
	      (if (= n-written n-read)
		  (lp (+ total-bytes n-written))
		  (error "can this happen?: n-read != n-written:" 
n-read n-written))))))))

(define (copy-file from-path to-path)
   (call-with-input-file from-path
     (lambda (in)
       (call-with-output-file to-path
	(lambda (out)
	  (port-copy-bytes in out))))))

Maybe it could be optimized some more by using lowlevel internals.

You could run this in a thread if you want:

; some thread usage simplification:

(define (make-future thunk)
   (let ((t (make-thread thunk)))
     (thread-start! t)
     t))

(define-macro (future arg)
   `(make-future (lambda ()
		  ,arg)))

(define future-force thread-join!)

; use it:

>  (define f (future (copy-file "foo" "bar")))
>  (future-force f)
3342


If you're on a unix with sendfile(2), you should be able to use the C 
interface and use that, but in this case be aware of the following:
- Gambit switches file descriptors to non-blocking IO (so that it 
doesn't risk blocking the thread scheduler). I don't know how 
sendfile(2) handles fd's in non-blocking mode; either you block until 
it finishes (good for simplicity, bad for the following point) or not 
(in which case coding will be a bit more complicated).
- for the duration of the C calls, other scheme threads (if there are 
any) do not have a chance to run. So iff sendfile(2) blocks, you 
might want to create a separate pthread and let that do the work (I 
still haven't tried this).

(BTW here are my thoughts on the {read,write}-sub* interface:
- I assume that write-subu8vector never returns with fewer than the 
requested number of bytes written; correct?
- I hope that those functions never return with 0 unless eof or 
timeout has happened (also not if being signalled); probably correct.
- the way for setting a timeout thunk does not seem to be flexible: 
there's no documented way getting the current thunk for restoring a 
previous one. Why not use dynamic parameter instead? (Or: an 
additional lexical argument to the read/write-sub* functions.))


>I was guessing that something to do with the procedures 
>"call-with-input-file" or "with-output-to-file" might be something 
>towards what might be most succinctly expressed as some sort of UNIX 
>"sendfile" functionality

If you're into elegance, you might want a curried version of the 
port-copy-bytes function (but it won't buy you that much; one has to 
be careful to keep the dynamic nesting of the call-with-* functions 
intact):

(define (port-byte-copier from)
   (lambda (to)
     (port-copy-bytes from to)))

(define (copy-file-from.II from-path to-path)
   (call-with-input-file from-path
     (lambda (in)
       (call-with-output-file to-path
	(port-byte-copier in)))))

Christian.