Ah Richard, note that all of these four file output/input routines mentioned now, can take a settings structure argument in the place where you passed filenames up to now. Using those settings, you can specify things like character encoding to use for the read/written file.

2012/10/27 Marc Feeley feeley@iro.umontreal.ca

Le 2012-10-27 à 4:31 AM, Richard Prescott rdprescott@gmail.com a écrit :

...

Good morning everybody,

Forgive my ignorance, I am just learning.

In order to embrace the "code is data" philosophy [1], I want to load

and save data as s-expressions within files. I figured the load using (read (open-input-file "data.scm")).

...

How can I save?

Thanks in advance.

Richard

While it does work, your code does not close the input-port and relies on the garbage collector to do this, which is a bad idea (because you may run out of file descriptors before a garbage collection is triggered). It is better to use call-with-input-file which closes the port.

Here are better save and restore functions:

;; File: save-restore.scm

(define (save-to-file filename obj) (call-with-output-file filename (lambda (port) (write obj port))))

(define (restore-from-file filename) (call-with-input-file filename (lambda (port) (read port))))

(save-to-file "data.txt" '#(1 2 3))

(pp (restore-from-file "data.txt"))

Marc

---

Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

Le 2012-10-27 à 8:11 PM, Mikael mikael.rcv@gmail.com a écrit :

Marc,

Do all ports GC? (am thinking especially about file and tcp client and server ports)

Yes. For all ports, the OS ressources which are attached to a port are freed (including closing file descriptors) when the port is GCed.

I suppose that the fact that with-*-port closes the port on finalization cannot be relied on to always work, because what if there's an exception during the thunk's execution and it's handled in a way that never leaves control back, or the thread is thread-terminate!:d (could happen a dynamic-wind finalizer would not be trigged right)?

Yes. If you need to force closing a file descriptor then you need to catch exceptions and explicitly close the port.

Here's a small program you can use to test limits on your particular OS.

;; File: fd-autoclose.scm

(define (on-gc thunk) (make-will (cons 1 2) ;; garbage (lambda (obj) (thunk) (on-gc thunk))))

(define counter 0)

(on-gc (lambda () (pp counter) (set! counter 0)))

(with-exception-catcher (lambda (e) (##gc) (display-exception e)) (lambda () (let loop ((n 5000)) (if (> n 0) (begin (set! counter (+ counter 1)) (let ((port (open-tcp-client "localhost:22"))) (loop (- n 1))))))))

It will try to open 5000 TCP connections in a loop, relying on the GC to close the connections. The program displays the number of connections that were open (and closed) since the last garbage collection. If the heap size is small, then the file descriptors never run out. But if a large heap is used, the program will reach the OS limit on the number of open file descriptors.

Here's a test on an OS X machine:

% gsi -:d2 fd-autoclose.scm *** GC: 0 ms, 180K alloc, 203K heap, 37.1K live (18% 18528+19448) 14 *** GC: 0 ms, 347K alloc, 209K heap, 60.4K live (29% 36752+25140) 19 *** GC: 1 ms, 495K alloc, 209K heap, 60.4K live (29% 36752+25140) 19 *** GC: 1 ms, 644K alloc, 209K heap, 60.4K live (29% 36752+25140) 19 *** GC: 5 ms, 792K alloc, 209K heap, 60.4K live (29% 36752+25140) 19 ...etc, until the normal end of the program (note the small 209 KB heap)

and with a 19 MB heap the program works properly:

% gsi -:d2,m19000 fd-autoclose.scm 2502 *** GC: 78 ms, 19.0M alloc, 18.6M heap, 63.4K live (0% 37696+27248) 2470 *** GC: 45 ms, 37.5M alloc, 18.6M heap, 63.4K live (0% 37696+27248)

but with a 20 MB heap the program terminates with an error:

% gsi -:d2,m20000 fd-autoclose.scm 2556 *** GC: 73 ms, 19.4M alloc, 19.9M heap, 60.3K live (0% 36600+25140) Too many open files (open-tcp-client "localhost:22")

It would seem that the maximum number of open file descriptor on this machine is about 2560 (considering a few file descriptors are open for stdin, stdout, ...).

Marc

Le 2012-10-27 à 8:44 AM, Marc Feeley feeley@iro.umontreal.ca a écrit :

Le 2012-10-27 à 4:31 AM, Richard Prescott rdprescott@gmail.com a écrit :

...

Good morning everybody,

Forgive my ignorance, I am just learning.

In order to embrace the "code is data" philosophy [1], I want to load and save data as s-expressions within files. I figured the load using (read (open-input-file "data.scm")).

How can I save?

Thanks in advance.

Richard

While it does work, your code does not close the input-port and relies on the garbage collector to do this, which is a bad idea (because you may run out of file descriptors before a garbage collection is triggered). It is better to use call-with-input-file which closes the port.

Here are better save and restore functions:

;; File: save-restore.scm

(define (save-to-file filename obj) (call-with-output-file filename (lambda (port) (write obj port))))

(define (restore-from-file filename) (call-with-input-file filename (lambda (port) (read port))))

(save-to-file "data.txt" '#(1 2 3))

(pp (restore-from-file "data.txt"))

Marc

Here's an improved version of those functions which allows any serializable object, including functions, to be saved/restored. The code uses the functions object->u8vector and u8vector->object which encode/decode any object to/from a vector of bytes. In the case of interpreted functions, the source code locations are saved in the encoding (so that any error in a restored function will give an error message which points to the correct source code location). See the example below.

Marc

;; File: save-restore.scm

(define (write-file filename obj) (call-with-output-file filename (lambda (port) (write obj port))))

(define (read-file filename) (call-with-input-file filename (lambda (port) (read port))))

(define (save-to-file filename obj) (write-file filename (object->u8vector obj)))

(define (restore-from-file filename) (u8vector->object (read-file filename)))

;; test it

(save-to-file "code.txt" (lambda (x) (/ 1.0 (* x x))))

(define inverse-square (restore-from-file "code.txt"))

(pp (inverse-square 10))

(pp (inverse-square "foo"))

;; gives: ;; ;; .01 ;; *** ERROR IN inverse-square, "save-restore.scm"@21.45 -- (Argument 1) NUMBER expected ;; (* "foo" "foo")