[gambit-list] Re: Reading data from mixed ASCII/binary files

Marc Feeley feeley at iro.umontreal.ca
Thu Jan 12 09:25:04 EST 2006


On 12-Jan-06, at 12:59 AM, Bradley Lucier wrote:

> Marc:
>
> The file formats used by the image library netpbm (pbm, pgm, ppm,  
> pnm, ...) are mixed ascii/binary format; the first few lines are  
> ASCII that have a magic number for the file, the size of the image  
> (rows and columns) and the maximum value of the pixels (for  
> greyscale images).  Then, after a single newline, comes the binary  
> data.
>
> So:
>
> [lindv2:Arrays/srfi2/stacey] lucier% head -3 2314.pgm
> P5
> 1605 2244
> 255
> <binary data follows here>
>
> Now, in beta 16 and beta 17, I get the error:
>
>> [lindv2:Arrays/srfi2/stacey] lucier% gsc
>> Gambit Version 4.0 beta 17
>>
>> > (load "convex2.scm")
>> *** ERROR IN #<procedure #2>, "../generic-arrays.scm"@1171.44 --  
>> Input port character buffer is not empty
>> (read-u8 '#<input-port #3 "/Users/lucier/lang/scheme/Arrays/srfi2/ 
>> stacey/2314.pgm">)
>> 1> ,b
>> 0  #<procedure #2>           "../generic-arrays.scm"@1171:44  
>> (domain-getter i j)
>> 1  #<procedure #4>           "../generic-arrays.scm"@283:9 (f i j)
>> 2  Array->Fixed-array        "../generic-arrays.scm"@1169:6  
>> (Interval-for-each (case (##Interval-dimension domain) ((1)  
>> (lambda (i) (range-setter (domain-...
>> 3  #<procedure #5>           "../pgm.scm"@44:5       (Array->Fixed- 
>> array generic-array-manipulators (build-Array (build-Interval '#(0  
>> 0) (vector rows colu...
>> 4  call-with-input-file
>> 5  (interaction)             "convex2.scm"@77:21     (read-pgm  
>> (string-append name ".pgm"))
>> 6  ##load
>> 7  ##load
>> 8  (interaction)             (console)@1:1           (load  
>> "convex2.scm")
>> 1> ,e
>> i = 0
>> j = 0
>> range-setter = (lambda (value i j) (manipulator-setter body  
>> (indexer i j) value))
>> domain-getter = (lambda (i j) (read-u8 port))
>> result = '#<Array-base #6 domain: #<Interval #7 lower-bounds: #(0  
>> 0) upper-bounds: #(2244 1605)> getter: #<procedure #8> setter:  
>> #<procedure #9> manipulat...
>> domain = '#<Interval #7 lower-bounds: #(0 0) upper-bounds: #(2244  
>> 1605)>
>> result-manipulators = '#<Array-manipulators #10 getter:  
>> #<procedure #11> setter: #<procedure #12> maker: #<procedure #13  
>> make-vector> length: #<procedure ...
>> array = '#<Array-base #15 domain: #<Interval #7 lower-bounds: #(0  
>> 0) upper-bounds: #(2244 1605)> getter: #<procedure #16> setter: #f  
>> manipulators: #f body...
>> (current-exception-handler) = primordial-exception-handler
>> (current-input-port) = '#<input-output-port #17 (console)>
>> (current-output-port) = '#<input-output-port #17 (console)>
>> (current-directory) = "/Users/lucier/lang/scheme/Arrays/srfi2/ 
>> stacey/"
>> 1>
>
> and I read from the manual:
>
>> procedure: read-u8 [port]
>>
>>     This procedure reads the byte input-port port and returns the  
>> byte at the current byte location and advances the current byte  
>> location to the next byte, unless the port is already at end-of- 
>> file in which case read-u8 returns the end-of-file object. If it  
>> is not specified, port defaults to the current input-port.
>>
>>     This procedure must be called before any use of the port in a  
>> character input operation (i.e. a call to the procedures read,  
>> read-char, peek-char, etc) because otherwise the character-stream  
>> and byte-stream may be out of sync due to the port buffering.
>
> So, how do I deal with meta-data like this?  Read the u8 numbers,  
> call integer->char, put them all into a string, call read on the  
> string, ...?  Why should I have to do this instead of the runtime  
> doing this?  Or perhaps you have a better suggestion.
>
> Brad

The specification in the manual is more restrictive than the  
implementation (I will change the manual).  You can only read a byte  
with read-u8 if the character buffer is empty.  This can be  
guaranteed by reading the character from a port that is not buffered  
(i.e. buffering: #f).  Then you can switch to reading bytes with read- 
u8 or read-subu8vector.  So what you want looks like this (tested on  
Mac OS X and Linux with beta 16 and 17):

#! /usr/bin/env gsi-script

; File: "text-bin-io.scm"

(define (open-input-file-for-text-bin-io path)
   (open-input-file (list path: path
                          char-encoding: 'latin1
                          eol-encoding: 'lf
                          buffering: #f)))

(define (read-pgm-P5 path)
   (let ((p (open-input-file-for-text-bin-io path)))
     (let* ((char1 (read-char p))
            (char2 (read-char p)))
       (if (not (and (char? char1)
                     (char? char2)
                     (char=? char1 #\P)
                     (char=? char2 #\5)))
           (error "P5 magic number expected")
           (let* ((width (read p))
                  (height (read p))
                  (maxval (read p))
                  (char3 (read-char p)))
             (if (not (and (char? char3) (char-whitespace? char3)))
                 (error "whitespace expected after maxval")
                 (let* ((bytes-per-pixel (if (< maxval 256) 1 2))
                        (nb-bytes (* bytes-per-pixel width height))
                        (image (make-u8vector nb-bytes)))
                   (let ((n (read-subu8vector image 0 nb-bytes p)))
                     (if (= n nb-bytes)
                         (vector width
                                 height
                                 maxval
                                 image)
                         (error "truncated file"))))))))))

; create a pgm P5 file we can read...
(with-output-to-file "foo.pgm"
   (lambda () (display "P5\n2 3\n255\n123456")))

(define (main)
   (let ((image (read-pgm-P5 "foo.pgm")))
     (pp image)))


Marc




More information about the Gambit-list mailing list