[gambit-list] fast csv parser (libcsv wrapped for gambit)

Marc Feeley feeley at iro.umontreal.ca
Thu Feb 8 19:26:16 EST 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On 8-Feb-07, at 4:03 PM, Phil Dawes wrote:

> Hi Gambit List,
>
> I needed a fast csv parser for parsing large (multi-gig) files so I've
> wrapped the libcsv c code by Robert Gamble[1]. The first cut of  
> csv.scm
> is here:  http://phildawes.net/2007/gambit-csv/0.1/csv.scm
>
> Example usage:
>
> (define it (csv-make-iterator fname))
>
> (it)   ; returns the first row as a list
>
> (it)   ; returns 2nd row ..etc..
>
> ; (it) returns '() when it hits the end of the file.
>
>
> I'm very new to gambit/scheme - is this a reasonable interface or is
> there a more schemey idiom I should be presenting?

Interesting.  But why would you do it in C (in 600 lines of code)  
when you can do it in 20 lines of Scheme?

(define (csv-make-iterator fname)
   (let ((port (open-input-file fname)))
     (lambda ()
       (read-csv port))))

(define (read-all-csv port)
   (read-all port read-csv))

(define (read-csv port)
   (let ((line (read-line port)))
     (if (eof-object? line)
         line
         (split-csv line))))

(define (split-csv str)
   (call-with-input-string
     str
     (lambda (port)
       (read-all port
                 (lambda (p)
                   (read-line p #\,))))))

(define it (csv-make-iterator "test"))

(it) => ("11" "22" "33")
(it) => ("a" "b")

(call-with-input-string
   "11,22,33\na,b\n"
   read-all-csv) => (("11" "22" "33") ("a" "b"))

Marc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFFy7+o//V9Zc2T/v4RAgNiAJ9hG5p4oPjYinPmg5s6B6+51OC/ZQCgmDx3
zxcA3D08d/QgjEBOgG6HexE=
=6jmG
-----END PGP SIGNATURE-----



More information about the Gambit-list mailing list