[gambit-list] fast csv parser (libcsv wrapped for gambit)

Bradley Lucier lucier at math.purdue.edu
Sat Feb 10 13:40:38 EST 2007


On Feb 9, 2007, at 1:36 PM, Phil Dawes wrote:

> Ok - that prompted me to do a little speed testing on a large file  
> (6000
> odd records). Note that Marc's comma splitter doesn't actually do the
> parsing properly because many records are split over multiple  
> lines, so
> I'm guessing this is an upper performance limit on pure scheme?.
>
> wrapped libcsv  ~240ms
> Marc comma splitter  ~510ms
> Phil Bewig csv parser  ~1008ms

Here's what I get with libcsv and the buffered-output version of Phil  
Bewig's code + expanding the fast-path of read-char inline; the  
scheme version seems about 10% slower.

Brad

libcsv:

(time (fn))
     95 ms real time
     94 ms cpu time (77 user, 17 system)
     no collections
     19938808 bytes allocated
     6156 minor faults
     no major faults
(time (fn))
     102 ms real time
     101 ms cpu time (99 user, 2 system)
     no collections
     19938808 bytes allocated
     28 minor faults
     no major faults
(time (fn))
     102 ms real time
     100 ms cpu time (99 user, 1 system)
     no collections
     19938808 bytes allocated
     8 minor faults
     no major faults
(time (fn))
     102 ms real time
     100 ms cpu time (99 user, 1 system)
     no collections
     19938808 bytes allocated
     8 minor faults
     no major faults
(time (fn))
     101 ms real time
     101 ms cpu time (99 user, 2 system)
     no collections
     19938808 bytes allocated
     8 minor faults
     no major faults

bewig2+macro-read-char.scm:

(time (fn))
     112 ms real time
     112 ms cpu time (99 user, 13 system)
     no collections
     45280848 bytes allocated
     3148 minor faults
     no major faults
(time (fn))
     122 ms real time
     122 ms cpu time (90 user, 32 system)
     no collections
     45280848 bytes allocated
     7886 minor faults
     no major faults
(time (fn))
     111 ms real time
     110 ms cpu time (91 user, 19 system)
     no collections
     45280848 bytes allocated
     4716 minor faults
     no major faults
(time (fn))
     111 ms real time
     111 ms cpu time (88 user, 23 system)
     no collections
     45280848 bytes allocated
     4716 minor faults
     no major faults
(time (fn))
     111 ms real time
     111 ms cpu time (95 user, 16 system)
     no collections
     45280848 bytes allocated
     4716 minor faults
     no major faults




More information about the Gambit-list mailing list