[gambit-list] fast csv parser (libcsv wrapped for gambit)
Bradley Lucier
lucier at math.purdue.edu
Sat Feb 10 13:40:38 EST 2007
On Feb 9, 2007, at 1:36 PM, Phil Dawes wrote:
> Ok - that prompted me to do a little speed testing on a large file
> (6000
> odd records). Note that Marc's comma splitter doesn't actually do the
> parsing properly because many records are split over multiple
> lines, so
> I'm guessing this is an upper performance limit on pure scheme?.
>
> wrapped libcsv ~240ms
> Marc comma splitter ~510ms
> Phil Bewig csv parser ~1008ms
Here's what I get with libcsv and the buffered-output version of Phil
Bewig's code + expanding the fast-path of read-char inline; the
scheme version seems about 10% slower.
Brad
libcsv:
(time (fn))
95 ms real time
94 ms cpu time (77 user, 17 system)
no collections
19938808 bytes allocated
6156 minor faults
no major faults
(time (fn))
102 ms real time
101 ms cpu time (99 user, 2 system)
no collections
19938808 bytes allocated
28 minor faults
no major faults
(time (fn))
102 ms real time
100 ms cpu time (99 user, 1 system)
no collections
19938808 bytes allocated
8 minor faults
no major faults
(time (fn))
102 ms real time
100 ms cpu time (99 user, 1 system)
no collections
19938808 bytes allocated
8 minor faults
no major faults
(time (fn))
101 ms real time
101 ms cpu time (99 user, 2 system)
no collections
19938808 bytes allocated
8 minor faults
no major faults
bewig2+macro-read-char.scm:
(time (fn))
112 ms real time
112 ms cpu time (99 user, 13 system)
no collections
45280848 bytes allocated
3148 minor faults
no major faults
(time (fn))
122 ms real time
122 ms cpu time (90 user, 32 system)
no collections
45280848 bytes allocated
7886 minor faults
no major faults
(time (fn))
111 ms real time
110 ms cpu time (91 user, 19 system)
no collections
45280848 bytes allocated
4716 minor faults
no major faults
(time (fn))
111 ms real time
111 ms cpu time (88 user, 23 system)
no collections
45280848 bytes allocated
4716 minor faults
no major faults
(time (fn))
111 ms real time
111 ms cpu time (95 user, 16 system)
no collections
45280848 bytes allocated
4716 minor faults
no major faults
More information about the Gambit-list
mailing list