On Feb 9, 2007, at 1:36 PM, Phil Dawes wrote:
Ok - that prompted me to do a little speed testing on a large file (6000 odd records). Note that Marc's comma splitter doesn't actually do the parsing properly because many records are split over multiple lines, so I'm guessing this is an upper performance limit on pure scheme?.
wrapped libcsv ~240ms Marc comma splitter ~510ms Phil Bewig csv parser ~1008ms
Here's what I get with libcsv and the buffered-output version of Phil Bewig's code + expanding the fast-path of read-char inline; the scheme version seems about 10% slower.
Brad
libcsv:
(time (fn)) 95 ms real time 94 ms cpu time (77 user, 17 system) no collections 19938808 bytes allocated 6156 minor faults no major faults (time (fn)) 102 ms real time 101 ms cpu time (99 user, 2 system) no collections 19938808 bytes allocated 28 minor faults no major faults (time (fn)) 102 ms real time 100 ms cpu time (99 user, 1 system) no collections 19938808 bytes allocated 8 minor faults no major faults (time (fn)) 102 ms real time 100 ms cpu time (99 user, 1 system) no collections 19938808 bytes allocated 8 minor faults no major faults (time (fn)) 101 ms real time 101 ms cpu time (99 user, 2 system) no collections 19938808 bytes allocated 8 minor faults no major faults
bewig2+macro-read-char.scm:
(time (fn)) 112 ms real time 112 ms cpu time (99 user, 13 system) no collections 45280848 bytes allocated 3148 minor faults no major faults (time (fn)) 122 ms real time 122 ms cpu time (90 user, 32 system) no collections 45280848 bytes allocated 7886 minor faults no major faults (time (fn)) 111 ms real time 110 ms cpu time (91 user, 19 system) no collections 45280848 bytes allocated 4716 minor faults no major faults (time (fn)) 111 ms real time 111 ms cpu time (88 user, 23 system) no collections 45280848 bytes allocated 4716 minor faults no major faults (time (fn)) 111 ms real time 111 ms cpu time (95 user, 16 system) no collections 45280848 bytes allocated 4716 minor faults no major faults