[gambit-list] Speed on XML-parsing
Bradley Lucier
lucier at math.purdue.edu
Sat Aug 16 17:34:16 EDT 2008
OK, I got sucked into this ... If you have 8 gigs of ram for the
compile on x86-64 you can try this version of SSAX. Since I don't have
any test data and I haven't ever tried expat I haven't done much more
than compile this.
I started with SSAX-SXML by Kirill Lisovsky (updated by Dominique
Boucher) from the dumping grounds at
http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Dumping_Grounds
I uncommented this line
./stx/libmisc.scm:(define (self x) x)
because it was needed later. I also commented the calls to "gambitize"
things because I don't think they help so much. (They don't eliminate
any inter-module procedure calls.)
./libs/input-parse.scm:;;(gambitize (read-char port))
./libs/input-parse.scm:;; (gambitize (peek-char port))
./libs/input-parse.scm:;; (gambitize (eof-object? port))
The Makefile has some mistakes (which can't be fixed in a
machine/OS-independent way, that's why Gambit has the configure
command), but it has a useful list of files needed to build
ssax-sxml.o1.
I include a file ssax-sxml.scm that I used to build SSAX on Ubuntu 8.0.4
on x86-64; if somebody's mail client munges this message, at least
you'll have the included file. (The c->.o1 compile took about 6.3GB of
RAM.) Here is an explanation of what it does.
First, I load the Gambit internal headers, which are always available at
~~/lib/, together with the names of all the standard procedures in r5rs:
(##include "~~/lib/gambit#.scm")
(##include "~~/lib/_gambit#.scm")
(##include "~~/lib/r5rs#.scm")
Then I set some declarations, mainly to reduce the size of the .c file
so recent versions of gcc can compile it. (Supposedly, the introduction
of "factored definition-use chains" will reduce the memory requirement
for compiling Gambit-generated C code in gcc 4.4.*, but that change
hasn't been committed to SVN gcc yet.):
(declare (standard-bindings)(extended-bindings)(block)(fixnum))
(declare (not safe))
(declare (inlining-limit 134))
Then I want to redefine read-char and peek-char to local versions that
just contain the fast path from _io.scm. I also want to redefine
eof-object? because that isn't expanded by gsc. (Marc: Can you fix this
last thing without me filing a bug report?) So I remove read-char,
peek-char, and eof-object? from the list of standard procedures the
compiler knows about, and I tell gsc to not do runtime expansion of
eof-object?
(declare (not standard-bindings read-char peek-char eof-object?))
(declare (not run-time-bindings eof-object?))
Then I redefine peek-char, read-char, and eof-object? as I decided to:
(define (ssax#peek-char port)
(##declare (not interrupts-enabled))
(macro-port-mutex-lock! port) ;; get exclusive access to port
(let ((char-rlo (macro-character-port-rlo port))
(char-rhi (macro-character-port-rhi port)))
(if (##fixnum.< char-rlo char-rhi)
;; the next character is in the character read buffer
(let ((c (##string-ref (macro-character-port-rbuf port) char-rlo)))
(macro-port-mutex-unlock! port)
c)
(if (macro-character-port-peek-eof? port)
(begin
(macro-port-mutex-unlock! port)
#!eof)
(begin
(macro-port-mutex-unlock! port)
(##peek-char port))))))
(define (peek-char
#!optional
(port (macro-absent-obj)))
(macro-force-vars (port)
(let ((p
(if (##eq? port (macro-absent-obj))
(macro-current-input-port)
port)))
(macro-check-character-input-port p 1 (peek-char p)
(ssax#peek-char p)))))
(define (ssax#read-char port)
(##declare (not interrupts-enabled))
(macro-port-mutex-lock! port) ;; get exclusive access to port
(let loop ()
(let ((char-rlo (macro-character-port-rlo port))
(char-rhi (macro-character-port-rhi port)))
(if (##fixnum.< char-rlo char-rhi)
;; the next character is in the character read buffer
(let ((c (##string-ref (macro-character-port-rbuf port) char-rlo)))
(if (##not (##char=? c #\newline))
;; frequent simple case, just advance rlo
(begin
(macro-character-port-rlo-set! port (##fixnum.+ char-rlo 1))
(macro-port-mutex-unlock! port)
c)
(begin
(macro-port-mutex-unlock! port)
(##read-char port))))))))
(define (read-char
#!optional
(port (macro-absent-obj)))
(macro-force-vars (port)
(let ((p
(if (##eq? port (macro-absent-obj))
(macro-current-input-port)
port)))
(macro-check-character-input-port p 1 (read-char p)
(ssax#read-char p)))))
(define (ssax#eof-object? x)
(##eq? x #!eof))
(define (eof-object? x)
(macro-force-vars (x)
(ssax#eof-object? x)))
Then I just include the list of .scm files included in ssax-sxml.o1:
(include "libs/gambit/common.scm")
(include "libs/gambit/myenv.scm")
(include "libs/srfi-13-local.scm")
(include "libs/util.scm")
(include "libs/gambit/parse-error.scm")
(include "libs/input-parse.scm")
(include "libs/look-for-str.scm")
(include "ssax/char-encoding.scm")
(include "ssax/SSAX-code.scm")
(include "ssax/SXML-tree-trans.scm")
(include "sxml-tools/sxpathlib.scm")
(include "multi-parser/id/srfi-12.scm")
(include "multi-parser/id/mime.scm")
(include "multi-parser/id/http.scm")
(include "multi-parser/id/access-remote.scm")
(include "multi-parser/id/id.scm")
(include "sxml-tools/xlink-parser.scm")
(include "multi-parser/ssax-prim.scm")
(include "multi-parser/multi-parser.scm")
(include "html-prag/htmlprag.scm")
(include "sxml-tools/sxml-tools.scm")
(include "sxml-tools/sxpath-ext.scm")
(include "sxml-tools/xpath-parser.scm")
(include "sxml-tools/txpath.scm")
(include "sxml-tools/sxpath.scm")
(include "sxml-tools/xpath-ast.scm")
(include "sxml-tools/xpath-context.scm")
(include "sxml-tools/xlink.scm")
(include "sxml-tools/ddo-axes.scm")
(include "sxml-tools/ddo-txpath.scm")
(include "sxml-tools/lazy-xpath.scm")
(include "ssax/lazy-ssax.scm")
(include "sxml-tools/modif.scm")
(include "sxml-tools/serializer.scm")
(include "sxml-tools/guides.scm")
(include "stx/libmisc.scm")
(include "stx/stx-engine.scm")
Then we get
frying-pan:~/programs/gambc-v4_2_3/ssax-sxml> time gsc -keep-c -expansion ssax-sxml > ! expansion.scm
150.589u 4.680s 2:36.47 99.2% 0+0k 16+46104io 0pf+0w
frying-pan:~/programs/gambc-v4_2_3/ssax-sxml> ll
total 13320
drwxr-xr-x 2 lucier lucier 4096 2008-08-16 16:54 apidoc/
-rw------- 1 lucier lucier 7623 2006-05-29 09:27 doc.txt
-rw------- 1 lucier lucier 1092 2006-05-29 09:27 example.sch
-rw------- 1 lucier lucier 5447 2006-05-29 09:27 example.scm
-rw-r--r-- 1 lucier lucier 3065182 2008-08-16 17:22 expansion.scm
drwxr-xr-x 2 lucier lucier 4096 2008-08-16 16:55 html-prag/
drwxr-xr-x 3 lucier lucier 4096 2008-08-16 16:55 libs/
-rw------- 1 lucier lucier 8361 2008-05-02 21:12 Makefile
drwxr-xr-x 3 lucier lucier 4096 2008-08-16 16:55 multi-parser/
-rw------- 1 lucier lucier 10081 2006-05-29 09:27 prj-ssax-sxml.s
drwxr-xr-x 2 lucier lucier 4096 2008-08-16 16:55 ssax/
-rw-r--r-- 1 lucier lucier 7505995 2008-08-16 17:20 ssax-sxml.c
-rwxr-xr-x 1 lucier lucier 2956306 2008-08-16 17:22 ssax-sxml.o1*
-rw-r--r-- 1 lucier lucier 3887 2008-08-16 16:56 ssax-sxml.scm
drwxr-xr-x 2 lucier lucier 4096 2008-08-16 16:55 stx/
drwxr-xr-x 3 lucier lucier 4096 2008-08-16 16:55 sxml-tools/
-rw------- 1 lucier lucier 234 2006-05-29 09:27 test-sxml.scm
drwxr-xr-x 2 lucier lucier 4096 2006-05-29 09:27 XML/
If there is some data around that I can benchmark this with, I'll try
it. It seems to pass the tests in test-sxml.scm.
Brad
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssax-sxml.scm
Type: text/x-scheme
Size: 3887 bytes
Desc: not available
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20080816/edae6314/attachment.bin>
More information about the Gambit-list
mailing list