[gambit-list] Speed on XML-parsing

Bradley Lucier lucier at math.purdue.edu
Sat Aug 16 17:34:16 EDT 2008


OK, I got sucked into this ...  If you have 8 gigs of ram for the
compile on x86-64 you can try this version of SSAX.  Since I don't have
any test data and I haven't ever tried expat I haven't done much more
than compile this.

I started with SSAX-SXML by Kirill Lisovsky (updated by Dominique
Boucher) from the dumping grounds at

http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Dumping_Grounds

I uncommented this line

./stx/libmisc.scm:(define (self x) x)

because it was needed later.  I also commented the calls to "gambitize"
things because I don't think they help so much.  (They don't eliminate
any inter-module procedure calls.)

./libs/input-parse.scm:;;(gambitize (read-char port))
./libs/input-parse.scm:;; (gambitize (peek-char port))
./libs/input-parse.scm:;; (gambitize (eof-object? port))

The Makefile has some mistakes (which can't be fixed in a
machine/OS-independent way, that's why Gambit has the configure
command), but it has a useful list of files needed to build
ssax-sxml.o1.

I include a file ssax-sxml.scm that I used to build SSAX on Ubuntu 8.0.4
on x86-64; if somebody's mail client munges this message, at least
you'll have the included file.  (The c->.o1 compile took about 6.3GB of
RAM.)  Here is an explanation of what it does.

First, I load the Gambit internal headers, which are always available at
~~/lib/, together with the names of all the standard procedures in r5rs:

(##include "~~/lib/gambit#.scm")
(##include "~~/lib/_gambit#.scm")
(##include "~~/lib/r5rs#.scm")

Then I set some declarations, mainly to reduce the size of the .c file
so recent versions of gcc can compile it.  (Supposedly, the introduction
of "factored definition-use chains" will reduce the memory requirement
for compiling Gambit-generated C code in gcc 4.4.*, but that change
hasn't been committed to SVN gcc yet.):

(declare (standard-bindings)(extended-bindings)(block)(fixnum))
(declare (not safe))
(declare (inlining-limit 134))

Then I want to redefine read-char and peek-char to local versions that
just contain the fast path from _io.scm.  I also want to redefine
eof-object? because that isn't expanded by gsc.  (Marc: Can you fix this
last thing without me filing a bug report?)  So I remove read-char,
peek-char, and eof-object? from the list of standard procedures the
compiler knows about, and I tell gsc to not do runtime expansion of
eof-object?

(declare (not standard-bindings read-char peek-char eof-object?))
(declare (not run-time-bindings eof-object?))

Then I redefine peek-char, read-char, and eof-object? as I decided to:

(define (ssax#peek-char port)

  (##declare (not interrupts-enabled))

  (macro-port-mutex-lock! port) ;; get exclusive access to port

    (let ((char-rlo (macro-character-port-rlo port))
          (char-rhi (macro-character-port-rhi port)))
      (if (##fixnum.< char-rlo char-rhi)

        ;; the next character is in the character read buffer

        (let ((c (##string-ref (macro-character-port-rbuf port) char-rlo)))
          (macro-port-mutex-unlock! port)
          c)

        (if (macro-character-port-peek-eof? port)

          (begin
            (macro-port-mutex-unlock! port)
            #!eof)
	  
	  (begin
	    (macro-port-mutex-unlock! port)
	    (##peek-char port))))))


(define (peek-char
	 #!optional
	 (port (macro-absent-obj)))
  (macro-force-vars (port)
    (let ((p
           (if (##eq? port (macro-absent-obj))
             (macro-current-input-port)
             port)))
      (macro-check-character-input-port p 1 (peek-char p)
        (ssax#peek-char p)))))


(define (ssax#read-char port)

  (##declare (not interrupts-enabled))

  (macro-port-mutex-lock! port) ;; get exclusive access to port

  (let loop ()

    (let ((char-rlo (macro-character-port-rlo port))
          (char-rhi (macro-character-port-rhi port)))
      (if (##fixnum.< char-rlo char-rhi)

        ;; the next character is in the character read buffer

        (let ((c (##string-ref (macro-character-port-rbuf port) char-rlo)))
          (if (##not (##char=? c #\newline))

            ;; frequent simple case, just advance rlo

            (begin
              (macro-character-port-rlo-set! port (##fixnum.+ char-rlo 1))
              (macro-port-mutex-unlock! port)
              c)

	    (begin
	      (macro-port-mutex-unlock! port)
	      (##read-char port))))))))


(define (read-char
              #!optional
              (port (macro-absent-obj)))
  (macro-force-vars (port)
    (let ((p
           (if (##eq? port (macro-absent-obj))
             (macro-current-input-port)
             port)))
      (macro-check-character-input-port p 1 (read-char p)
        (ssax#read-char p)))))

(define (ssax#eof-object? x)
  (##eq? x #!eof))

(define (eof-object? x)
  (macro-force-vars (x)
    (ssax#eof-object? x)))

Then I just include the list of .scm files included in ssax-sxml.o1:

(include "libs/gambit/common.scm")
(include "libs/gambit/myenv.scm")
(include "libs/srfi-13-local.scm")
(include "libs/util.scm")
(include "libs/gambit/parse-error.scm")
(include "libs/input-parse.scm")
(include "libs/look-for-str.scm")
(include "ssax/char-encoding.scm")
(include "ssax/SSAX-code.scm")
(include "ssax/SXML-tree-trans.scm")
(include "sxml-tools/sxpathlib.scm")
(include "multi-parser/id/srfi-12.scm")
(include "multi-parser/id/mime.scm")
(include "multi-parser/id/http.scm")
(include "multi-parser/id/access-remote.scm")
(include "multi-parser/id/id.scm")
(include "sxml-tools/xlink-parser.scm")
(include "multi-parser/ssax-prim.scm")
(include "multi-parser/multi-parser.scm")
(include "html-prag/htmlprag.scm")
(include "sxml-tools/sxml-tools.scm")
(include "sxml-tools/sxpath-ext.scm")
(include "sxml-tools/xpath-parser.scm")
(include "sxml-tools/txpath.scm")
(include "sxml-tools/sxpath.scm")
(include "sxml-tools/xpath-ast.scm")
(include "sxml-tools/xpath-context.scm")
(include "sxml-tools/xlink.scm")
(include "sxml-tools/ddo-axes.scm")
(include "sxml-tools/ddo-txpath.scm")
(include "sxml-tools/lazy-xpath.scm")
(include "ssax/lazy-ssax.scm")
(include "sxml-tools/modif.scm")
(include "sxml-tools/serializer.scm")
(include "sxml-tools/guides.scm")
(include "stx/libmisc.scm")
(include "stx/stx-engine.scm")

Then we get

frying-pan:~/programs/gambc-v4_2_3/ssax-sxml> time gsc -keep-c -expansion ssax-sxml > ! expansion.scm
150.589u 4.680s 2:36.47 99.2%	0+0k 16+46104io 0pf+0w
frying-pan:~/programs/gambc-v4_2_3/ssax-sxml> ll
total 13320
drwxr-xr-x 2 lucier lucier    4096 2008-08-16 16:54 apidoc/
-rw------- 1 lucier lucier    7623 2006-05-29 09:27 doc.txt
-rw------- 1 lucier lucier    1092 2006-05-29 09:27 example.sch
-rw------- 1 lucier lucier    5447 2006-05-29 09:27 example.scm
-rw-r--r-- 1 lucier lucier 3065182 2008-08-16 17:22 expansion.scm
drwxr-xr-x 2 lucier lucier    4096 2008-08-16 16:55 html-prag/
drwxr-xr-x 3 lucier lucier    4096 2008-08-16 16:55 libs/
-rw------- 1 lucier lucier    8361 2008-05-02 21:12 Makefile
drwxr-xr-x 3 lucier lucier    4096 2008-08-16 16:55 multi-parser/
-rw------- 1 lucier lucier   10081 2006-05-29 09:27 prj-ssax-sxml.s
drwxr-xr-x 2 lucier lucier    4096 2008-08-16 16:55 ssax/
-rw-r--r-- 1 lucier lucier 7505995 2008-08-16 17:20 ssax-sxml.c
-rwxr-xr-x 1 lucier lucier 2956306 2008-08-16 17:22 ssax-sxml.o1*
-rw-r--r-- 1 lucier lucier    3887 2008-08-16 16:56 ssax-sxml.scm
drwxr-xr-x 2 lucier lucier    4096 2008-08-16 16:55 stx/
drwxr-xr-x 3 lucier lucier    4096 2008-08-16 16:55 sxml-tools/
-rw------- 1 lucier lucier     234 2006-05-29 09:27 test-sxml.scm
drwxr-xr-x 2 lucier lucier    4096 2006-05-29 09:27 XML/

If there is some data around that I can benchmark this with, I'll try
it.  It seems to pass the tests in test-sxml.scm.

Brad
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssax-sxml.scm
Type: text/x-scheme
Size: 3887 bytes
Desc: not available
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20080816/edae6314/attachment.bin>


More information about the Gambit-list mailing list