[gambit-list] Speed on XML-parsing
Christian Jaeger
christian at pflanze.mine.nu
Sat Aug 16 11:31:29 EDT 2008
Mikael More wrote:
> I see your point - if one writes a string-port, with completely custom open,
> read, write, close operations, one can achieve a lot higher speeds than if
> using Gambit's built-in I/O.
>
> Though, are the higher speeds Marc addressed reachable while still using
> Gambit built-in I/O operations, such as read, write, read-char, display,
> etc.?
>
> In order to make for instance SSAX-SXML use a custom string port
> implementation without
>
> - modifying its code (i.e. changing all I/O operations there are in it to
> custom ones)
>
That's easy, just alias the new read-char in place of the built-in one
-- you've already using a separate ssax-sxml# namespace after all.
> - dismaking its compatibility with Gambit's internal I/O functionality
> (i.e. file ports, TCP ports, etc.)
>
Did you look at my example? You could just dump it in place of the
built-in read-char *iff* you are sure you don't need thread-safe access
to those ports.
> one needs to run this port implementation atop/behind/under/using Gambit's
> built-in I/O system. So, is it possible to increase the speed of the string
> port implementation
My example was even using file input ports, not string ports.
> a lot, while still running on Gambit's built-in I/O?
>
(My read-char example *is* using all of Gambit's built-in I/O except
that it does not do the mutex locking.)
The Gambit mutex implementation could maybe be sped up, or complemented
by some faster variant, in the current threading model (i.e. running in
one system thread only), I did play with a Scheme-level spinlock
implementation some time ago (which did atomic increments/decrements of
a boxed integer through disabling interrupts / by using the C ffi, and
loop running thread-yield! until the mutex is granted), but it only
worked if all threads were running under the same priority since
thread-yield! wouldn't yield to a thread with a lower priority; anyway
should the thread system incorporate multiple system threads another
mutex implementation (iff mutexes can be passed between system threads)
will have to be made again so... (I could imagine to (help) work on this
but don't currently see when this would be).
> If not, perhaps one would benefit from writing a custom buffered IO layer
> atop Gambit-s built-in IO, and then patch SSAX-SXML to use it. That ought to
> give blazingly high speeds, correct?
>
If you want to make it guaranteed correct, you should find out (by
reading the SSAX-SXML sources, or at least studying the api precisely
and hoping you're taking correct conclusions from the study) to find out
whether they don't somehow directly or indirectly allow different
threads to read from the same port, or/and introduce the port mutex
locking at a coarser granularity than character based.
I remember the discussion of the SSAX-SXML author boosting with his
library being as fast as expat, but it was pointed out that this was
only true if expat was run in character-reading mode too (smile), at
which point he said that it's enjoyable to read in characters since this
allows to precisely finish reading from a stream when an xml document
has finished. So if you want to keep this philosophy, block-wise reading
of input data is out. I guess you could still achieve that goal if you
take the mutex when starting to read from a stream, then release it when
the last piece of the document has been read. Where exactly you would
introduce those lock/unlock calls into the work flow, I don't know as I
haven't really worked with SSAX-SXML yet.
Christian.
More information about the Gambit-list
mailing list