[gambit-list] Speed on XML-parsing

Christian Jaeger christian at pflanze.mine.nu
Sat Aug 16 11:31:29 EDT 2008


Mikael More wrote:
> I see your point - if one writes a string-port, with completely custom open,
> read, write, close operations, one can achieve a lot higher speeds than if
> using Gambit's built-in I/O.
>
> Though, are the higher speeds Marc addressed reachable while still using
> Gambit built-in I/O operations, such as read, write, read-char, display,
> etc.?
>
> In order to make for instance SSAX-SXML use a custom string port
> implementation without
>
>  - modifying its code (i.e. changing all I/O operations there are in it to
> custom ones)
>   

That's easy, just alias the new read-char in place of the built-in one 
-- you've already using a separate ssax-sxml# namespace after all.

>  - dismaking its compatibility with Gambit's internal I/O functionality
> (i.e. file ports, TCP ports, etc.)
>   

Did you look at my example? You could just dump it in place of the 
built-in read-char *iff* you are sure you don't need thread-safe access 
to those ports.

> one needs to run this port implementation atop/behind/under/using Gambit's
> built-in I/O system. So, is it possible to increase the speed of the string
> port implementation

My example was even using file input ports, not string ports.

>  a lot, while still running on Gambit's built-in I/O?
>   

(My read-char example *is* using all of Gambit's built-in I/O except 
that it does not do the mutex locking.)

The Gambit mutex implementation could maybe be sped up, or complemented 
by some faster variant, in the current threading model (i.e. running in 
one system thread only), I did play with a Scheme-level spinlock 
implementation some time ago (which did atomic increments/decrements of 
a boxed integer through disabling interrupts / by using the C ffi, and 
loop running thread-yield! until the mutex is granted), but it only 
worked if all threads were running under the same priority since 
thread-yield! wouldn't yield to a thread with a lower priority; anyway 
should the thread system incorporate multiple system threads another 
mutex implementation (iff mutexes can be passed between system threads) 
will have to be made again so... (I could imagine to (help) work on this 
but don't currently see when this would be).

> If not, perhaps one would benefit from writing a custom buffered IO layer
> atop Gambit-s built-in IO, and then patch SSAX-SXML to use it. That ought to
> give blazingly high speeds, correct?
>   

If you want to make it guaranteed correct, you should find out (by 
reading the SSAX-SXML sources, or at least studying the api precisely 
and hoping you're taking correct conclusions from the study) to find out 
whether they don't somehow directly or indirectly allow different 
threads to read from the same port, or/and introduce the port mutex 
locking at a coarser granularity than character based.

I remember the discussion of the SSAX-SXML author boosting with his 
library being as fast as expat, but it was pointed out that this was 
only true if expat was run in character-reading mode too (smile), at 
which point he said that it's enjoyable to read in characters since this 
allows to precisely finish reading from a stream when an xml document 
has finished. So if you want to keep this philosophy, block-wise reading 
of input data is out. I guess you could still achieve that goal if you 
take the mutex when starting to read from a stream, then release it when 
the last piece of the document has been read. Where exactly you would 
introduce those lock/unlock calls into the work flow, I don't know as I 
haven't really worked with SSAX-SXML yet.

Christian.




More information about the Gambit-list mailing list