Mikael More wrote:
I see your point - if one writes a string-port, with completely custom open, read, write, close operations, one can achieve a lot higher speeds than if using Gambit's built-in I/O.
Though, are the higher speeds Marc addressed reachable while still using Gambit built-in I/O operations, such as read, write, read-char, display, etc.?
In order to make for instance SSAX-SXML use a custom string port implementation without
- modifying its code (i.e. changing all I/O operations there are in it to
custom ones)
That's easy, just alias the new read-char in place of the built-in one -- you've already using a separate ssax-sxml# namespace after all.
- dismaking its compatibility with Gambit's internal I/O functionality
(i.e. file ports, TCP ports, etc.)
Did you look at my example? You could just dump it in place of the built-in read-char *iff* you are sure you don't need thread-safe access to those ports.
one needs to run this port implementation atop/behind/under/using Gambit's built-in I/O system. So, is it possible to increase the speed of the string port implementation
My example was even using file input ports, not string ports.
a lot, while still running on Gambit's built-in I/O?
(My read-char example *is* using all of Gambit's built-in I/O except that it does not do the mutex locking.)
The Gambit mutex implementation could maybe be sped up, or complemented by some faster variant, in the current threading model (i.e. running in one system thread only), I did play with a Scheme-level spinlock implementation some time ago (which did atomic increments/decrements of a boxed integer through disabling interrupts / by using the C ffi, and loop running thread-yield! until the mutex is granted), but it only worked if all threads were running under the same priority since thread-yield! wouldn't yield to a thread with a lower priority; anyway should the thread system incorporate multiple system threads another mutex implementation (iff mutexes can be passed between system threads) will have to be made again so... (I could imagine to (help) work on this but don't currently see when this would be).
If not, perhaps one would benefit from writing a custom buffered IO layer atop Gambit-s built-in IO, and then patch SSAX-SXML to use it. That ought to give blazingly high speeds, correct?
If you want to make it guaranteed correct, you should find out (by reading the SSAX-SXML sources, or at least studying the api precisely and hoping you're taking correct conclusions from the study) to find out whether they don't somehow directly or indirectly allow different threads to read from the same port, or/and introduce the port mutex locking at a coarser granularity than character based.
I remember the discussion of the SSAX-SXML author boosting with his library being as fast as expat, but it was pointed out that this was only true if expat was run in character-reading mode too (smile), at which point he said that it's enjoyable to read in characters since this allows to precisely finish reading from a stream when an xml document has finished. So if you want to keep this philosophy, block-wise reading of input data is out. I guess you could still achieve that goal if you take the mutex when starting to read from a stream, then release it when the last piece of the document has been read. Where exactly you would introduce those lock/unlock calls into the work flow, I don't know as I haven't really worked with SSAX-SXML yet.
Christian.