[gambit-list] Your feedback would be much appreciated re: Proposal for enabling IO errors to be reported through returning of a custom value instead of by throwing exception, through DSL with exports: ##io-error-behavior param, ##default-io-error-behavior unique value, ##last-io-error param.

Mikael mikael.rcv at gmail.com
Sun Mar 10 08:55:16 EDT 2013


Dear Marc and Gambit community,

Some while ago I emailed a suggestion for improvement to IO error handling
for IO-intensive realworld applications (post at
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006322.html
,
"Proposal: Addition of a parameter option for signalling IO error as #f/
#!eof result value instead of as exception").

The update provides a way for higher IO performance and higher specificity
of IO error reporting through the removal of need for up to as many
with-exception-catcher calls as the CPU can perform IO procedure calls i.e.
up to ~25,000,000 per second.

This makes great sense as with-exception-catcher calls everywhere really
clutters code, and, it's a medium-expensive procedure taking approx 0.344
seconds per million calls (test file attached).

Secondarily this update makes using Gambit's IO more holistic through
guaranteeing that IO calls will never lead to an exception except if it's
such a serious error that manual administrator or programmer intervention
would reasonably be required, as for instance in the case of heap overflow.

(As I got it, IO exceptions are thrown in corner cases of
read/write-u8/char/substring/subu8vector even while generally #!eof or 0
are returned.)

The previous post worked at a level of concept, and I now wish to propose a
specific implementation specification as to further the progress on this
topic:


*Implementation suggestion*

Three exports are needed to make it spin:

     ##io-error-behavior
     ##default-io-error-behavior
     ##last-io-error

##io-error-behavior is a parameter object whose content determines Gambit's
IO procedures' behavior in case of an IO error.

It is set by default to ##default-io-error-behavior which is a unique
object (an uninterned symbol or something).

When an IO error happens,

 * If ##io-error-behavior is set to ##default-io-error-behavior, then the
error is handled as per Gambit's current behavior (exception or #!eof or 0
depending on procedure).

 * If ##io-error-behavior is set to another value, then that value is
returned.

##last-io-error is a parameter object whose value is set on IO error to the
exception value or return value that Gambit produces in its default
behavior. This way the caller can inspect the error further if interested.


*Example use*
; => boolean, #t = success.
(define (handle-request port)
  (parameterize ((##io-error-behavior #f))

    ; Read input data up to end of input stream
    (let loop ()
      (let ((c (read-u8 port)))
        (if c
          (begin
            ; [Handle char]
            (loop))

          ; End of input stream reached.
          ;
          ; Acknowledge reception
          (and (display "OK\n" port) (force-output port)))))))

The benefit here is the saving of a with-exception-catcher calls for each
read-u8 call, and depending on how you'd do it otherwise, the saving of one
or two with-exception-catcher calls for the display and force-output calls
and saving of the possibility of doing force-output on port when
displayalready reported it as closed.



*Concluding notes re implementing this*
I may very well get into implementing a Gambit commit for providing this
functionality.

On my end, it will make ground for doing well-deserved cleanup to Sack and
the to-be-released OpenSSL SSL channels module, both IO-intensive modules
for production use, that currently use version-unsafe hacks to simulate the
functionality proposed above.

I do not claim any copyright on suggestions or commits I send to Gambit but
leave them in public domain.


*Bigger picture, for reference*
What motivated me to propose this as well as recently proposing

 1) A ##os-device-get-fd procedure (
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-March/006477.html
)

 2) A ##device-port-wait-for-output! procedure to complement
##device-port-wait-for-input (https://github.com/feeley/gambit/pull/6)

 3) Some way to implement low-level application-level ports (
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006321.html
)

is that I've gotten to the conclusion that Gambit's builtin IO system is
strikingly good as it is and has the potential to scale for any practical
usecase I can imagine with only minor-ish tweaks.


The usecases to tweak for that I see relevant are

 a) Realworld IO-intense use (which the proposal in this email seeks to
improve in a quite fundamental way through a quite trivial tweak),

 b) Deep C integration out of the box (which 1 and 2 above regard, really
trivial tweaks those two too), and

 c) Delivering low-level application-level ports e.g. SSL and GZip (3 above
regard this, I didn't get to any specific proposal for this til now but
from reviewing _io.scm it cannot be a very big deal to do)

 d) Later I'd guess: Hybrid character and byte access of ports (for
ISO-8859-1 encoding, I believe it's already in the box, maybe some
clarification needed and possibility for support for more encodings could
be looked into)

And last

 e) Making the IO system's UTF-8 coder suitable for any use at all, because
it's not due to a freeze-crash bug!!
     (I believe it's at
http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=127 though
Bugzilla is down right now)

 f) Perhaps later: Higher IO procedure call performance through lower
overhead on primarily the per-byte/character calls i.e. read-byte
read-char, perhaps by making them automatically inlined and/or by making
their mutex use configurable.
   read-u8 currently delivers ~350KB/sec and an increase to 25MB/sec should
be realistic.
   For now though this can easily be worked around by application-level
buffers and use of read/write-subu8vector, though that does come with
drawbacks i.e. doesn't integrate well with hybrid byte-character use etc.


These are the only practical improvements that have come to my mind in
total from several years of intense use of this functionality. Zooming out
a bit, this is impressively little really.



I would kindly ask you for your feedback on the suggestion regarded by this
email. :)


Best regards,
Mikael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://webmail.iro.umontreal.ca/pipermail/gambit-list/attachments/20130310/49fdad53/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: with-exception-catcher speed test, test results inlined.scm
Type: application/octet-stream
Size: 1236 bytes
Desc: not available
Url : http://webmail.iro.umontreal.ca/pipermail/gambit-list/attachments/20130310/49fdad53/attachment.obj 


More information about the Gambit-list mailing list