[gambit-list] Your feedback would be much appreciated re: Proposal for enabling IO errors to be reported through returning of a custom value instead of by throwing exception, through DSL with exports: ##io-error-behavior param, ##default-io-error-behavior unique value, ##last-io-error param.

Mikael mikael.rcv at gmail.com
Fri Mar 15 09:51:32 EDT 2013

A port-specific flag makes perfect sense, it matches the intention of
providing specific, direct, fundamental error handling behavior exactly!

So this is as easy as it gets to do fwrite() < 0 checks :)

When zooming out and comparing, this makes perfect sense while a parameter
object does not, in that it's not as specific (e.g. web server switching
|##io-error-behavior| parameter to a value suiting its configuration, would
mean that user code that it invokes in turn, would inherit the server's
setting for its port use - not good at all).

Re the with-exception-handler possibility, presuming that it actually does
work - and indeed it can be made to work by just hammering until it
perfectly does - it is a workable solution. Considering though that IO
error handling are really local and simple phenomena within the IO DSL/API
and designed to happen All the time, involvement of the completely indirect
code path of using the exception system and reliance on an
execution-scope-global exception filtering procedure that does come with an
element of execution overhead global to the entire execution scope, seems
much less direct and fundamental than justified.

The port flag can be named |io-error| and have a procedure |io-error| as
default value. The procedure takes one argument being an error object, and
the procedure's return value is passed as return value of the Gambit IO
procedure that failed. This way any IO user can both specify IO primitive
return value and get a copy of the actual underlying IO error for later
reference (debug etc), this is all that's needed.

What do you say?

(Side reflections: This should scale very well also - this flag is quite
analogous to ports' timeout length and handler settings: Any code that uses
a port, must be pre-adapted to handle this configuration. In the long run,
if a port would be passed to be used in a completely unrelated setting,
then this would be done by passing a software port that abstracts on the
previous port: E.g., a HTTP client has TCP ports with certain IO error
handling and timeout configuration, and to give streaming access to HTTP
response body reception, the HTTP client provides a custom "HTTP response
body" port that the user may read-substring/etc. from directly. This custom
IO error handling is also useful for propagating errors within such a 'port
hierarchy' right, with involve code kept performant, concise, and

Best regards,

2013/3/13 Marc Feeley <feeley at iro.umontreal.ca>

> I'm not keen to add a parameter object for the handling of IO errors
> because there is an efficient solution with the current API.  It seems that
> the performance problem in your code is related to the cost of
> with-exception-catcher, that invokes call/cc implicitly, and which your
> code calls for every byte read.
> The solution I propose has no overhead for calls to IO primitives. The
> idea is to install an exception handler with the primitive
> with-exception-handler, for the execution of the code which performs IO.
>  It is important to remember that (almost all) primitives call exception
> handlers in tail position, so if the exception handler returns a value,
> that value will be the result of the primitive which raised an exception.
>  For example, if you want read-u8 to return #f when it raises an exception
> you could do this:
>   (continuation-capture
>    (lambda (cont)
>      (with-exception-handler  ;; <<<<< handler NOT catcher
>       (lambda (e)
>         (if (and (os-exception? e)
>                  (eq? (os-exception-procedure e) read-u8))
>             #f  ;; <<<<<< value which read-u8 will return
>             (continuation-graft cont (lambda () (raise e)))))
>       (lambda ()
>         ...the code which calls read-u8
>         ))))
> With this, a read-u8 will return #f instead of raising an os-exception.
> If you need this behavior for other IO procedures, change the test of the
> os-exception-procedure.  If you need to restrict this behavior to a
> specific port, you could add in the "and" the condition (eq? (car
> (os-exception-arguments e)) port) .
> Does this satisfy your requirements?  If my solution isn't adequate, then
> I'd like to explore the addition of a port specific flag because parameter
> objects have performance issues.
> Marc
> On 2013-03-10, at 8:55 AM, Mikael <mikael.rcv at gmail.com> wrote:
> > Dear Marc and Gambit community,
> >
> > Some while ago I emailed a suggestion for improvement to IO error
> handling for IO-intensive realworld applications (post at
> https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006322.html, "Proposal: Addition of a parameter option for signalling IO error as
> #f/#!eof result value instead of as exception").
> >
> > The update provides a way for higher IO performance and higher
> specificity of IO error reporting through the removal of need for up to as
> many with-exception-catcher calls as the CPU can perform IO procedure calls
> i.e. up to ~25,000,000 per second.
> >
> > This makes great sense as with-exception-catcher calls everywhere really
> clutters code, and, it's a medium-expensive procedure taking approx 0.344
> seconds per million calls (test file attached).
> >
> > Secondarily this update makes using Gambit's IO more holistic through
> guaranteeing that IO calls will never lead to an exception except if it's
> such a serious error that manual administrator or programmer intervention
> would reasonably be required, as for instance in the case of heap overflow.
> >
> > (As I got it, IO exceptions are thrown in corner cases of
> read/write-u8/char/substring/subu8vector even while generally #!eof or 0
> are returned.)
> >
> > The previous post worked at a level of concept, and I now wish to
> propose a specific implementation specification as to further the progress
> on this topic:
> >
> >
> > Implementation suggestion
> >
> > Three exports are needed to make it spin:
> >
> >      ##io-error-behavior
> >      ##default-io-error-behavior
> >      ##last-io-error
> >
> > ##io-error-behavior is a parameter object whose content determines
> Gambit's IO procedures' behavior in case of an IO error.
> >
> > It is set by default to ##default-io-error-behavior which is a unique
> object (an uninterned symbol or something).
> >
> > When an IO error happens,
> >
> >  * If ##io-error-behavior is set to ##default-io-error-behavior, then
> the error is handled as per Gambit's current behavior (exception or #!eof
> or 0 depending on procedure).
> >
> >  * If ##io-error-behavior is set to another value, then that value is
> returned.
> >
> > ##last-io-error is a parameter object whose value is set on IO error to
> the exception value or return value that Gambit produces in its default
> behavior. This way the caller can inspect the error further if interested.
> >
> >
> > Example use
> > ; => boolean, #t = success.
> > (define (handle-request port)
> >   (parameterize ((##io-error-behavior #f))
> >
> >     ; Read input data up to end of input stream
> >     (let loop ()
> >       (let ((c (read-u8 port)))
> >         (if c
> >           (begin
> >             ; [Handle char]
> >             (loop))
> >
> >           ; End of input stream reached.
> >           ;
> >           ; Acknowledge reception
> >           (and (display "OK\n" port) (force-output port)))))))
> >
> > The benefit here is the saving of a with-exception-catcher calls for
> each read-u8 call, and depending on how you'd do it otherwise, the saving
> of one or two with-exception-catcher calls for the display and force-output
> calls and saving of the possibility of doing force-output on port when
> display already reported it as closed.
> >
> >
> >
> > Concluding notes re implementing this
> > I may very well get into implementing a Gambit commit for providing this
> functionality.
> >
> > On my end, it will make ground for doing well-deserved cleanup to Sack
> and the to-be-released OpenSSL SSL channels module, both IO-intensive
> modules for production use, that currently use version-unsafe hacks to
> simulate the functionality proposed above.
> >
> > I do not claim any copyright on suggestions or commits I send to Gambit
> but leave them in public domain.
> >
> >
> > Bigger picture, for reference
> > What motivated me to propose this as well as recently proposing
> >
> >  1) A ##os-device-get-fd procedure (
> https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-March/006477.html
> )
> >
> >  2) A ##device-port-wait-for-output! procedure to complement
> ##device-port-wait-for-input (https://github.com/feeley/gambit/pull/6)
> >
> >  3) Some way to implement low-level application-level ports (
> https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006321.html
> )
> >
> > is that I've gotten to the conclusion that Gambit's builtin IO system is
> strikingly good as it is and has the potential to scale for any practical
> usecase I can imagine with only minor-ish tweaks.
> >
> >
> > The usecases to tweak for that I see relevant are
> >
> >  a) Realworld IO-intense use (which the proposal in this email seeks to
> improve in a quite fundamental way through a quite trivial tweak),
> >
> >  b) Deep C integration out of the box (which 1 and 2 above regard,
> really trivial tweaks those two too), and
> >
> >  c) Delivering low-level application-level ports e.g. SSL and GZip (3
> above regard this, I didn't get to any specific proposal for this til now
> but from reviewing _io.scm it cannot be a very big deal to do)
> >
> >  d) Later I'd guess: Hybrid character and byte access of ports (for
> ISO-8859-1 encoding, I believe it's already in the box, maybe some
> clarification needed and possibility for support for more encodings could
> be looked into)
> >
> > And last
> >
> >  e) Making the IO system's UTF-8 coder suitable for any use at all,
> because it's not due to a freeze-crash bug!!
> >      (I believe it's at
> http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=127 though
> Bugzilla is down right now)
> >
> >  f) Perhaps later: Higher IO procedure call performance through lower
> overhead on primarily the per-byte/character calls i.e. read-byte
> read-char, perhaps by making them automatically inlined and/or by making
> their mutex use configurable.
> >    read-u8 currently delivers ~350KB/sec and an increase to 25MB/sec
> should be realistic.
> >    For now though this can easily be worked around by application-level
> buffers and use of read/write-subu8vector, though that does come with
> drawbacks i.e. doesn't integrate well with hybrid byte-character use etc.
> >
> >
> > These are the only practical improvements that have come to my mind in
> total from several years of intense use of this functionality. Zooming out
> a bit, this is impressively little really.
> >
> >
> >
> > I would kindly ask you for your feedback on the suggestion regarded by
> this email. :)
> >
> >
> > Best regards,
> > Mikael
> >
> > <with-exception-catcher speed test, test results inlined.scm>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://webmail.iro.umontreal.ca/pipermail/gambit-list/attachments/20130315/ad88af20/attachment.html 

More information about the Gambit-list mailing list