Your feedback would be much appreciated re: Proposal for enabling IO errors to be reported through returning of a custom value instead of by throwing exception, through DSL with exports: ##io-error-behavior param, ##default-io-error-behavior unique value, ##last-io-error param.
Dear Marc and Gambit community,
Some while ago I emailed a suggestion for improvement to IO error handling for IO-intensive realworld applications (post at https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006322.h... , "Proposal: Addition of a parameter option for signalling IO error as #f/ #!eof result value instead of as exception").
The update provides a way for higher IO performance and higher specificity of IO error reporting through the removal of need for up to as many with-exception-catcher calls as the CPU can perform IO procedure calls i.e. up to ~25,000,000 per second.
This makes great sense as with-exception-catcher calls everywhere really clutters code, and, it's a medium-expensive procedure taking approx 0.344 seconds per million calls (test file attached).
Secondarily this update makes using Gambit's IO more holistic through guaranteeing that IO calls will never lead to an exception except if it's such a serious error that manual administrator or programmer intervention would reasonably be required, as for instance in the case of heap overflow.
(As I got it, IO exceptions are thrown in corner cases of read/write-u8/char/substring/subu8vector even while generally #!eof or 0 are returned.)
The previous post worked at a level of concept, and I now wish to propose a specific implementation specification as to further the progress on this topic:
*Implementation suggestion*
Three exports are needed to make it spin:
##io-error-behavior ##default-io-error-behavior ##last-io-error
##io-error-behavior is a parameter object whose content determines Gambit's IO procedures' behavior in case of an IO error.
It is set by default to ##default-io-error-behavior which is a unique object (an uninterned symbol or something).
When an IO error happens,
* If ##io-error-behavior is set to ##default-io-error-behavior, then the error is handled as per Gambit's current behavior (exception or #!eof or 0 depending on procedure).
* If ##io-error-behavior is set to another value, then that value is returned.
##last-io-error is a parameter object whose value is set on IO error to the exception value or return value that Gambit produces in its default behavior. This way the caller can inspect the error further if interested.
*Example use* ; => boolean, #t = success. (define (handle-request port) (parameterize ((##io-error-behavior #f))
; Read input data up to end of input stream (let loop () (let ((c (read-u8 port))) (if c (begin ; [Handle char] (loop))
; End of input stream reached. ; ; Acknowledge reception (and (display "OK\n" port) (force-output port)))))))
The benefit here is the saving of a with-exception-catcher calls for each read-u8 call, and depending on how you'd do it otherwise, the saving of one or two with-exception-catcher calls for the display and force-output calls and saving of the possibility of doing force-output on port when displayalready reported it as closed.
*Concluding notes re implementing this* I may very well get into implementing a Gambit commit for providing this functionality.
On my end, it will make ground for doing well-deserved cleanup to Sack and the to-be-released OpenSSL SSL channels module, both IO-intensive modules for production use, that currently use version-unsafe hacks to simulate the functionality proposed above.
I do not claim any copyright on suggestions or commits I send to Gambit but leave them in public domain.
*Bigger picture, for reference* What motivated me to propose this as well as recently proposing
1) A ##os-device-get-fd procedure ( https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-March/006477.htm... )
2) A ##device-port-wait-for-output! procedure to complement ##device-port-wait-for-input (https://github.com/feeley/gambit/pull/6)
3) Some way to implement low-level application-level ports ( https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006321.h... )
is that I've gotten to the conclusion that Gambit's builtin IO system is strikingly good as it is and has the potential to scale for any practical usecase I can imagine with only minor-ish tweaks.
The usecases to tweak for that I see relevant are
a) Realworld IO-intense use (which the proposal in this email seeks to improve in a quite fundamental way through a quite trivial tweak),
b) Deep C integration out of the box (which 1 and 2 above regard, really trivial tweaks those two too), and
c) Delivering low-level application-level ports e.g. SSL and GZip (3 above regard this, I didn't get to any specific proposal for this til now but from reviewing _io.scm it cannot be a very big deal to do)
d) Later I'd guess: Hybrid character and byte access of ports (for ISO-8859-1 encoding, I believe it's already in the box, maybe some clarification needed and possibility for support for more encodings could be looked into)
And last
e) Making the IO system's UTF-8 coder suitable for any use at all, because it's not due to a freeze-crash bug!! (I believe it's at http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=127 though Bugzilla is down right now)
f) Perhaps later: Higher IO procedure call performance through lower overhead on primarily the per-byte/character calls i.e. read-byte read-char, perhaps by making them automatically inlined and/or by making their mutex use configurable. read-u8 currently delivers ~350KB/sec and an increase to 25MB/sec should be realistic. For now though this can easily be worked around by application-level buffers and use of read/write-subu8vector, though that does come with drawbacks i.e. doesn't integrate well with hybrid byte-character use etc.
These are the only practical improvements that have come to my mind in total from several years of intense use of this functionality. Zooming out a bit, this is impressively little really.
I would kindly ask you for your feedback on the suggestion regarded by this email. :)
Best regards, Mikael
Afficher les réponses par date
I'm not keen to add a parameter object for the handling of IO errors because there is an efficient solution with the current API. It seems that the performance problem in your code is related to the cost of with-exception-catcher, that invokes call/cc implicitly, and which your code calls for every byte read.
The solution I propose has no overhead for calls to IO primitives. The idea is to install an exception handler with the primitive with-exception-handler, for the execution of the code which performs IO. It is important to remember that (almost all) primitives call exception handlers in tail position, so if the exception handler returns a value, that value will be the result of the primitive which raised an exception. For example, if you want read-u8 to return #f when it raises an exception you could do this:
(continuation-capture (lambda (cont) (with-exception-handler ;; <<<<< handler NOT catcher (lambda (e) (if (and (os-exception? e) (eq? (os-exception-procedure e) read-u8)) #f ;; <<<<<< value which read-u8 will return (continuation-graft cont (lambda () (raise e))))) (lambda () ...the code which calls read-u8 ))))
With this, a read-u8 will return #f instead of raising an os-exception.
If you need this behavior for other IO procedures, change the test of the os-exception-procedure. If you need to restrict this behavior to a specific port, you could add in the "and" the condition (eq? (car (os-exception-arguments e)) port) .
Does this satisfy your requirements? If my solution isn't adequate, then I'd like to explore the addition of a port specific flag because parameter objects have performance issues.
Marc
On 2013-03-10, at 8:55 AM, Mikael mikael.rcv@gmail.com wrote:
Dear Marc and Gambit community,
Some while ago I emailed a suggestion for improvement to IO error handling for IO-intensive realworld applications (post at https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006322.h... , "Proposal: Addition of a parameter option for signalling IO error as #f/#!eof result value instead of as exception").
The update provides a way for higher IO performance and higher specificity of IO error reporting through the removal of need for up to as many with-exception-catcher calls as the CPU can perform IO procedure calls i.e. up to ~25,000,000 per second.
This makes great sense as with-exception-catcher calls everywhere really clutters code, and, it's a medium-expensive procedure taking approx 0.344 seconds per million calls (test file attached).
Secondarily this update makes using Gambit's IO more holistic through guaranteeing that IO calls will never lead to an exception except if it's such a serious error that manual administrator or programmer intervention would reasonably be required, as for instance in the case of heap overflow.
(As I got it, IO exceptions are thrown in corner cases of read/write-u8/char/substring/subu8vector even while generally #!eof or 0 are returned.)
The previous post worked at a level of concept, and I now wish to propose a specific implementation specification as to further the progress on this topic:
Implementation suggestion
Three exports are needed to make it spin:
##io-error-behavior ##default-io-error-behavior ##last-io-error##io-error-behavior is a parameter object whose content determines Gambit's IO procedures' behavior in case of an IO error.
It is set by default to ##default-io-error-behavior which is a unique object (an uninterned symbol or something).
When an IO error happens,
If ##io-error-behavior is set to ##default-io-error-behavior, then the error is handled as per Gambit's current behavior (exception or #!eof or 0 depending on procedure).
If ##io-error-behavior is set to another value, then that value is returned.
##last-io-error is a parameter object whose value is set on IO error to the exception value or return value that Gambit produces in its default behavior. This way the caller can inspect the error further if interested.
Example use ; => boolean, #t = success. (define (handle-request port) (parameterize ((##io-error-behavior #f))
; Read input data up to end of input stream (let loop () (let ((c (read-u8 port))) (if c (begin ; [Handle char] (loop)) ; End of input stream reached. ; ; Acknowledge reception (and (display "OK\n" port) (force-output port)))))))The benefit here is the saving of a with-exception-catcher calls for each read-u8 call, and depending on how you'd do it otherwise, the saving of one or two with-exception-catcher calls for the display and force-output calls and saving of the possibility of doing force-output on port when display already reported it as closed.
Concluding notes re implementing this I may very well get into implementing a Gambit commit for providing this functionality.
On my end, it will make ground for doing well-deserved cleanup to Sack and the to-be-released OpenSSL SSL channels module, both IO-intensive modules for production use, that currently use version-unsafe hacks to simulate the functionality proposed above.
I do not claim any copyright on suggestions or commits I send to Gambit but leave them in public domain.
Bigger picture, for reference What motivated me to propose this as well as recently proposing
A ##os-device-get-fd procedure (https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-March/006477.htm...)
A ##device-port-wait-for-output! procedure to complement ##device-port-wait-for-input (https://github.com/feeley/gambit/pull/6)
Some way to implement low-level application-level ports (https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006321.h...)
is that I've gotten to the conclusion that Gambit's builtin IO system is strikingly good as it is and has the potential to scale for any practical usecase I can imagine with only minor-ish tweaks.
The usecases to tweak for that I see relevant are
a) Realworld IO-intense use (which the proposal in this email seeks to improve in a quite fundamental way through a quite trivial tweak),
b) Deep C integration out of the box (which 1 and 2 above regard, really trivial tweaks those two too), and
c) Delivering low-level application-level ports e.g. SSL and GZip (3 above regard this, I didn't get to any specific proposal for this til now but from reviewing _io.scm it cannot be a very big deal to do)
d) Later I'd guess: Hybrid character and byte access of ports (for ISO-8859-1 encoding, I believe it's already in the box, maybe some clarification needed and possibility for support for more encodings could be looked into)
And last
e) Making the IO system's UTF-8 coder suitable for any use at all, because it's not due to a freeze-crash bug!! (I believe it's at http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=127 though Bugzilla is down right now)
f) Perhaps later: Higher IO procedure call performance through lower overhead on primarily the per-byte/character calls i.e. read-byte read-char, perhaps by making them automatically inlined and/or by making their mutex use configurable. read-u8 currently delivers ~350KB/sec and an increase to 25MB/sec should be realistic. For now though this can easily be worked around by application-level buffers and use of read/write-subu8vector, though that does come with drawbacks i.e. doesn't integrate well with hybrid byte-character use etc.
These are the only practical improvements that have come to my mind in total from several years of intense use of this functionality. Zooming out a bit, this is impressively little really.
I would kindly ask you for your feedback on the suggestion regarded by this email. :)
Best regards, Mikael
<with-exception-catcher speed test, test results inlined.scm>
A port-specific flag makes perfect sense, it matches the intention of providing specific, direct, fundamental error handling behavior exactly!
So this is as easy as it gets to do fwrite() < 0 checks :)
When zooming out and comparing, this makes perfect sense while a parameter object does not, in that it's not as specific (e.g. web server switching |##io-error-behavior| parameter to a value suiting its configuration, would mean that user code that it invokes in turn, would inherit the server's setting for its port use - not good at all).
Re the with-exception-handler possibility, presuming that it actually does work - and indeed it can be made to work by just hammering until it perfectly does - it is a workable solution. Considering though that IO error handling are really local and simple phenomena within the IO DSL/API and designed to happen All the time, involvement of the completely indirect code path of using the exception system and reliance on an execution-scope-global exception filtering procedure that does come with an element of execution overhead global to the entire execution scope, seems much less direct and fundamental than justified.
The port flag can be named |io-error| and have a procedure |io-error| as default value. The procedure takes one argument being an error object, and the procedure's return value is passed as return value of the Gambit IO procedure that failed. This way any IO user can both specify IO primitive return value and get a copy of the actual underlying IO error for later reference (debug etc), this is all that's needed.
What do you say?
(Side reflections: This should scale very well also - this flag is quite analogous to ports' timeout length and handler settings: Any code that uses a port, must be pre-adapted to handle this configuration. In the long run, if a port would be passed to be used in a completely unrelated setting, then this would be done by passing a software port that abstracts on the previous port: E.g., a HTTP client has TCP ports with certain IO error handling and timeout configuration, and to give streaming access to HTTP response body reception, the HTTP client provides a custom "HTTP response body" port that the user may read-substring/etc. from directly. This custom IO error handling is also useful for propagating errors within such a 'port hierarchy' right, with involve code kept performant, concise, and version-safe.)
Best regards, Mikael
2013/3/13 Marc Feeley feeley@iro.umontreal.ca
I'm not keen to add a parameter object for the handling of IO errors because there is an efficient solution with the current API. It seems that the performance problem in your code is related to the cost of with-exception-catcher, that invokes call/cc implicitly, and which your code calls for every byte read.
The solution I propose has no overhead for calls to IO primitives. The idea is to install an exception handler with the primitive with-exception-handler, for the execution of the code which performs IO. It is important to remember that (almost all) primitives call exception handlers in tail position, so if the exception handler returns a value, that value will be the result of the primitive which raised an exception. For example, if you want read-u8 to return #f when it raises an exception you could do this:
(continuation-capture (lambda (cont) (with-exception-handler ;; <<<<< handler NOT catcher (lambda (e) (if (and (os-exception? e) (eq? (os-exception-procedure e) read-u8)) #f ;; <<<<<< value which read-u8 will return (continuation-graft cont (lambda () (raise e))))) (lambda () ...the code which calls read-u8 ))))
With this, a read-u8 will return #f instead of raising an os-exception.
If you need this behavior for other IO procedures, change the test of the os-exception-procedure. If you need to restrict this behavior to a specific port, you could add in the "and" the condition (eq? (car (os-exception-arguments e)) port) .
Does this satisfy your requirements? If my solution isn't adequate, then I'd like to explore the addition of a port specific flag because parameter objects have performance issues.
Marc
On 2013-03-10, at 8:55 AM, Mikael mikael.rcv@gmail.com wrote:
Dear Marc and Gambit community,
Some while ago I emailed a suggestion for improvement to IO error
handling for IO-intensive realworld applications (post at https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006322.h..., "Proposal: Addition of a parameter option for signalling IO error as #f/#!eof result value instead of as exception").
The update provides a way for higher IO performance and higher
specificity of IO error reporting through the removal of need for up to as many with-exception-catcher calls as the CPU can perform IO procedure calls i.e. up to ~25,000,000 per second.
This makes great sense as with-exception-catcher calls everywhere really
clutters code, and, it's a medium-expensive procedure taking approx 0.344 seconds per million calls (test file attached).
Secondarily this update makes using Gambit's IO more holistic through
guaranteeing that IO calls will never lead to an exception except if it's such a serious error that manual administrator or programmer intervention would reasonably be required, as for instance in the case of heap overflow.
(As I got it, IO exceptions are thrown in corner cases of
read/write-u8/char/substring/subu8vector even while generally #!eof or 0 are returned.)
The previous post worked at a level of concept, and I now wish to
propose a specific implementation specification as to further the progress on this topic:
Implementation suggestion
Three exports are needed to make it spin:
##io-error-behavior ##default-io-error-behavior ##last-io-error##io-error-behavior is a parameter object whose content determines
Gambit's IO procedures' behavior in case of an IO error.
It is set by default to ##default-io-error-behavior which is a unique
object (an uninterned symbol or something).
When an IO error happens,
- If ##io-error-behavior is set to ##default-io-error-behavior, then
the error is handled as per Gambit's current behavior (exception or #!eof or 0 depending on procedure).
- If ##io-error-behavior is set to another value, then that value is
returned.
##last-io-error is a parameter object whose value is set on IO error to
the exception value or return value that Gambit produces in its default behavior. This way the caller can inspect the error further if interested.
Example use ; => boolean, #t = success. (define (handle-request port) (parameterize ((##io-error-behavior #f))
; Read input data up to end of input stream (let loop () (let ((c (read-u8 port))) (if c (begin ; [Handle char] (loop)) ; End of input stream reached. ; ; Acknowledge reception (and (display "OK\n" port) (force-output port)))))))The benefit here is the saving of a with-exception-catcher calls for
each read-u8 call, and depending on how you'd do it otherwise, the saving of one or two with-exception-catcher calls for the display and force-output calls and saving of the possibility of doing force-output on port when display already reported it as closed.
Concluding notes re implementing this I may very well get into implementing a Gambit commit for providing this
functionality.
On my end, it will make ground for doing well-deserved cleanup to Sack
and the to-be-released OpenSSL SSL channels module, both IO-intensive modules for production use, that currently use version-unsafe hacks to simulate the functionality proposed above.
I do not claim any copyright on suggestions or commits I send to Gambit
but leave them in public domain.
Bigger picture, for reference What motivated me to propose this as well as recently proposing
- A ##os-device-get-fd procedure (
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-March/006477.htm... )
- A ##device-port-wait-for-output! procedure to complement
##device-port-wait-for-input (https://github.com/feeley/gambit/pull/6)
- Some way to implement low-level application-level ports (
https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006321.h... )
is that I've gotten to the conclusion that Gambit's builtin IO system is
strikingly good as it is and has the potential to scale for any practical usecase I can imagine with only minor-ish tweaks.
The usecases to tweak for that I see relevant are
a) Realworld IO-intense use (which the proposal in this email seeks to
improve in a quite fundamental way through a quite trivial tweak),
b) Deep C integration out of the box (which 1 and 2 above regard,
really trivial tweaks those two too), and
c) Delivering low-level application-level ports e.g. SSL and GZip (3
above regard this, I didn't get to any specific proposal for this til now but from reviewing _io.scm it cannot be a very big deal to do)
d) Later I'd guess: Hybrid character and byte access of ports (for
ISO-8859-1 encoding, I believe it's already in the box, maybe some clarification needed and possibility for support for more encodings could be looked into)
And last
e) Making the IO system's UTF-8 coder suitable for any use at all,
because it's not due to a freeze-crash bug!!
(I believe it's athttp://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=127 though Bugzilla is down right now)
f) Perhaps later: Higher IO procedure call performance through lower
overhead on primarily the per-byte/character calls i.e. read-byte read-char, perhaps by making them automatically inlined and/or by making their mutex use configurable.
read-u8 currently delivers ~350KB/sec and an increase to 25MB/sec
should be realistic.
For now though this can easily be worked around by application-level
buffers and use of read/write-subu8vector, though that does come with drawbacks i.e. doesn't integrate well with hybrid byte-character use etc.
These are the only practical improvements that have come to my mind in
total from several years of intense use of this functionality. Zooming out a bit, this is impressively little really.
I would kindly ask you for your feedback on the suggestion regarded by
this email. :)
Best regards, Mikael
<with-exception-catcher speed test, test results inlined.scm>
On 2013-03-15, at 9:51 AM, Mikael mikael.rcv@gmail.com wrote:
A port-specific flag makes perfect sense, it matches the intention of providing specific, direct, fundamental error handling behavior exactly!
So this is as easy as it gets to do fwrite() < 0 checks :)
When zooming out and comparing, this makes perfect sense while a parameter object does not, in that it's not as specific (e.g. web server switching |##io-error-behavior| parameter to a value suiting its configuration, would mean that user code that it invokes in turn, would inherit the server's setting for its port use - not good at all).
Re the with-exception-handler possibility, presuming that it actually does work - and indeed it can be made to work by just hammering until it perfectly does - it is a workable solution. Considering though that IO error handling are really local and simple phenomena within the IO DSL/API and designed to happen All the time, involvement of the completely indirect code path of using the exception system and reliance on an execution-scope-global exception filtering procedure that does come with an element of execution overhead global to the entire execution scope, seems much less direct and fundamental than justified.
The port flag can be named |io-error| and have a procedure |io-error| as default value. The procedure takes one argument being an error object, and the procedure's return value is passed as return value of the Gambit IO procedure that failed. This way any IO user can both specify IO primitive return value and get a copy of the actual underlying IO error for later reference (debug etc), this is all that's needed.
What do you say?
The problem I see with the approach you propose (whether it uses a parameter or a port specific flag) is that the semantics of an "io-error" is vague. What is an IO error? The definition is important because exceptions that are IO errors are going to be processed using this new mechanism, and non-IO exceptions will use a different mechanism (normal exception handling).
The approach I propose does not have this problem because the programmer has complete control over the definition of an IO error. The exception object can be inspected to see if it qualifies as an IO error and an appropriate action can be taken. The definition of IO error can depend on the type of port, the type of primitive which caused the exception (read-u8, read-char, read), etc.
Marc
Hi Marc!
2013/3/18 Marc Feeley feeley@iro.umontreal.ca
The problem I see with the approach you propose (whether it uses a parameter or a port specific flag) is that the semantics of an "io-error" is vague. What is an IO error? The definition is important because exceptions that are IO errors are going to be processed using this new mechanism, and non-IO exceptions will use a different mechanism (normal exception handling).
The approach I propose does not have this problem because the programmer has complete control over the definition of an IO error. The exception object can be inspected to see if it qualifies as an IO error and an appropriate action can be taken. The definition of IO error can depend on the type of port, the type of primitive which caused the exception (read-u8, read-char, read), etc.
Marc
Aha. To really get this, there are two quite fundamental things about the applicability of the with-exception-handler possibility that I maybe don't get yet or at least would benefit of clarification, can we have a look at it?
Also last a question re semantics of io-error.
So, for the with-exception-handler mechanism to work out, it needs to go together with other use of exception handling use that's being done in the same scope.
In a setup where there's another exception handler *outside* the IO with-exception-handler, there would be no issue as the IO w.e.h. 's handler would be invoked first for any exception that occurs, and do its matching and handling.
In a setup where the other exception handler is made *inside* the IO w.e.h. though, any IO exception that arises within that exception handler will be picked up by it first, and for the IO w.e.h. thing to work out, that handler needs to be able to pass on the exception to the parent exception handler (which is the IO w.e.h.) completely in its original shape, in such a way that if the IO w.e.h. handler returns a value, that will be passed as return value to the original |raise| call.
An example of this would be a web-based PI calculator that uses exception handling locally in its PI calculate request thunk to pick up invalid user input.
So let's ask how this could be done.
Let's say below that we have a procedure (install-io-w.e.h. port thunk) that installs the IO w.e.h. that makes IO primitives return #f on exception, so that thunk is invoked with that w.e.h. installed.
Then, we have application logics (logics) that, aside from using IO, internally uses exception handling.
So a setup something like,
(define (logics) (with-exception-catcher (lambda (e) "Logics failed due to invalid user input!") (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
The desired behavior here is to print #f to the console and for logics to then return "Logics failed due to invalid user input!".
The issue now becomes, how would this local exception handler need to be implemented as for this to work out.
The local exception handler needs to be specific about what kind of error it looks for as to know which to handle locally and which to re-raise. This might be a complete PITA in some situations as you're looking for a catch-all behavior, as in the example above!
Perhaps the order of exception handlers could be tweaked somehow, so that the IO w.e.h. would get highest priority or something, though how could that be made as a 'clean' abstraction? I mean, who's in a place to claim at a general level that one exception is of higher prio than another? - different classes could be introduced, like, "user exceptions" and "io exceptions" or "system exception" (this would lead to an at least almost functional equivalent of the port specific flag solution, just with a more indirect code path!), or, the exception matching procedure could be exported to a separate mechanism, and the exception handler claiming the highest specificity in the matching would get to handle it e.g. (with-exception-handler exception-match exception-handler thunk) where exception-match takes an exception argument e and returns how well it matches the exception, i duno as a boolean or 0-10 or a symbol.
Or, a global parameter object |is-nonlocal-exception?| could be introduced that any exception handler is free to invoke as to check if it should re-raise the exception, and to overlap.. hmm, a more general solution would be better.
Do you see any general solution to this specificity problem?
Now to get to the next thing, let's just presume there's a solution to this and we represent it in this example as a (if (is-nonlocal-exception? e) (raise e) condition in the local exception handler, again not because this would necessarily represent a general solution but just to get on to the next thing in the reasoning; so now we have
(define (logics) (with-exception-catcher (lambda (e) (if (is-nonlocal-exception? e) (raise e) "Logics failed due to invalid user input!")) (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
Now, how do you make this *parent* exception handler (the IO w.e.h.) that got the exception passed to it, able to pass a return value to the original |raise| call?
This is required for the exception handling-based IO error handling behavior we're looking for to work.
The problem reduces to
(define (logics) (raise "Please return 'properly-handled!"))
(define (proxy-exception-handler/catcher2 thunk) (lambda () (with-exception-catcher (lambda (e) (raise e)) thunk)))
(define (proxy-exception-handler/catcher1 thunk) (lambda () (with-exception-handler (lambda (e) (raise e)) thunk)))
(define (parent-exception-handler thunk) (continuation-capture (lambda (cont) (with-exception-handler (lambda (e) 'properly-handled!) thunk))))
where proxy-exception-handler/catcher 1 & 2 are to represent an arbitrary chain of exception handlers that logics code may come up with. The intended behavior is
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
Actually evaluating these three tests showed:
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
To start with, great that we see that with-exception-catcher delivers out of the box for this usecase!
Thus we have with-exception-handler left. I guess the best way would be if with-exception-handler inherently somehow would deliver for this, so that support for this kind of use would be transparent and not require possible updates of user code (e.g. any typical user code such as a PI calculator etc. could without needing code review just be run within a web server that uses this special IO error handling).
Is there any way to make with-exception-handler deliver for this usecase?
Last, what about that for introducing a port specific flag there would be the issue that the semantics of an IO error is vague -
Maybe you see something here I didn't get. As IO error for a port would count any error reported from the OS about the port, as well as timeouts within Gambit's IO system.
So this would correspond to any OS IO primitive invocation that gives an error return value (other than one that asks for a reiteration of the procedure, which some of them come with).
Another way to relate to it would be that such a port specific flag would serve to protect from requirement of programmer or admin intervention for any IO error that could possibly come up regarding the addressed ports.
What do you see here?
Best regards, Mikael
Dear Marc,
Spontaneously I think that the prospect of using with-exception-handler as you propose sounds better than adding custom error behavior code to the IO, as, the w.e.h. route would be more holistic in that it maintains the error handling mechanism in Gambit one in total in number; also,
It would work from a performance point of view as there is no overhead per IO primitive call that's successful, which accounts for almost all of them. The only potential issue with performance would be if the exception handler part would take a lot of time, though I guess it can be generalized that that is not an issue.
Then I guess the last point would be the design aspect that if somehow IO code would be run outside the current environment that the w.e.h. is installed in, the custom error behavior would disappear; that would indeed be a benefit with a port specific flag, that it's not subject to the same limitation. Though, from the practical use I see today, that more or less does not happen so it's fine.
I didn't think deeper about the possibility of w.e.h. before as I found myself without clarity on how the things I addressed in the previous email could be solved, as they need to be solved for it to be a practically viable solution.
Looking forward a lot to hear your take on those two-three things and hopefully get to a reliable io-error => #f soon =)
Thanks, Mikael
2013/3/20 Mikael
Hi Marc!
2013/3/18 Marc Feeley feeley@iro.umontreal.ca
The problem I see with the approach you propose (whether it uses a parameter or a port specific flag) is that the semantics of an "io-error" is vague. What is an IO error? The definition is important because exceptions that are IO errors are going to be processed using this new mechanism, and non-IO exceptions will use a different mechanism (normal exception handling).
The approach I propose does not have this problem because the programmer has complete control over the definition of an IO error. The exception object can be inspected to see if it qualifies as an IO error and an appropriate action can be taken. The definition of IO error can depend on the type of port, the type of primitive which caused the exception (read-u8, read-char, read), etc.
Marc
Aha. To really get this, there are two quite fundamental things about the applicability of the with-exception-handler possibility that I maybe don't get yet or at least would benefit of clarification, can we have a look at it?
Also last a question re semantics of io-error.
So, for the with-exception-handler mechanism to work out, it needs to go together with other use of exception handling use that's being done in the same scope.
In a setup where there's another exception handler *outside* the IO with-exception-handler, there would be no issue as the IO w.e.h. 's handler would be invoked first for any exception that occurs, and do its matching and handling.
In a setup where the other exception handler is made *inside* the IO w.e.h. though, any IO exception that arises within that exception handler will be picked up by it first, and for the IO w.e.h. thing to work out, that handler needs to be able to pass on the exception to the parent exception handler (which is the IO w.e.h.) completely in its original shape, in such a way that if the IO w.e.h. handler returns a value, that will be passed as return value to the original |raise| call.
An example of this would be a web-based PI calculator that uses exception handling locally in its PI calculate request thunk to pick up invalid user input.
So let's ask how this could be done.
Let's say below that we have a procedure (install-io-w.e.h. port thunk) that installs the IO w.e.h. that makes IO primitives return #f on exception, so that thunk is invoked with that w.e.h. installed.
Then, we have application logics (logics) that, aside from using IO, internally uses exception handling.
So a setup something like,
(define (logics) (with-exception-catcher (lambda (e) "Logics failed due to invalid user input!") (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
The desired behavior here is to print #f to the console and for logics to then return "Logics failed due to invalid user input!".
The issue now becomes, how would this local exception handler need to be implemented as for this to work out.
The local exception handler needs to be specific about what kind of error it looks for as to know which to handle locally and which to re-raise. This might be a complete PITA in some situations as you're looking for a catch-all behavior, as in the example above!
Perhaps the order of exception handlers could be tweaked somehow, so that the IO w.e.h. would get highest priority or something, though how could that be made as a 'clean' abstraction? I mean, who's in a place to claim at a general level that one exception is of higher prio than another? - different classes could be introduced, like, "user exceptions" and "io exceptions" or "system exception" (this would lead to an at least almost functional equivalent of the port specific flag solution, just with a more indirect code path!), or, the exception matching procedure could be exported to a separate mechanism, and the exception handler claiming the highest specificity in the matching would get to handle it e.g. (with-exception-handler exception-match exception-handler thunk) where exception-match takes an exception argument e and returns how well it matches the exception, i duno as a boolean or 0-10 or a symbol.
Or, a global parameter object |is-nonlocal-exception?| could be introduced that any exception handler is free to invoke as to check if it should re-raise the exception, and to overlap.. hmm, a more general solution would be better.
Do you see any general solution to this specificity problem?
Now to get to the next thing, let's just presume there's a solution to this and we represent it in this example as a (if (is-nonlocal-exception? e) (raise e) condition in the local exception handler, again not because this would necessarily represent a general solution but just to get on to the next thing in the reasoning; so now we have
(define (logics) (with-exception-catcher (lambda (e) (if (is-nonlocal-exception? e) (raise e) "Logics failed due to invalid user input!")) (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
Now, how do you make this *parent* exception handler (the IO w.e.h.) that got the exception passed to it, able to pass a return value to the original |raise| call?
This is required for the exception handling-based IO error handling behavior we're looking for to work.
The problem reduces to
(define (logics) (raise "Please return 'properly-handled!"))
(define (proxy-exception-handler/catcher2 thunk) (lambda () (with-exception-catcher (lambda (e) (raise e)) thunk)))
(define (proxy-exception-handler/catcher1 thunk) (lambda () (with-exception-handler (lambda (e) (raise e)) thunk)))
(define (parent-exception-handler thunk) (continuation-capture (lambda (cont) (with-exception-handler (lambda (e) 'properly-handled!) thunk))))
where proxy-exception-handler/catcher 1 & 2 are to represent an arbitrary chain of exception handlers that logics code may come up with. The intended behavior is
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
Actually evaluating these three tests showed:
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
To start with, great that we see that with-exception-catcher delivers out of the box for this usecase!
Thus we have with-exception-handler left. I guess the best way would be if with-exception-handler inherently somehow would deliver for this, so that support for this kind of use would be transparent and not require possible updates of user code (e.g. any typical user code such as a PI calculator etc. could without needing code review just be run within a web server that uses this special IO error handling).
Is there any way to make with-exception-handler deliver for this usecase?
Last, what about that for introducing a port specific flag there would be the issue that the semantics of an IO error is vague -
Maybe you see something here I didn't get. As IO error for a port would count any error reported from the OS about the port, as well as timeouts within Gambit's IO system.
So this would correspond to any OS IO primitive invocation that gives an error return value (other than one that asks for a reiteration of the procedure, which some of them come with).
Another way to relate to it would be that such a port specific flag would serve to protect from requirement of programmer or admin intervention for any IO error that could possibly come up regarding the addressed ports.
What do you see here?
Best regards, Mikael
Hi Marc!
Ah, one thing struck me: in some cases, IO primitives don't raise an exception on error but instead return a special value, I think it's #!eof always.
This would not transform to returning #f by a with-exception-handler wrapper, but it could with a port specific flag |io-error| or |eof-value| or |treat-eof-as-error|.
Do you have any thoughts on the previous email on with-exception-handler and IO error semantics yet?
Of course there's time though would be great to get this question about how to reliably do general IO error handling settled.
The two possible ways I have the impression that there are now are
1) By: * Add a |treat-eof-as-error| port flag * Add an optional third arg |first?| to |with-exception-catcher| and -handler that makes the thunk be invoked first in case of exception. Internally there's two exception handler chains, the non-first that works just like the one today - an exception handler added makes it be invoked as first line in case of exception, and another chain that has priority over the non-first chain, that's invoked before it and where new handlers are added at the end and not at the beginning. (Perhaps internally they can be implemented as one chain only.) * Make |with-exception-handler| exception handlers recursive (now they make an infinite loop).
or
2): (Less general solution, so probably not desirable) By: * Add a port-specific flag |io-error| to IO ports that's invoked in the current place of both |raise| and eof.
In either of these two, the IO primitives need to be checked so that the IO error handling/|raise| calls are done in tail position / place that produces the primitive's return value anyhow.
Thanks and best regards, Mikael
2013/3/22 Mikael mikael.rcv@gmail.com
Dear Marc,
Spontaneously I think that the prospect of using with-exception-handler as you propose sounds better than adding custom error behavior code to the IO, as, the w.e.h. route would be more holistic in that it maintains the error handling mechanism in Gambit one in total in number; also,
It would work from a performance point of view as there is no overhead per IO primitive call that's successful, which accounts for almost all of them. The only potential issue with performance would be if the exception handler part would take a lot of time, though I guess it can be generalized that that is not an issue.
Then I guess the last point would be the design aspect that if somehow IO code would be run outside the current environment that the w.e.h. is installed in, the custom error behavior would disappear; that would indeed be a benefit with a port specific flag, that it's not subject to the same limitation. Though, from the practical use I see today, that more or less does not happen so it's fine.
I didn't think deeper about the possibility of w.e.h. before as I found myself without clarity on how the things I addressed in the previous email could be solved, as they need to be solved for it to be a practically viable solution.
Looking forward a lot to hear your take on those two-three things and hopefully get to a reliable io-error => #f soon =)
Thanks, Mikael
2013/3/20 Mikael
Hi Marc!
2013/3/18 Marc Feeley feeley@iro.umontreal.ca
The problem I see with the approach you propose (whether it uses a parameter or a port specific flag) is that the semantics of an "io-error" is vague. What is an IO error? The definition is important because exceptions that are IO errors are going to be processed using this new mechanism, and non-IO exceptions will use a different mechanism (normal exception handling).
The approach I propose does not have this problem because the programmer has complete control over the definition of an IO error. The exception object can be inspected to see if it qualifies as an IO error and an appropriate action can be taken. The definition of IO error can depend on the type of port, the type of primitive which caused the exception (read-u8, read-char, read), etc.
Marc
Aha. To really get this, there are two quite fundamental things about the applicability of the with-exception-handler possibility that I maybe don't get yet or at least would benefit of clarification, can we have a look at it?
Also last a question re semantics of io-error.
So, for the with-exception-handler mechanism to work out, it needs to go together with other use of exception handling use that's being done in the same scope.
In a setup where there's another exception handler *outside* the IO with-exception-handler, there would be no issue as the IO w.e.h. 's handler would be invoked first for any exception that occurs, and do its matching and handling.
In a setup where the other exception handler is made *inside* the IO w.e.h. though, any IO exception that arises within that exception handler will be picked up by it first, and for the IO w.e.h. thing to work out, that handler needs to be able to pass on the exception to the parent exception handler (which is the IO w.e.h.) completely in its original shape, in such a way that if the IO w.e.h. handler returns a value, that will be passed as return value to the original |raise| call.
An example of this would be a web-based PI calculator that uses exception handling locally in its PI calculate request thunk to pick up invalid user input.
So let's ask how this could be done.
Let's say below that we have a procedure (install-io-w.e.h. port thunk) that installs the IO w.e.h. that makes IO primitives return #f on exception, so that thunk is invoked with that w.e.h. installed.
Then, we have application logics (logics) that, aside from using IO, internally uses exception handling.
So a setup something like,
(define (logics) (with-exception-catcher (lambda (e) "Logics failed due to invalid user input!") (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
The desired behavior here is to print #f to the console and for logics to then return "Logics failed due to invalid user input!".
The issue now becomes, how would this local exception handler need to be implemented as for this to work out.
The local exception handler needs to be specific about what kind of error it looks for as to know which to handle locally and which to re-raise. This might be a complete PITA in some situations as you're looking for a catch-all behavior, as in the example above!
Perhaps the order of exception handlers could be tweaked somehow, so that the IO w.e.h. would get highest priority or something, though how could that be made as a 'clean' abstraction? I mean, who's in a place to claim at a general level that one exception is of higher prio than another? - different classes could be introduced, like, "user exceptions" and "io exceptions" or "system exception" (this would lead to an at least almost functional equivalent of the port specific flag solution, just with a more indirect code path!), or, the exception matching procedure could be exported to a separate mechanism, and the exception handler claiming the highest specificity in the matching would get to handle it e.g. (with-exception-handler exception-match exception-handler thunk) where exception-match takes an exception argument e and returns how well it matches the exception, i duno as a boolean or 0-10 or a symbol.
Or, a global parameter object |is-nonlocal-exception?| could be introduced that any exception handler is free to invoke as to check if it should re-raise the exception, and to overlap.. hmm, a more general solution would be better.
Do you see any general solution to this specificity problem?
Now to get to the next thing, let's just presume there's a solution to this and we represent it in this example as a (if (is-nonlocal-exception? e) (raise e) condition in the local exception handler, again not because this would necessarily represent a general solution but just to get on to the next thing in the reasoning; so now we have
(define (logics) (with-exception-catcher (lambda (e) (if (is-nonlocal-exception? e) (raise e) "Logics failed due to invalid user input!")) (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
Now, how do you make this *parent* exception handler (the IO w.e.h.) that got the exception passed to it, able to pass a return value to the original |raise| call?
This is required for the exception handling-based IO error handling behavior we're looking for to work.
The problem reduces to
(define (logics) (raise "Please return 'properly-handled!"))
(define (proxy-exception-handler/catcher2 thunk) (lambda () (with-exception-catcher (lambda (e) (raise e)) thunk)))
(define (proxy-exception-handler/catcher1 thunk) (lambda () (with-exception-handler (lambda (e) (raise e)) thunk)))
(define (parent-exception-handler thunk) (continuation-capture (lambda (cont) (with-exception-handler (lambda (e) 'properly-handled!) thunk))))
where proxy-exception-handler/catcher 1 & 2 are to represent an arbitrary chain of exception handlers that logics code may come up with. The intended behavior is
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
Actually evaluating these three tests showed:
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
To start with, great that we see that with-exception-catcher delivers out of the box for this usecase!
Thus we have with-exception-handler left. I guess the best way would be if with-exception-handler inherently somehow would deliver for this, so that support for this kind of use would be transparent and not require possible updates of user code (e.g. any typical user code such as a PI calculator etc. could without needing code review just be run within a web server that uses this special IO error handling).
Is there any way to make with-exception-handler deliver for this usecase?
Last, what about that for introducing a port specific flag there would be the issue that the semantics of an IO error is vague -
Maybe you see something here I didn't get. As IO error for a port would count any error reported from the OS about the port, as well as timeouts within Gambit's IO system.
So this would correspond to any OS IO primitive invocation that gives an error return value (other than one that asks for a reiteration of the procedure, which some of them come with).
Another way to relate to it would be that such a port specific flag would serve to protect from requirement of programmer or admin intervention for any IO error that could possibly come up regarding the addressed ports.
What do you see here?
Best regards, Mikael
I've added a port-specific exception handler that can be set with the new "port-io-exception-handler-set!" procedure. I/O exceptions which are detected by the I/O primitives are passed to this handler (if one is set) by calling it in tail position with respect to the primitive. That way, you can define the specific behavior you need in your application. In particular, if the handler returns a value (instead of raising an exception), then that value will be the result of the primitive.
Here's an example:
(let ((p (open-tcp-client "localhost:9999"))) (port-io-exception-handler-set! p (lambda (e) (display-exception e (current-output-port)) #f)) (pp (list 'return-value= (read-u8 p))))
that prints:
Connection refused (read-u8 '#<input-output-port #4 (tcp-client "localhost" 9999)>) (return-value= #f)
Note that timeouts are not considered to be io-exceptions. That shouldn't be a problem because a timeout handler can be set explicitly for any port in addition to the io-exception handler.
Marc
On 2013-03-24, at 2:12 PM, Mikael mikael.rcv@gmail.com wrote:
Hi Marc!
Ah, one thing struck me: in some cases, IO primitives don't raise an exception on error but instead return a special value, I think it's #!eof always.
This would not transform to returning #f by a with-exception-handler wrapper, but it could with a port specific flag |io-error| or |eof-value| or |treat-eof-as-error|.
Do you have any thoughts on the previous email on with-exception-handler and IO error semantics yet?
Of course there's time though would be great to get this question about how to reliably do general IO error handling settled.
The two possible ways I have the impression that there are now are
- By:
- Add a |treat-eof-as-error| port flag
- Add an optional third arg |first?| to |with-exception-catcher| and -handler that makes the thunk be invoked first in case of exception. Internally there's two exception handler chains, the non-first that works just like the one today - an exception handler added makes it be invoked as first line in case of exception, and another chain that has priority over the non-first chain, that's invoked before it and where new handlers are added at the end and not at the beginning. (Perhaps internally they can be implemented as one chain only.)
- Make |with-exception-handler| exception handlers recursive (now they make an infinite loop).
or
2): (Less general solution, so probably not desirable) By:
- Add a port-specific flag |io-error| to IO ports that's invoked in the current place of both |raise| and eof.
In either of these two, the IO primitives need to be checked so that the IO error handling/|raise| calls are done in tail position / place that produces the primitive's return value anyhow.
Thanks and best regards, Mikael
2013/3/22 Mikael mikael.rcv@gmail.com Dear Marc,
Spontaneously I think that the prospect of using with-exception-handler as you propose sounds better than adding custom error behavior code to the IO, as, the w.e.h. route would be more holistic in that it maintains the error handling mechanism in Gambit one in total in number; also,
It would work from a performance point of view as there is no overhead per IO primitive call that's successful, which accounts for almost all of them. The only potential issue with performance would be if the exception handler part would take a lot of time, though I guess it can be generalized that that is not an issue.
Then I guess the last point would be the design aspect that if somehow IO code would be run outside the current environment that the w.e.h. is installed in, the custom error behavior would disappear; that would indeed be a benefit with a port specific flag, that it's not subject to the same limitation. Though, from the practical use I see today, that more or less does not happen so it's fine.
I didn't think deeper about the possibility of w.e.h. before as I found myself without clarity on how the things I addressed in the previous email could be solved, as they need to be solved for it to be a practically viable solution.
Looking forward a lot to hear your take on those two-three things and hopefully get to a reliable io-error => #f soon =)
Thanks, Mikael
2013/3/20 Mikael
Hi Marc!
2013/3/18 Marc Feeley feeley@iro.umontreal.ca The problem I see with the approach you propose (whether it uses a parameter or a port specific flag) is that the semantics of an "io-error" is vague. What is an IO error? The definition is important because exceptions that are IO errors are going to be processed using this new mechanism, and non-IO exceptions will use a different mechanism (normal exception handling).
The approach I propose does not have this problem because the programmer has complete control over the definition of an IO error. The exception object can be inspected to see if it qualifies as an IO error and an appropriate action can be taken. The definition of IO error can depend on the type of port, the type of primitive which caused the exception (read-u8, read-char, read), etc.
Marc
Aha. To really get this, there are two quite fundamental things about the applicability of the with-exception-handler possibility that I maybe don't get yet or at least would benefit of clarification, can we have a look at it?
Also last a question re semantics of io-error.
So, for the with-exception-handler mechanism to work out, it needs to go together with other use of exception handling use that's being done in the same scope.
In a setup where there's another exception handler *outside* the IO with-exception-handler, there would be no issue as the IO w.e.h. 's handler would be invoked first for any exception that occurs, and do its matching and handling.
In a setup where the other exception handler is made *inside* the IO w.e.h. though, any IO exception that arises within that exception handler will be picked up by it first, and for the IO w.e.h. thing to work out, that handler needs to be able to pass on the exception to the parent exception handler (which is the IO w.e.h.) completely in its original shape, in such a way that if the IO w.e.h. handler returns a value, that will be passed as return value to the original |raise| call.
An example of this would be a web-based PI calculator that uses exception handling locally in its PI calculate request thunk to pick up invalid user input.
So let's ask how this could be done.
Let's say below that we have a procedure (install-io-w.e.h. port thunk) that installs the IO w.e.h. that makes IO primitives return #f on exception, so that thunk is invoked with that w.e.h. installed.
Then, we have application logics (logics) that, aside from using IO, internally uses exception handling.
So a setup something like,
(define (logics) (with-exception-catcher (lambda (e) "Logics failed due to invalid user input!") (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
The desired behavior here is to print #f to the console and for logics to then return "Logics failed due to invalid user input!".
The issue now becomes, how would this local exception handler need to be implemented as for this to work out.
The local exception handler needs to be specific about what kind of error it looks for as to know which to handle locally and which to re-raise. This might be a complete PITA in some situations as you're looking for a catch-all behavior, as in the example above!
Perhaps the order of exception handlers could be tweaked somehow, so that the IO w.e.h. would get highest priority or something, though how could that be made as a 'clean' abstraction? I mean, who's in a place to claim at a general level that one exception is of higher prio than another? - different classes could be introduced, like, "user exceptions" and "io exceptions" or "system exception" (this would lead to an at least almost functional equivalent of the port specific flag solution, just with a more indirect code path!), or, the exception matching procedure could be exported to a separate mechanism, and the exception handler claiming the highest specificity in the matching would get to handle it e.g. (with-exception-handler exception-match exception-handler thunk) where exception-match takes an exception argument e and returns how well it matches the exception, i duno as a boolean or 0-10 or a symbol.
Or, a global parameter object |is-nonlocal-exception?| could be introduced that any exception handler is free to invoke as to check if it should re-raise the exception, and to overlap.. hmm, a more general solution would be better.
Do you see any general solution to this specificity problem?
Now to get to the next thing, let's just presume there's a solution to this and we represent it in this example as a (if (is-nonlocal-exception? e) (raise e) condition in the local exception handler, again not because this would necessarily represent a general solution but just to get on to the next thing in the reasoning; so now we have
(define (logics) (with-exception-catcher (lambda (e) (if (is-nonlocal-exception? e) (raise e) "Logics failed due to invalid user input!")) (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user input. ))) (install-io-w.e.h. broken-port logics)
Now, how do you make this *parent* exception handler (the IO w.e.h.) that got the exception passed to it, able to pass a return value to the original |raise| call?
This is required for the exception handling-based IO error handling behavior we're looking for to work.
The problem reduces to
(define (logics) (raise "Please return 'properly-handled!"))
(define (proxy-exception-handler/catcher2 thunk) (lambda () (with-exception-catcher (lambda (e) (raise e)) thunk)))
(define (proxy-exception-handler/catcher1 thunk) (lambda () (with-exception-handler (lambda (e) (raise e)) thunk)))
(define (parent-exception-handler thunk) (continuation-capture (lambda (cont) (with-exception-handler (lambda (e) 'properly-handled!) thunk))))
where proxy-exception-handler/catcher 1 & 2 are to represent an arbitrary chain of exception handlers that logics code may come up with. The intended behavior is
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
Actually evaluating these three tests showed:
(parent-exception-handler (proxy-exception-handler/catcher1 (proxy-exception-handler/catcher2 logics))) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) => 'properly-handled!
To start with, great that we see that with-exception-catcher delivers out of the box for this usecase!
Thus we have with-exception-handler left. I guess the best way would be if with-exception-handler inherently somehow would deliver for this, so that support for this kind of use would be transparent and not require possible updates of user code (e.g. any typical user code such as a PI calculator etc. could without needing code review just be run within a web server that uses this special IO error handling).
Is there any way to make with-exception-handler deliver for this usecase?
Last, what about that for introducing a port specific flag there would be the issue that the semantics of an IO error is vague -
Maybe you see something here I didn't get. As IO error for a port would count any error reported from the OS about the port, as well as timeouts within Gambit's IO system.
So this would correspond to any OS IO primitive invocation that gives an error return value (other than one that asks for a reiteration of the procedure, which some of them come with).
Another way to relate to it would be that such a port specific flag would serve to protect from requirement of programmer or admin intervention for any IO error that could possibly come up regarding the addressed ports.
What do you see here?
Best regards, Mikael
Great news! :-D
Thanks!
2013/4/4 Marc Feeley feeley@iro.umontreal.ca
I've added a port-specific exception handler that can be set with the new "port-io-exception-handler-set!" procedure. I/O exceptions which are detected by the I/O primitives are passed to this handler (if one is set) by calling it in tail position with respect to the primitive. That way, you can define the specific behavior you need in your application. In particular, if the handler returns a value (instead of raising an exception), then that value will be the result of the primitive.
Here's an example:
(let ((p (open-tcp-client "localhost:9999"))) (port-io-exception-handler-set! p (lambda (e) (display-exception e (current-output-port)) #f)) (pp (list 'return-value= (read-u8 p))))
that prints:
Connection refused (read-u8 '#<input-output-port #4 (tcp-client "localhost" 9999)>) (return-value= #f)
Note that timeouts are not considered to be io-exceptions. That shouldn't be a problem because a timeout handler can be set explicitly for any port in addition to the io-exception handler.
Marc
On 2013-03-24, at 2:12 PM, Mikael mikael.rcv@gmail.com wrote:
Hi Marc!
Ah, one thing struck me: in some cases, IO primitives don't raise an
exception on error but instead return a special value, I think it's #!eof always.
This would not transform to returning #f by a with-exception-handler
wrapper, but it could with a port specific flag |io-error| or |eof-value| or |treat-eof-as-error|.
Do you have any thoughts on the previous email on with-exception-handler
and IO error semantics yet?
Of course there's time though would be great to get this question about
how to reliably do general IO error handling settled.
The two possible ways I have the impression that there are now are
- By:
- Add a |treat-eof-as-error| port flag
- Add an optional third arg |first?| to |with-exception-catcher| and
-handler that makes the thunk be invoked first in case of exception.
Internally there's two exception handler chains, the non-first that
works just like the one today - an exception handler added makes it be invoked as first line in case of exception, and another chain that has priority over the non-first chain, that's invoked before it and where new handlers are added at the end and not at the beginning. (Perhaps internally they can be implemented as one chain only.)
- Make |with-exception-handler| exception handlers recursive (now they
make an infinite loop).
or
2): (Less general solution, so probably not desirable) By:
- Add a port-specific flag |io-error| to IO ports that's invoked in the
current place of both |raise| and eof.
In either of these two, the IO primitives need to be checked so that the
IO error handling/|raise| calls are done in tail position / place that produces the primitive's return value anyhow.
Thanks and best regards, Mikael
2013/3/22 Mikael mikael.rcv@gmail.com Dear Marc,
Spontaneously I think that the prospect of using with-exception-handler
as you propose sounds better than adding custom error behavior code to the IO, as, the w.e.h. route would be more holistic in that it maintains the error handling mechanism in Gambit one in total in number; also,
It would work from a performance point of view as there is no overhead
per IO primitive call that's successful, which accounts for almost all of them. The only potential issue with performance would be if the exception handler part would take a lot of time, though I guess it can be generalized that that is not an issue.
Then I guess the last point would be the design aspect that if somehow
IO code would be run outside the current environment that the w.e.h. is installed in, the custom error behavior would disappear; that would indeed be a benefit with a port specific flag, that it's not subject to the same limitation. Though, from the practical use I see today, that more or less does not happen so it's fine.
I didn't think deeper about the possibility of w.e.h. before as I found
myself without clarity on how the things I addressed in the previous email could be solved, as they need to be solved for it to be a practically viable solution.
Looking forward a lot to hear your take on those two-three things and
hopefully get to a reliable io-error => #f soon =)
Thanks, Mikael
2013/3/20 Mikael
Hi Marc!
2013/3/18 Marc Feeley feeley@iro.umontreal.ca The problem I see with the approach you propose (whether it uses a
parameter or a port specific flag) is that the semantics of an "io-error" is vague. What is an IO error? The definition is important because exceptions that are IO errors are going to be processed using this new mechanism, and non-IO exceptions will use a different mechanism (normal exception handling).
The approach I propose does not have this problem because the programmer
has complete control over the definition of an IO error. The exception object can be inspected to see if it qualifies as an IO error and an appropriate action can be taken. The definition of IO error can depend on the type of port, the type of primitive which caused the exception (read-u8, read-char, read), etc.
Marc
Aha. To really get this, there are two quite fundamental things about
the applicability of the with-exception-handler possibility that I maybe don't get yet or at least would benefit of clarification, can we have a look at it?
Also last a question re semantics of io-error.
So, for the with-exception-handler mechanism to work out, it needs to go
together with other use of exception handling use that's being done in the same scope.
In a setup where there's another exception handler *outside* the IO
with-exception-handler, there would be no issue as the IO w.e.h. 's handler would be invoked first for any exception that occurs, and do its matching and handling.
In a setup where the other exception handler is made *inside* the IO
w.e.h. though, any IO exception that arises within that exception handler will be picked up by it first, and for the IO w.e.h. thing to work out, that handler needs to be able to pass on the exception to the parent exception handler (which is the IO w.e.h.) completely in its original shape, in such a way that if the IO w.e.h. handler returns a value, that will be passed as return value to the original |raise| call.
An example of this would be a web-based PI calculator that uses
exception handling locally in its PI calculate request thunk to pick up invalid user input.
So let's ask how this could be done.
Let's say below that we have a procedure (install-io-w.e.h. port thunk)
that installs the IO w.e.h. that makes IO primitives return #f on exception, so that thunk is invoked with that w.e.h. installed.
Then, we have application logics (logics) that, aside from using IO,
internally uses exception handling.
So a setup something like,
(define (logics) (with-exception-catcher (lambda (e) "Logics failed due to invalid user input!") (lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid user
input.
)))(install-io-w.e.h. broken-port logics)
The desired behavior here is to print #f to the console and for logics
to then return "Logics failed due to invalid user input!".
The issue now becomes, how would this local exception handler need to be
implemented as for this to work out.
The local exception handler needs to be specific about what kind of
error it looks for as to know which to handle locally and which to re-raise. This might be a complete PITA in some situations as you're looking for a catch-all behavior, as in the example above!
Perhaps the order of exception handlers could be tweaked somehow, so
that the IO w.e.h. would get highest priority or something, though how could that be made as a 'clean' abstraction? I mean, who's in a place to claim at a general level that one exception is of higher prio than another?
- different classes could be introduced, like, "user exceptions" and "io
exceptions" or "system exception" (this would lead to an at least almost functional equivalent of the port specific flag solution, just with a more indirect code path!), or, the exception matching procedure could be exported to a separate mechanism, and the exception handler claiming the highest specificity in the matching would get to handle it e.g. (with-exception-handler exception-match exception-handler thunk) where exception-match takes an exception argument e and returns how well it matches the exception, i duno as a boolean or 0-10 or a symbol.
Or, a global parameter object |is-nonlocal-exception?| could be
introduced that any exception handler is free to invoke as to check if it should re-raise the exception, and to overlap.. hmm, a more general solution would be better.
Do you see any general solution to this specificity problem?
Now to get to the next thing, let's just presume there's a solution to
this and we represent it in this example as a (if (is-nonlocal-exception? e) (raise e) condition in the local exception handler, again not because this would necessarily represent a general solution but just to get on to the next thing in the reasoning; so now we have
(define (logics) (with-exception-catcher (lambda (e) (if (is-nonlocal-exception? e) (raise e) "Logics failed
due to invalid user input!"))
(lambda () (pp (read-u8 broken-port)) (/ 0 0) ; This is to simulate an exception due to invalid userinput.
)))(install-io-w.e.h. broken-port logics)
Now, how do you make this *parent* exception handler (the IO w.e.h.)
that got the exception passed to it, able to pass a return value to the original |raise| call?
This is required for the exception handling-based IO error handling
behavior we're looking for to work.
The problem reduces to
(define (logics) (raise "Please return 'properly-handled!"))
(define (proxy-exception-handler/catcher2 thunk) (lambda ()
(with-exception-catcher (lambda (e) (raise e)) thunk)))
(define (proxy-exception-handler/catcher1 thunk) (lambda ()
(with-exception-handler (lambda (e) (raise e)) thunk)))
(define (parent-exception-handler thunk) (continuation-capture (lambda
(cont) (with-exception-handler (lambda (e) 'properly-handled!) thunk))))
where proxy-exception-handler/catcher 1 & 2 are to represent an
arbitrary chain of exception handlers that logics code may come up with. The intended behavior is
(parent-exception-handler (proxy-exception-handler/catcher1
(proxy-exception-handler/catcher2 logics))) => 'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) =>
'properly-handled!
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) =>
'properly-handled!
Actually evaluating these three tests showed:
(parent-exception-handler (proxy-exception-handler/catcher1
(proxy-exception-handler/catcher2 logics))) => infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher1 logics)) =>
infinite loop (!)
(parent-exception-handler (proxy-exception-handler/catcher2 logics)) =>
'properly-handled!
To start with, great that we see that with-exception-catcher delivers
out of the box for this usecase!
Thus we have with-exception-handler left. I guess the best way would be
if with-exception-handler inherently somehow would deliver for this, so that support for this kind of use would be transparent and not require possible updates of user code (e.g. any typical user code such as a PI calculator etc. could without needing code review just be run within a web server that uses this special IO error handling).
Is there any way to make with-exception-handler deliver for this usecase?
Last, what about that for introducing a port specific flag there would
be the issue that the semantics of an IO error is vague -
Maybe you see something here I didn't get. As IO error for a port would
count any error reported from the OS about the port, as well as timeouts within Gambit's IO system.
So this would correspond to any OS IO primitive invocation that gives an
error return value (other than one that asks for a reiteration of the procedure, which some of them come with).
Another way to relate to it would be that such a port specific flag
would serve to protect from requirement of programmer or admin intervention for any IO error that could possibly come up regarding the addressed ports.
What do you see here?
Best regards, Mikael
-
Marc Feeley -
Mikael