Dear Marc and Gambit community,
Here's a reflection on Gambit's IO routines' use of exceptions and a proposition for a tweak. For context, please check the previous email. Here we go:
I've come to relate to exceptions as a suitable medium pretty much solely for conveying an error state that requires manual administrator/programmer intervention.
Exceptions to the "solely" would be if there's some rare invocation of some routine, where a catch-all "it failed" kind of result would be satisfactory for error handling.
Thus, for instance if you make a laboratory project of connecting one computer to another and suddenly a read-u8 operation fails, that's a good reason to throw an exception. But, if you do read-u8:s all the time in a context where some of them are expected to fail, I suggest the view that making that result in an exception is a terrible idea because of the reasons below.
So to sum up, Gambit's IO routines currently throw a quite wide variety of exceptions. At least a bunch of them essentially convey the same thing as a #f or #!eof result would have done.
Lots of cases when exception on IO error is a bad idea
A typical example of when this makes a difference, is when you want to make some IO work in Gambit, where some form of eof-like IO failure is an expected outcome. Examples:
* read-u8 from a TCP port that may or may not be closed now
* write-substring to a TCP port that may or may not be closed now
* A HTTP request, in particular if it's to a host that's not under your control
All these calls cannot be wrapped individually in with-exception-catcher blocks, you'd need a 1Thz core for per-byte reading of large amounts of data with each read-u8 wrapped in a with-exception-catcher to not be too expensive, so my best understanding today is that the locations in Gambit's runtime that currently always produce exceptions in these cases, would benefit of a tweak:
What about: Introduce a global parameter object that introduces the option of #f/#!eof result instead of exception on IO error
As to keep code simple and effective, I would propose the introduction of a global parameter object io-prefer-eof-result-to-exception to control this behavior, and perhaps one or two more global parameters to be assigned detailed error info in the event of io error.
Whenever a user needs to use IO with performance and in a might-fail context, he/she just switches this parameter to #t.
What's important is that IO can be done at high performance when it works. Unlike individual invocations wrapped in with-exception-catcher, the presence of a parameter value has no overhead on individual procedure invocations. IO operation failures are extremely rare compared to IO operation successes (maybe 1:10,000), so the error handling code will be executed quite rarely and thus unwrapping of a parameter object is a no brainer expense in the context.
As to get to this suggestion, I experimented over a long period with with-exception-catcher and with extracting Gambit IO routine code as to make custom exception-safe variants of them, this lead to the understanding that that cannot be reliably done, and involves copying of so much code that it's by far better to make a Gambit patch for this in the first place anyhow.
#f preferable over #!eof as IO operation return value in case of EOF due to its more central position in the type system?
Last, there is some #!eof returning in case of EOF already in Gambit's IO API. I have the impression that it's not completely consistent.
It's struck me that #!eof not is so universal as EOF signal afterall: In a great many cases, #f would be better due to its role in the type system - to do meaningful stuff with #!eof you need to typecast it to boolean (i.e. (eq? result-value #!eof)) to do anything meaningful with it anyhow, and all the regarded procedures never throw #f within their current definition anyhow.
Because of this at most or all of
read-subu8vector read-substring read-u8 read-char peek-char read-line
write-subu8vector write-substring write-u8 write-char
force-output close-input-port close-output-port close-port
input-port-timeout-set! output-port-timeout-set!
would do better with #f than #!eof as return value to signal EOF.
Preliminary thought about next step: Make a patch and when it's mature, include it in the Gambit repo
Preliminarily I think this would be a good idea to pursue by making a Gambit patch that incorporates this functionality, and when it's mature, incorporating that patch into Gambit's master branch.
The value of having it in the master branch is that this way, Gambit-based libraries such as a HTTP client, can be written in the coding style that follows from this, which would clean up the code *alot* and calibrate design decisions in an advantageous direction:
Basically, error handling is done more explicitly but is much more specific - which is completely OK for IO-dense code for real world use, as this is what that kind of code does effectively anyhow - and there's some kind of performance increase.
Please feel free to share any thoughts you have on this topic, what do you say about the idea?
Best regards,
Mikael