Dear Marc and Gambit community,
Here comes a subject of meditation for the weekend. As to maximize clarity and discussability I'll split it in two emails. Here we go:
How do we make Gambit port-based protocol wrapper Gambit ports with great performance and as solid abstractions?
Various forms of byte- and character-stream based protocols, such as communication protocols like SSL channels, or encodings like GZIP, probably accounts for in the range ~35% of all Gambit use, so it's central and even critical stuff.
If all of this can be not just whimsical special-purpose modules but be implemented in the united format of Gambit ports, *alot* would be won: Gambit's IO API is really neat and complete in nature: Being able to deliver also these completely central uses through it would be of enormous practical value.
Perhaps even everything needed for this is in Gambit already and we pretty much only need to make a bit of documentation and examples and this way we can all enjoy easily pluggable modules for the rich diversity of uses that are in this field.
To stimulate the conversation and add context, below I'll reflect my long experience of this kind of use and a proposal for qualities that such an abstraction mechanism needs.
I hope you share the enjoyment and appreciation of this topic, and look forward to any discussion that ensues.
*Experience of protocol wrapper Gambit ports using open-u8vector-pipe**: Super easy to implement, and delivers beyond unacceptably badly in the respect that it's not suitable for real-world use ever.* Briefly, over the last years I used Gambit for doing SSL over TCP. A typical, elementary and critical use case for this is HTTPS client/server.
Til now I've essentially made two SSL modules for Gambit, both atop OpenSSL. Total time invested is somewhere in the range 100 - 150 man hours, a lot due great complexity of and lack of documentation in how the OpenSSL library is used in particular in nonblocking mode.
The first one was made based on a bidirectional byte channel Gambit port as generated by Gambit's |open-u8vector-pipe|.
Put very briefly, my experience with it was that
* It had a low transfer speed performance and high latency - probably because of the extra round through Gambit's scheduler needed for every piece of IO done, as compared with just using a native Gambit port such as a TCP client port directly -, and,
* It had all the pains of a leaky abstraction, such as, that when either party would close the channel, it was not necessarily propagated to this channel as the |read-subu8vector| would not necessarily return straight away nor have a return value clearly reflecting that the connection was closed, so actual freeing of this port could take a while. I don't remember the exact details of this at the moment, but the constant feeling was like, uh oh, OMG what's that object on the heap doing now really plus the discomfort of knowing it's possible it's not quite right and of not having a clue - I believe you have similar experiences from your own encounters with leaky abstractions.
So briefly, my experience of |open-u8vector-pipe| for providing a protocol wrapper atop Gambit ports was completely unsatisfactory, and provided motivation for exploring an alternative implementation path later. So..
*Experience of protocol wrapper Gambit ports using custom "io-primitives" structures with a closure for each IO routine: Delivers great but will never be universal between projects* The second one, completed very recently, was built on the concept of that the Gambit port is wrapped into an "io-primitives" structure that contains one closure for each IO method: IO of u8vector chunks, and of ISO-8859-1/ANSI/8-bit encoded characters at the level of character, line and string, for complete definition see [1].
The benefits here are quite tremendously good: The io-primitives code itself accesses Gambit's IO routines directly, and thus
* Latency is ~zero, excellent. Also, thanks to the same thing,
* Both the beneficiary code and the io-primitives code follows IO behavior directly, making the entire abstraction not leak but be solid. Introspection for instance is piece of cake as execution passes directly between the layers of port abstraction - that's easy to understand due to the sequential nature of the execution, it's easy to log meaningfully and to work with in the interactive debugger.
Now, after having implemented this and had these basically very good experiences, it became/was completely evident that this is and will remain a special purpose code, as Gambit's built-in IO API is so good in itself. And really, these io-primitives could be mapped to Gambit's IO exports, it's quite the same thing really, so if that can be done in a way that really delivers it's completely preferable. Thus this proposition:
*Proposed characteristics for Gambit protocol wrapper port functionality* So, now here are the primary things that come to my mind about what protocol wrapper support ought to include:
* Provision of application-layer ports exported through Gambit's IO routines
* These ports can export individual Gambit IO routines i.e. can provide a specific read-char, read-u8, read-subu8vector, read-line behavior, so that a call e.g. (read-subu8vector u8v start end port) leads directly to |mycustomimplementation-read-subu8vector|. Part of the point with this is that,
* Double-buffering and copying must be kept to a minimum as it brings all kinds of expense and complexity - a "protocol wrapper" like this, for example an SSL wrapper, must not be required to work with the beneficiary code via a buffer, but must be let to deal with buffering as it wants to. Furthermore I'd say that,
* Due to the dual nature of a lot of protocols, the API must be in a hybrid binary-character mode i.e. binary IO (i.e. read/write-subu8vector) must be combinable with character IO, at least for 8bit-fixedsize-character i.e. ISO-8859-1 encoding.
This is not a must though - even if a bit of work, binary mode only would be fine, and the user can just wrap read-u8 to read-char etc., this is easy enough to do really.
And last, I'd guess that
* There must be a way to implement these ports in such a way that any IO made of them does not use mutexes at all (i.e. can be run in non-threadsafe mode), and, can be done with not so many trampolines happening. The reason for this is performance:
The default |read-char| implementation whose design involves both mutex locking and trampolines on each call, delivers unacceptably low performance (approximately 200KB/second on laptop CPU core), while a direct implementation approach would deliver approx 25MB/second on the same core which is really good. (Disclaimer: these two numbers are taken off the top of my head from 5 years ago, though should apply now.)
Mutexes can be addressed now, and trampolines probably better wait til the native backend is around, where module-to-module calls don't involve trampolines anymore anyhow, and I guess these parts of Gambit's runtime can be run under that backend at that time.
*Rough proposition of API, draft*
(make-custom-port #!key read-subu8vector read-substring read-u8 read-char peek-char read-line write-subu8vector write-substring write-u8 write-char force-output close-input-port close-output-port close-port input-port-timeout-set! output-port-timeout-set! ) => gambit-port
Note the absence of read and write here - those deal with objects and are not in the scope of this port abstraction mechanism.
It would not be an issue that a particular protocol wrapper port implementation would only implement some of the involved functionality, such as only byte and no character IO; if anything is needed it can be implemented later, the important thing here is for Gambit to provide a format for this that encompasses all this area of use in a performant and qualitative way and thus is suitable for cross-module, cross-project use.
*Example* With this approach it'd be something like
(let* ((client (open-tcp-client "wikipedia.org:443")) (ssl-channel (open-ssl-channel client))) (display "POST /someform HTTP/1.1\nHost: wikipedia.org\nContent-Length: 100\n\n" ssl-channel) (write-subu8vector postdata 0 (u8vector-length postdata) ssl-channel) (force-output ssl-channel 1) (let ((hdr (read-line ssl-channel)) [..]
*Related implementation/code example* For an example of an implementation of a set of IO operation closures akin to what's suggested in this email, see line 61 - 89 of openssl-sack.scm and row 1621 to 1969 of openssl.scm , both attached.
This suits as an example, as as it exports an API extremely similar to Gambit's IO, it is essentially built atop Gambit's IO API in itself, it adds no latency, delivers high performance and performs minimum buffering and copying while providing some hybrid byte and character IO, and can be run in a thread-unsafe mode.
*What do you say?* Now I wish to query you about your take on this:
* What are your thoughts on this topic?
and:
* Marc, how usable is Gambit's included IO abstraction for this kind of use today?
Spontaneously, would you say this can be done right away just given people know how to do it (i.e. documentation), or would adaptations of the IO code be required, if so what kind of adaptations would it be?
Feel free to reflect at high detail what of this can be done already now, perhaps all is ready and it's just a question of getting going with it.
With best regards, Mikael
[1] io-primitives definition used in the particular wrapper described above:
; Name Arguments & Result Behavior ; read-subu8vector (u8v start end #!optional (need 1)) Read u8vector data ; => bytes read or #!eof ; read-substring (str start end #!optional (need 1)) Read string data as ISO-8859-1 ; => bytes read or #!eof ; read-u8 () => byte or #!eof Read a byte ; write-subu8vector (u8v start end) Write u8vector data ; display (v) Write string, number or symbol as ISO-8859-1 ; force-output () Flush any write buffer to network ; close-port () Close port ; make-read-line-until-crlf (#!optional return-eof-on-eof?) Read line until reaching crlf. Permissive behavior. ; => procedure taking no args The procedure returned may be used in a single thread only and thus ; => string or maybe #!eof, see desc reuse the same buffers across invocations. ; If return-eof-on-eof? is set then #!eof is returned in case of EOF. ; If not, string reading is just ended and result is returned.