Gambit-list Avril 2013

gambit-list@iro.umontreal.ca

21 participants
51 discussions

[PATCH] Poll Support for I/O
by Dimitris Vyzovitis 07 Avr '13

07 Avr '13

This is a patch for enabling poll/ppoll support for I/O: https://github.com/feeley/gambit/pull/7 This allows the system to use more than 1024 file descriptors without crashing. Branch in my repo in case you want to fork: https://github.com/vyzo/gambit/tree/enable-poll-support -- vyzo

2 1

Unable to build v4.6.7 on the base of v4.6.6
by Paul Wolneykien 07 Avr '13

07 Avr '13

Hi, all, Currently I'm unable to build v4.6.7 when I set up gsc v4.6.6 as gsc-boot: _num.c:1410:1: error: '___SYMCOUNT' undeclared here (not in a function) _num.c: In function '___H__20___num': _num.c:15316:1: error: '___lp' undeclared (first use in this function) _num.c:15316:1: note: each undeclared identifier is reported only once for each function it appears in _num.c: In function '___init_proc': _num.c:74427:1: error: '___lp' undeclared (first use in this function) make[1]: *** [_num.o] Error 1 make[1]: *** Waiting for unfinished jobs.... If the gsc-boot dummy script (exit 0) is not replaced by the real gsc and the _num.c and _io.c aren't regenerated, the situation is much worse: all of the above errors and much more... My building scheme was already tested on 4.6.4 -> 4.6.5 and 4.6.5 -> 4.6.6 builds. Regards, Paul.

2 1

Your feedback would be much appreciated re: Proposal for enabling IO errors to be reported through returning of a custom value instead of by throwing exception, through DSL with exports: ##io-error-behavior param, ##default-io-error-behavior unique value, ##last-io-error param.
by Mikael 05 Avr '13

05 Avr '13

Dear Marc and Gambit community, Some while ago I emailed a suggestion for improvement to IO error handling for IO-intensive realworld applications (post at https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006322.… , "Proposal: Addition of a parameter option for signalling IO error as #f/ #!eof result value instead of as exception"). The update provides a way for higher IO performance and higher specificity of IO error reporting through the removal of need for up to as many with-exception-catcher calls as the CPU can perform IO procedure calls i.e. up to ~25,000,000 per second. This makes great sense as with-exception-catcher calls everywhere really clutters code, and, it's a medium-expensive procedure taking approx 0.344 seconds per million calls (test file attached). Secondarily this update makes using Gambit's IO more holistic through guaranteeing that IO calls will never lead to an exception except if it's such a serious error that manual administrator or programmer intervention would reasonably be required, as for instance in the case of heap overflow. (As I got it, IO exceptions are thrown in corner cases of read/write-u8/char/substring/subu8vector even while generally #!eof or 0 are returned.) The previous post worked at a level of concept, and I now wish to propose a specific implementation specification as to further the progress on this topic: *Implementation suggestion* Three exports are needed to make it spin: ##io-error-behavior ##default-io-error-behavior ##last-io-error ##io-error-behavior is a parameter object whose content determines Gambit's IO procedures' behavior in case of an IO error. It is set by default to ##default-io-error-behavior which is a unique object (an uninterned symbol or something). When an IO error happens, * If ##io-error-behavior is set to ##default-io-error-behavior, then the error is handled as per Gambit's current behavior (exception or #!eof or 0 depending on procedure). * If ##io-error-behavior is set to another value, then that value is returned. ##last-io-error is a parameter object whose value is set on IO error to the exception value or return value that Gambit produces in its default behavior. This way the caller can inspect the error further if interested. *Example use* ; => boolean, #t = success. (define (handle-request port) (parameterize ((##io-error-behavior #f)) ; Read input data up to end of input stream (let loop () (let ((c (read-u8 port))) (if c (begin ; [Handle char] (loop)) ; End of input stream reached. ; ; Acknowledge reception (and (display "OK\n" port) (force-output port))))))) The benefit here is the saving of a with-exception-catcher calls for each read-u8 call, and depending on how you'd do it otherwise, the saving of one or two with-exception-catcher calls for the display and force-output calls and saving of the possibility of doing force-output on port when displayalready reported it as closed. *Concluding notes re implementing this* I may very well get into implementing a Gambit commit for providing this functionality. On my end, it will make ground for doing well-deserved cleanup to Sack and the to-be-released OpenSSL SSL channels module, both IO-intensive modules for production use, that currently use version-unsafe hacks to simulate the functionality proposed above. I do not claim any copyright on suggestions or commits I send to Gambit but leave them in public domain. *Bigger picture, for reference* What motivated me to propose this as well as recently proposing 1) A ##os-device-get-fd procedure ( https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-March/006477.ht… ) 2) A ##device-port-wait-for-output! procedure to complement ##device-port-wait-for-input (https://github.com/feeley/gambit/pull/6) 3) Some way to implement low-level application-level ports ( https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006321.… ) is that I've gotten to the conclusion that Gambit's builtin IO system is strikingly good as it is and has the potential to scale for any practical usecase I can imagine with only minor-ish tweaks. The usecases to tweak for that I see relevant are a) Realworld IO-intense use (which the proposal in this email seeks to improve in a quite fundamental way through a quite trivial tweak), b) Deep C integration out of the box (which 1 and 2 above regard, really trivial tweaks those two too), and c) Delivering low-level application-level ports e.g. SSL and GZip (3 above regard this, I didn't get to any specific proposal for this til now but from reviewing _io.scm it cannot be a very big deal to do) d) Later I'd guess: Hybrid character and byte access of ports (for ISO-8859-1 encoding, I believe it's already in the box, maybe some clarification needed and possibility for support for more encodings could be looked into) And last e) Making the IO system's UTF-8 coder suitable for any use at all, because it's not due to a freeze-crash bug!! (I believe it's at http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=127 though Bugzilla is down right now) f) Perhaps later: Higher IO procedure call performance through lower overhead on primarily the per-byte/character calls i.e. read-byte read-char, perhaps by making them automatically inlined and/or by making their mutex use configurable. read-u8 currently delivers ~350KB/sec and an increase to 25MB/sec should be realistic. For now though this can easily be worked around by application-level buffers and use of read/write-subu8vector, though that does come with drawbacks i.e. doesn't integrate well with hybrid byte-character use etc. These are the only practical improvements that have come to my mind in total from several years of intense use of this functionality. Zooming out a bit, this is impressively little really. I would kindly ask you for your feedback on the suggestion regarded by this email. :) Best regards, Mikael

2 8

Edge triggered I/O with libevent
by Adam Langley 04 Avr '13

04 Avr '13

Gambit uses the select call on POSIX systems to multiplex IO operations. This is fine for small numbers of file descriptors, but falls apart quickly with large numbers[1]. libevent[2] is a wrapper library around a number of modern IO multiplexing systems (epoll on Linux, kqueue on the BSDs as well as poll and select). Attached is a patch which converts Gambit to using edge triggered I/O with epoll and libevent. This also requires a patch (attached) of libevent since the standard distribution doesn't include edge triggering support and the upstream appears to be on holiday. (epoll is required because I don't have a BSD box on which to develop edge triggering support for kqueue) This patch (for Gambit) preprocesses away to nothing unless --enable-libevent is passed to the configure script. (Note that, since this patches the configure.ac script, you'll need to run autoconf to regenerate). I hope I've not broken anything, but at this stage I've only tested that the console and tcp ports work. This patch (in the future) should also allow people, who write programs which use socket calls directly, to integrate into the main event loop at some point. Testing is more than welcome and, if you find a form of IO which is broken, please email me. [1] http://monkey.org/~provos/libevent/libevent-benchmark2.jpg [2] http://monkey.org/~provos/libevent/ -- Adam Langley agl(a)imperialviolet.org http://www.imperialviolet.org (+44) (0)7906 332512 PGP: 9113 256A CC0F 71A6 4C84 5087 CDA5 52DF 2CB6 3D60

2 2

If you have a copy of the ML post 2005-08-19 09:54 " Edge triggered I/O with libevent" can you please re-post it here? :)
by Mikael 04 Avr '13

04 Avr '13

https://mercure.iro.umontreal.ca/pipermail/gambit-list/2005-August/000367.h… The ML web mirror's attachment downloads is currently broken for old ML posts, believed it worked two years ago, emailed the admin anyhow. Queried Mario about it and he subscribed 2005-08-20 20:16, so he missed it by about 34 hours :) So if you have it feel free to re-post, Thanks! Mikael

1 0

Re: [gambit-list] current-directory // problem on Windows 7
by sales.creditscore＠creditscore.co.nz 04 Avr '13

04 Avr '13

Sorry guys. I've had you chasing a red herring on this one. The problems I've been having had nothing to do with Gambit. Turns out that modern browsers do not allow server generated pages to access the local file system. Opera gives you an error message saying the path is wrong, which started me down this slippery slope. The others just hang. Might be an idea to remove that function from the web-server example. Lots to learn. No doubt I'll have other questions shortly. Cheers, Rob. ---- Mikael <mikael.rcv(a)gmail.com> wrote: > Hi Rob, > > Welcome! > > 2013/4/3 <sales.creditscore(a)creditscore.co.nz> > > > Never thought of just typing the function straight into the interpreter > > :-| Anyway this is what it produces. > > > > "C:\\MinGW\\msys\\1.0\\local\\Gambit-C\\bin" > > > > > That's the serialized form, for seeing a verbatim copy of the string > content: > > > (print (current-directory) "\n") > C:\Program Files\Gambit-C\v4.6.6\bin\ > > > ..or in pieces.. > > > (string->list (current-directory)) > (#\C > #\: > #\\ > #\P > #\r > #\o > #\g > #\r > #\a > #\m > #\space > #\F > #\i > #\l > #\e > #\s > #\\ > #\G > #\a > #\m > #\b > #\i > #\t > #\- > #\C > #\\ > #\v > #\4 > #\. > #\6 > #\. > #\6 > #\\ > #\b > #\i > #\n > #\\) > > > In the code in your previous email, you can leave object->string out as > what you want right there is a verbatim copy of the string's content, and > not what that procedure produces, which is a serialization though with a > string as output rather than the console (as you showed an example of above) > . > > Brgds

1 0

Re: [gambit-list] current-directory // problem on Windows 7
by sales.creditscore＠creditscore.co.nz 03 Avr '13

03 Avr '13

Never thought of just typing the function straight into the interpreter :-| Anyway this is what it produces. "C:\\MinGW\\msys\\1.0\\local\\Gambit-C\\bin" ---- Marc Feeley <feeley(a)iro.umontreal.ca> wrote: > > Can you start "gsi" and at the REPL type (current-directory) to see what is returned. > > Marc > > _______________________________________________ > Gambit-list mailing list > Gambit-list(a)iro.umontreal.ca > https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

3 2

Custom software port query
by Mikael 03 Avr '13

03 Avr '13

Dear Marc, It would rock to be able to define custom software ports in Gambit, with complete control over the IO primitive behavior on its use, in contrast with the bidirectional u8vector ports available now, which provide this in a very limited way. Possible applications are any kinds of stream or channel protocol or mechanism, like, SSL server, SSL client, GZIP, HTTP. Data to transfer over them would be both binary, character and object. (I'm well aware that Gambit contains all structures for doing this internally, so it is a quite straightforward usecase, though it would utilize things Gambit does not export today so such attempts would become hacks today, so this is why I brought it to conversation now; perhaps a general DSL could be achievable that exports it to userland.) Do you have any thoughts on this topic, or see any straight-enough way today of doing this? Thanks and best regards, Mikael

1 0

How best cancel a read/write-subu8vector or ##device-port-wait-for-input/output! from another thread gracefully? (for abort operation on active HTTP request etc.)
by Mikael 03 Avr '13

03 Avr '13

Dear Marc and list, A universal HTTP client is taking form, it does connect & logics & IO in a singlethread setting now properly. Would be happy to be able to cancel blocking read/write-subu8vector or ##device-port-wait-for-input/output! calls gracefully as to be able to abort HTTP connections directly and gracefully like this: (define r (http-request url)) ((r 'abort!)) ; From another thread than the one performing the HTTP request logics & IO. ; (Even if the abort is done at the level of HTTP request, due to the invalid ; protocol state the abort produces the connection needs to be taken down.) The ways I can see this could be done by would be using - close-port - thread-interrupt! or ##thread-int! ?? - A low timeout on any read and write IO primitive invoked, like, 0.05 seconds, and check for the cancel condition on each timeout and if not reached, invoke the IO primitive again. - thread-terminate! (The HTTP request may be running in the user's thread, so this would require redesign) It'd be most preferable if this cancel could be done in such a way that 1) The abort action's effects are limited within the scope of the code for the respective HTTP request or connection so user code executed after the HTTP request code is not affected by it. This includes taking care automatically of any "mutex abandoned" exceptions or alike that would otherwise come as surprises later. 2) The thread performing the abort action does not get blocked in turn. (Something like that, the write-subu8vector would block a mutex in the IO system that would make the close-port call used to interrupt it, block until timeout.) To help 1), I can ensure that a thread-interrupt! or alike call is not done after that the HTTP request code is done and the thread is back in user code, through making a round of locking some mutex at the end of the HTTP request and guaranteeing a thread-interrupt! never is made after that mutex was locked. It's OK if the interrupt appears to the read/write-subu8vector caller as an IO failure i.e. #f return value. While I believe I could come up with a solution, I'd be very happy to learn to know your take on it as to do this the best way? Thanks! Mikael

2 5

current-directory // problem on Windows 7
by sales.creditscore＠creditscore.co.nz 03 Avr '13

03 Avr '13

Hi, I'm a newbie to both Scheme and Gambit which I'm using on Windows 7. I've managed to download and compile Gambit 4.6.7's source code using MinGW and MSys. Now I'm playing with it a little. I've just successfully compiled the web-server example, but when I run it the phrase (<li> (<a> href: (object->string (current-directory)) "Browse the web server's filesystem")) in web-server.scm produces: file:///c://MinGW//msys//1.0//local//Gambit-C//web-server/ which causes an error in the browser. Anybody know why the double forward slashes are appearing in the directory path. Is the extra / an escape character? Cheers, Rob.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Gambit-list Avril 2013