Gambit-list Mars 2013

gambit-list@iro.umontreal.ca

21 participants
30 discussions

Efficient Compilation of Tail Calls and Continuations to JavaScript
by Marc Feeley 17 Sep '13

17 Sep '13

Eric Thivierge is in Copenhagen today, at the Scheme Workshop, presenting our latest paper: "Efficient Compilation of Tail Calls and Continuations to JavaScript" You can get the paper on the Scheme Worshop web page: http://users-cs.au.dk/danvy/sfp12/programme-sfp12.html It describes the implementation of tail calls and continuations in Gambit-JS and evaluates its performance. Even if you are not specifically interested in the JavaScript back-end the paper may be interesting to read as it provides a brief explanation of the Gambit Virtual Machine (GVM) and compiler front-end, and a complete example of compilation from Scheme, to GVM, to JavaScript. The performance evaluation (see attached tables) shows that the approach used by Gambit-JS is consistently faster than Scheme2JS and Spock. Marc Execution times using V8 (Chrome 21.0.1180.89): Gambit-JS Scheme2JS Spock fib35 .80 1.54 1.9x 2.40 3.0x nqueens12 .72 .76 1.1x 2.33 3.3x oddeven .83 1.92 2.3x 5.62 6.8x ctak .18 17.64 95.9x .59 3.2x contfib30 1.17 106.01 90.9x 3.38 2.9x btsearch2000 1.35 25.40 18.8x 7.93 5.9x threads10 1.34 24.68 18.5x 4.40 3.3x Execution times using JagerMonkey (Firefox 15.0.1): Gambit-JS Scheme2JS Spock fib35 1.07 7.66 7.2x 20.73 19.4x nqueens12 1.49 2.62 1.8x 12.97 8.7x oddeven 1.09 1.93 1.8x 33.91 31.1x ctak 1.25 30.38 24.3x 1.86 1.5x contfib30 5.88 156.79 26.7x 10.61 1.8x btsearch2000 11.19 27.16 2.4x 16.54 1.5x threads10 6.97 34.36 4.9x 22.11 3.2x Execution times using Nitro (Safari 6.0): Gambit-JS Scheme2JS Spock fib35 1.32 6.53 4.9x 84.69 64.2x nqueens12 1.31 1.89 1.4x 45.15 34.4x oddeven 2.52 1.55 .6x 143.97 57.1x ctak .30 52.88 177.4x 4.18 14.0x contfib30 1.60 2311.69 1443.0x 25.98 16.2x btsearch2000 3.46 102.98 29.7x 92.02 26.6x threads10 3.03 74.21 24.5x 74.88 24.7x

4 10

Evaluation in reader?
by Nathan Sorenson 02 Mai '13

02 Mai '13

I'm the author of the Clojure->Scheme compiler. I'm looking into the possibility making the compiler not require the JVM for macro-expansion. Clojure makes use of vectors and maps directly as syntax -- I was looking into using Gambit's bracket-keyword to produce literal vectors from the reader, but it seems the reader just attaches a symbol to the head of a list, as opposed to evaluating the forms. I'd like the macro-expansion stage to have access to the actual maps, vectors etc. as some Clojure macros behave differently depending on whether the forms are lists or vectors. Is it possible to hook into the reader this way? (Perhaps as a fallback I could just tree-walk the forms to replace instances of the vector-literal and hash-literal markers with actual vectors and maps before macro-expansion.)

2 4

Your feedback would be much appreciated re: Proposal for enabling IO errors to be reported through returning of a custom value instead of by throwing exception, through DSL with exports: ##io-error-behavior param, ##default-io-error-behavior unique value, ##last-io-error param.
by Mikael 05 Avr '13

05 Avr '13

Dear Marc and Gambit community, Some while ago I emailed a suggestion for improvement to IO error handling for IO-intensive realworld applications (post at https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006322.… , "Proposal: Addition of a parameter option for signalling IO error as #f/ #!eof result value instead of as exception"). The update provides a way for higher IO performance and higher specificity of IO error reporting through the removal of need for up to as many with-exception-catcher calls as the CPU can perform IO procedure calls i.e. up to ~25,000,000 per second. This makes great sense as with-exception-catcher calls everywhere really clutters code, and, it's a medium-expensive procedure taking approx 0.344 seconds per million calls (test file attached). Secondarily this update makes using Gambit's IO more holistic through guaranteeing that IO calls will never lead to an exception except if it's such a serious error that manual administrator or programmer intervention would reasonably be required, as for instance in the case of heap overflow. (As I got it, IO exceptions are thrown in corner cases of read/write-u8/char/substring/subu8vector even while generally #!eof or 0 are returned.) The previous post worked at a level of concept, and I now wish to propose a specific implementation specification as to further the progress on this topic: *Implementation suggestion* Three exports are needed to make it spin: ##io-error-behavior ##default-io-error-behavior ##last-io-error ##io-error-behavior is a parameter object whose content determines Gambit's IO procedures' behavior in case of an IO error. It is set by default to ##default-io-error-behavior which is a unique object (an uninterned symbol or something). When an IO error happens, * If ##io-error-behavior is set to ##default-io-error-behavior, then the error is handled as per Gambit's current behavior (exception or #!eof or 0 depending on procedure). * If ##io-error-behavior is set to another value, then that value is returned. ##last-io-error is a parameter object whose value is set on IO error to the exception value or return value that Gambit produces in its default behavior. This way the caller can inspect the error further if interested. *Example use* ; => boolean, #t = success. (define (handle-request port) (parameterize ((##io-error-behavior #f)) ; Read input data up to end of input stream (let loop () (let ((c (read-u8 port))) (if c (begin ; [Handle char] (loop)) ; End of input stream reached. ; ; Acknowledge reception (and (display "OK\n" port) (force-output port))))))) The benefit here is the saving of a with-exception-catcher calls for each read-u8 call, and depending on how you'd do it otherwise, the saving of one or two with-exception-catcher calls for the display and force-output calls and saving of the possibility of doing force-output on port when displayalready reported it as closed. *Concluding notes re implementing this* I may very well get into implementing a Gambit commit for providing this functionality. On my end, it will make ground for doing well-deserved cleanup to Sack and the to-be-released OpenSSL SSL channels module, both IO-intensive modules for production use, that currently use version-unsafe hacks to simulate the functionality proposed above. I do not claim any copyright on suggestions or commits I send to Gambit but leave them in public domain. *Bigger picture, for reference* What motivated me to propose this as well as recently proposing 1) A ##os-device-get-fd procedure ( https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-March/006477.ht… ) 2) A ##device-port-wait-for-output! procedure to complement ##device-port-wait-for-input (https://github.com/feeley/gambit/pull/6) 3) Some way to implement low-level application-level ports ( https://mercure.iro.umontreal.ca/pipermail/gambit-list/2013-January/006321.… ) is that I've gotten to the conclusion that Gambit's builtin IO system is strikingly good as it is and has the potential to scale for any practical usecase I can imagine with only minor-ish tweaks. The usecases to tweak for that I see relevant are a) Realworld IO-intense use (which the proposal in this email seeks to improve in a quite fundamental way through a quite trivial tweak), b) Deep C integration out of the box (which 1 and 2 above regard, really trivial tweaks those two too), and c) Delivering low-level application-level ports e.g. SSL and GZip (3 above regard this, I didn't get to any specific proposal for this til now but from reviewing _io.scm it cannot be a very big deal to do) d) Later I'd guess: Hybrid character and byte access of ports (for ISO-8859-1 encoding, I believe it's already in the box, maybe some clarification needed and possibility for support for more encodings could be looked into) And last e) Making the IO system's UTF-8 coder suitable for any use at all, because it's not due to a freeze-crash bug!! (I believe it's at http://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=127 though Bugzilla is down right now) f) Perhaps later: Higher IO procedure call performance through lower overhead on primarily the per-byte/character calls i.e. read-byte read-char, perhaps by making them automatically inlined and/or by making their mutex use configurable. read-u8 currently delivers ~350KB/sec and an increase to 25MB/sec should be realistic. For now though this can easily be worked around by application-level buffers and use of read/write-subu8vector, though that does come with drawbacks i.e. doesn't integrate well with hybrid byte-character use etc. These are the only practical improvements that have come to my mind in total from several years of intense use of this functionality. Zooming out a bit, this is impressively little really. I would kindly ask you for your feedback on the suggestion regarded by this email. :) Best regards, Mikael

2 8

Edge triggered I/O with libevent
by Adam Langley 04 Avr '13

04 Avr '13

Gambit uses the select call on POSIX systems to multiplex IO operations. This is fine for small numbers of file descriptors, but falls apart quickly with large numbers[1]. libevent[2] is a wrapper library around a number of modern IO multiplexing systems (epoll on Linux, kqueue on the BSDs as well as poll and select). Attached is a patch which converts Gambit to using edge triggered I/O with epoll and libevent. This also requires a patch (attached) of libevent since the standard distribution doesn't include edge triggering support and the upstream appears to be on holiday. (epoll is required because I don't have a BSD box on which to develop edge triggering support for kqueue) This patch (for Gambit) preprocesses away to nothing unless --enable-libevent is passed to the configure script. (Note that, since this patches the configure.ac script, you'll need to run autoconf to regenerate). I hope I've not broken anything, but at this stage I've only tested that the console and tcp ports work. This patch (in the future) should also allow people, who write programs which use socket calls directly, to integrate into the main event loop at some point. Testing is more than welcome and, if you find a form of IO which is broken, please email me. [1] http://monkey.org/~provos/libevent/libevent-benchmark2.jpg [2] http://monkey.org/~provos/libevent/ -- Adam Langley agl(a)imperialviolet.org http://www.imperialviolet.org (+44) (0)7906 332512 PGP: 9113 256A CC0F 71A6 4C84 5087 CDA5 52DF 2CB6 3D60

2 2

How best cancel a read/write-subu8vector or ##device-port-wait-for-input/output! from another thread gracefully? (for abort operation on active HTTP request etc.)
by Mikael 03 Avr '13

03 Avr '13

Dear Marc and list, A universal HTTP client is taking form, it does connect & logics & IO in a singlethread setting now properly. Would be happy to be able to cancel blocking read/write-subu8vector or ##device-port-wait-for-input/output! calls gracefully as to be able to abort HTTP connections directly and gracefully like this: (define r (http-request url)) ((r 'abort!)) ; From another thread than the one performing the HTTP request logics & IO. ; (Even if the abort is done at the level of HTTP request, due to the invalid ; protocol state the abort produces the connection needs to be taken down.) The ways I can see this could be done by would be using - close-port - thread-interrupt! or ##thread-int! ?? - A low timeout on any read and write IO primitive invoked, like, 0.05 seconds, and check for the cancel condition on each timeout and if not reached, invoke the IO primitive again. - thread-terminate! (The HTTP request may be running in the user's thread, so this would require redesign) It'd be most preferable if this cancel could be done in such a way that 1) The abort action's effects are limited within the scope of the code for the respective HTTP request or connection so user code executed after the HTTP request code is not affected by it. This includes taking care automatically of any "mutex abandoned" exceptions or alike that would otherwise come as surprises later. 2) The thread performing the abort action does not get blocked in turn. (Something like that, the write-subu8vector would block a mutex in the IO system that would make the close-port call used to interrupt it, block until timeout.) To help 1), I can ensure that a thread-interrupt! or alike call is not done after that the HTTP request code is done and the thread is back in user code, through making a round of locking some mutex at the end of the HTTP request and guaranteeing a thread-interrupt! never is made after that mutex was locked. It's OK if the interrupt appears to the read/write-subu8vector caller as an IO failure i.e. #f return value. While I believe I could come up with a solution, I'd be very happy to learn to know your take on it as to do this the best way? Thanks! Mikael

2 5

with-exception-handler supports recursion?
by Mikael 31 Mar '13

31 Mar '13

Dear Marc, (with-exception-handler (lambda (e) (print "Outer.\n") 'outer-handler-result) (lambda () (with-exception-handler (lambda (e) (print "Inner.\n") (let ((r (raise e))) (print "Passing on " r ".\n") r)) (lambda () (list 'raise-returned: (raise 'err)))))) prints out "Inner.\n" in an infinite loop currently. Is it not supposed to output Inner. Outer. Passing on outer-handled-result. and then return '(raise-returned: outer-handler-result) ? (Found something that may be very related and filed it as https://www.iro.umontreal.ca/~gambit/bugzilla/show_bug.cgi?id=175 .) Thanks and best regards, Mikael

2 2

How can I allocate a vector in a function?
by Jason Felice 26 Mar '13

26 Mar '13

I don't have the environment to use the ___BEGIN_ALLOC_VECTOR() macros. (No ___hp). I know the size ahead of time. If possible, I'd like to avoid passing ___hp all the way down. Actually, this is one place where I'd love documentation - the environment for writing C functions and how things are protected from gc, etc. I can put it together after I figure it out.

3 3

Re: [gambit-list] Compilation of gambit (git master) on/for Windows 7 64 bit using mingw64 x86_64 fails
by panos stergiotis 25 Mar '13

25 Mar '13

according to config.log HAVE_SELECT is defined as 1. I changed the priority of HAVE_SELECT and HAVE_MSGWAITFORMULTIPLEOBJECTS by changing the ifdef in os.h accordingly and was then able to compile gambit-c successfully. The self test 'make check' failed in the error.scm test case: (make-string 536870911) causes a segfault in gsi.exe. (make-string 536870910) works. For (make-vector <big-number-here>) gsi segfaults too. tests 4,5 and 6 fail with "Signed 32 bit exact integer expected". >In os_io.h, ___device_select_state contains those two slots only if the USE_MsgWaitForMultipleObjects define is enabled. >os.h defines it like this >#ifdef HAVE_SELECT >#define USE_select >#else >#ifdef HAVE_MSGWAITFORMULTIPLEOBJECTS >#define USE_MsgWaitForMultipleObjects >#endif >#endif >That's funny, on Windows it's supposed to always use MsgWaitForMultipleObjects rather than select.. >can you see what your HAVE_SELECT and HAVE_MSGWAITFORMULTIPLEOBJECTS configuration is? >>Hi list >>My attempt to compile the current git version (same for 4.6.6 and 4.6.7) using mingw64 failed with >>os_io.c: In function '___device_event_queue_select_virt': >>os_io.c:4980:16: error: '___device_select_state' has no member named 'message_queue_mask' >>os_io.c:4981:16: error: '___device_select_state' has no member named 'message_queue_dev_pos' >>I used i686 -> x86_64 cross compilation under mingw32, msys1 and Windows 7 64bit. Following the advice of >>http://www.gaia-gis.it/gaia-sins/mingw64_how_to.html and using a x86_64 compiler didn't help neighter. >>Hints are greatly appreciated, thank you in advance >>Panos Stergiotis _______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca[Gambit-list@iro.umontreal.ca] https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

1 0

Compilation of gambit (git master) on/for Windows 7 64 bit using mingw64 x86_64 fails
by panos stergiotis 25 Mar '13

25 Mar '13

2 1

Gambit crash with over 1024 file descriptors
by Dimitris Vyzovitis 25 Mar '13

25 Mar '13

I have a program uses large numbers of file descriptors (over 1024), and it crashes miserably in __fortify_fail. Turning off FORTIFY_SOURCE/stack-protector results in a plain old segfault. Reproducing with the attached program (crash-it.scm): Platform is GNU/Linux x86-64, gcc is 4.6.3. You need to set your ulimit over 1024 to trigger the crash (otherwise you'll just get "Too many open files" and rain of "integer expected" from the looping threads) $ ulimit -n 4096 $ gsi crash-it.scm -e "(go 1500)" *** buffer overflow detected ***: gsi terminated ======= Backtrace: ========= /lib64/libc.so.6(__fortify_fail+0x37)[0x7fb775187dd7] /lib64/libc.so.6(+0xfdbe0)[0x7fb775185be0] /lib64/libc.so.6(+0xffd8e)[0x7fb775187d8e] /usr/local/gambit/current/lib/libgambc.so(___device_select_add_fd+0x55)[0x7fb775d625c5] /usr/local/gambit/current/lib/libgambc.so(+0x227a91)[0x7fb775d62a91] /usr/local/gambit/current/lib/libgambc.so(___device_select+0x104)[0x7fb775d62324] /usr/local/gambit/current/lib/libgambc.so(___os_condvar_select+0x141)[0x7fb775d64d61] /usr/local/gambit/current/lib/libgambc.so(+0x2310a1)[0x7fb775d6c0a1] /usr/local/gambit/current/lib/libgambc.so(___call+0xfa)[0x7fb775d4d2ba] /usr/local/gambit/current/lib/libgambc.so(___setup+0xfc0)[0x7fb775d4f760] /usr/local/gambit/current/lib/libgambc.so(___main+0x2a7)[0x7fb775d4bf57] /usr/local/gambit/current/lib/libgambc.so(___main_char+0xb0)[0x7fb775d59200] /lib64/libc.so.6(__libc_start_main+0xed)[0x7fb7750ac60d] gsi[0x403199] Build config: ./configure --prefix=/usr/local/gambit --enable-single-host --enable-c-opt --enable-gcc-opts --enable-multiple-versions --enable-shared To build without fortifications: ./configure CFLAGS="-U_FORTIFY_SOURCE -fno-stack-protector" ... And you'll get a segfault. -- vyzo

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Gambit-list Mars 2013