August 2006 - Gambit-list - mailman.iro.umontreal.ca

Edge triggered I/O with libevent
by Adam Langley 04 Apr '13

04 Apr '13

Gambit uses the select call on POSIX systems to multiplex IO operations. This is fine for small numbers of file descriptors, but falls apart quickly with large numbers[1]. libevent[2] is a wrapper library around a number of modern IO multiplexing systems (epoll on Linux, kqueue on the BSDs as well as poll and select). Attached is a patch which converts Gambit to using edge triggered I/O with epoll and libevent. This also requires a patch (attached) of libevent since the standard distribution doesn't include edge triggering support and the upstream appears to be on holiday. (epoll is required because I don't have a BSD box on which to develop edge triggering support for kqueue) This patch (for Gambit) preprocesses away to nothing unless --enable-libevent is passed to the configure script. (Note that, since this patches the configure.ac script, you'll need to run autoconf to regenerate). I hope I've not broken anything, but at this stage I've only tested that the console and tcp ports work. This patch (in the future) should also allow people, who write programs which use socket calls directly, to integrate into the main event loop at some point. Testing is more than welcome and, if you find a form of IO which is broken, please email me. [1] http://monkey.org/~provos/libevent/libevent-benchmark2.jpg [2] http://monkey.org/~provos/libevent/ -- Adam Langley agl(a)imperialviolet.org http://www.imperialviolet.org (+44) (0)7906 332512 PGP: 9113 256A CC0F 71A6 4C84 5087 CDA5 52DF 2CB6 3D60

2 2

Gambit-C vs. Bigloo vs. MzScheme
by Marc Feeley 28 Aug '06

28 Aug '06

After I saw Eli Barzilay's post to comp.lang.scheme about the performance of MzScheme compared to other Scheme systems (http:// groups.google.com/group/comp.lang.scheme/msg/473f91e7065b6f73? dmode=source) I was tempted to do some benchmarking of my own using the Gambit Scheme benchmark suite which goes beyond the classical Gabriel benchmarks (it has close to 60 Scheme benchmarks). Most benchmarks are straightforward R4RS code and so should work on just about any Scheme system. I have updated the benchmarking infrastructure to accomodate other Scheme systems, and MzScheme and Bigloo in particular. I also wrote a script to run all benchmarks and automatically generate an HTML table of the results. The table can be viewed here: http://www.iro.umontreal.ca/~gambit/bench.html And the benchmark suite is available here: http://www.iro.umontreal.ca/~gambit/bench.tar.gz As you can see from the table, Gambit-C is faster than Bigloo and MzScheme on roughly 3 out of 4 benchmarks. I have used the same assumptions as Eli, which approximate the semantics of R6RS (see the last section of the result page for details). It would be really interesting to extend this experiment to other Scheme systems. It should not be too difficult to modify the benchmarking scripts to test other systems. The hardest parts are installing the Scheme system, figuring out which compiler options are equivalent to the ones used by the other systems (i.e. approximate R6RS semantics), and work out unexpected problems. The following systems are particularly interesting to compare to: - Chicken - Scheme 48 - Larceny - Chez Scheme Are there any volunteers? Marc

4 6

Re: [gambit-list] Fwd: Gambit-C vs. Bigloo vs. MzScheme
by Manuel Serrano 27 Aug '06

27 Aug '06

On Sat, 26 Aug 2006 21:31:16 -0400 Marc Feeley <feeley(a)iro.umontreal.ca> wrote: > Désolé j'ai oublié de te mettre en CC. > > Marc > > > Begin forwarded message: > > > From: Marc Feeley <feeley(a)iro.umontreal.ca> > > Date: August 26, 2006 9:28:15 PM EDT (CA) > > To: Gambit List <Gambit-list(a)iro.umontreal.ca> > > Cc: Eli Barzilay <eli(a)barzilay.org>, Matthew Flatt > > <mflatt(a)cs.utah.edu> > > Subject: Gambit-C vs. Bigloo vs. MzScheme > > > > After I saw Eli Barzilay's post to comp.lang.scheme about the > > performance of MzScheme compared to other Scheme systems (http:// > > groups.google.com/group/comp.lang.scheme/msg/473f91e7065b6f73? > > dmode=source) I was tempted to do some benchmarking of my own > > using the Gambit Scheme benchmark suite which goes beyond the > > classical Gabriel benchmarks (it has close to 60 Scheme > > benchmarks). Most benchmarks are straightforward R4RS code and so > > should work on just about any Scheme system. > > > > I have updated the benchmarking infrastructure to accomodate other > > Scheme systems, and MzScheme and Bigloo in particular. I also > > wrote a script to run all benchmarks and automatically generate an > > HTML table of the results. The table can be viewed here: > > > > http://www.iro.umontreal.ca/~gambit/bench.html > > > > And the benchmark suite is available here: > > > > http://www.iro.umontreal.ca/~gambit/bench.tar.gz > > > > As you can see from the table, Gambit-C is faster than Bigloo and > > MzScheme on roughly 3 out of 4 benchmarks. I have used the same > > assumptions as Eli, which approximate the semantics of R6RS (see > > the last section of the result page for details). > > > > It would be really interesting to extend this experiment to other > > Scheme systems. It should not be too difficult to modify the > > benchmarking scripts to test other systems. The hardest parts are > > installing the Scheme system, figuring out which compiler options > > are equivalent to the ones used by the other systems (i.e. > > approximate R6RS semantics), and work out unexpected problems. > > > > The following systems are particularly interesting to compare to: > > > > - Chicken > > - Scheme 48 > > - Larceny > > - Chez Scheme > > > > Are there any volunteers? > > > > Marc > Hello Marc, Ah, benchmarking... Definitely we can't escape benchmarks :-). I had more or less sweared not to benchmark any more because this is an activity that can consume all your time and for which the result is not necessarily very useful. However, because I didn't want to spend a bad Sunday :-) I have downloaded your benchmarks and make some measurements with Bigloo. Here is what I got: -----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|----- General command line: bigloo -O6 OFP=-copt -fomit-frame-pointer fixnum= /+/+fx/, /-/-fx/, ... unsafe=bigloo -O6 -unsafe FLOAT= re-implementing FLOAT+, FLOAT-, ... ====================================================================== tak1: Marc's time: real: 11.22s sys: 0.02s user: 11.13s fixnum: real: 3.61s sys: 0.01s user: 3.57s fixnum, omit-frame-pointer: real: 2.80s sys: 0.01s user: 2.78s ====================================================================== fft: Marc's time: real: 11.71s sys: 0.03s user: 11.32s FLOAT fixed: real: 4.39s sys: 0.02s user: 4.10s FLOAT fixed, unsafe real: 3.44s sys: 0.01s user: 3.33s FLOAT fixed, unsafe, $OFP real: 3.02s sys: 0.01s user: 2.92s ====================================================================== pnpoly: Marc's time: real: 16.39s sys: 0.02s user: 15.79s FLOAT: real: 15.81s sys: 0.02s user: 14.50s FLOAT, unsafe: real: 1.86s sys: 0.01s user: 1.82s FLOAT, unsafe, $OFP: real: 1.49s sys: 0.00s user: 1.47s ====================================================================== sum: Marc's time: real: 5.44s sys: 0.01s user: 5.36s FLOAT, unsafe, $OFP: real: 5.29s sys: 0.01s user: 5.22s FIXNUM, unsafe, $OFP: real: 0.29s sys: 0.00s user: 0.27s ====================================================================== sumfp: Marc's time: real: 9.76s sys: 0.01s user: 9.57s FLOAT: real: 0.55s sys: 0.01s user: 0.52s FLOAT, unsafe, $OFP: real: 0.54s sys: 0.00s user: 0.52s -----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|----- As you can notice, changing the compilation options and improving your prefix-bigloo.scm file has a huge impact. As you can notice some benchmarks now go 18 times faster! Here is a description of the modiication I have applied: 1- I have re-implemented your macros FLOATxxx that were badling implemented for Bigloo. They used to expanse into call to the generic functions. They now expanse to the floating point operator. The improvement for benchmark using floating applications is in between 2 and 5 times faster. 2- I have used the proper benchmarking compilation options. a- The produced C code must be compiled with -fomit-frame-pointer (when using GCC). b- The option -unsafe has to be used. 3- For fixnum benchmarks, I have replaced the call to the generic functions with calls to fixnum functions. Please, find enclosed the new version of prefix-bigloo.scm. I would appreciate if you could re-execute the Bigloo benchmarks with the new prefix and update your web pages with the new Bigloo results. Thanks in advance. -- Manuel -----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|----- (module prefix (main main-entry) (extern (include "sys/times.h")) (extern (include "time.h")) (option (set! *unsafe-rgc* #t)) (option (set! *unsafe-library* #t)) (option (set! *unsafe-arity* #t)) (option (set! *unsafe-type* #t)) (option (set! *unsafe-struct* #t)) (option (set! *unsafe-range* #t)) (option (set! *unsafe-version* #t)) (option (set! *cc-options* (string-append "-fomit-frame-pointer " *cc-options*)))) (define (clock-to-msecs x) (quotient (* x 1000) (pragma::int "CLOCKS_PER_SEC"))) (define (process-times) (let () (pragma "struct tms buf") (let ((real::long (pragma::long "times(&buf)"))) (cons real (+ (pragma::long "buf.tms_utime") (pragma::long "buf.tms_stime")))))) (define (time* thunk) (let ((start (process-times))) (let ((result (thunk))) (let ((end (process-times))) (let ((cpu (clock-to-msecs (- (cdr end) (cdr start)))) (real (clock-to-msecs (- (car end) (car start))))) (display "cpu time: ") (display cpu) (display " real time: ") (display real) (newline) result))))) (define (run-benchmark name count ok? run-maker . args) (let* ((run (apply run-maker args)) (result (time* (lambda () (run-bench name count ok? run))))) (if (not (ok? result)) (begin (display "*** wrong result ***") (newline) (display "*** got: ") (write result) (newline))))) (define (run-bench name count ok? run) (let loop ((i 0) (result (list 'undefined))) (if (< i count) (loop (+ i 1) (run)) result))) (define (fatal-error . args) (for-each display args) (newline) (exit 1)) (define (main-entry args) (main)) ;------------------------------------------------------------------------------ ; Macros... (define-macro (def-macro form . body) `(define-macro ,form (let () ,@body))) ; Flonum arithmetic. (begin (def-macro (FLOATvector-const . lst) `',(list->vector lst)) (def-macro (FLOATvector? x) `(vector? ,x)) (def-macro (FLOATvector . lst) `(vector ,@lst)) (def-macro (FLOATmake-vector n . init) `(make-vector ,n ,@init)) (def-macro (FLOATvector-ref v i) `(vector-ref ,v ,i)) (def-macro (FLOATvector-set! v i x) `(vector-set! ,v ,i ,x)) (def-macro (FLOATvector-length v) `(vector-length ,v)) (def-macro (nuc-const . lst) `',(list->vector lst)) ) (begin (def-macro (+ . lst) (if (null? (cdr lst)) (car lst) `(+fx ,(car lst) (+ ,@(cdr lst))))) (def-macro (- . lst) (if (null? (cdr lst)) (car lst) `(-fx ,(car lst) (- ,@(cdr lst))))) (def-macro (* . lst) (if (null? (cdr lst)) (car lst) `(*fx ,(car lst) (* ,@(cdr lst))))) (def-macro (/ . lst) (if (null? (cdr lst)) (car lst) `(/fx ,(car lst) (/ ,@(cdr lst))))) (def-macro (FLOAT+ . lst) (if (null? (cdr lst)) (car lst) `(+fl ,(car lst) (FLOAT+ ,@(cdr lst))))) (def-macro (FLOAT- . lst) (if (null? (cdr lst)) (car lst) `(-fl ,(car lst) (FLOAT- ,@(cdr lst))))) (def-macro (FLOAT* . lst) (if (null? (cdr lst)) (car lst) `(*fl ,(car lst) (FLOAT* ,@(cdr lst))))) (def-macro (FLOAT/ . lst) (if (null? (cdr lst)) (car lst) `(/fl ,(car lst) (FLOAT/ ,@(cdr lst))))) (def-macro (FLOAT= a b) `(=fl ,a ,b)) (def-macro (FLOAT< a b) `(<fl ,a ,b)) (def-macro (FLOAT<= a b) `(<= ,a ,b)) (def-macro (FLOAT> a b) `(> ,a ,b)) (def-macro (FLOAT>= a b) `(>= ,a ,b)) (def-macro (FLOATnegative? a) `(negativefl? ,a)) (def-macro (FLOATpositive? a) `(positivefl? ,a)) (def-macro (FLOATzero? a) `(zerofl? ,a)) (def-macro (FLOATabs a) `(abs ,a)) (def-macro (FLOATsin a) `(sin ,a)) (def-macro (FLOATcos a) `(cos ,a)) (def-macro (FLOATatan a) `(atan ,a)) (def-macro (FLOATsqrt a) `(sqrt ,a)) (def-macro (FLOATmin a) `(min ,a)) (def-macro (FLOATmax a) `(max ,a)) (def-macro (FLOATround a) `(round ,a)) (def-macro (FLOATinexact->exact a) `(inexact->exact ,a)) ) ; Generic arithmetic. (begin (def-macro (GENERIC+ . lst) `(+ ,@lst)) (def-macro (GENERIC- . lst) `(- ,@lst)) (def-macro (GENERIC* . lst) `(* ,@lst)) (def-macro (GENERIC/ . lst) `(/ ,@lst)) (def-macro (GENERICquotient . lst) `(quotient ,@lst)) (def-macro (GENERICremainder . lst) `(remainder ,@lst)) (def-macro (GENERICmodulo . lst) `(modulo ,@lst)) (def-macro (GENERIC= . lst) `(= ,@lst)) (def-macro (GENERIC< . lst) `(< ,@lst)) (def-macro (GENERIC<= . lst) `(<= ,@lst)) (def-macro (GENERIC> . lst) `(> ,@lst)) (def-macro (GENERIC>= . lst) `(>= ,@lst)) (def-macro (GENERICexpt . lst) `(expt ,@lst)) ) (begin (def-macro (bitwise-or x y) `(bit-or ,x ,y)) (def-macro (bitwise-and x y) `(bit-and ,x ,y)) (def-macro (bitwise-not x) `(bit-not ,x)) ) -----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|----- ps: benchmarks using vector of float could be improved by using Bigloo typed vectors. I have not tried to apply this because as I said, I don't want to allocate to much time to benchmarking. However, if you want accurate Bigloo measurement. You should try to add the following to prefix-bigloo.scm: (module prefix (main main-entry) (extern (include "sys/times.h")) (extern (include "time.h")) ... (type (tvector f64vector (double)))) (def-macro (FLOATvector? x) `(f64vector? ,x)) (def-macro (FLOATvector-ref v i) `(f64vector-ref ,v ,i)) ...

3 2

Cygwin installation
by Alexander S. Petrov 24 Aug '06

24 Aug '06

Hello, I have problem installing Gambit under Cygwin. "make" command runs whole "configure" again and again. Is it normal?

1 0

MySQL FFI (Version 0.2 ready), FFI questions: strings not terminated with null, passed in strings
by Jonathan Arkell 19 Aug '06

19 Aug '06

Okay, I know it was only last week, but I have a new version of the MySQL FFI ready for testing. As soon as a Gambit module system is in place, I will totally be making this a module. There are much improvements made here. There are a few more higher level functions (mysql-map-rows, mysql-map-fields, get-mysql-datum, mysql-row->list, mysql-column->list, mysql-result->alist, mysql-result->list) and better testing and error recovery. I have also commented the source a lot more, licensed it under the Apache 2.0 license. As Usual, play with it, make suggestions, comments bug reports, etc. Blog entry about it (including personal motivations for writing it! Wow!): http://blog.jonnay.net/archives/744-More-Gambit-MySQL-Mana-From-Heaven,-due… Link to Download: http://blog.jonnay.net/index.php?serendipity[subpage]=downloadmanager&thisc… I also have a few questions regarding how the C functions handle strings with null values and how they handle references, The C Function mysql_real_escape_string is defined like this: unsigned long mysql_real_escape_string(MYSQL *mysql, char *to,const char *from, unsigned long length); "to" is the variable where the result of mysql_real_escape_string is stored, "from" is the string to escape which may contain null bytes, and length is the length of "from". If I define my c-lambda in similar terms, will the string I pass in be properly mutated? Or will I have to build a wrapper function that will return the "to" string instead? (which might be the better option anyway) Also, what happens when I have a Scheme string that contains a null character, and I pass it into a c-function? Will the passed in string be terminated at the first null byte? If so, how do I get around this? Thanks in Advance! -- ,-------. The Cute Grim Floating Skull of Hello Cthulu Says / | | \ / _ ^ _ \ Listen To Crazy Shaman Techno \_/ }|||{ \_/ http://www.jonnay.net '^"^` I'm Jonnay!

1 0

No echo of input to gsi
by bn 19 Aug '06

19 Aug '06

Dear list readers, I have a problem with gambit 40b17 on cygwin/WinXP. I compiled gambit with gcc3.4.4 and it seemed to work. But gsi.exe does not echo the characters I type into the terminal (bash or DOS cmd). I tried various -:- and -:t options but with no success. However, the result of the evaluation of what I typed gets printed. Any help would be appreciated Burkhard

2 1

open-tcp-client
by Lang Martin 17 Aug '06

17 Aug '06

I note that UDP and RAW socket support is on it's way -- does this include support for specifying other parts of the address request? I need tcp clients and servers to bind to source addresses that aren't the default address on the machine. I started digging into working on this myself, but it seems a bit more complicated than I was hoping. Is support for the full set of socket options planned? Soon? Thanks.

3 3

Introduction, MySQL FFI
by Jonathan Arkell 16 Aug '06

16 Aug '06

Hi List I've been playing around with Gambit for the past little while, and I really like it. Congrats and thanks to Marc Feeley (and others) for all their hard work. I have just finished putting together an alpha-quality MySQL FFI for gambit, available here: http://blog.jonnay.net/archives/742-A-Gambit-MySQL-Gambit-FFI.html Direct Link to source: http://blog.jonnay.net/plugin/dlfile_4 It still needs plenty of work (a proper test suite, better handling of field data, documentation, implementing mysql_real_escape_string, implementing the rest of the MySQL c API) but it should be enough to get you connected to a myself database, and run some queries. I've also built some simple wrappers and plan to build more, taking into account the work that Oleg Kiselyov has done in this area (http://okmij.org/ftp/Scheme/index.html#databases) Any comments, suggestions, bug fixes or enhancements are welcome. Here is a simple example of how to use the FFI (define test-server (make-mysql-server "127.0.0.1" "test" "test" "test" 0 "" 0)) (define test-mysql-result->list (lambda () (print-line "testing mysql-result->list") (with-mysql-connection test-server (lambda (mysql-handle) (mysql-query-with-result mysql-handle "select * from test" (lambda (result) (print-line "Number of rows: " (mysql-num-rows result)) (pp (mysql-result->list result)) (pp (mysql-result->list result)) (pp (mysql-result->list result)))))))) Cheers! -- ,-------. The Cute Grim Floating Skull of Hello Cthulu Says / | | \ / _ ^ _ \ Listen To Crazy Shaman Techno \_/ }|||{ \_/ http://www.jonnay.net '^"^` I'm Jonnay!

2 1

Re: [gambit-list] Can Gambit programs be safely fork()'ed?
by TJay 01 Aug '06

01 Aug '06

On 7/31/06, Christian <christian(a)pflanze.mine.nu> wrote: > At 1:04 Uhr +0800 31.07.2006, TJay wrote: > >I wasn't thinking about file descriptors in particular. I was > >wondering whether gambit itself has any internal state that's not > >friendly with fork(). (I have no idea what these states might > >include.) > > I think I've already given my non-authoritative answer (just a > guess): if you have multiple gambit threads running, then call fork, > both processes will continue running all the threads, and if you > print some stuff to buffered ports but do not flush nor close > (through the POSIX call) them, you'll get output being printed twice. > I don't know how I would handle the former problem (I think there's > no place where Gambit is doing this already; the only place where > Gambit calls fork is in implements > ___device_stream_setup_from_process in os_io.c and there is no need > to shut down other threads as exec is called immediately). I see. I've not thought of how threads would be affected, and I'd hate to think of the kinds of strange, hidden bugs I'd get from unflushed buffered ports. I'll have to be super careful if I do any forking. > BTW using fork as opposed to starting subprocesses using open-process > will probably not give you much of an advantage, if you load all code > as compiled shared libraries, since almost all shareable memory will > then be shared anyway, and you don't seem to want to fork chilren at > a high frequency. You're right. It might be easier to just call open-process. Thanks for the heads up. tj

2 1

creating scheme ports from filehandles
by Christian 01 Aug '06

01 Aug '06

Hello Is there a way to turn a POSIX filehandle into a Gambit scheme port, or could it be added? I'm imaging something like: (posix-fd->port 0) => returning an input-output port or (open-port (list posix-fd: 0 direction: 'input)) ;; in case one wants ;; the direction to be preserved. or similar. This would allow things like implement functions creating unix sockets, or maybe even UDP ports; well for the latter maybe more changes would actually be needed. Maybe I should ask (instead of the above) how one can, from a C function, read/write from/to a file descriptor without blocking the system (using the scheduler, thus possibly running other scheme code in the mean time). Thanks Christian.

2 3