October 2005 - Gambit-list - mailman.iro.umontreal.ca

Edge triggered I/O with libevent
by Adam Langley 04 Apr '13

04 Apr '13

Gambit uses the select call on POSIX systems to multiplex IO operations. This is fine for small numbers of file descriptors, but falls apart quickly with large numbers[1]. libevent[2] is a wrapper library around a number of modern IO multiplexing systems (epoll on Linux, kqueue on the BSDs as well as poll and select). Attached is a patch which converts Gambit to using edge triggered I/O with epoll and libevent. This also requires a patch (attached) of libevent since the standard distribution doesn't include edge triggering support and the upstream appears to be on holiday. (epoll is required because I don't have a BSD box on which to develop edge triggering support for kqueue) This patch (for Gambit) preprocesses away to nothing unless --enable-libevent is passed to the configure script. (Note that, since this patches the configure.ac script, you'll need to run autoconf to regenerate). I hope I've not broken anything, but at this stage I've only tested that the console and tcp ports work. This patch (in the future) should also allow people, who write programs which use socket calls directly, to integrate into the main event loop at some point. Testing is more than welcome and, if you find a form of IO which is broken, please email me. [1] http://monkey.org/~provos/libevent/libevent-benchmark2.jpg [2] http://monkey.org/~provos/libevent/ -- Adam Langley agl(a)imperialviolet.org http://www.imperialviolet.org (+44) (0)7906 332512 PGP: 9113 256A CC0F 71A6 4C84 5087 CDA5 52DF 2CB6 3D60

2 2

handling of macros
by Christian 14 Nov '05

14 Nov '05

Hello Some questions: (the first two of those may be rather general scheme philosophy questions) - why are DSSSL style argument declarations not allowed in define-macro? - why is there no (define-macro foo (lambda(...)...)) form? Is there a lowlevel variant for setting runtime and compiletime macro bindings to some lambda? - is there a way to get at the converter lambda from a (runtime-)macro definition? - I need a way to unset a macro definition, so that the corresponding name can be used as normal function/variable binding from there on. - I'd like to be able to scan a namespace for all bindings and macro bindings. For example for readline completion. (And to be able to remove all runtime macros.) - how can I macro-expand code with the original context? Until now I've used (caddr (##decompile (eval `(lambda() ,code)))) but code is eval'ed in a new lexical context of course. (I haven't understood how to use ##macro-expand. I would be grateful for any help for understanding the sources. How do the cte (=compile-time environment?) and such work? What's the exact concept?) I want macro expansion for writing macros which expand their arguments before working on them. For playing with optimizations (kind of like optimizing mini-compilers). - is there a way to preserve the line/column numbering of source code which is "wrapped" by macros? For something like for example (receive (a b) (code1) code2) it would be nice if when an error happens in code1 or code2, the error message didn't just mention the line/column of the opening paren of the receive form, but the line/column where the relevant part of code1 or code2 is in the original file. Thanks, Christian.

3 4

One other thing to help builders
by Pupeno 09 Nov '05

09 Nov '05

One other thing that can be added to Gambit-C to help automatic builders is a '-v' parameter that prints the name and version of the compiler (without colors or anything, so it is easily parseable). -- Pupeno <pupeno(a)pupeno.com> (http://pupeno.com)

2 2

gc problems with streams/recursions
by Christian 31 Oct '05

31 Oct '05

Hello While playing with streams, I think I've noticed that the stream "head" is not garbage collected while traversing the stream even if it is not referenced by user data structures anymore. There also seems to be a (independent?) problem releasing memory when streams are long (thus recursion is deep). I'm getting out of memory errors when "there shouldn't be", and need to run (##gc) manually to be able to continue calculations (entering ,c in the debugger resumes calculation correctly): it looks like gambit is starting to fill up all available memory if the recursion depth is bigger than something between 20000 and 35000. The two issues happen in both interpreted and compiled code. $ ulimit -S -v 300000 $ gsc > ; (compile-file "test.scm") > (load "test") > (test 35000) repeat the test call several times, it will give *** ERROR IN ##bignum.make -- Heap overflow after about 10 invocations. Same thing with > (test-2 35000) This can be repeated "forever" without hitting out of memory errors: > (test 20000) or this > (test-2 20000) (There are two implementations of stream-take below (one commented out), choosing the other one doesn't change the situation.) This is with Gambit Version 4.0 beta 14 on Debian ppc. With beta 15 on Debian x86, the out of memory situation already happens with a test argument of 25000. What I'm especially wondering is this: the output list from take only needs about 600000 bytes. So if the stream would be deallocated while stream-take is still walking it, much longer streams should be possible (at least 10 times longer). Do you have an explanation or a solution? Thanks Christian. ;; test.scm (declare (block) (standard-bindings) (extended-bindings) ) ;; stream library (define *stream-strict* #f) (set! *stream-strict* #t) (define @force ##force) (define @promise? ##promise?) (define @car ##car) (define @cdr ##cdr) (define @pair? ##pair?) (define @null? ##null?) (define (stream-force strm) (if (@promise? strm) (@force strm) (if *stream-strict* (error "not a stream: expected promise, got:" strm) strm))) (define stream-car (lambda (strm) (let ((p (stream-force strm))) (if (@pair? p) (@car p) (error "stream-car: not a pair:" p))))) (define stream-cdr (lambda (strm) (let ((p (stream-force strm))) (if (@pair? p) (@cdr p) (error "stream-cdr: not a pair:" p))))) (define (null-stream? strm) (let ((l (stream-force strm))) (cond ((##pair? l) #f) ((##null? l) #t) (else (error "null-stream?: argument out of domain:" l))))) ; (define (stream-take strm k) ;; returns a list, not a stream. ; (if (and (##fixnum? k) (##fixnum.>= k 0)) ; (if (##fixnum.= k 0) ; '() ; (or (and (not (null-stream? strm)) ; (let ((res (cons (stream-car strm) '()))) ; (let iter ((res-tail res) ; (strm (stream-cdr strm)) ; (k (##fixnum.- k 1))) ; (if (##fixnum.zero? k) res ; (let ((p (stream-force strm))) ; (and (##pair? p) ; (let ((res-next (##cons (##car p) '()))) ; (##set-cdr! res-tail res-next) ; (iter res-next ; (##cdr p) ; (##fixnum.- k 1))))))))) ; (error "stream-take: stream too short"))) ; (error "stream-take: k is not a positive fixnum:" k))) (define (stream-take strm k) ;; returns a list, not a stream. (if (and (##fixnum? k) (##fixnum.>= k 0)) (let recur ((strm strm) (k k)) (if (##fixnum.zero? k) '() (let ((p (stream-force strm))) (if (##pair? p) (##cons (##car p) (recur (##cdr p) (##fixnum.- k 1))) (error "stream-take: stream too short"))))) (error "stream-take: k is not a positive fixnum:" k))) (define-macro (scons a r) `(delay (cons ,a ,r))) ;; the actual program (define (fib-stream) (letrec ((strm (scons 0 (scons 1 (let recur ((2behind (let ((s strm)) ;(set! strm #f); make gc possible? no, doesn't help. s))) (delay (let ((a (stream-car 2behind)) (r (stream-cdr 2behind)) ) (cons (+ a (stream-car r)) (recur r))))))))) strm)) (define (fib-stream-2) ;; (misses the first two values of the fib series) (let recur ((pprev 0) (prev 1)) (delay (let ((cur (+ prev pprev))) (cons cur (recur prev cur)))))) (define t #f) (define (test n) (time (set! t (stream-take (fib-stream) n)))) (define (test-2 n) (time (set! t (stream-take (fib-stream-2) n)))) ;;end

1 1

Reentrant calling into Gambit object code
by Marko Cetina 30 Oct '05

30 Oct '05

I'd like to use Gambit-compiled object code from a multithreaded C application. However, I'm unsure if Gambit supports reentrant calling from multithreaded C code. Could anybody help me with this? Thanks a lot, Marko Cetina _________________________________________________________________ Dont just search. Find. Check out the new MSN Search! http://search.msn.click-url.com/go/onm00200636ave/direct/01/

3 2

extracting information from socket ports
by david rush 30 Oct '05

30 Oct '05

Maybe I'm just missing it in the documentation, but how can I extract some of the core networking information from a tcp-port returned from reading a port opened with open-tcp-server? I'm hoping to extract at least the IP address and port number of the inbound connection for authorization purposes. david rush -- DIsruptive Technology!

2 1

Re: Calling blocking C libraries from scheme threads
by Christian 28 Oct '05

28 Oct '05

At 15:45 Uhr +0200 18.10.2005, Johannes Groedem wrote: >The libraries often have provisions for this. See for example >PostgreSQL's client library and its PQconsumeInput, PQisBusy, etc. >But I guess the C libraries you're using don't support this, then? The MySQL client library doesn't seem to support this. >It could be that the best solution is to just implement the protocol >in Scheme. It might be tempting to try this and see how fast Gambit can get with it. And see whether one is able to grok and imitate the protocol. But I think that doesn't scale in two ways: first, if Mysql is extended with new functionality, that interface might be going to break much sooner than if it did use the C libs. Second, there are more libraries than only this one. Imagine if you could run all of Perl's CPAN without change and not block any other scheme thread. Christian.

2 1

Need different compiler behavior to debug arithmetic
by Bradley Lucier 26 Oct '05

26 Oct '05

Marc: Here's something to think about for later. I know of no way, at the Scheme level, to debug incorrect declarations of fixnum or flonum. If you run an application interpreted, it just works, which is OK. If you run something compiled in safe mode, then it works whether the declarations are correct or not. And when you run it in unsafe mode then it generally fails with a segmentation fault. The only way I know to debug this type of fault is to compile Gambit and the application itself with - g, run the application inside gdb, then determine (a) what operation is failing and (b) the location of that operation in the scheme code by diagnosing the C output of gsc. Not fun, even for me ;-). I'l like to suggest the following compilation model for compiling numerics in safe mode: Declaration: generic, mostly-* Result: Generates code to determine the types of arguments and do the correct thing, including calling the generic operations if necessary. Declaration: fixnum, flonum Result: check the types of arguments, and generate an exception of the arguments do not match the correct types. I don't think there needs to be any change in unsafe mode. This way I could debug mistaken declarations without going through gdb. Brad

2 4

Calling blocking C libraries from scheme threads
by Christian 18 Oct '05

18 Oct '05

Hello For writing interfaces to C libraries which have socket function calls hard wired (like database client libraries), how can I make foreign calls so that they don't block other scheme threads? At first, I've thought about hacking those libraries so that they allow the socket functions to be replaced by callbacks (to functions which go into the gambit scheduler / select loop), but then I realized that this won't solve the problem, since for each scheme thread that is calling a foreign function a separate C stack would be necessary, or there would be no way to let threads continue which don't have their C call frame at the top of the stack. So the only solution seems to be using a separate pthread for each foreign function call done by a scheme thread. (The pthread could be cached and reused for subsequent foreign calls by the same scheme thread, or even be reused for different scheme threads after the scheme thread has finished.) I don't know yet the details about how to set up the connection between those pthreads and the scheme calls. I guess by using some mutex which can be accessed by both scheme and C and passing the arguments through some thread-local storage. (I've read that the GHC Haskell compiler does something like this. I could take a more detailed look there.) Has anyone done something like this already? Any hints, ideas, suggestions? (Would there be interest to include a solution into the Gambit distribution?) Thanks Christian.

2 1

Changing the name of the compiler
by Pupeno 06 Oct '05

06 Oct '05

Hello Gambiters, This may be a strange thing to ask, but could you change the name of the compiler, that is, the binary (I would put this a bug tracker, but there's none for Gambit, am I wrong ?). The problem is that there's another widely used program with a binary with the same name: $ gsc ESP Ghostscript 7.07 (2003-07-12) ... In my distro, Gambit's gsc is renamed to gsc-gambit. This makes writting automated building software very hard. I have to check if gsc exists, then check if it is Gambit or not, if it is not, check if gsc-gambit exist, and so on. Do you think it'll be possible to change the name of the compiler with something that won't conflict with something as common as Ghostscript ? Thank you. -- Pupeno <pupeno(a)pupeno.com> (http://pupeno.com)

1 0