Gambit-list Avril 2008

gambit-list@iro.umontreal.ca

17 participants
22 discussions

Edge triggered I/O with libevent
by Adam Langley 04 Avr '13

04 Avr '13

Gambit uses the select call on POSIX systems to multiplex IO operations. This is fine for small numbers of file descriptors, but falls apart quickly with large numbers[1]. libevent[2] is a wrapper library around a number of modern IO multiplexing systems (epoll on Linux, kqueue on the BSDs as well as poll and select). Attached is a patch which converts Gambit to using edge triggered I/O with epoll and libevent. This also requires a patch (attached) of libevent since the standard distribution doesn't include edge triggering support and the upstream appears to be on holiday. (epoll is required because I don't have a BSD box on which to develop edge triggering support for kqueue) This patch (for Gambit) preprocesses away to nothing unless --enable-libevent is passed to the configure script. (Note that, since this patches the configure.ac script, you'll need to run autoconf to regenerate). I hope I've not broken anything, but at this stage I've only tested that the console and tcp ports work. This patch (in the future) should also allow people, who write programs which use socket calls directly, to integrate into the main event loop at some point. Testing is more than welcome and, if you find a form of IO which is broken, please email me. [1] http://monkey.org/~provos/libevent/libevent-benchmark2.jpg [2] http://monkey.org/~provos/libevent/ -- Adam Langley agl(a)imperialviolet.org http://www.imperialviolet.org (+44) (0)7906 332512 PGP: 9113 256A CC0F 71A6 4C84 5087 CDA5 52DF 2CB6 3D60

2 2

New Gambit wiki
by Marc Feeley 18 Mai '08

18 Mai '08

I have just installed a recent distribution of MediaWiki and PHP, and the reCaptcha extension. In order to do this I had to use a different machine (dynamo.iro.umontreal.ca) and copy all the pages from the old wiki to the new wiki. When a page edit is saved a visual captcha will now appear to make sure the edit has been submitted by a human user. Please let me know if you find broken links, missing information, or other issues. Marc

3 6

What are your experiences of Termite?
by Mikael More 01 Mai '08

01 Mai '08

Hi! I'm interested in hearing what uses you and others have had of termite, in production as well as experimentally. What is your general experience? Is it mature for production use? If not, then what needs to be improved? If so, what are the areas that would benefit from further development? By what reasons would you or would you not go with Erlang, or any other environment, to fill the same purposes? If any other environment, then what environment? Thanks! Mikael

5 5

Termite & Butterfly Lessons Learned
by D.McClain 30 Avr '08

30 Avr '08

Hi, While our code base for Butterfly is currently proprietary, I thought I would share a few lessons learned from my own re-implementation of Termite. We have had Butterfly successfully running and spawning remote threads and performing "RPC" across our own local area network among 6 different computers, running for hours on end, passing several millions of messages between them, with no problems, save for a few cautionary notes. Our Butterfly images were written in domain specific Lisp, with all compiler dependencies relegated to a single source module of 80 lines of code. Compilers used were Lispworks for Mac and Windows, and Franz Allegro for Windows. We have a port to SBCL about 99% completed, but we got diverted from this activity before fully resolving our mapping to PThreads. 1. We found you do not need 3 full tasks to manage the socket proxy on each end of a network connection. We found that we needed only a reader and writer thread, with the reader in charge of spawning the writer. Out of order message delivery is a must for ease of protocol for the clients. The proxies perform an initial handshake and exchange of local identities so that each proxy can map incoming and outgoing messages between locally known ID's and the correct socket port. 2. The Lisp reader needs to be extended so that composite objects can be interchanged - e.g., service ID's, remote addresses, etc. We used CLOS to represent some of these higher-level objects, and extended our reader using prefix #U to allow for interchange of printable representations of these kinds of objects. 3. Machine image ID's do need to be UUID's for interchange with other computers, or even between separate processes on the same computer. Each running image has its own notion of process ID's that will possibly conflict with those of other processes. The only way to unabmiguously inform each other about PID's is to attach a UUID as part of the identifier. We use simple cardinal integers, symbols, and most often, keyword symbols, for local identifiers. Mapping tables must exist to translate any of these representations to actual process ID's and the thread object representing the running thread and its mailbox queue. 4. Exception processing needs careful supervision. A separate queue is attached to each thread to hold pending exceptions that can be generated by another running thread calling for termination of a sibling thread. When the afflicted thread resumes, it must check this exception queue before accessing normal messages. To avoid imperative and side-effecting notions such as Erlang's use of Flags to represent supervisor tasks that should become safe against the failure of spawned and linked threads, we use an enclosing macro WITH-TRAPPED-EXCEPTIONS to make this more FPL in style. 5. An agent thread is needed at each end of a socket connection to do the bidding of the opposite end. For example, remote spawning requires a remote agent to perform a local spawn. Termite already has that. We extended the duties of the Agent to include remote killing of threads, remote lookup of various ID's, etc. 6. Despite what Erlang would have you believe, it is not always possible to have direct communications between remote threads. It is often the case that they are segregated from each other, e.g., on different sub-nets, or through a serial chain of servers. So direct replies to a RPC request are often impossible. Hence the need for forwarding proxies at each intermediate server location. Our proxy threads are transparent to the applications running on each node. This is made possible by the use of the PID translation table and the proxy substitution PID table mentioned above. 7. Under heavy traffic conditions it is important for the proxy threads to avoid excessive consing, thereby leading to excessive GC cycles. We use local queues of recyclable envelopes (vectors) that can be stuffed by the socket reader and writer threads for use in forwarding. Once these envelopes are no longer needed they are recycled back into the local queue. Our initial tests among all the participating computers used a simple Echo server at the application level. That performed enough consing at each end to cause occasional long pauses in responses. Even though we are on a local area network, with typical average packet Ping response times of around 1-4 ms, we would see as much as 3-5 second pauses while one node or another performs a large GC cycle. So our timeouts had to be adjusted accordingly. But the median RPC response time was actually around 7 ms over several million cycles. I'm sure there are a bunch of other ideas we came up with to solve each problem as it was encountered. On the whole the re-design effort was an enjoyable experience. The net was around 2300 LOC of Lisp after several rounds of refinements and speedup modifications. Of that, only 80 lines need to be examined for adapting to another compiler. We are now in the process of wrapping the communications with SSL to get necessary levels of commercial security. Our lab is one thing -- an environment for scientists -- but the real world can be a nasty place at times. I hope this is helpful... Dr. David McClain Sr. VP, Embedded Systems Asyrmatos Inc. Boston & Tucson phone: 520-529-2437 cell: 520-390-3995 web: www.asyrmatos.com e-mail: dbm(a)asyrmatos.com

1 0

Object duplication
by David St-Hilaire 29 Avr '08

29 Avr '08

Hi all! Is there a gambit function that can duplicate an arbitrary scheme object available or do I have to write my own for each of my data structures? Thanks! David

2 1

Linker problems
by Burkhard Neppert 29 Avr '08

29 Avr '08

Hello, I've trouble installing gambit 4.2.6 on Win XP with cygwin with gcc 4.2.2 I did ./configure --enable-cplusplus CXX=g++; make Linking of gsi then gives many undefined references: g++ _gsi.o _gsi_.o libgambcgsi.a ../lib/libgambc.a -o gsi.exe _gsi.o:_gsi.c:(.text+0x1d): undefined reference to `____G_main' _gsi.o:_gsi.c:(.text+0x22): undefined reference to `____G_main' _gsi.o:_gsi.c:(.text+0x43): undefined reference to `____S__23__23_type_2d_17_2d_2babe060_2d_9af6_2d_456f_2d_a26e_2d_40b592f690ec' _gsi.o:_gsi.c:(.text+0x4d): undefined reference to `____S__23__23_type_2d_33_2d_bebee95d_2d_0da2_2d_401d_2d_a33a_2d_c1afc75b9e43' _gsi.o:_gsi.c:(.text+0x57): undefined reference to `____S__23__23_type_2d_35_2d_85099702_2d_35ec_2d_4cb8_2d_ae55_2d_13c4b9b05d10' _gsi.o:_gsi.c:(.text+0x61): undefined reference to `____S__23__23_type_2d_5' _gsi.o:_gsi.c:(.text+0x6b): undefined reference to `____S_angle_2d_keyword' _gsi.o:_gsi.c:(.text+0x75): undefined reference to `____S_brace_2d_keyword' _gsi.o:_gsi.c:(.text+0x7f): undefined reference to `____S_bracket_2d_keyword' _gsi.o:_gsi.c:(.text+0x89): undefined reference to `____S_case_2d_conversion_3f_' _gsi.o:_gsi.c:(.text+0x93): undefined reference to `____S_char_2d_delimiter_3f__2d_table' ... Anyone any idea what I'm doing wrong? Thanks Burkhard

1 0

Using mercurial to update Gambit
by James Long 29 Avr '08

29 Avr '08

I'm trying to get on the track of updating Gambit with mercurial. Unfortunately I can't get it to work. Here's the most "official" documentation I could find: http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Contributing_Patches_… I followed those steps and during the `make update` command, I get the following error: james% make update hg pull pulling from http://www.iro.umontreal.ca/~gambit/repo/gambit/ searching for changes no changes found make update-nopull next_version=`hg tags | sed -n -e '/v4.2.6 /{g;1!p;};h' | sed -e 's/\ ([^ ]*\)-bootstrap.*/\1/'`; \ next_version=`hg tags | sed -n -e '/v4.2.6 /{g;1!p;};h' | grep -e 'v.*-bootstrap' | sed -e 's/$[^ ]*$-bootstrap.*/\1/'`; \ if test "$next_version" = ""; then \ hg update; \ else \ hg update -r $next_version-bootstrap; \ make bootstrap; \ hg update -r $next_version; \ make clean bootstrap update; \ fi 0 files updated, 0 files merged, 0 files removed, 0 files unresolved make making all in include making all in lib gcc -I../include -I. -no-cpp-precomp -Wall -W -Wno-unused -O1 -fno- math-errno -fschedule-insns2 -fno-trapping-math -fno-strict-aliasing - fwrapv -fmodulo-sched -freschedule-modulo-scheduled-loops -fomit-frame- pointer -fPIC -fno-common -mieee-fp -DHAVE_CONFIG_H -D___PRIMAL - D___LIBRARY -D___GAMBCDIR="\"/usr/local/Gambit-C/v4.2.6\"" - D___SYS_TYPE_CPU="\"i386\"" -D___SYS_TYPE_VENDOR="\"apple\"" - D___SYS_TYPE_OS="\"darwin9.2.2\"" -c _io.c In file included from ../include/gambit.h:12, from _io.c:1248: ../include/gambit-not402006.h:1:2: error: #error "The version indicated in ___VERSION is not supported by gambit.h" ../include/gambit-not402006.h:2:2: error: #error "You are probably compiling C code generated by a Gambit compiler earlier or later than v4.2.6" ../include/gambit-not402006.h:3:2: error: #error "___VERSION will be set to 402006 to reduce other errors" make[2]: *** [_io.o] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [update] Error 2 Thanks - james

4 6

Listing of wiki pages
by David St-Hilaire 29 Avr '08

29 Avr '08

Hello! I think that someone asked not long ago (last week?) how to list all the pages in the wiki... I don't think that this question was answered and I was also interested in knowing the answer because I could be very usefull and the search does not seem to be really efficient. Sorry for the bumping ;) David

2 1

Access to Gambit version inside Gambit
by François Magnan 29 Avr '08

29 Avr '08

Hi, Here is a simple one... Is there a clean way to access the gambit version number from gsi and compiled code? Thank you, Francois Magnan

2 2

(read-char port) eof question
by Francisco 26 Avr '08

26 Avr '08

According to r5rs the read-char procedure returns the eof object. I asume that eof-object? returns true when called with the eof object returned by the read-char procedure. I have run this tests with string ports and network ports But when I run my program I eventually get the following error: *** ERROR IN ##thread-deadlock-action! -- Deadlock detected It never finds the eof object, is gambit returning the eof object? (I am using v4.2.4) How should I test if a port has reached the end? Thanks, Francisco

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Gambit-list Avril 2008