Gambit-list Mars 2012

gambit-list@iro.umontreal.ca

25 participants
24 discussions

Edge triggered I/O with libevent
by Adam Langley 04 Avr '13

04 Avr '13

Gambit uses the select call on POSIX systems to multiplex IO operations. This is fine for small numbers of file descriptors, but falls apart quickly with large numbers[1]. libevent[2] is a wrapper library around a number of modern IO multiplexing systems (epoll on Linux, kqueue on the BSDs as well as poll and select). Attached is a patch which converts Gambit to using edge triggered I/O with epoll and libevent. This also requires a patch (attached) of libevent since the standard distribution doesn't include edge triggering support and the upstream appears to be on holiday. (epoll is required because I don't have a BSD box on which to develop edge triggering support for kqueue) This patch (for Gambit) preprocesses away to nothing unless --enable-libevent is passed to the configure script. (Note that, since this patches the configure.ac script, you'll need to run autoconf to regenerate). I hope I've not broken anything, but at this stage I've only tested that the console and tcp ports work. This patch (in the future) should also allow people, who write programs which use socket calls directly, to integrate into the main event loop at some point. Testing is more than welcome and, if you find a form of IO which is broken, please email me. [1] http://monkey.org/~provos/libevent/libevent-benchmark2.jpg [2] http://monkey.org/~provos/libevent/ -- Adam Langley agl(a)imperialviolet.org http://www.imperialviolet.org (+44) (0)7906 332512 PGP: 9113 256A CC0F 71A6 4C84 5087 CDA5 52DF 2CB6 3D60

2 2

Gambit on Android
by Steve Graham 10 Jui '12

10 Jui '12

I understand that Android apps are typically written in Java. How then is Gambit able to run on that platform? If it is possible to write GUI apps for the Android using Gambit, are there any pointers on how to get started? Thanks, Steve

4 23

Native threading with C++?
by Patrick Bene 21 Avr '12

21 Avr '12

I'm interested in using multithreaded C++ code along with Gambit. From what I understand, Gambit does not support native multithreading, but what about the FFI? Am I allowed to have multiple C++ threads running alongside Gambit? Additionally (in case I missed something), is there anyway to get Gambit itself running in multiple native threads? I don't need full-blown functionality, but I'll take anything that's availible. Many thanks, -- Patrick

7 11

Proposal for lightweight real hygienic macros in Gambit
by Per Eckerdal 02 Avr '12

02 Avr '12

Hi, Background A few years ago, I started working on a module system for Gambit called Black Hole. I have still not managed to make it work well, and I do not have time to work on it right now, but my work on Black Hole has taught me quite a bit about Gambit and Scheme, and macro systems in particular. Most of my Gambit related work has been centered around Black Hole, and I have made no attempts to push ideas in Black Hole back to Gambit, mostly because I have felt that Black Hole has a narrower scope than Gambit. I recently realized that some of the ideas that I have stumbled upon might actually be useful for the Gambit community as a whole. This email is about one of them. Gambit and hygienic macros Gambit ships with a generic implementation of syntax-case hygienic macros, psyntax, but in practise it does not work very well, especially in conjunction with features that are specific to Gambit. Marc has repeatedly stated that he does not intend to add syntax-case to the default Gambit language, not only because of the aforementioned reasons but also because it would increase the size of the Gambit system unacceptably. syntax-case is a rather large piece of software, and is also IMO bloated conceptually. (I should add that my Black Hole system is not much better in this regard.) In fact, most Scheme macro systems that address the hygiene problem, especially the modern ones, as defined by R6RS and related systems, add a huge amount of conceptual overhead to the language. The Gambit way of things My impression of Gambit is that it's spirit is fairly "close-to-the-metal", being a tightly integrated system that works well on a wide variety of platforms including ones with limited resources. The mechanisms provided by Gambit for organizing code into modules and writing macros match this thinness: |define-macro| is a very simple construct for non-hygienic macros and |##namespace| is a fairly low level tool for avoiding name clashes. Proposal This is more of a straw man than a proper proposal, and I am fully aware that this will probably conflict with many ideas that R7RS introduces. This proposal is about a way of achieving hygienic macros that is far more lightweight than any other hygienic Scheme macro system that I know of. In fact, Gambit already includes most of the machinery that is required by this system. I did not invent this technique myself. The basic idea is described here: http://www.p-cos.net/documents/hygiene.pdf Here's a description of the two related, but separate, issues that hygienic macro systems try to solve: http://pereckerdal.com/2010/05/18/the-two-faces-of-the-hygiene-problem/ If you are not familiar with hygienic macro systems, reading this blog post might make it easier to understand this proposal. In order to explain the concepts as clearly as possible, I will begin with explaining the core problem, then describe way to write macros with the current Gambit system that ensures hygiene, and then continue to generalize the approach. A quick summary of the two hygiene problems As the blog post I linked to earlier describes, there are two distinct types of unhygiene in macros. One of them can be avoided through careful use of |gemsym|, but the other cannot. Here's an illustrative example: (define-macro (ten-times . x) `(let loop ((i 10)) ,@x (if (> i 1) (loop (- i 1))))) This code defines a macro that makes a piece of code run ten times. An example use is (ten-times (println "Hello")). The definition has two different kinds of errors in it. The first one shows up if you try to do (let ((i "i")) (ten-times (println "The letter i: " i))). The code is intended to print "The letter i: i" ten times, but instead it counts from ten to one. This problem can be solved by replacing |i| in the macro with a gensym. The second problem shows up if you try to do (let ((if list)) (ten-times (println "Hi"))). The code is intended to print "Hi" ten times, but instead it prints nothing. This second problem what this proposal addresses. I think that's enough, because I don't think the "gemsym fixable" problem is that serious. Not making gensym mistakes is fairly easy, but there is no obvious way to circumvent the second problem. Manually emulating hygiene with Gambit's current define-macro macros With these two problems in mind, we can define three rules for writing Gambit code: 1. When writing macros, use gensym to avoid unintentional variable capture wherever necessary and possible. 2. In all code, not just macros, never lexically bind an identifier with a # in it. For instance, even though Gambit would not reject it, never write (let ((##let 3)) (code-goes-here)). 3. Qualify every function, macro or global variable that you use in a macro expansion with its namespace (that is, all names that you use must contain a #). For instance, instead of writing (define-macro (two-list a b) `(list ,a ,b)), write (define-macro (two-list a b) `(##list ,a ,b)). If these rules are observed, you will have full hygiene. The rest of the email is dedicated to describing how one could change Gambit to perform this process in a semi-automatic way. This does not really work with the current version of Gambit A minor sidetrack about how this would work with Gambit in practise: Doing this with the current version of Gambit is not quite possible, even if you do it fully manually. The main problem is that not all of Gambit's built in functions have a version with a namespace prefix. To be able to apply this systematically, all built-in functions would have to be renamed to get a namespace prefix. The namespace system would then make sure to expand all occurrences of eg |car| to |g#car| or something similar. If all other names are prefixed (including local bindings), one can get away with not adding a namespace prefix to Gambit's built in functions. Black Hole works this way, but it would nicer if Gambit also followed the convention. The alias macro To avoid having to type namespace prefixes all over macro code, and also as a basis for further helper macros, I propose to add a new special form called |alias|. What it does is that it takes an unqualified symbol and expands it into a quoted fully qualified symbol with a namespace prefix, according to the namespace declarations that are currently in scope. For instance, assuming that Gambit's built-in functions use the "g#" namespace and a fresh REPL: > (alias car) g#car > `(,(alias if) #t 'yes 'no) (##if #t 'yes 'no) When writing macros, it would be used like this: (define-macro (add a b) `(,(alias +) ,a ,b)) Note that |add| is a fully hygienic macro. The call to |+| will expand into |g#+|, which always has the same meaning. If we would have omitted |alias|, and just written |+|, the macro would not have worked in lexical environments where |+| means something different from what it means in the context where the macro is defined. Implementing |alias| in Gambit should be trivial, since Gambit already does this transformation internally with its namespace machinery. Restrictions on name binding For hygiene to work, shadowing bindings must be disallowed. This code snippet illustrates the problem: (let ((a 0)) (define-macro (mac name) `(list ,(alias a) ,name)) (let ((a 1)) ;; This should return (0 1), but it will return (1 1). (mac a))) To make this possible, defining a name that is already defined must be disallowed. I would make it a compilation error. Auto-aliasing bindings The restriction of not being allowed to bind a name that's already bound is clearly too limiting; we want the code above with the two |a|s to work. To circumvent this, we introduce auto-aliasing to all binding language constructs. Auto-aliasing means that binding a name, for instance (let ((a 0)) (code-goes-here)) is internally expanded to becoming equivalent to (let ((0#a 0)) (##namespace "0#" a) (code-goes-here)) The choice of "0#" as a namespace prefix is arbitrary, any prefix could be chosen, as long as all bindings end up unique. In fact, I think it should be possible to not prefix the first occurence of each name in the scope, so that only bindings that shadow another binding is prefixed. This might reduce clutter. (Note that this only works if *all* top level bindings are namespaced, though. I'm not even sure if just looking up the name in the global namespace on macro expansion time and prefixing if a global variable with that name is found is sufficient) Auto-aliasing should, like |alias|, be fairly straightforward to implement. Note that it is sufficient to implement auto-aliasing in the built-in binding special forms (like let, lambda, parameterize), any user-defined macro that does name binding must use the underlying special forms and will thus automatically be auto-aliasing. Normally, the programmer will not have to think about the auto-aliasing. The only time it will be visible is when inspecting code in the REPL/debugger. If the aliasing turns out to be too invasive, it should be possible to do some lower-level magic to properly hide this from the user. I personally think it is not without benefit to have shadow-free code: You will always be able to access all variables from the debugger. Note that Gambit's current #!key argument implementation makes it impossible to implement auto-aliasing as a macro, because renaming keyword arguments changes the procedure's interface. But then again I have never seen code with #!key arguments that shadow each other. Simply ignoring to auto-alias #!key parameters, and emitting a compiler error when they shadow each other (according to the name binding restriction rule) might be sufficient. Namespaces Probably the largest difference this would do to the Gambit language is that it would require all code to specify a namespace. I'm not sure about how to best do this. One possibility might be to implicitly prepend a |##namespace| directive to all files, with the namespace matching the file name. The file could then override the default if desired. The REPL could have its own reserved "r#" namespace. Generalized alias It is possible to expand the definition of |alias| so that it can take an arbitrary s-expression as argument. The semantics would be like quasiquote, except that all symbols on the first quasiquotation level would be alias-expanded. Some examples: (alias +) => g#+ (alias (+ 1 2)) => (g#+ 1 2) (alias (eq? 'a a)) => (g#eq? 'a current-namespace#a) ;; Note that quasiquote is a macro, with a corresponding g#quasiquote fully qualified name, while unquote is not actually a macro, so it does not have a fully qualified name (alias (list 'a a `(a b ,c))) => (g#list 'a current-namespace#a (g#quasiquote a b (unquote current-namespace#c))) |quasiquote| like functionality is nontrivial to implement, but it should be possible to implement |alias| like this by piggybacking on the |quasiquote| implementation, so it should not be that difficult to do. Another option is to let |alias| remain as taking a single symbol as argument, and instead call the quasiquoting alias |alias-quasiquote| or something similar. Here's a more intricate (and properly hygienic) example of the |ten-times| macro, implemented with the generalized |alias| and |gensym|: (define-macro (ten-times . x) (let ((loop-gs (gensym)) (i-gs (gensym))) (alias (let ,loop-gs ((,i-gs 10)) ,@x (if (> ,i-gs 1) (,loop-gs (- ,i-gs 1))))))) This is how a typical macro definition will look like. It is also possible to give |alias| a syntactic shortcut, like what has been done to quote, quasiquote and unquote. Summary In this email I have described a hygienic macro system that is easy to use, is very similar to the current non-hygienic system (it is, in fact, source level compatible), allows the user to choose whether to break hygiene or not, is lightweight, has semantics that is very easy to reason about and does not introduce any fundamentally new concepts to the language. Furthermore, it is very simple to implement and no special algorithmic tricks are required to implement a macro expander that takes linear time with respect to code size. Making the changes I suggest to Gambit would make it possible and easy to write truly hygienic macros in Gambit, without compromising what I feel is the spirit of the Gambit system. The main problem with this system is that, just like Gambit's current define-macro macros, it discards source code location information. Furthermore, writing macros that give good looking error messages on incorrect inputs is tedious. I can think of a couple of ways to work around these problems, though. But solving this is necessarily a bit complicated, so I will not discuss that here. Fortunately, these issues are orthogonal to the hygiene problem. I would love to hear what you think about this. Best Regards, Per Eckerdal

5 10

Thread switch timer, hook?
by Mikael More 31 Mar '12

31 Mar '12

Dear Marc, On switches between threads or to/from select(), is the system/epoch time loaded anywhere then so that one could get a copy? Is there any "on thread/select() switch" Scheme procedure where one could add code to execute on each thread switch? Any particular things to take into consideration if hooking it? (i remember the rbtree used for getting the next thread has constant stack use as a feature) Kind regards, Mikael

2 1

Make update from github?
by Bradley Lucier 29 Mar '12

29 Mar '12

Marc: What should we do after building 4.6.5 to get "make update" to work? I get [Bradley-Luciers-MacBook-Pro:~/programs/gambc-v4_6_5-devel] lucier% make update The authenticity of host 'github.com (207.97.227.239)' can't be established. RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'github.com,207.97.227.239' (RSA) to the list of known hosts. Permission denied (publickey). fatal: The remote end hung up unexpectedly Permission denied (publickey). fatal: The remote end hung up unexpectedly make: *** [pull] Error 128 [Bradley-Luciers-MacBook-Pro:~/programs/gambc-v4_6_5-devel] lucier% make update Permission denied (publickey). fatal: The remote end hung up unexpectedly Permission denied (publickey). fatal: The remote end hung up unexpectedly make: *** [pull] Error 128 Thanks. Brad

3 5

Erlang to Scheme compiler (ETOS) on github
by Marc Feeley 27 Mar '12

27 Mar '12

I've just put ETOS (the Erlang to Scheme compiler) on github. The repo is at this URL: https://github.com/feeley/etos ETOS supports a large chunk of the Erlang language, but very few of its libraries. I can give push access to anyone interested in improving the system (improving the compiler, porting it to other Scheme systems, adding libraries, etc). The README file is attached below. Enjoy! Marc ETOS is an Erlang to Scheme compiler designed at the Universite de Montreal. The Scheme code it generates is compiled to native code with the Gambit Scheme compiler, which allows it to achieve good performance. Here are some papers and presentations on ETOS: - "Compiling Erlang to Scheme", PLILP 1998 (http://www.iro.umontreal.ca/~feeley/papers/FeeleyLarosePLILP98.pdf) - "Status report on the ETOS Erlang to Scheme compiler", Erlang User Conference 1999 (http://www.erlang.org/euc/99/ETOS.ps) - "Etos: an Erlang to Scheme compiler", Technical Report 1079, 1997, DIRO, Universite de Montreal (http://www.iro.umontreal.ca/~feeley/papers/FeeleyLaroseDIRO1079.pdf) 1. INSTALLING Before installing ETOS, you must install the Gambit Scheme compiler (http://gambit.iro.umontreal.ca/). ETOS v2.4 has been tested with Gambit v4.6.5 (older versions of Gambit will probably work too, but don't go too far back). Then build ETOS with the command % make install This will place the following files in the bin/ directory: etos the ETOS compiler estart the program loader rt-gambit.scm a Scheme file with macro definitions required by the Gambit Scheme compiler to compile the Scheme files produced by ETOS If you wish to install these files somewhere else, then change the definition of INSTALL_DIR in the Makefile before the "make install". Note that on some platforms it can take a long time to compile etos. On Mac OS X with LLVM gcc, it takes roughly half an hour on a fast machine (with GNU gcc it takes only a few minutes). 2. USAGE Erlang files, with extension .erl, can be compiled by invoking etos: % bin/etos hw.erl By default this will produce the files hw.scm, the Scheme code generated, and hw.o1 (or hw.o2, etc) which is the dynamically loadable object file produced from hw.scm by the Gambit Scheme compiler. To execute the program, simply invoke estart with the name of the module, "hw" in this case: % bin/estart hw "Hello World!" You can get usage information on these programs with: % bin/etos --help or man pages with: % man man/man1/etos.1 % man man/man1/estart.1 3. BENCHMARKS The benchmarks can be run with the command: % make bench

2 2

References between 'foreign' objects.
by Estevo 27 Mar '12

27 Mar '12

I'm trying to write a macro to wrap C structures and unions [1] in an interface similar to that of Scheme records. It defines an appropriate foreign type, creates a constructor, utility functions to take and dereference pointers, and member accessors and mutators. Here is the definition [2]: https://github.com/euccastro/gambit-SDL2/blob/master/ffi.scm and here is a sample of how you are supposed to use it: https://github.com/euccastro/gambit-SDL2/blob/master/test-macro-struct.scm So far I've been letting Gambit take care of cleaning up structs, except for those that were obtained via a non-pointer accessor to an existing struct. So disposing of your last Scheme reference to a "child" struct will not try to clean up memory in the "parent" struct. I did this in an unclean way (I'm wrapping each struct twice, with different Scheme names, one of them with a bogus cleanup function; I'm also assuming that every "voidstar" member is a struct or union that has been wrapped using this macro) and I'd be thankful for any suggestions on how to improve this, but this is not my main question. I still have the somewhat converse problem: when Gambit loses sight of your last Scheme reference to a "parent" struct, it will clean it up even if there are Scheme references to "child" structs. I would like to somehow make a Scheme reference to the "parent" struct from the "child" structs, so that children will keep the parent alive as long as they are around. This reference needs not be visible from Scheme; it's only to help the garbage collector do the right thing. I can think of a couple ways to do this, but they either complicate my macro considerably, or make things more cumbersome for the user -- or both. I'm asking the list just in case I'm missing an easy way to create these links. [1] I say "structs" in the rest of this message to refer to both C `struct` and `union`s. [2] All my code in github is under the WTFPL license (profanity warning): http://sam.zoy.org/wtfpl/

3 8

KCopy
by Álvaro Castro-Castilla 25 Mar '12

25 Mar '12

Hi, Compiling the Linux kernel 3.2.12, I've seen this new feature: KCOPY: "High-performance inter-process memory copies. Can often save a memory copy to shared memory in the application. Useful at least for MPI applications where the point-to-point nature of vmsplice and pipes can be a limiting factor in performance." Could Gambit easily exploit this feature to optimize message passing in multicore machines with several Gambit instances, at least under linux?

1 0

[ANN] Gambit-C v4.6.5 released
by gambit＠iro.umontreal.ca 24 Mar '12

24 Mar '12

Gambit-C v4.6.5 is now available. Changelog: http://www.iro.umontreal.ca/~gambit/repo/.cgit.cgi/Gambit/log/ The sources and prebuilt distributions can be obtained from the Gambit web site by visiting one of the following links. sources: http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/source/gambc-v4_6_… (for typical users) http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/source/gambc-v4_6_… (for developers) prebuilt: http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/prebuilt/gambc-v4_… http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/prebuilt/gambc-v4_… http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/prebuilt/gambc-v4_… http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/prebuilt/gambc-v4_…

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Gambit-list Mars 2012